DWhite Chat (DWC) is a reference application using DWhite. DWC nodes represent a chat room with possibly thousands or millions of small messages.
The two primary node types in DWC are the Channel node and the Message node. A secondary node type, an Index node, is used to partition the tree into smaller, more easily managed portions. In the following description of properties, those properties marked with an asterisk (*) are copied with partial nodes.
A Channel node contains the following properties:
* DW:Id The channel's DWhite identifier. * DW:Signature The channel's DWhite signature, will change over time. * Name The channel name of the chat room. * Topic The topic of the chat room. * Contents The contents of the chat room, a list of either Index nodes or Message nodes (not mixed). * FirstMsgNum The message number of the first message. * LastMsgNum The message number of the last message. * FirstMsgDate The date and time of the first message. * LastMsgDate The date and time of the last message.
A Message node contains the following properties:
* DW:Id The message's DWhite identifier. * DW:Signature The message's DWhite signature, will never change. * From An identifier (email? URI? UUID?) of the sender. * Date The date when the message entered the tree. * MsgNum The sequence number of the message. Body A string (mixed content?) of up to 500 characters containing the body of the message.
An Index node contains the following properties:
* DW:Id The index's DWhite identifier. * DW:Signature The index's DWhite signature, will never change after the 50th subnode is inserted. * Contents The contents of the index, a list of either Index nodes or Message nodes (not mixed). * FirstMsgNum The message number of the first message. * LastMsgNum The message number of the last message. * FirstMsgDate The date and time of the first message. * LastMsgDate The date and time of the last message.
The Contents of a Channel or Index node will, at any time, only contain up to 50 subnodes. As soon as the 51st subnode would have been created, an Index node is created and the 50 subnodes are moved to the new Index node. Another Index node will be created and the 51st subnode will be placed in the new Index node. The two Index nodes together replace the Contents of the Channel or Index node that contained the subnodes. In this manner, a Channel's Contents may contain 0-N levels of Index nodes, with leaf Index nodes containing messages. Index nodes and Message nodes are never mixed within the Contents of a Channel or Index node; the Contents will contain only all Index nodes or only all Message nodes.
DWC has getMessages(), a "View" method, and postMessage() and setTopic() "Controller" methods. DWC methods are directed towards the root of the tree.
At startup, the client application is expected to use getMessage(-n, -1) (n between 50-500) to retrieve partial copies of the last n messages, and then periodically call getMessage(m, -1), where m is the message number of the last retrieved message plus one. The application could also use checkNode() on the Index or Channel nodes. After calling getMessage(), the application then calls getNode() for each message to retrieve the fully populated Message node.
Notes


Product
Inventory
OrderHistory
Sign Out
Product Lists
ProductId
Name
Description
Number Of TeaBags
Price
Name:
Description:
Number of Tea Bags:
Price:


AIADMK scared of losing mayoral posts: Kanimozhi
MP brushes aside Rajini’s remark
by Special CorrespondentDMK Member of Parliament Kanimozhi has stressed the need for holding civic polls in Tamil Nadu in a fair and just manner without giving room for any confusion or objectionable practices.
The MP told presspersons here on Thursday that the Tamil Nadu Government had promulgated the ordinance for indirect election of mayors and chairpersons of municipal corporations out of frustration as the ruling AIADMK was afraid of losing the posts in case of direct elections.
Ms. Kanimozhi said that the fiscal position of the Union government was so poor that MPs were yet to be sanctioned their local area development fund (MPLAD). “This is a sorry spectacle,” she said.
Rajini’s comments
Referring to the claim of actor Rajinikanth that there was a vacuum in leadership in Tamil Nadu, she said the DMK had already proved that there was no vacuum with the outcome in the Lok Sabha polls. “There needs no further explanation or clarification,” she said.
She said the media and the people’s representatives were not permitted to visit Jammu and Kashmir to have a first hand information on the prevailing situation. But parliamentarians from other countries were invited and taken on a visit to the Valley.
“We have made a plea to the Centre to release National Conference (NC) patron Farooq Abdullah and leaders of other parties but nothing has been heard from them,” she said.


Death Row inmate Ray Cromartie killed by lethal injection after humongous last meal
Ray "Jeff" Cromartie, 52, was put to death for murdering shop owner Richard Slysz in Georgia, USA, in 1994 before fleeing with two 12-packs of beer - and he ordered a gigantic last meal before his execution
by Bradley Jolly, Matthew Dresch, https://www.mirror.co.uk/authors/bradley-jolly/, https://www.mirror.co.uk/authors/matthew-dresch/Death row inmate Ray "Jeff" Cromartie has been executed for gunning down a shop owner during a robbery.
Cromartie, 52, had always admitted he and his co-defendant Corey Clark were guilty of the raid, but claimed neither he or his pal knew who pulled the trigger to shoot Richard Slysz in Georgia, USA, in 1994.
Cromartie was originally scheduled to die on October 30 but his execution was delayed as an appeal was still pending with Georgia's Supreme Court.
However, his argument was dismissed and he died by lethal injection at 10.59pm local time in Georgia Diagnostic Prison today.
He declined to make a final statement.
His last meal - the second he has had - consisted of macaroni and cheese, steak, rice and gravy, a steak and cheese sandwich, a double cheeseburger, French fries, a side of ranch dressing, a strawberry milkshake and layered cake with white icing.
He had a previous last meal in October before his execution was delayed.
Mr Slysz, 50, was murdered at Junior Food Store in Thomasville, near the Florida border.
But Clark and getaway driver, Thad Lucas, were both cleared of murder charges.
They were imprisoned for the roles in the heist but released in the early 2000s.
Speaking previously after Cromartie made his appeal, Melven Johnson, of the Thomasville Police Department, said: "A jury has found him guilty and sentenced him to death.
"I think to give these families and this victim some satisfaction and let them get this off their minds, it needs to happen."
In April 1994, Cromartie borrowed a gun from his cousin and went to the Madison Street Deli in Thomasville and shot and wounded store clerk, Dan Wilson, in the face, court papers said.
Three days later, Cromartie and his friends Corey Clark and Thaddeus Lucas went to Junior Food Store in Thomasville, Georgia to steal beer.
When they entered the store, Cromartie shot Mr Slysz twice in the head, killing him, prosecutors said.
The two criminals then tried to open the cash register but were unsuccessful.
Cromartie took two 12-packs of Budweiser beer and the men fled, according to court papers.
He is the third person to be executed in Georgia this year.


Home
About Us
All Team
Marketing Page
Front End
Back End
Data Science
PM
Petar Vlaisavljevic
Aspiring programmer interested in marketing and bitcoin.
Jay Maas
Climbing the mountain into the tech industry with the best Sherpa in the game - Lambda.
Kimberlee Johnson
Worked at Stripe for four years before deciding to become a developer.
Timothy Shores
Former marketer at a fortune 500 company turned full-stack web developer.
Cole Hudson
Data ninja.
Dmitriy Kavyazin
Generalist Data Scientist and avid learner with evolving interests.
Clint Kunz
"Not great at karate, but I still kick butt"


Differences from standard testing
Single-sourced:
looking mostly for common-ish vulns
not competing with others
incentivized for count
payment guaranteed and quality check based on approximation
Crowdsourced: - looking for vulns that aren’t as easy to find
racing vs. time
competitive vs. others
incentivized to find unique bugs
payment based on impact not number of findings


Introduction to Omar Data Structures
New to Open edX? Learn all about Open edX’ core data structures, and see how to apply them in the real world.


A responsive navbar
A login form
A card
A brief summary of the content of your card
typography
p
A button that highlights


Source code for image.image
from astropy.io import fits
import numpy as np
import logging
from datetime import date
log = logging.getLogger('main')
[docs]class Image:
"""
Image is the basic class for raw and partially reduced CHARIS data.
It must have at least the following boolean attribute references:
self.destriped (default False)
self.flatfielded (default False)
It must have at least the following other attribute references:
self.data (default None)
self.ivar (default None)
self.header (default None)
self.data, self.ivar, and self.header should be numpy ndarrays,
which can be read from and written to a fits file with the load
and write methods. If not ndarrays, they should be None.
Image may be initialized with the name of the raw file to read,
through a call to Image.load().
"""
def __init__(self, filename='', data=None, ivar=None, chisq=None,
header=fits.PrimaryHDU().header, extrahead=None,
reads=None, flags=None):
'''
Image initialization
Parameters
----------
filename: string
Name of input file
data: ndarray
Numpy ndarray containing your data. Can be multi-dimensional.
ivar: ndarray
Numpy ndarray containing the inverse variance of the data. Should be same shape as data
chisq: ndarray
Numpy ndarray containing chisq value for each ramp fit to the data. Should be same shape as data
header: `PrimaryHDU` header
Empty header instance
extraheader: `PrimaryHDU` header
Placeholder for header from original ramp
reads: ndarray
flags: ndarray
'''
self.data = data
self.ivar = ivar
self.chisq = chisq
self.header = header
self.reads = reads
self.flags = flags
self.filename = filename
self.extrahead = extrahead
if data is None and filename != '':
self.load(filename)
[docs] def load(self, filename, loadbadpixmap=False):
"""
Image.load(outfilename)
Read the first HDU with data from filename into self.data, and
HDU[0].header into self.header. If there is more than one HDU
with data, attempt to read the second HDU with data into
self.ivar.
Parameters
----------
filename: string
Name of input file
loadbadpixmap: boolean
When True, loads the bad pixel map at `calibrations/mask.fits`
"""
try:
self.filename = filename
hdulist = fits.open(filename)
self.header = hdulist[0].header
if hdulist[0].data is not None:
i_data = 0
else:
i_data = 1
self.data = hdulist[i_data].data
log.info("Read data from HDU " + str(i_data) + " of " + filename)
if len(hdulist) > i_data + 1:
self.ivar = hdulist[i_data + 1].data
if self.ivar.shape != self.data.shape:
log.error("Error: data (HDU " + str(i_data) +\
") and inverse variance (HDU " + str(i_data +\
1) + ") have different shapes in file " + filename)
self.ivar = None
else:
log.info("Read inverse variance from HDU " + str(i_data + 1) + " of " + filename)
elif loadbadpixmap:
self.ivar = fits.open('calibrations/mask.fits')[0].data
else:
self.ivar = None
except:
log.error("Unable to read data and header from " + filename)
self.data = None
self.header = None
self.ivar = None
[docs] def write(self, filename, clobber=True):
"""
Image.write(outfilename, clobber=True)
Creates a primary HDU using self.data and self.header, and
attempts to write to outfilename. If self.ivar is not None,
append self.ivar as a second HDU before writing to a file.
clobber is provided as a keyword to fits.HDUList.writeto.
Parameters
----------
filename: string
Name of destination file
clobber: boolean
When True, overwrites if file already exists
"""
hdr = fits.PrimaryHDU().header
today = date.today().timetuple()
yyyymmdd = '%d%02d%02d' % (today[0], today[1], today[2])
hdr['date'] = (yyyymmdd, 'File creation date (yyyymmdd)')
for i, key in enumerate(self.header):
hdr.append((key, self.header[i], self.header.comments[i]), end=True)
out = fits.HDUList(fits.PrimaryHDU(None, hdr))
out.append(fits.PrimaryHDU(self.data.astype(np.float32),hdr))
if self.ivar is not None:
out.append(fits.PrimaryHDU(self.ivar.astype(np.float32),hdr))
if self.chisq is not None:
out.append(fits.PrimaryHDU(self.chisq.astype(np.float32),hdr))
if self.flags is not None:
out.append(fits.PrimaryHDU(self.flags),hdr)
if self.extrahead is not None:
try:
out.append(fits.PrimaryHDU(None, self.extrahead))
except:
log.warn("Extra header in image class must be a FITS header.")
try:
out.writeto(filename, clobber=clobber)
log.info("Writing data to " + filename)
except:
log.error("Unable to write FITS file " + filename)


Current Temperature is
Power Saving Mode:
On
Up
Down
Reset
Power Saving Mode on
Power Saving Mode off


Maths Constants
Math constants are E, PI, SQR2, SQR1_2, LN2, LN10, LOG2E, LOG10E
Try it
Calculator Program Lite
Calculator Program Pro
Exclusive to some people (10 cents)


| Content: (annotation?, (simpleContent | complexContent | ((group | all | choice | sequence)?, ((attribute | attributeGroup)*, anyAttribute?)))) |
| </complexType> |
Notation
The following auxiliary grammar productions are used to describe the content of a complex type definition.
| [57 (Formal)] | ComplexTypeContent |
::= | "annotation"? ("simpleContent" | "complexContent" |
(ChildrenContent AttributeContent)) |
| [60 (Formal)] | AttributeContent |
::= | ("attribute" | "attributeGroup")*
"anyAttribute"? |
| [58 (Formal)] | ChildrenContent |
::= | ("group" | "all" | "choice" | "sequence")? |
Schema import distinguishes between global complex types (which are mapped to sort declarations) and local complex types (which are mapped to type definitions).
Schema mapping
In the case of global complex types, the mapping rule which applies is denoted by []definition(targetNCName).
[
|
|||||
| == | |||||
| define type targetNCName:NCName [MixedAttribute ComplexTypeContent]mixed_content(targetNCName) |
Note that the mixed
is passed along in the
normalization rules, in order to map it later on to the appropriate
indication in the [XPath/XQuery] type system.
Schema mapping
In the case of a local complex types, there must not be a name attribute and the mapping rule which applies is denoted by []content(targetNCName).
[
|
||||
| == | ||||
| [MixedAttribute ComplexTypeContent]mixed_content(targetNCName) |
Note that the mixed
is passed along in the
normalization rules, in order to map it later on to the appropriate
indication in the [XPath/XQuery] type system.
Schema component
A complex type can be of simple content. A simple content is represented in XML by the following structure.
| <simpleContent |
| [ ignored ] id = ID |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (restriction | extension)) |
| </simpleContent> |
Derivation by restriction inside a simple content is represented in XML by the following structure.
| <restriction |
| base = QName |
| [ ignored ] id = ID |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (simpleType?, (minExclusive | minInclusive | maxExclusive | maxInclusive | totalDigits | fractionDigits | length | minLength | maxLength | enumeration | whiteSpace | pattern)*)?, ((attribute | attributeGroup)*, anyAttribute?)) |
| </restriction> |
Derivation by extension inside a simple content is represented in XML by the following structure.
| <extension |
| base = QName |
| [ ignored ] id = ID |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, ((attribute | attributeGroup)*, anyAttribute?)) |
| </extension> |
Notation
The normalization rule
| [MixedAttribute ComplexTypeContent]mixed_content(targetNCName) |
| == |
| TypeDerivation |
maps a pair of mixed attribute and complex type content to a type derivation.
Schema mapping
A complex types with simple content must not have a
mixed
attribute set to "true".
If the simple content is derived by restriction, it is mapped into a simple type restriction in the [XPath/XQuery] type system. Only the name of the base atomic type and attributes are mapped, while the actual simple type restriction is ignored. (Remember that facets are not captured in the [XPath/XQuery] type system.)
[
|
|||||||||
| == | |||||||||
| restricts QName { [AttributeContent]content(targetNCName) QName } |
If the simple type is derived by extension, it is mapped into an extended type specifier into the [XPath/XQuery] type system.
[
|
|||||||||
| == | |||||||||
| extends QName { [AttributeContent]content(targetNCName) } |
Schema component
A complex type can be of complex content. A complex content is represented in XML by the following structure.
| <complexContent |
| [ ignored ] id = ID |
| mixed = boolean : false |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (restriction | extension)) |
| </complexContent> |
Derivation by restriction inside a complex content is represented in XML by the following structure.
| <restriction |
| base = QName |
| [ ignored ] id = ID |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (group | all | choice | sequence)?, ((attribute | attributeGroup)*, anyAttribute?)) |
| </restriction> |
Derivation by extension inside a complex content is represented in XML by the following structure.
| <extension |
| base = QName |
| [ ignored ] id = ID |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, ((group | all | choice | sequence)?, ((attribute | attributeGroup)*, anyAttribute?))) |
| </extension> |
Schema mapping
If the complex content is derived by restriction, it is mapped into a type restriction in the [XPath/XQuery] type system, and the
[
|
|||||||||
| == | |||||||||
| restricts QName [MixedAttribute]mixed { [AttributeContent]content(targetNCName) [ChildrenContent]content(targetNCName) } |
If the complex content is derived by extension, it is mapped into an extended type specifier into the [XPath/XQuery] type system.
[
|
|||||||||
| == | |||||||||
| extends QName [MixedAttribute]mixed { [AttributeContent]content(targetNCName) [ChildrenContent]content(targetNCName) } |
Mapping for attribute uses is given in [D.1.4 Special attributes].
Schema component
Model group definitions are represented in XML by the following structure.
| <attributeGroup |
| [ ignored ] id = ID |
| name = NCame |
| ref = QName |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, ((attribute | attributeGroup)*, anyAttribute?)) |
| </attributeGroup> |
Schema mapping
Attribute group definitions are not currently handled by the mapping. See Issue 501 (FS-Issue-0158).
Schema component
Model group definitions are represented in XML by the following structure.
| <group |
| name = NCame > |
| Content: (annotation?, (all | choice | sequence)) |
| </group> |
Schema mapping
Model group definitions are not currently handled by the mapping. See Issue 501 (FS-Issue-0158).
Model groups are either "all", "sequence" or "choice". One can also refer to a model group definition.
Schema component
All groups are represented in XML by the following structure.
| <all |
| [ ignored ] id = ID |
| maxOccurs = 1 : 1 |
| minOccurs = (0 | 1) : 1 |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, element*) |
| </all> |
Schema mapping
All groups are mapped into the "&" operation in the [XPath/XQuery] type system.
[
|
||||
| == | ||||
| ([Element1]content(targetNCName) & ... & [Elementn]content(targetNCName)) [OccursAttributes]occurs |
Schema component
Choice groups are represented in XML by the following structure.
| <choice |
| [ ignored ] id = ID |
| maxOccurs = (nonNegativeInteger | unbounded) : 1 |
| minOccurs = nonNegativeInteger : 1 |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (element | group | choice | sequence | any)*) |
| </choice> |
Notation
The following auxiliary grammar productions are used to describe group components.
| [59 (Formal)] | GroupComponent |
::= | "element" | "group" | "choice" | "sequence" |
"any" |
Schema mapping
Choice groups are mapped into the "|" operation in the [XPath/XQuery] type system.
[
|
||||
| == | ||||
| ([GroupComponent1]content(targetNCName) | ... | [GroupComponentn]content(targetNCName)) [OccursAttributes]occurs |
Schema component
Sequence groups are represented in XML by the following structure.
| <sequence |
| [ ignored ] id = ID |
| maxOccurs = (nonNegativeInteger | unbounded) : 1 |
| minOccurs = nonNegativeInteger : 1 |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (element | group | choice | sequence | any)*) |
| </sequence> |
Schema mapping
Sequence groups are mapped into the "," operation in the [XPath/XQuery] type system.
[
|
||||
| == | ||||
| ([GroupComponent1]content(targetNCName) , ... , [GroupComponentn]content(targetNCName)) [OccursAttributes]occurs |
Particles contribute to the definition of content models.
A particle can be either an element reference, a group reference or a wildcard.
Schema component
Element reference particles are represented in XML by the following structure.
| <element |
| ref = QName |
| maxOccurs = (nonNegativeInteger | unbounded) : 1 |
| minOccurs = nonNegativeInteger : 1 |
| [ ignored ] {any attributes with non-schema namespace ...} > |
Schema mapping
Element references are mapped into element references in the [XPath/XQuery] type system.
[
|
|||
| == | |||
| element QName [OccursAttributes]occurs |
Schema component
Group reference particles are represented in XML by the following structure.
| <group |
| ref = QName |
| maxOccurs = (nonNegativeInteger | unbounded) : 1 |
| minOccurs = nonNegativeInteger : 1 |
| [ ignored ] {any attributes with non-schema namespace ...} > |
Schema mapping
Model group references are not currently handled by the mapping.
Schema component
Attribute wildcards are represented in XML by the following structure.
| <anyAttribute |
| [ ignored ] id = ID |
| [ not handled ] namespace = ((##any | ##other) | List of (anyURI | (##targetNamespace | ##local)) ) : ##any |
| processContents = (lax | skip | strict) : strict |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?) |
| </anyAttribute> |
Schema mapping
An attribute wildcard with a "skip" process content is mapped as an attribute wildcard in the [XPath/XQuery] type system.
[
|
||||
| == | ||||
(attribute (*,
xs:untypedAtomic ))* |
[
|
||||
| == | ||||
| attribute * |
[
|
||||
| == | ||||
| attribute * |
| Editorial note | |
| Namespace wildcards are not handled by the mapping. |
Schema component
Element wildcards are represented in XML by the following structure.
| <any |
| [ ignored ] id = ID |
| maxOccurs = (nonNegativeInteger | unbounded) : 1 |
| minOccurs = nonNegativeInteger : 1 |
| [ not handled ] namespace = ((##any | ##other) | List of (anyURI | (##targetNamespace | ##local)) ) : ##any |
| processContents = (lax | skip | strict) : strict |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?) |
| </any> |
Schema mapping
An element wildcard with a "skip" process content is mapped as an element wildcard in the [XPath/XQuery] type system.
[
|
|||||
| == | |||||
( element (*, xs:untyped ) )[OccursAttributes]occurs |
[
|
|||||
| == | |||||
( element (*,
xs:anyType ) )[OccursAttributes]occurs |
| Editorial note | |
| Element wildcards with a "lax" or "strict" process content are not handled by the mapping. |
| Editorial note | |
| Namespace wildcards are not handled by the mapping. |
All identity-constraints definitions are ignored when mapping into the [XPath/XQuery] type system.
All notation declarations are ignored when mapping into the [XPath/XQuery] type system.
Schema component
A simple type is represented in XML by the following structure.
| <simpleType |
| [ ignored ] final = (#all | (list | union | restriction)) |
| [ ignored ] id = ID |
| name = NCName |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| name = NCName |
| </simpleType> |
Derivation by restriction inside a simple type is represented in XML by the following structure.
| <restriction |
| base = QName |
| [ ignored ] id = ID |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (simpleType?, (minExclusive | minInclusive | maxExclusive | maxInclusive | totalDigits | fractionDigits | length | minLength | maxLength | enumeration | whiteSpace | pattern)*)?) |
| </restriction> |
Derivation by list inside a simple type is represented in XML by the following structure.
| <list |
| [ ignored ] id = ID |
| itemType = QName |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (simpleType?)) |
| </list> |
Derivation by union inside a simple type is represented in XML by the following structure.
| <union |
| [ ignored ] id = ID |
| memberTypes = List of QName |
| [ ignored ] {any attributes with non-schema namespace ...} > |
| Content: (annotation?, (simpleType*)) |
| </union> |
Schema import distinguishes between global simple types (which are mapped to sort declarations) and local simple types (which are mapped to type definitions).
Schema mapping
In the case of global simple types, the mapping rule which applies is denoted by []definition(targetNCName).
[
|
||||
| == | ||||
| define type targetNCName:NCName [SimpleTypeContent]simple_content(targetNCName) |
Schema mapping
In the case of global simple types, the mapping rule which applies is denoted by []content(targetNCName).
[
|
|||
| == | |||
| [SimpleTypeContent]simple_content(targetNCName) |
Notation
The normalization rule []simple_content(targetNCName) maps a simple type content to a type specifier and an optional occurrence indicator.
Schema mapping
If the simple type is derived by restriction, it is mapped into a simple type restriction in the [XPath/XQuery] type system. The name of the base atomic type and attributes are mapped. Only the minLength, maxLength, and length facets in the simple type restriction are handled. All other properties of the simple-type restriction are ignored.
[
|
||||
| == | ||||
| restricts QName { QName } [simpleContentRestriction]occurs |
If the simple type is derived by list, and its content type does not constrain the length of the list, it is mapped into a zero-or-more repetition type into the [XPath/XQuery] type system.
[
|
|||
| == | |||
| { Type * } |
If the simple type is derived by list, and its content type does constrain the length of the list, then it is mapped into a zero-or-more repetition type into the [XPath/XQuery] type system.
[
|
|||
| == | |||
| { Type · OccurrenceIndicator } |
[
|
||
| == | ||
| { QName* } |
If the simple type is derived by union, it is mapped into a union type into the [XPath/XQuery] type system.
[
|
|||
| == | |||
| { ([SimpleType]content(targetNCName) | ... | [SimpleTypen]content(targetNCName)) } |
[
|
||
| == | ||
| { QName1 | ... | QNamen } |
XQuery supports XML Schema validation using the validate expression. This section gives a non-normative formal semantics of XML Schema validation, solely for the purpose of specifying its usage in XQuery.
Specifying XML Schema validation requires a fairly large number of auxiliary judgments. There are two main judgments used to describe the semantics of validation.
The "erase" judgment takes a value and removes all type information from it. This operation is necessary since, in XQuery, validation can occur both on well-formed or already validated documents.
The "annotate" operation takes an untyped value and a type and either fails or succeeds by returning a new -validated- value.
Before defining these two judgments, we first introduce the auxiliary judgments used to describe specific parts of XML Schema's semantics.
Notation
The judgments
and
hold when a type reference (resp. a type derivation) resolves to the given type name and type content.
Semantics
Those judgments are specified by the following rules.
If the type is omitted, it is resolved as the empty sequence type.
| statEnv |- OptDerivation OptMixed { empty } resolves to TypeName { Type } |
|
|
| statEnv |- OptDerivation OptMixed { } resolves to TypeName { Type } |
In case of a type reference, then the type name is the name of that type, and the type is taken by resolving the type declaration of the global type.
|
||||
|
|
||||
| statEnv |- of type TypeName resolves to TypeName { Type } |
In the above inference rule, note that BaseTypeName is the base type of the type referred to. So this is indeed the original type name, TypeName, which must be returned, and eventually used to annotated the corresponding element or attribute. However, the type needs to be obtained through a second application of the resolves to judgment.
If the type derivation is a restriction, then the type name is the name of the base type, and the type is taken from the type derivation.
| statEnv |- OptMixed Type adjusts to AdjustedType |
|
|
| statEnv |- restricts TypeName OptMixed { Type } resolves to TypeName { AdjustedType } |
If the type derivation is an extension, then the type name is the name of the base type, and the type is the base type extended by the type in the type derivation.
|
|||||
|
|
|||||
| statEnv |- extends TypeName OptMixed { Type } resolves to TypeName { AdjustedType } |
Notation
The judgment
holds if some interleaving of Value1 and Value2 yields Value3. Interleaving is non-deterministic; it is used for processing all groups.
Semantics
This judgment is specified by the following rules.
Interleaving two empty sequences yields the empty sequence.
|
|
| statEnv |- () interleave () yields () |
Otherwise, pick an item from the head of one of the sequences, and recursively interleave the remainder.
Introduction
Finally, we introduce an auxiliary judgment which extracts the value of a given attribute if it exists. This judgment is not used in the semantics of step expressions, but in [8.3 Judgments for type matching], and is based on the other filter judgments.
Notation
The judgment
holds if there are no occurrences of the attribute QName in Value. The judgment
holds if there is one occurrence of the attribute QName in Value, and the value of that attribute is SimpleValue.
Semantics
The filter judgments are defined as follows.
Notation
To define erasure, an auxiliary judgment is needed. The judgment
holds when SimpleValue erases to the string String.
Semantics
This judgment is specified by the following rules.
The empty sequence erases to the empty string.
|
|
| () simply erases to "" |
The concatenation of two non-empty sequences of values erases to the concatenation of their erasures with a separating space.
|
|||||
|
|
|||||
SimpleValue1,SimpleValue2 simply erases to
fn:concat (String1," ",String2) |
An atomic value erases to its string representation as an
instance of xs:untypedAtomic
.
|
|
AtomicValueContent of
type AtomicTypeName
simply erases
to dm:string-value (AtomicValueContent) |
Notation
The erases to judgment is used in the definition of the dynamic semantics of validation. The normative dynamic semantics of validation is specified in Section 3.13 Validate ExpressionsXQ. The effect of the validate expression is equivalent to:
serialization of the data model, as described in [XSLT 2.0 and XQuery 1.0 Serialization (Second Edition)], followed by
validation of the serialized value into a Post-Schema Validated Infoset, as described in [Schema Part 1], followed by
construction of a new data model value, as described in [XQuery 1.0 and XPath 2.0 Data Model (Second Edition)].
Erasure is the formal equivalent of serialization followed by
construction of a new data model value in which all element nodes
are labeled with xs:untyped
and all attribute
nodes with xs:untypedAtomic
.
The judgment
holds when the erasure of Value1 is Value2.
Semantics
This judgment is specified by the following rules.
The empty sequence erases to itself.
|
|
| () erases to () |
The erasure of the concatenation of two values is the concatenation of their erasure, so long as neither of the two original values is simple.
|
|||
|
|
|||
| Value1,Value2 erases to Value1',Value2' |
The erasure of an element is an element that has the same name
and the type xs:untyped
and the erasure of the original content.
|
||
|
|
||
element ElementName
of type TypeName { Value1 } erases to element ElementName of type xs:untyped { Value2 } |
The erasure of an attribute is an attribute that has the same
name and the type xs:untypedAtomic
and the
simple erasure of the original content labeled with xs:untypedAtomic
.
|
||
|
|
||
attribute AttributeName of type TypeName { SimpleValue } erases to attribute
AttributeName of type
xs:untypedAtomic {
String of type xs:untypedAtomic } |
The erasure of a document is a document with the erasure of the original content.
The erasure of a text or comment or processing-instruction node is itself.
|
|
| text { String } erases to text { String } |
|
|
| comment { String } erases to comment { String } |
|
|
| processing-instruction NCName { String } erases to processing-instruction NCName { String } |
The erasure of a simple value is the corresponding text node.
|
||
|
|
||
| SimpleValue erases to text { String } |
The annotate as judgment is used in the definition of the dynamic semantics of validation. The normative dynamic semantics of validation is specified in Section 3.13 Validate ExpressionsXQ. The effect of the validate expression is equivalent to:
serialization of the data model, as described in [XSLT 2.0 and XQuery 1.0 Serialization (Second Edition)], followed by
parsing of the serialized value into the Infoset
validation of the Infoset into a Post-Schema Validated Infoset, as described in [Schema Part 1], followed by
construction of a new data model value, as described in [XQuery 1.0 and XPath 2.0 Data Model (Second Edition)].
Annotation is the formal equivalent of schema validation of an
Infoset value into the PSVI followed by construction of a new data
model value. Because the Formal Semantics is defined on data model
values, not the Infoset, annotation is applied to data model values
in which all element nodes are labeled with xs:untyped
and all attribute
nodes with xs:untypedAtomic
-- that
is, the result of erasure.
Notation
The judgment
holds if the result of casting the SimpleValue1 to SimpleType is SimpleValue2.
Semantics
This judgment is specified by the following rules.
Simply annotating a simple value to a union type yields the result of simply annotating the simple value to either the first or second type in the union. Note that simply annotating to the second type is attempted only if simply annotating to the first type fails.
|
||
|
|
||
| statEnv |- simply annotate as SimpleType1|SimpleType2 (SimpleValue1) => SimpleValue2 |
|
|||
|
|
|||
| statEnv |- simply annotate as SimpleType1|SimpleType2 (SimpleValue1) => SimpleValue2 |
The simple annotation rules for ?, +, * are similar.
|
|
| statEnv |- simply annotate as SimpleType? ( () ) => () |
|
||
|
|
||
| statEnv |- simply annotate as SimpleType? (SimpleValue1) => SimpleValue2 |
|
|
| statEnv |- simply annotate as SimpleType* ( () ) => () |
|
|||
|
|
|||
| statEnv |- simply annotate as SimpleType* (SimpleValue1,SimpleValue2) => SimpleValue1',SimpleValue2' |
|
|||
|
|
|||
| statEnv |- simply annotate as SimpleType+ (SimpleValue1,SimpleValue2) => SimpleValue1',SimpleValue2' |
Simply annotating an atomic value to xs:string
yields its string representation.
|
|
statEnv |- simply annotate as xs:string
(AtomicValue) =>
dm:string-value(AtomicValue) |
Simply annotating an atomic value to xs:decimal
yields the decimal that results from parsing its string
representation.
|
|
statEnv |- simply annotate as xs:decimal
(AtomicValue) =>
xs:decimal (dm:string-value (AtomicValue)) |
Similar rules are assumed for the rest of the 19 XML Schema primitive types.
Notation
The judgment
holds if it is possible to annotate value Value1 as if it had the nillable type Type and Value2 is the corresponding annotated value.
Semantics
This judgment is specified by the following rules.
If the type is not nillable, then the xsi:nil attribute must not appear in the value, and it must be possible to annotate value Value as if it had the type Type.
|
|||
|
|
|||
| statEnv |- nil-annotate as Type ( Value1 ) => Value2 |
If the type is nillable, and the xsi:nil attribute does not appear or is false, then it must be possible to annotate value Value1 as if it had the type Type.
|
||||
|
|
||||
| statEnv |- nil-annotate as nillable Type ( Value1 ) => Value2 |
If the type is nillable, and the xsi:nil attribute is true, then it must be possible to annotate value Value1 as if it had a type where the attributes in the type are kept and the element content of the type is ignored.
|
|||
|
|
|||
| statEnv |- nil-annotate as nillable (AttributeModel, ElementModel) ( Value1 ) => Value2 |
The annotate as judgment is used in the definition of the dynamic semantics of validation. The normative dynamic semantics of validation is specified in Section 3.13 Validate ExpressionsXQ. The effect of the validate expression is equivalent to:
serialization of the data model, as described in [XSLT 2.0 and XQuery 1.0 Serialization (Second Edition)], followed by
parsing of the serialized value into the Infoset
validation of the Infoset into a Post-Schema Validated Infoset, as described in [Schema Part 1], followed by
construction of a new data model value, as described in [XQuery 1.0 and XPath 2.0 Data Model (Second Edition)].
Erasure is the formal equivalent of serialization followed by
construction of a new data model value in which all element nodes
are labeled with xs:untyped
and all attribute
nodes with xs:untypedAtomic
.
Notation
The judgment
holds if it is possible to annotate value Value1 as if it had type Type and Value2 is the corresponding annotated value.
Note
Assume an XML Infoset instance X1 is validated against an XML Schema S, yielding PSVI instance X2. Then if X1 corresponds to Value1 and S corresponds to Type and X2 corresponds to Value2, the following should hold: annotate as Type ( Value1 ) => Value2.
Semantics
This judgment is specified by the following rules.
Annotating the empty sequence as the empty type yields the empty sequence.
|
|
| statEnv |- annotate as empty (()) => () |
Annotating a concatenation of values as a concatenation of types yields the concatenation of the annotated values.
|
|||
|
|
|||
| statEnv |- annotate as Type1,Type2 (Value1,Value2) => Value1',Value2' |
Annotating a value as a choice type yields the result of annotating the value as either the first or second type in the choice.
| statEnv |- annotate as Type1 (Value1) => Value2 |
|
|
| statEnv |- annotate as Type1|Type2 (Value1) => Value2 |
| statEnv |- annotate as Type2 (Value1) => Value2 |
|
|
| statEnv |- annotate as Type1|Type2 (Value1) => Value2 |
Annotating a value as an all group uses interleaving to decompose the original value and recompose the annotated value.
| Editorial note | |
| Jerome and Phil: Note that this may reorder the original sequence. Perhaps we should disallow such reordering. Specifying that formally is not as easy as we would like. |
|
|||||
|
|
|||||
| statEnv |- annotate as Type1 & Type2 ( Value ) => Value' |
The annotation rules for ?, +, * are similar.
| statEnv |- annotate as (Type | empty)(Value1) => Value2 |
|
|
| statEnv |- annotate as Type? (Value1) => Value2 |
| statEnv |- annotate as Type (Value1) => Value1' statEnv |- annotate as Type* (Value2) => Value2' |
|
|
| statEnv |- annotate as Type+ (Value1,Value2) => (Value1',Value2') |
|
|
| statEnv |- annotate as Type* ( () ) => () |
| statEnv |- annotate as Type (Value1) => Value1' statEnv |- annotate as Type* (Value2) => Value2' |
|
|
| statEnv |- annotate as Type* (Value1,Value2) => (Value1',Value2') |
To annotate an element with no xsi:type attribute, first look up the element type, next resolve the resulting type reference, then annotate the value against the resolved type, and finally return a new element with the name of the original element, the resolved type name, and the annotated value.
|
|||||
|
|
|||||
statEnv |- annotate as ElementType ( element ElementName of type
xs:anyType { Value } ) => element ElementName of type TypeName { Value' } |
To annotate an element with an xsi:type attribute, define a type reference corresponding to the xsi:type. Look up the element type, yielding a type reference, and check that the xsi:type reference derives from this type reference. Resolve the xsi:type reference, then annotate the value against the resolved type, and finally return a new element with the name of the original element, the resolved type name, and the annotated value.
|
|||||||
|
|
|||||||
statEnv |- annotate as ElementType ( element ElementName of type
xs:anyType { Value } ) => element ElementName of type TypeName { Value' } |
The rule for attributes is similar to the first rule for elements.
|
||||
|
|
||||
statEnv |- annotate as AttributeType ( attribute
AttributeName of type
xs:anySimpleType { SimpleValue1 } ) => attribute AttributeName of type TypeName { SimpleValue2 } |
Annotating a document node yields a document with the annotation of its contents.
| statEnv |- annotate as Type (Value) => Value' |
|
|
| statEnv |- annotate as document { Type } ( document { Value } ) => document { Value' } |
Annotating a text node as text yields itself.
|
|
| statEnv |- annotate as text (text { String }) => text { String } |
Annotating a text nodes as a simple type is identical to casting.
| statEnv |- simply annotate as SimpleType ( String ) => SimpleValue' |
|
|
| statEnv |- annotate as SimpleType ( text { String } ) => SimpleValue' |
Annotating a simple value as a simple type is identical to casting.
| statEnv |- simply annotate as SimpleType ( SimpleValue ) => SimpleValue' |
|
|
| statEnv |- annotate as SimpleType ( SimpleValue ) => SimpleValue' |
The changes made to this document are described in detail in the Errata to the first edition. The rationale for each erratum is explained in the corresponding Bugzilla database entry. The following table summarizes the errata that have been applied.
| Erratum | Bugzilla | Category | Description |
| E001 | 1641 3896 | editorial | Make normalization of DirAttributeValue more explicit. |
| E002 | 3864 | editorial | Both Core and Formal grammars had symbols 'NamespaceBinding' and 'NamespaceBindings'. To avoid confusion, we rename the Core symbols 'NamespaceBinding[s]' as 'LocalNamespaceDecl[s]', changing all occurrences of the former to the latter. At the same time, we introduce a production for LocalNamespaceDecls (currently assumed). |
| E003 | 1647 | editorial | Complete the changes entailed by adding a local-namespaces component to the Core CompElemConstructor. |
| E004 | 1746 | editorial | Fix typo in function signature. |
| E005 | 1660 | editorial | (superseded) |
| E006 | 3875 3184 3885 3194 1715 | editorial | Fix problems involving the domains of statEnv.funcType and dynEnv.funcDefn: Although section 3.1.1 declares the domain of statEnv.funcType to be (expanded-QName, arity), some inference rules neglect the arity. And although section 3.1.2 declares the domain of dynEnv.funcDefn to be expanded-QName(Type1, ..., Typen), this leads to misunderstandings and mistakes; dynEnv.funcDefn should have the same domain as statEnv.funcType. To simplify some of these changes, we introduce the Formal symbol FunctionKey to represent the common domain. |
| E007 | 1694 | editorial | Fix miscellaneous small errors in section 5. |
| E008 | 1680 | editorial | Fix a bug in the normalization of function calls. |
| E009 | 3142 | editorial | Fix some errors in the productions for FunctionSig and TypeList. |
| E010 | 3670 | editorial | Clean up due to the removal of op:anyURI-equal from the F+O spec. |
| E011 | 3758 3760 | editorial | Correct/simplify/complete the Normalization rules defining []ElementContent and []AttributeContent, and simplify the corresponding Dynamic Evaluation rules. |
| E012 | 3771 | editorial | Static Type Analysis must allow for empty text nodes introduced during normalization of direct element constructors. |
| E013 | 1754 | editorial | Fix small errors in sections 7.2.1 and 7.2.2. |
| E014 | 1756 | editorial | (superseded) |
| E015 | 1783 | editorial | Fix some small errors in section 8.4. |
| E016 | 3847 | substantive | For internal function fs:idiv, correct the operand types and supporting op: function given in the Binary Operators table. |
| E017 | 4371 | editorial | Update the "Processing Model" diagram. |
| E018 | 4242 4261 4512 4581 5129 3269 | substantive | Correct various errors in section <IP_ADDRESS>.1. |
| E019 | 4578 | substantive | Correct a normalization error in section 4.3.2 Filter Expressions. |
| E020 | 4766 | editorial | Correct errors in examples in section 2.4.4. |
| E021 | 3818 | substantive | Avoid loss of type information in normalized path expressions. |
| E022 | 3946 | substantive | Make the static typing of processing-instructions consistent with their dynamic typing. |
| E023 | 3269 | editorial | ElementNameOrWildcard and AttributeNameOrWildcard do not derive empty. |
| E024 | 4841 | editorial | Introduce fs:item-at() function. |
| E025 | 5601 | substantive | For internal function fs:div, correct the result type in the Binary Operators table. |
| E026 | 1757 | editorial | Fix small errors in section 7.2.4 [The fn:boolean function]. |
| E027 | 5651 | substantive | Provide (non-default) static typing for fn:not(). |
| E028 | 5670 | substantive | Call fn:boolean() in normalization of Where clause |
| E029 | 3655 3771 4869 | substantive | Don't use the "interleave with empty text nodes" technique for element constructors. |
| E030 | 5986 | editorial | Fix cross-reference in <IP_ADDRESS>. |
| E031 | 3771 | substantive | Don't use the "interleave with empty text nodes" technique for attribute constructors. |
| E032 | 5747 | substantive | Rework the rules for CastableExpr to avoid raising type errors (and fix various other problems). |
| E033 | 5459 5460 | substantive | Fix the static typing for numeric and aggregate functions. |
| E034 | 1776 | editorial | Adjust the formatting of a rule in <IP_ADDRESS> [Dynamic semantics of axes]. |
| E035 | 3193 | editorial | Fix typo in 5.14 [Variable Declaration]. |
| E036 | 3268 | editorial | Introduce Formal symbol OptDerivation. |
| E037 | 3861 | editorial | Fix typo in 2.1.4 [Notations for inference rules]. |
| E038 | 3864 | editorial | Fix typos in 2.3.1 [Formal values]. |
| E039 | 3865 | editorial | Fix typo in 2.3.2 [Examples of values]. |
| E040 | 3866 | editorial | Fix typos in 2.4.1 [XML Schema and the [XPath/XQuery] Type System]. |
| E041 | 3868 | editorial | Fix typos in 2.4.3 [Content models]. |
| E042 | 3871 | editorial | In the production for Definitions, add a semicolon after each Definition. Adjust the rest of the spec accordingly. |
| E043 | 3871 | editorial | Fix typos in 2.4.4 [Top level definitions]. |
| E044 | 3873 | editorial | Fix typo in 2.5 [Functions and operators]. |
| E045 | 3876 | editorial | Fix typo in 3.2.2 [Normalization mapping rules]. |
| E046 | 3879 | editorial | Fix typos in 4 [Expressions]. |
| E047 | 3882 | editorial | Change some wording in 4.1.5 [Function Calls] / Normalization. |
| E048 | 3883 | editorial | Change some wording in 4.1.5 [Function Calls] / Static Type Analysis. |
| E049 | 3885 | editorial | Minor changes in 4.1.5 [Function Calls] / Dynamic Evaluation. |
| E050 | 3886 | editorial | Fix typo in 4.2.1 [Steps]. |
| E051 | 6005 | markup | Correctly render metavariables in mapping subscripts. |
| E052 | 3895 | editorial | Change some wording in 4.7.1 [Direct Element Constructors]. |
| E053 | 4447 | editorial | Fix incorrect references to standard functions/operators. |
| E054 | 4593 | editorial | Fix typo in 8.4 [Judgments for FLWOR and other expressions on sequences]. |
| E055 | 4601 | editorial | Change wording re fs:convert-operand in 4.5.2 [General Comparisons]. |
| E056 | 6007 | markup | Fix italicization in C.2 [Mapping of Overloaded Internal Functions]. |
| E057 | 6160 | substantive | Fix the static typing for fs:convert-operand(). |
| E058 | 5254 | substantive | Fix the static typing for ValidateExpr. |
| E059 | 4189 | substantive | Fix various bugs in the 'union interpretation' rules. |
| E060 | 6538 | substantive | Fix behaviour of following-sibling axis re attribute nodes. |
| E061 | 4273 | substantive | Fix unsoundness of 'data on' judgment. |
| E062 | 5452 | substantive | Don't define the static type of the namespace axis. |


{% extends "base.html" %} {% block title %} GAE Persona {% endblock %} {% block container_content %}
"Sign into sites across the Web quickly and easily with a single email of your choice."
For more information about Persona, Mozilla's implementation of Browser ID for secure authentication, see: https://www.mozilla.org/en-US/persona/
You can use mockmyid to test. See: https://mockmyid.com/


{% extends "home/base.html" %} {% load static %} {% load i18n %} {% block content %}
A new email has been associated with your GPCRmd account.
For more information on this form, see the docs.
Your email account has been set to {{ email }}. You can now access to your account.
{{ error_message }}


This site does not collect any information from you. It uses Gmail API directly from the browser to find train tickets from your inbox, and displays them to you. The data will only be saved locally in your browser.


Sorry, bad request
Error code: 400
If you believe this is an error, contact us
Name:
E-mail:
Comment:


Last updated: 2017-11-15
Code version: d35db9f
Spearman = Pearson on ranks
set.seed(3)
n = 20
noise = .4
x = runif(n)
y = x + rnorm(n, sd = noise)
plot(x,y)
cor.test(~x+y, method = "p")
Pearson's product-moment correlation
data: x and y
t = 3.3407, df = 18, p-value = 0.003639
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.2425222 0.8330969
sample estimates:
cor
0.6186489
cor.test(~x+y, method = "s")
Spearman's rank correlation rho
data: x and y
S = 422, p-value = 0.001239
alternative hypothesis: true rho is not equal to 0
sample estimates:
rho
0.6827068
cor.test(~rank(x)+rank(y), method = "p")
Pearson's product-moment correlation
data: rank(x) and rank(y)
t = 3.964, df = 18, p-value = 0.0009099
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.3441612 0.8641558
sample estimates:
cor
0.6827068
Note that the estimate is equal for spearman vs pearson of rank, however, the p-value for pearson on the ranks is lower.
Is this always the case?
set.seed(2)
nsim = 100
n = 20
noise = .4
sim <- data.frame(
rp = numeric(nsim),
rs = numeric(nsim),
rpr = numeric(nsim),
blm = numeric(nsim),
sqr_rsq = numeric(nsim),
pp = numeric(nsim),
ps = numeric(nsim),
ppr = numeric(nsim),
plm = numeric(nsim)
)
for (i in 1:nsim) {
x = runif(min = -1, max = 1, n = n)
y = x + rnorm(n, sd = noise)
cor_p <- cor.test(~x+y, method = "p")
cor_s <- cor.test(~x+y, method = "s")
cor_pr <- cor.test(~rank(x)+rank(y), method = "p")
lm_fit <- lm(y~x)
sum_fit<- summary(lm_fit)
sim[i, ] <- c(cor_p$estimate, cor_s$estimate, cor_pr$estimate,
lm_fit$coefficients[2],
sqrt(sum_fit$r.squared),
cor_p$p.value, cor_s$p.value, cor_pr$p.value,
sum_fit$coefficients[2,4])
}
plot(rs~rpr, data = sim)
plot(ps~ppr, data = sim)
plot(rp~sqr_rsq, data = sim)
plot(sim$ps-sim$ppr)
plot(sim$pp-sim$plm)
Conclusions
set.seed(3)
n = 10
noise = .4
x = runif(min = -2, max = 2, n)
x = rnorm(n)
y = x + rnorm(n, sd = noise)
fit <- lm(y~x)
summary(fit)
Call:
lm(formula = y ~ x)
Residuals:
Min 1Q Median 3Q Max
-0.46551 -0.17179 0.00419 0.05663 0.57650
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.1815 0.1015 -1.789 0.111
x 0.9223 0.1232 7.486 7.02e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.3189 on 8 degrees of freedom
Multiple R-squared: 0.8751, Adjusted R-squared: 0.8595
F-statistic: 56.04 on 1 and 8 DF, p-value: 7.018e-05
plot(x,y)
abline(fit, lty = 2)
x_preds <- seq(from = floor(min(x)), to = ceiling(max(x)), length.out = 50)
yhat_pred <- predict(fit, newdata = data.frame(x = x_preds),
what = "terms", interval = "confidence")
y_pred <- predict(fit, newdata = data.frame(x = x_preds),
what = "response", interval = "prediction")
points(x_preds, yhat_pred[,2], pch = 3, col = "blue")
points(x_preds, yhat_pred[,3], pch = 3, col = "blue")
points(x_preds, y_pred[,2], pch = 3, col = "red")
points(x_preds, y_pred[,3], pch = 3, col = "red")
sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.6
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] backports_1.1.0 magrittr_1.5 rprojroot_1.2 tools_3.3.2
[5] htmltools_0.3.6 yaml_2.1.14 Rcpp_0.12.11 stringi_1.1.5
[9] rmarkdown_1.6 knitr_1.17 git2r_0.19.0 stringr_1.2.0
[13] digest_0.6.12 evaluate_0.10.1
This R Markdown site was created with workflowr


A symptom checker is a system allowing the detection of any subjective evidence of an ailment or medical condition. Subjective evidence is defined as a symptom, as only patients can discover them. The symptom checker may be run by the user/patient or with the assistance of medical personnel, as symptoms are, by definition, known to the patient only. In opposition, a sign is objectively determined by a medical professional. For example, blood coming out of the nose is a sign. It is apparent to the patient, physician, and others. Dizziness, low back pain, tingling in the feet or anxiety are all symptoms, as only the user/patient can perceive them. A symptom checker can be of different forms. The most frequent ones are either keyword based or artificial intelligence based. All symptom checkers will try to determine a differential diagnosis (DDX). We consider MEDoctor's symptom checker to be of a superior type, as it is patented, with its patents granted (see US Patent #7,149,756 A - December, 2006 - ,and EPO Patent #EP 1284639 B1 - July, 2008.
Our mission is to empower you to help yourself determine your health problems. The MEDoctor Symptom Checker won't replace a medical visit to your doctor to have the final diagnosis and the correct treatment, but will be useful to identify which pain, disease or illness can correspond to the symptoms you have. The MEDoctor Symptom Checker is a guide that will help you for all kinds of problems, for a benign disease as a cold or something more serious as a cancer. MEDoctor is indeed one of the world's most advanced symptom checkers. And it is the only patented symptom checker on the Web.
You the individual are the answer to the challenges of health care. The right to protect the health and well-being of every person, of those we love, is a basic human right. Health care costs are the number 1 leading cause for personal bankruptcy in the USA today. 40% to 60% of patients are not compliant for a diagnosis, because they can't afford the bill, because they have a high deductible. Today the patient receives no copies of their diagnosis process '... because it could be dangerous...'. And they cannot see a physician without prior approval by their insurance.
We see a world where every person knows what potential disease he or she might have prior to seeing a clinician and thereby make the affordable choices for their health. We want to give you access to actionable health information at the time and frequency it matters. Every person needs to understand the risks of their condition before they undergo any costs. Patient engagement comes with knowledge. And knowledge comes with access to information. We are redefining the paradigm of diagnosis. All information becomes accessible before seeing the clinician, at any place, any time, any frequency. Technology is transforming the world. With that, we will begin to change our health care system and our lives. MEDoctor delivers 24/7, immediate, accurate health assessments for FREE and from any place a connection is available (Web, smart phone). This health assessment is generated by a sophisticated and patented algorithm. All information becomes accessible before seeing the clinician, at any place, at any frequency, at the time it matters. Early health recommendations are provided for your better choices.
MEDoctor practices policies that are intended to protect the confidentiality of personal information obtained from users in the course of its regular business functions. We are committed to protecting information about users, especially the confidential nature of any personal information (PI) we collect. Personal information is information that is capable of being associated with an individual through one or more identifiers including but not limited to, a Social Security number, a driver's license number, a state identification card number, an account number, a credit or debit card number, a passport number, an alien registration number or a health insurance identification number, and does not include publicly available information that is lawfully made available to the general public from federal, state or local government records, or widely distributed media. To this end, the MEDoctor Symptom Checker does not require that you identify yourself in any way other than an account name and password that you yourself choose. While the name you choose may be associated with you or your activities on the Internet (such as an email address), there is no requirement that it be so. MEDoctor considers the account name, password, and entered data to be sensitive information and restricts access to it.


Anim pariatur cliche reprehenderit, enim eiusmod high life accusamus terry richardson ad squid. 3 wolf moon officia aute, non cupidatat skateboard dolor brunch. Food truck quinoa nesciunt laborum eiusmod. Brunch
If you have a short hairstyle and want to maintain that, you should get a trim every 3-4 weeks
If you are sporting a longer haircut and want to maintain it that way, you should get a trim not later than 5-6 weeks


--- layout: post title: "Functional architecture is Ports and Adapters" description: "Functional architecture tends to fall into a pit of success that looks a lot like Ports and Adapters." date: 2016-03-18 08:53 UTC tags: [Software Design, Architecture, F#, Haskell, Functional Programming] image: "/content/binary/ports-and-adapters-conceptual-diagram.png" image_alt: "A conceptual diagram of Ports and Adapters architecture: coloured boxes in concentric circles." --- {% include JB/setup %}
{{ page.description }}
In object-oriented architecture, we often struggle towards the ideal of the Ports and Adapters architecture, although we often call it something else: layered architecture, onion architecture, hexagonal architecture, and so on. The goal is to decouple the business logic from technical implementation details, so that we can vary each independently.
This creates value because it enables us to manoeuvre nimbly, responding to changes in business or technology.
The idea behind the Ports and Adapters architecture is that ports make up the boundaries of an application. A port is something that interacts with the outside world: user interfaces, message queues, databases, files, command-line prompts, etcetera. While the ports constitute the interface to the rest of the world, adapters translate between the ports and the application model.
The word adapter is aptly chosen, because the role of the Adapter design pattern is exactly to translate between two different interfaces.
You ought to arrive at some sort of variation of Ports and Adapters if you apply Dependency Injection, as I've previously attempted to explain.
The problem with this architecture, however, is that it seems to take a lot of explaining:
It's possible to implement a Ports and Adapters architecture with object-oriented programming, but it takes so much effort. Does it have to be that difficult?
Someone recently asked me: how do I know I'm being sufficiently Functional?
I was wondering that myself, so I decided to learn Haskell. Not that Haskell is the only Functional language out there, but it enforces purity in a way that neither F#, Clojure, nor Scala does. In Haskell, a function must be pure, unless its type indicates otherwise. This forces you to be deliberate in your design, and to separate pure functions from functions with (side) effects.
If you don't know Haskell, code with side effects can only happen inside of a particular 'context' called IO
. It's a monadic type, but that's not the most important point. The point is that you can tell by a function's type whether or not it's pure. A function with the type ReservationRendition -> Either Error Reservation
is pure, because IO
appears nowhere in the type. On the other hand, a function with the type ConnectionString -> ZonedTime -> IO Int
is impure because its return type is IO Int
. This means that the return value is an integer, but that this integer originates from a context where it could change between function calls.
There's a fundamental distinction between a function that returns Int
, and one that returns IO Int
. Any function that returns Int
is, in Haskell, referentially transparent. This means that you're guaranteed that the function will always return the same value given the same input. On the other hand, a function returning IO Int
doesn't provide such a guarantee.
In Haskell programming, you should strive towards maximising the amount of pure functions you write, pushing the impure code to the edges of the system. A good Haskell program has a big core of pure functions, and a shell of IO
code. Does that sound familiar?
It basically means that Haskell's type system enforces the Ports and Adapters architecture. The ports are all your IO
code. The application's core is all your pure functions. The type system automatically creates a pit of success.
Haskell is a great learning aid, because it forces you to explicitly make the distinction between pure and impure functions. You can even use it as a verification step to figure out whether your F# code is 'sufficiently Functional'. F# is a Functional first language, but it also allows you to write object-oriented or imperative code. If you write your F# code in a Functional manner, though, it's easy to translate to Haskell. If your F# code is difficult to translate to Haskell, it's probably because it isn't Functional.
Here's an example.
In my Test-Driven Development with F# Pluralsight course (a free, condensed version is also available), I demonstrate how to implement an HTTP API that accepts reservation requests for an on-line restaurant booking system. One of the steps when handling the reservation request is to check whether the restaurant has enough remaining capacity to accept the reservation. The function looks like this:
// int // -> (DateTimeOffset -> int) // -> Reservation // -> Result<Reservation,Error> let check capacity getReservedSeats reservation = let reservedSeats = getReservedSeats reservation.Date if capacity < reservation.Quantity + reservedSeats then Failure CapacityExceeded else Success reservation
As the comment suggests, the second argument, getReservedSeats
, is a function of the type DateTimeOffset -> int
. The check
function calls this function to retrieve the number of already reserved seats on the requested date.
When unit testing, you can supply a pure function as a Stub; for example:
let getReservedSeats _ = 0 let actual = Capacity.check capacity getReservedSeats reservation
When finally composing the application, instead of using a pure function with a hard-coded return value, you can compose with an impure function that queries a database for the desired information:
let imp = Validate.reservation >> bind (Capacity.check 10 (SqlGateway.getReservedSeats connectionString)) >> map (SqlGateway.saveReservation connectionString)
Here, SqlGateway.getReservedSeats connectionString
is a partially applied function, the type of which is DateTimeOffset -> int
. In F#, you can't tell by its type that it's impure, but I know that this is the case because I wrote it. It queries a database, so isn't referentially transparent.
This works well in F#, where it's up to you whether a particular function is pure or impure. Since that imp
function is composed in the application's Composition Root, the impure functions SqlGateway.getReservedSeats and SqlGateway.saveReservation are only pulled in at the edge of the system. The rest of the system is nicely protected against side-effects.
It feels Functional, but is it?
In order to answer that question, I decided to re-implement the central parts of this application in Haskell. My first attempt to check the capacity was this direct translation:
checkCapacity :: Int -> (ZonedTime -> Int) -> Reservation -> Either Error Reservation checkCapacity capacity getReservedSeats reservation = let reservedSeats = getReservedSeats $ date reservation in if capacity < quantity reservation + reservedSeats then Left CapacityExceeded else Right reservation
This compiles, and at first glance seems promising. The type of the getReservedSeats
function is ZonedTime -> Int
. Since IO
appears nowhere in this type, Haskell guarantees that it's pure.
On the other hand, when you need to implement the function to retrieve the number of reserved seats from a database, this function must, by its very nature, be impure, because the return value could change between two function calls. In order to enable that in Haskell, the function must have this type:
getReservedSeatsFromDB :: ConnectionString -> ZonedTime -> IO Int
While you can partially apply the first ConnectionString argument, the return value is IO Int
, not Int
.
A function with the type ZonedTime -> IO Int
isn't the same as ZonedTime -> Int
. Even when executing inside of an IO context, you can't convert ZonedTime -> IO Int
to ZonedTime -> Int
.
You can, on the other hand, call the impure function inside of an IO context, and extract the Int
from the IO Int
. That doesn't quite fit with the above checkCapacity function, so you'll need to reconsider the design. While it was 'Functional enough' for F#, it turns out that this design isn't really Functional.
If you consider the above checkCapacity function, though, you may wonder why it's necessary to pass in a function in order to determine the number of reserved seats. Why not simply pass in this number instead?
checkCapacity :: Int -> Int -> Reservation -> Either Error Reservation checkCapacity capacity reservedSeats reservation = if capacity < quantity reservation + reservedSeats then Left CapacityExceeded else Right reservation
That's much simpler. At the edge of the system, the application executes in an IO context, and that enables you to compose the pure and impure functions:
import Control.Monad.Trans (liftIO) import Control.Monad.Trans.Either (EitherT(..), hoistEither) postReservation :: ReservationRendition -> IO (HttpResult ()) postReservation candidate = fmap toHttpResult $ runEitherT $ do r <- hoistEither $ validateReservation candidate i <- liftIO $ getReservedSeatsFromDB connStr $ date r hoistEither $ checkCapacity 10 i r >>= liftIO . saveReservation connStr
(Complete source code is available here.)
Don't worry if you don't understand all the details of this composition. The highlights are these:
The postReservation function takes a ReservationRendition (think of it as a JSON document) as input, and returns an IO (HttpResult ())
as output. The use of IO
informs you that this entire function is executing within the IO monad. In other words: it's impure. This shouldn't be surprising, since this is the edge of the system.
Furthermore, notice that the function liftIO
is called twice. You don't have to understand exactly what it does, but it's necessary to use in order to 'pull out' a value from an IO
type; for example pulling out the Int
from an IO Int
. This makes it clear where the pure code is, and where the impure code is: the liftIO function is applied to the functions getReservedSeatsFromDB and saveReservation. This tells you that these two functions are impure. By exclusion, the rest of the functions (validateReservation, checkCapacity, and toHttpResult) are pure.
It's interesting to observe how you can interleave pure and impure functions. If you squint, you can almost see how the data flows from the pure validateReservation function, to the impure getReservedSeatsFromDB function, and then both output values (r
and i
) are passed to the pure checkCapacity function, and finally to the impure saveReservation function. All of this happens within an (EitherT Error IO) () do
block, so if any of these functions return Left
, the function short-circuits right there and returns the resulting error. See e.g. Scott Wlaschin's excellent article on railway-oriented programming for an exceptional, lucid, clear, and visual introduction to the Either monad.
The value from this expression is composed with the built-in runEitherT function, and again with this pure function:
toHttpResult :: Either Error () -> HttpResult () toHttpResult (Left (ValidationError msg)) = BadRequest msg toHttpResult (Left CapacityExceeded) = StatusCode Forbidden toHttpResult (Right ()) = OK ()
The entire postReservation function is impure, and sits at the edge of the system, since it handles IO. The same is the case of the getReservedSeatsFromDB and saveReservation functions. I deliberately put the two database functions in the bottom of the below diagram, in order to make it look more familiar to readers used to looking at layered architecture diagrams. You can imagine that there's a cylinder-shaped figure below the circles, representing a database.
You can think of the validateReservation and toHttpResult functions as belonging to the application model. While pure functions, they translate between the external and internal representation of data. Finally, the checkCapacity function is part of the application's Domain Model, if you will.
Most of the design from my first F# attempt survived, apart from the Capacity.check function. Re-implementing the design in Haskell has taught me an important lesson that I can now go back and apply to my F# code.
Since the required change is so little, it's easy to apply the lesson learned from Haskell to the F# code base. The culprit was the Capacity.check function, which ought to instead be implemented like this:
let check capacity reservedSeats reservation = if capacity < reservation.Quantity + reservedSeats then Failure CapacityExceeded else Success reservation
This simplifies the implementation, but makes the composition slightly more involved:
let imp = Validate.reservation >> map (fun r -> SqlGateway.getReservedSeats connectionString r.Date, r) >> bind (fun (i, r) -> Capacity.check 10 i r) >> map (SqlGateway.saveReservation connectionString)
This almost looks more complicated than the Haskell function. Haskell has the advantage that you can automatically use any type that implements the Monad
typeclass inside of a do
block, and since (EitherT Error IO) ()
is a Monad instance, the do
syntax is available for free.
You could do something similar in F#, but then you'd have to implement a custom computation expression builder for the Result type. Perhaps I'll do this in a later blog post...
Good Functional design is equivalent to the Ports and Adapters architecture. If you use Haskell as a yardstick for 'ideal' Functional architecture, you'll see how its explicit distinction between pure and impure functions creates a pit of success. Unless you write your entire application to execute within the IO
monad, Haskell will automatically enforce the distinction, and push all communication with the external world to the edges of the system.
Some Functional languages, like F#, don't explicitly enforce this distinction. Still, in F#, it's easy to informally make the distinction and compose applications with impure functions pushed to the edges of the system. While this isn't enforced by the type system, it still feels natural.
Comments
Great post as always. However, I saw that you are using IO monad. I wonder if you shouldn't use IO action instead? To be honest, I'm not proficient in Haskell. But according to that post they are encouraging to use IO action rather than IO monad if we are not talking about monadic properties.
Arkadiusz, thank you for writing. It's the first time I see that post, but if I understand the argument correctly, it argues against the (over)use of the monad terminology when talking about IO actions that aren't intrinsically monadic in nature. As far as I can tell, it doesn't argue against the use of the
IO
type in Haskell.It seems a reasonable point that it's unhelpful to newcomers to throw them straight into a discussion about monads, when all they want to see is a hello world example.
As some of the comments to that post point out, however, you'll soon need to compose IO actions in Haskell, and it's the monadic nature of the
IO
type that enables the use ofdo
blocks. (..or, if you want to be more specific, it seems that sometimes, all you need is a Functor. It hardly helps to talk about the IO functor instead of the IO monad, though...)I don't claim to be a Haskell expert, so there may be more subtle nuances in that article that simply went over my head. Still, within my current understanding, the discussion in this particular post of mine does relate to the IO monad.
When I wrote the article, I chose my words with some caution. Notice that when I introduce the
IO
type my article, I mostly talk about it as a 'context'.When it comes to the discussion about the postReservation function, however, I pull out the stops and increasingly use the word monad. The reason is that this composition wouldn't be possible without the monadic properties of
IO
. Most of this function executes within a monadic 'stack':EitherT Error IO
. EitherT is a monad transformer, and in order to be useful (e.g. composable indo
blocks), the type it transforms must be a monad as well (or perhaps, as hinted above, a functor would be sufficient).I agree with Justin Le's article that overuse of monad is likely counterproductive. On the other hand, one can also fall into the opposite side. There are some educators who seem to avoid the word at any cost. That's not my policy. I try to use as precise language as possible. That means that I'll use monad when I talk about monads, and I'll avoid it when it isn't necessary. That's my goal, but I may not have achieved it. Do you think I used it incorrectly here?
Mark, thank you for your comprehensive explanation. I was rather just asking you about opinion on this topic. I didn't try to claim that you had used it incorrectly. Now, I see that as you wrote you are first talking about IO context and then about IO monad when it comes to use monadic properties.
Hello, thanks so much for your website and pluralsight library -- it's been a tremendous help to me getting started with F# and functional programming (and TDD to a degree). I'm having some problems with the big picture question of how to structure applications, and this post is leaving me even more confused.
I can't understand how some of the original code, and especially the changes made in this post could possibly scale to a larger or more complicated application. Many real-life domain operations are going to need to query some data, make decisions based on that data, map that data into different domain objects, persist something, and so on.
It seems like the end result is that we end up copying much of the meaningful internal logic of the domain out to this boundary area in the Web API project. In this code we've extracted:
Does that logic not properly belong 'behind the wall' inside the domain? I would have expected to see a simple 'makeReserveration r' function as the public face to the domain. Suppose the restaurant also had a desktop/kiosk application in their building to manage their reservation system. Wouldn't we now be forced to duplicate this internal logic in the Composition Root of that application?
What I instinctively want to gravitate towards is something like this:
With this, the SqlGateway code needs be visible to the reservation code, putting it at the center of the app not on the edges. However, are there any practical disadvantages to this if the code is disciplined to only use it within this sort of dependency injection? What problems would I be likely to run into?
At least as a beginner, this seems to me to have a couple advantages:
But most notable to me, dependencies are 'injected' right where they were needed. Imagine a much more complicated aggregate operation like 'performOvernightCloseout' that needs to do many things like read cash register totals and send them to accounting, send timeclock info to HR, write order statistics to a reporting database, check inventory and create new orders for the following day. Of course we'd break this down into a hierarchy of functions, with top level orchestration calling down into increasingly specific functions, no function individually being too complicated.
The original demo application would have to approach this scenario by passing in a huge number of functions to the operation root, which then parcels them out level by level where they are needed. This would seem very brittle since any change in a low level function would require changes to the parameter list all the way up.
With newer code shown in this post, it would seem impossible. There might be a handful of small functions that can be in the central library, but the bulk of the logic is going to need to be extracted into a now-500-line imp method in the Web API project, or whatever service is launching this operation.
John, thank you for writing. I'm not sure I can answer everything to your satisfaction in single response, but we can always view this as a beginning of longer interchange. Some of the doubts you express are ambiguous, so I'm not sure how to interpret them, but I'll do my best to be as explicit as possible.
In the beginning of your comment, you write:
This is exactly the reason why I like the reservation request example so much, because it does all of that. It queries data, because it reads the number of already reserved seats from a database. It makes decisions based on that data, because it decides whether or not it can accept the reservation based on the number of remaining seats. It maps from an external JSON (or XML) data format to a Reservation record type (belonging to the Domain Model, if you will); it also maps back to an HTTP response. Finally, the example also saves the reservation to the database, if it was accepted.When you say that we've extracted the logic, it's true, but I'm not sure what you mean by "behind the wall", but let's look at each of your bullet points in turn.
Querying existing reservations
You write that I've extracted the "logic of only checking existing reservations on the same day as the day on the request". It's a simple database query. In F#, one implementation may look like this:
In my opinion, there's no logic in this function, but this is an example where terminology can be ambiguous. To me, however, logic implies that decisions are being made. That's not the case here. This getReservedSeats function has a cyclomatic complexity of 1.
This function is an Adapter: it adapts the ADO.NET SDK (the port to SQL Server) to something the Domain Model can use. In this case, the answer is a simple integer.
To save, or not to save
You also write that I've extracted out the "logic of not saving a new reservation if the request failed the capacity check". Yes, that logic is extracted out to the
check
function above. Since that's a pure function, it's part of the Domain Model.The function that saves the reservation in the database, again, contains no logic (as I interpret the word logic):
Due to the way the Either monad works, however, this function is only going to be called if all previous functions returned Success (or Right in Haskell). That logic is an entirely reusable abstraction. It's also part of the glue that composes functions together.
Reuse of composition
Further down, you ask:
That's an excellent question! I'm glad you asked.A desktop or kiosk application is noticeably different from a web application. It may have a different application flow, and it certainly exposes a different interface to the outside world. Instead of handling incoming HTTP requests, and translating results to HTTP responses, it'll need to respond to click events and transform results to on-screen UI events. This means that validation may be different, and that you don't need to map results back to HTTP response values. Again, this may be clearer in the Haskell implementation, where the
toHttpResult
function would obviously be superfluous. Additionally, a UI may present a native date and time picker control, which can directly produce strongly typed date and time values, negating the need for validation.Additionally, a kiosk may need to be able to work in off-line, or occasionally connected, mode, so perhaps you'd prefer a different data store implementation. Instead of connecting directly to a database, such an application would need to write to a message queue, and read from an off-line snapshot of reservations.
That's a sufficiently different application that it warrants creating applications from fine-grained building blocks instead of course-grained building blocks. Perhaps the only part you'll end up reusing is the core business logic contained in the check[Capacity] function. Composition Roots aren't reusable.
(As an aside, you may think that if the check[Capacity] function is the only reusable part, then what's the point? This function is only a couple of lines long, so why even bother? That'd be a fair concern, if it wasn't for the context that all of this is only example code. Think of this example as a stand-in for much more complex business logic. Perhaps the restaurant will allow a certain percentage of over-booking, because some people never show up. Perhaps it will allow different over-booking percentages on different (week) days. Perhaps it'll need to handle reservations on per-table basis, instead of per-seat. Perhaps it'll need to handle multiple seatings on the same day. There are lots of variations you can throw into this business domain that will make it much more complex, and worthy of reuse.)
Brittleness of composition
Finally, you refer to a more realistic scenario, called performOvernightCloseout, and ask if a fine-grained composition wouldn't be brittle? In my experience, it wouldn't be more brittle than any alternatives I've identified. Whether you 'inject' functions into other functions, or you compose them in the Composition Root, doesn't change the underlying forces that act on your code. If you make substantial changes to the dependencies involved, it will break your code, and you'll need to address that. This is true for any manual composition, including Pure DI. The only way to avoid compilation errors when you redefine your dependency graphs is to use a DI Container, but that only makes the feedback loop worse, because it'd change a compile-time error into a run-time error.
This is a long answer, and even so, I'm not sure I've sufficiently addressed all of your concerns. If not, please write again for clarifications.
I believe there was a bit of miscommunication. When I talked about the 'logic of only checking existing reservations on the same day', I wasn't talking about the implementation of the SqlGateway. I was talking about the 'imp' function:
This implements the logic of 'Validation comes first. If that checks out, then get the number of reserved seats from the data store. Use the date on the request for that. Feed this into the capacity check routine and only if everything checks out should we persist the reservation.
That's all internal logic that in my mind ought to be encapsulated within the domain. That's what I meant by 'behind the wall'. There should simply be a function 'makeReservation' with the signature ReservationRequest -> ReservationResponse. That's it. Clients should not be worrying themselves with these extra details. That 'imp' function just seems deeply inappropriate as something that lives externally to the reservations module, and that must be supplied individually by each client accessing the module.
I guess I'm rejecting the idea of Onion/Hexagonal architecture, or at least the idea that all I/O always belongs at the outside. To me, a reservations module must depend on a reservations data store, because if you don't persist information to the db, you haven't actually made the reservation. Of course, I still want to depend on interfaces, not implementations (perhaps naming the reference ReservationDataStore.saveReservation). The data store IS a separate layer and should have a well defined interface.
I just don't understand why anyone would find this design more desirable than the straightforward UI -> Logic -> Storage chain of dependencies. Clearly there's some sort of appeal -- lots of smart people show excitement for it. But it's a bit mystifying to me.
John, thank you for writing. It's OK to reject the concept of ports and adapters; it's not a one-size-fits-all architecture. While I often use it, I don't always use it. One of my most successful customer engagements involved building a set of REST APIs on top of an existing IT infrastructure. All the business logic was already implemented elsewhere, so I deliberately chose a simpler, more coupled architecture, because there'd be no benefit from a full ports and adapters architecture. The code bases involved still run in production four years later, even though they're constantly being changed and enhanced.
If, on the other hand, you choose to adopt the ports and adapters architecture, you'll need to do it right so that you can harvest the benefits.
A Façade function with the type
ReservationRequest -> ReservationResponse
only makes sense in an HTTP request/response scenario, because both the ReservationRequest and ReservationResponse types are specific to the web context. You can't reuse such a function in a different setting.You still have such a function, because that's the function that handles the web request, but you'll need to compose it from smaller components, unless you want to write a Transaction Script within that function (which is also sometimes OK).
Granted, there's some 'logic' involved in this composition, in the sense that it might be possible to create a faulty composition, but that logic has to go somewhere.
Hi Mark, thanks for this post and the discussion.
I feel that I share some of the concerns that John is bringing up. The
postReservation
function is pretty simple now, but eventually we might want to make external calls based on some conditional logic from the domain model.Let's say there is a new feature request. The restaurant has struck a deal with the local caravan dealership, allowing them to rent a caravan to park outside the restaurant in order to increase the seating capacity for one evening. Of course, sometimes there are no caravans available, so we'll need to query the caravan database to see if there is a big enough caravan available that evening:
How would we integrate this? We could put a call to
findCaravan
inside the IO context ofpostReservation
, but since we only want to query the caravan database if the first capacity check failed, we'll need to add branching logic topostReservation
. We want to have tests for this logic, but since we are coupled to the IO context, we are in trouble.The original issue was that the F# version of
checkCapacity
could be made impure by injecting the impuregetReservedSeats
function to it, which is something that Haskell disallows. What if we changed the Haskell version ofcheckCapacity
to the following instead?Since we know that
getReservedSeats
will be effectful, we declare it so in its type signature. We changepostReservation
to become:(Or something like that. Monad transformers are the current step on my Haskell learning journey, so I'm not sure this will typecheck.)
This way, we can once more inject an IO typed function as the
getReservedSeats
argument ofcheckCapacity
like we did in F#. If we want to test it, we can create a stub database call by using the monad instance ofIdentity
, for example:The new feature request also becomes easy to implement. Just pass
findCaravan
as a new argument tocheckCapacity
, and make the type of that argument to beZonedTime -> m (Maybe Caravan)
. ThecheckCapacity
function can now do the initial capacity check, and if that fails, callfindCaravan
to see if it is still possible to allow the booking by renting a suitable caravan. The branching logic that is needed is now insidecheckCapacity
and can be tested outside of the IO context.I don't know if this would work well in practice, or if it would cause more trouble than what it's worth. What do you think?
Martin, thank you for writing. Your question prompted me to write a new article. It takes a slightly different approach to the one you suggest.
I haven't tried defining
checkCapacity
in the way you suggest, but that may be an exercise for another day. In general, though, I'm hesitant to introduce constraints not required by the implementation. In this case, I think it'd be a fair question to ask why thegetReservedSeats
argument has to have the typeZonedTime -> m Int
? Nothing in its implementation seems to indicate the need for that, so it looks like a leaky abstraction to me.I'd like to follow up my twitter comments here
Very interesting article! I'm sorry that some Haskell details are quite difficult for me at the moment and the F# code looks a bit too simplified or maybe just incomplete
So I'd prefer your approach here : it better translates Haskell to F# but I still have to study its implications. Anyway the concrete benefits of Async are evident, like the ones of F# computation expressions (to wrap exceptions etc...)
On the other hand I fail to see the benefit of purity vs other IoC patterns. Besides async and exceptions wrapping, purity seems an unnecessarily strict enforcement of Haskell (isn't it?): original F# was testable in isolation
What's missing if I only mock/stub/inject the dependency? When you say
and about DI, do you mean something related to this topic?Thank you so much for your interesting insights!
My bad, actually I don't know. I assume that purity's goal is to avoid side effects. It is very clear that this can be useful when I have a hidden side effect in the function, but it is less obvious (to me) what it does mean when the side effect is as evident as an argument (?).Anyway, I'll follow your implicit suggestion that with a pure function I can reach a more important goal than testability. Ok, practical example. Today I spent one hour to fix a bug that I introduced with a previous enhancement. My goal is to avoid this next time, by learning a better programming style from your article, but first I need to understand what went wrong. I can spot 3 problems:
- 1) I didn't see the bug when I wrote the code changes
- 2) I didn't go through that part of my app when I tested the upgrade
- 3) I needed a whole hour to fix that stupid bug, because I could not immediately find the point where it was
When I think that testability is important, it's because I hope it saves me from the issues 2 and 3, but let's take for granted that with a pure function I can avoid even point 1.Now unfortunately I struggle to interpret Haskell's details and I can't find enough F# to recover the whole meaning, but the main point I get is that a part of code (that could be bugged because the ability to reason is eroded there) has been moved from a finally 'pure' function to a new 'impure' one, that is introduced by a special 'liftIO' keyword (sorry for my poor recap)
Finally my question is, but what about that all the possible bugs (often at the interface between persistence and model layers) that have simply been migrated around this critical IO land? are they going to counterbalance the above said benefits? where is exactly the net, overall improvement?
First, I should make it specific that when I discuss object-oriented code in general, and encapsulation specifically, I tend to draw heavily on the work of Bertrand Meyer in Object-Oriented Software Construction. Some people may regard Alan Kay as the original inventor of object-orientation, but C# and Java's interpretation of object-oriented programming seems closer to Meyer's than to Kay's.
To Meyer, the most important motivation for object-orientation was reuse. He wanted to be able to reuse software that other people had written, instead of, as had been the norm until then, importing other people's source code into new code bases. I'm sure you can see how that goal was important.
In order to make binary libraries reusable, Meyer wanted to establish some techniques that would enable developers to (re)use the code without having access to the source code. His vision was that every object should come with a set of invariants, pre-, and post-conditions. He even invented a programming language, Eiffel, that included the ability to define such rules as programmable and enforceable checks. Furthermore, such rules could be exported as automatic documentation. This concept, which Meyer called contracts, is what we now know as encapsulation (I have a Pluralsight course that explains this in more details).
While the idea of formal contracts seemed like a good idea, neither Java nor C# have them. In C# and Java, you can still implement invariants with guard clauses, assertions, and so on, but there's no formal, externally visible way to communicate such invariants.
In other words, in modern object-oriented languages, you can't reason about the code from its API alone. Meyer's goal was that you should be able to reuse an object that someone else had written, without reading the source code. In a nutshell: if you have to read the source code, it means that encapsulation is broken.
One of the other concepts that Meyer introduced was Command-Query Separation (CQS). To Meyer, it was important to be able to distinguish between operations that had side-effects, and operations that didn't.
In my experience, a big category of defects are caused by unintended side-effects. One developer doesn't realise that the method call (s)he invokes has a side-effect that another developer put there. Languages like C# and Java have few constraints, so any method could potentially hide a side-effect. The only way you can know is by reading the code.
What if, then, there was a programming language where you could tell, at a glance, whether an operation had a side-effect? Such programming languages exist. Haskell is one of them. In Haskell, all functions are pure by default. This means that they are deterministic and have no side-effects. If you want to write an impure function, you must explicitly declare that in the function's type (which is done with the
IO
type).In Haskell, you don't have to read the source code in order to know whether a function is impure or not. You can simply look at the type!
This is one of the many benefits of pure functions: we know, by definition, that they have no side-effects. This property holds not only for the function itself, but for all functions that that pure function calls. This follows from the definition of purity. A pure function can call other pure functions, but it can't call impure functions, because, if it did, it would become impure itself. Haskell enforces this rule, so when you're looking at a pure function, you know that that entire subgraph of your code is pure. Otherwise, it wouldn't compile.
Not only does that relationship force us to push impure functions to the boundary of applications, as described in the present article. It also makes functions intrinsically testable. There are other advantages of pure functions besides these, such as easy composability, but in general, pure functions are attractive because it becomes easier to figure out what a function does without having to understand all the details. In other words, pure functions are better abstractions.
To be clear: functional programming doesn't protect you from defects, but in my experience, it helps you to avoid entire classes of bugs. I still write automated tests of my F# code, but I write fewer tests than when I wrote C# code. Despite that, I also seem to be producing fewer defects, and I rarely need to debug the functions.
What's the overall net improvement? I don't know. How would you measure that?
Consider a logic that, depending on a condition, performs one IO operation or another. In Java (I don’t know F# or Haskell):
Would you recommend pre-fetching the two values prior to calling the
foo
function even though only one would ultimately be required?Abel, thank you for writing. There's a discussion closely related to that question in the remarks sections of my dependency rejection article. It's the remark from 2017-02-18 19:54 UTC (I really ought to get around to add permalinks to comments...).
Additionally, there's a discussion on this page that lead to a new article that goes into some further details.
(This is rather a delayed reply, but...) @Martin Rykfors, you mentioned monad transformers as an option. A few months before your comment I recorded a livecoded session of applying MTL-style monad transformers to this code.
I'm not sure if you're interested (it's nearly an hour of me talking to myself), but just in case... 🙂


Create Table and Alter Table
Create Table and Alter Table are the mainstays of building a database, but I can never quite remember the syntax. SQL Server, Create Table [1][2], Alter Table [3].
CREATE TABLE Newspapers
(
NewspaperID INT NOT NULL,
NewspaperName VARCHAR(40) NOT NULL,
Format VARCHAR(10) NOT NULL
)
-- Adding a primary key adds a *clustered* index too.
ALTER TABLE Newspapers
ADD CONSTRAINT PK_Newspapers PRIMARY KEY (NewspaperID)
ALTER TABLE Newspapers
ADD CONSTRAINT CK_Newspapers_Format CHECK (Format IN ('Tabloid', 'Broadsheet'))
ALTER TABLE Newspapers
ADD CONSTRAINT DF_Newspapers_Format DEFAULT ('Tabloid') FOR Format
CREATE TABLE Articals
(
ArticalID INT CONSTRAINT PK_Articals PRIMARY KEY IDENTITY(1,1),
NewspaperID INT NOT NULL,
Title VARCHAR(40) NOT NULL,
WordCount INT NOT NULL CONSTRAINT DF_Articals_WordCount DEFAULT (0),
TS ROWVERSION
)
ALTER TABLE Articals
ADD CONSTRAINT FK_Articals_Newspapers FOREIGN KEY (NewspaperID)
REFERENCES Newspapers(NewspaperID)
-- Adding a foregin key does not automatically create an index
-- An index is nonclustered by default
CREATE INDEX IX_Articals_NewspaperID ON Articals(NewspaperID)
-- Adding a Unique constraint adds a unique non-clustered index automatically
ALTER TABLE Articals
ADD CONSTRAINT UC_Articles_Title UNIQUE (Title)
DROP TABLE Articals
DROP TABLE Newspapers


This is a service for the internal communications. You may use either XML-RPC to call this method (using Util. before the method) or posting data to /method_name.