All of the below functions can be found in the tribology
module of the
package.
The functions in this module are related to tribology in general.
fball3plates
(ax_force, plate_angle=1.5708)[source]¶Calculate the normal force per contact in ball-on-3-plates setup.
| Parameters: |
|
|---|---|
| Returns: | The normal force acting in each ball-plate contact. |
| Return type: | norm_force |
ffourball
(r_1, r_2, ax_force)[source]¶Calculate the normal force per contact in a 4-ball test setup.
| Parameters: |
|
|---|---|
| Returns: | norm_force – The normal force in a single ball-ball contact. |
| Return type: | ndarray, scalar |
gfourball
(r_1, r_2)[source]¶Geometric parameters of 4-ball setup
| Parameters: |
|
|---|---|
| Returns: |
|
profball
(x_axis, r_ball)[source]¶Create a 1D ball profile along a central axis.
| Parameters: |
|
|---|---|
| Returns: | prof_heights – The profile heights along the x_axis. |
| Return type: | ndarray |
profrevolve
(prof_1d, y_axis, y_diam)[source]¶Create a 2D surface (profile heights) by revolving a 1D profile around a central axis.
| Parameters: |
|
|---|---|
| Returns: |
|
profrolleriso
(x_axis, diam, length)[source]¶Generate a roller profile according to DIN 26281 for rollers with
length
\(\leq\) 2.5
\(\cdot\) diam
.
| Parameters: |
|
|---|---|
| Returns: | x_profile – The profile heights of the roller along |
| Return type: | ndarray |
radpersec2rpm
(vel_rad_per_sec)[source]¶Convert velocity from rotations per minute (rpm) to radians per second (rad/s).
| Parameters: | vel_rad_per_sec (ndarray, scalar) – The velcity in radians per second. |
|---|---|
| Returns: | vel_rpm – The velocity in rotations per minute. |
| Return type: | ndarray, scalar |
rball3plates
(r_ball, plate_angle=1.5708)[source]¶Calculate the sliding radius (lever arm) for a ball-on-3-plates test setup.
| Parameters: |
|
|---|---|
| Returns: | r_slide – The sliding radius. |
| Return type: | ndarray, scalar |
refix
(val, p_in='', p_out='')[source]¶Convert between different SI unit prefixes. Available options are:
'T'
Terra
'G'
Giga
'M'
Mega
'k'
Kilo
'm'
Milli
'mu'
Micro
'n'
Nano
'p'
Pico
| Parameters: |
|
|---|---|
| Returns: | val_refix – The value in units of prefix |
| Return type: | scalar |
rpm2radpersec
(vel_rpm)[source]¶Convert velocity from radians per second (rad/s) to rotations per minute (rpm).
| Parameters: | vel_rpm (ndarray, scalar) – The velcity in rotations per minute. |
|---|---|
| Returns: | vel_rad_per_sec – The velocity in radians per second. |
| Return type: | ndarray, scalar |
srr
(vel_1, vel_2)[source]¶Calculate the slide-to-roll ratio (SRR) in a tribological contact based on contact body velocities.
| Parameters: |
|
|---|---|
| Returns: | srr – The slide-to-roll ratio in the tribological contact. |
| Return type: | ndarray, scalar |
vroll
(vel_1, vel_2)[source]¶Calculate the rolling speed in a tribological contact based on contact body velocities.
| Parameters: |
|
|---|---|
| Returns: | vel_roll – The rolling velocity in the tribological contact. |
| Return type: | ndarray, scalar |
vslide
(vel_1, vel_2)[source]¶Calculate the sliding speed in a tribological contact based on contact body velocities.
| Parameters: |
|
|---|---|
| Returns: | vel_slide – The sliding velocity in the tribological contact. |
| Return type: | ndarray, scalar |


Click headings to sort; only candidates targeted with $50,000 or more in independent expenditures are listed below.
| Support or oppose |
Candidate |
Office |
Amount |
Result |
{% for candidate in spender.candidates %}
| {{ candidate.support_oppose }} |
{{ candidate.name}} {{ candidate.party}} |
{{ candidate.office}} |
${{ candidate.amount|intcomma}} |
{% if candidate.result %}{{ candidate.result}}{% else %}Won primary{% endif %} |
{% endfor %}
Click here to collapse


Home
About
Contact
(current)
Cart
Contact Us
Your name
Your email address
We'll never share your email
Your message
Submit


For the information you entered:
Your Height:
Your Weight:
Your BMI is
This indicates that your weight is in the Normal category of your height.
Some tips to Maintain a Healthy Weight:
- Eat good foods. Foods high in sugar and calories lead to weight gain if you are not careful
- Drink more water. Drinking water, along with eating good foods and exercising daily, can help you to maintain a healthy weight
- Exercise daily and get more sleep
Maintaining a healthy weight may reduce the risk of chronic diseases associated with overweight and obesity.


| MATLAB 文件帮助: cv.watershed | 查看 cv.watershed 的代码 | Index |
watershed Performs a marker-based image segmentation using the watershed algrorithm
marker = cv.watershed(image, marker)
image
.image
.The function implements one of the variants of watershed, non-parametric marker-based segmentation algorithm, described in [Meyer92].
Before passing the image to the function, you have to roughly outline the desired regions in the image markers with positive (>0) indices. So, every region is represented as one or more connected components with the pixel values 1, 2, 3, and so on. Such markers can be retrieved from a binary mask using cv.findContours and cv.drawContours. The markers are "seeds" of the future image regions. All the other pixels in markers, whose relation to the outlined regions is not known and should be defined by the algorithm, should be set to 0's. In the function output, each pixel in markers is set to a value of the "seed" components or to -1 at boundaries between the regions.
Any two neighbor connected components are not necessarily separated by a watershed boundary (-1 pixels); for example, they can touch each other in the initial marker image passed to the function.
[Meyer92]:
Fernand Meyer. "Color image segmentation". In Image Processing and its Applications, 1992, International Conference on, pages 303-306. IET, 1992.


You must be over 18 to enter!
By entering this site, you are certifying that
you are of legal adult age.
Yes, I'm over 18
No, I'm not


Oral hygiene has been everyone's daily routine. Oral Care electric toothbrush will be a suggestion for you. This is not the first electric toothbrush from Oral Care, but it differs from previous models with interesting features.
Oral Care electric toothbrush is quite thin and light. Weighing only 50g with a slim design like a pen, allow users to easily carry around anywhere. In addition, the product set includes 2 versions which are lovely pink suitable for girls, and fresh blue, suitable for boys.
Say Goodbye to brushing your teeth manually :)


| S | stats toolbox needed | I | interactive (user input required) |
| B | bioinformatics toolbox needed | M | matlab needed (will not work in octave) |
| O | optimization toolbox needed | W | windows needed (will not work in linux) |
| X | currently broken | * | slow (two stars indicates very slow) |
| File Name | Brief Description | Page Number(s) | Tags |
|---|---|---|---|
| biasVarianceFigure | Visualization of error versus amount of regularization | 697 | |
| biasVarModelComplexity | Illustration of the bias variance trade off as we vary the L2 regularizer. | 697 | |
| bootstrapDemoBer | Bootstrap demo for the MLE for a Bernoulli | 693 | |
| neymanPearson2 | Illustration of the Neyman-Pearson testing paradigm | 699 | |
| samplingDistGaussShrinkage | Plot the sampling distribution of mean shrinkage estimators | 696 | |
| twoPowerCurves | Two hypothetical two-sided power curves: B dominates A | 700 |


Report: Everton want Sheffield United defender John Egan
by Jordan HarrisAccording to a report from The Sun on Sunday (17/11; page 67), Everton are monitoring Sheffield United defender John Egan and plan to watch the centre-back play for Republic of Ireland on Monday.
It is little surprise that Marco Silva is keen to sign a new defender in January. The Toffees look to be extremely light in that area of the field. And it is only the five teams below them in the Premier League table who have conceded more goals.
The Blades meanwhile, boast the second-best record in the division. And Egan has played a key role in their success. Chris Wilder’s men have conceded just nine goals and sit fifth in the table.
Egan has started all 12 league games since United returned to England’s top table. He appeared 44 times last year as his first season at Bramall Lane culminated in automatic-promotion.
The Sun on Sunday (17/11; page 67) reports that Everton are interested in signing the 27-year-old and plan to watch how he gets on in a back four when Republic of Ireland face Denmark next week.
The report notes however, that Sheffield United will likely demand a ‘fortune’ for the former Brentford man. So it does not look set to be a simple chase for Everton.
TBR View
Everton certainly need to target a new centre-back in January. And Egan’s form at the start of this season suggests that he could be a very shrewd addition.
Despite the Blades’ success, it may be a good time to take a look at Wilder’s men. They are still going somewhat under the radar and probably will not get the full credit they deserve once the season reaches the business end.
That does not necessarily mean that United will make it any easier for Everton to land Egan. But perhaps, the Toffees will attempt to agree a lower price before Wilder’s side really receive the plaudits they deserve.
Have something to tell us about this article?
Let us know


Haxe Studio is a text editor for Haxe Project. It tries to integrate with your project by reading haxestudio.config
file in your project folder. Currently , Haxe Studio supports :
Haxe Studio is distributed using the update system, but you need the core file first. The core file are the node-webkit runtime and some script to help you run. Simply follow this guideline :
a.k.a c:\ or ~
Haxe-Studio-Update
(.bat/.sh/.command depends on your OS)Haxe-Studio-Update
Haxe-Studio
(.bat/.sh/.command depends on your OS)Reminder : Haxe Studio only works with a project. If you want to edit a File, please use ATOM editor with Haxe plugin, or any other editors.
Working with Haxe Studio might be different from your favourite editor. So, we suggest you follow this guideline
If you tried to open a non-compatible project (H/XML), you'll notice it didn't integrate well with Haxe Studio. To remedy this, You can simply add haxestudio.config
file to your non-compatible project to make it compatible.
{
"compiler":"lime",
"base_library":"openfl",
"source_folder":"Source"
}
The guy who compile your code to the respected target. Supported compiler are written in About Section
The library you depends on. This integrate with the new file menu and show you the template.
Your -src folder. This integrate with the new file menu and handle the file creation


--- layout: default title: Contact Us ---
Bristol, United Kingdom · 51°27'N 2°35'W
info(at)lewisbuckley.co.uk
07809 667611
We look forward to helping you design and create an awesome website or web application. We can also meet up if you prefer, contact us to arrange.


SEDNA Systems
senior software developer • april 2018 — present
alisdair led the team responsible for SEDNA's API, primary and secondary data stores (MySQL and Elasticsearch) and supporting services and infrastructure
responsibilities
worked with product managers to refine product goals into technical requirements
mentored other members of the team; pairing with them, reviewing their pull requests and working to clarify their tasks
communicated with integration partners and customers to ensure SEDNA's API was appropriate and functional for their requirements
worked with the entire development team and company leadership to establish and document service monitoring and on call and incident procedures
achievements
refactored a react client application and replaced a tangle of concurrent api requests that often encountered race conditions and data consistency violations with an rxjs based reactive approach that eliminated all protocol errors
migrated from an application specific private API to an application agnostic versioned public API
instrumented a react client application and kotlin, nodejs and elixir server applications with modern distributed system tooling including tracing (opentracing) and metrics (prometheus)
Askuity
senior software developer • may 2015 — march 2018
alisdair led the team responsible for the ingestion of customer data, the persistence of that data to data warehouses including postgresql, apache presto and aws redshift and the infrastructure and supporting libraries for querying that data
responsibilities
worked with data brokers and customers to automatically extract and ingest data from a variety of sources including sftp, apache kafka, edi brokerages and email
led and mentored a team of software developers and data analysts, working with them to ensure adherence to industry best practices like source control, code review and test driven development
achievements
led the design and implementation of a data ingestion pipeline capable of capturing billions of data points each day; aggregating, deduplicating and storing while maintaining data providence to faciliate the invalidation or replacement of already ingested data
wrote and maintained a framework and react application to allow data analysts and customer implementation specialists to submit and run spark sql queries over data stored across aws s3, aws redshift and postgresql and track outdated query results for which new or updated data existed
developed an API and angular application to allow customers to submit and validate supplemental data used to enrich the data automatically ingested via other means
Twentieth Century Fox Film Productions
scenic art department coordinator • january 2012 — april 2015
alisdair worked as a supervisor in the scenic art, paint, props and construction departments of various fox productions including Deadpool, Dawn of the Planet of the Apes and X-Men Days of Future Past
programming languages
typescript • javascript • java • kotlin • scala • erlang • elixir • python
databases and data storage
postgresql • mysql • elasticseach • aws redshift • apache spark • apache presto • apache kafka • aws kinesis
platforms and infrastructure
linux • aws • docker • opentracing • elk • cloudformation • terraform • ansible


exptSeq -t {int,double} <n> <filename>
This generator creates a sequence of n values with repeats that are distributed in an exponential distribution and outputs the in the sequence file format.
In particular it will first generate n possible values v1, v2, ..., vn uniformly at random from a given range (depending on the type) and then among those it will pick the ith value with probability (1/(i ln n)). The purpose of the distribution is to test codes on inputs with a varying number of duplicates, and with some values highly duplicated (e.g., approximately a 1/(ln n) fraction of the elements will have value v1).
The generator supports both double-precision floating-point values and integers. The integer version selects the n possible values uniformly at random from 0 up to the maximum possible value for a twos complement 32-bit integer (2,147,483,647). The double precision version selects the n possible values uniformly at random in the range [0:1].


China-US may still have a long way to go to end the tradewar – Global Times
by Dhwani MehtaWith risk-off creeping back amid fresh doubts over the preliminary trade truce, “China & the US may still have a long way to go to end the tradewar,” Global Times tweeted out.
The Chinese news outlet further said: “During an entrepreneurship roundtable in Beijing, the two sides had apparent disagreements on the nature of trade relationship, with the US insisting that it hasn't had enough benefits.”
These headlines are only going to temper the market mood further, with USD/JPY back in the red near 108.60 while S&P 500 futures pare back gains. The Hong Kong civil unrest also remains a drag on the sentiment.


“strive 8/13 ~ 8/20”
Use
-
[NPM] API Designer
-
[Gem] jbuilder
Read
-
[AWS] vpc aws
-
[API] raml introduction
Book
-
- Model
- AntiPattern: Voyeuristic Models
- follow the Law of Demeter
- In Rails, this could be summed up as “use only one dot.” => class-level delegate method, This method provides a shortcut fondicating that one or more methods that will be created on your object are actually provided by a related objectr
- Push All find() calls into finder on the model
- use scoped
- keep finders on their own model
- follow the Law of Demeter
- AntiPattern: Fat Models
- delegate responsibility to new classes
- composed_of
- make use of modules
- reduce the size of large transaction blocks
- callback, nested attributes
- delegate responsibility to new classes
- AntiPattern: spaghetti SQL
- use your ActiveRecord associations and finders effectively
- as much as possible using sql on own model, others use association to handle it
- leanr and love the scope method
- use a full-text search engine
- use your ActiveRecord associations and finders effectively
- AntiPattern: duplicate code duplication
- extract into module
- template pattarn => use module, and some must implement method => define and execute a class that is missing implementations of template methods
- write a plugin
- metaprograming
- extract into module
- AntiPattern: Voyeuristic Models
- Domain Modeling
- AntiPattern: authorization astronaut
- simplify with simple flags
- AntiPattern: the million-model march
- denormalize into text fields
- when model have the column like state, and need have some method to check state value => use meta-programing to define_method “#{state}?”
- make use of rails serialization
- save in hash
- denormalize into text fields
- AntiPattern: authorization astronaut
- Views
- AntiPattern: PHPitis
- learn about the view helpers that come with rails
- add useful accessors to your models
- extract into custom helpers
- AntiPattern: Markup Mayhem
- make use of the rails helpers
- use haml
- AntiPattern: PHPitis
- Controllers
- AntiPattern: Homemade Keys
- use clearance
- use authlogic
- AntiPattern: Fat Controller
- use active record callbacks and setters
- default values => on migration
- effective callback use
- identification of unnecessary code
- move to a presenter
- use active record callbacks and setters
- AntiPattern: Bloated Sessions
- store references instead of instances
- AntiPattern: Monolithic Controllers
- embrace REST
- AntiPattern: Controller of Many Faces
- refactor non-restful actions into a separate controller
- AntiPattern: A Lost Child Controller
- make use of nested resources
- AntiPattern: Rat’s Nest Resources
- use separate controllers for each nesting
- AntiPattern: Evil Twin Controllers
- use rails 3 responders
- AntiPattern: Homemade Keys
- Services
- AntiPattern: Fire and Forget
- konw what exceptions to look out for
- AntiPattern: Sluggish Services
- move the task to the background
- AntiPattern: Pitiful Page Parsing
- use a gem => nokogiri
- AntiPattern: Successful Failure
- obey the http codes
- AntiPattern: Kraken Code Base
- divide into confederated applications
- AntiPattern: Fire and Forget
-
Using Third-Party Code
- AntiPattern: Recutting the Gem
- look for a gem first
- AntiPattern: Amateur Gemologist
- follow TAM (tests, activity, maturity)
- AntiPattern: Vendor Junk Drawer
- prune irrelevant or unused gems
- AntiPattern: Miscreant Modification
- consider vendored code sacrosanct
- AntiPattern: Recutting the Gem
-
Testing
- AntiPattern: Fixture Blues
- make use of factories
- refactor into contexts
- AntiPattern: Lost in Isolation
- watch your integration points
- AntiPattern: Mock Suffocation
- tell, don’t ask
- AntiPattern: Untested Rake
- extract to a class method
- AntiPattern: Unprotexted Jewels
- write normal unit tests without rails
- load only the parts of rails you need
- brake out the atom bomb
- AntiPattern: Fixture Blues
- Scaling and Deploying
- AntiPattern: Scaling Roadblocks
- build to scale from the start
- AntiPattern: Scaling Roadblocks
- Databases
- AntiPattern: Disappearing Assets
- make use of the system directory
- AntiPattern: Sluggish SQL
- add indexes
- reassess your domain model
- AntiPattern: Painful Performance
- don’t do inruby what you can do in sql
- move processing into background jobs
- AntiPattern: Messy Migrations
- never modify the up method on a committed migration
- never use external code in a migration
- always provide a down method in migrations
- AntiPattern: Wet Validations
- eschew constraints in the database
- AntiPattern: Disappearing Assets
-
Building for Failure
- AntiPattern: Continual Catastrophe
- fail fast
- AntiPattern: Unaudible Failures
- never fail quietly
- AntiPattern: Continual Catastrophe
- Model


Overview and Quick Facts
The NTP Security Project, or "NTPsec", is an open source project that is security hardening and updating the widely used reference implementation of the Network Time Protocol.
NTP is how the Internet keeps correct time, which is critically important for cybersecurity, and is necessary for finance, cloud computing, media streaming, data storage, email, calendaring, and for keeping the clock on most personal computers set correctly.
NTP was invented by one of the original creators of the internet itself, Dr. David L. Mills, now Professor Emeritus at the University of Delaware. Dr. Mills was also the original architect of the original reference implementation, now called "NTP Classic", that the NTP Security Project is updating.
Development of NTPsec is currently being funded by the Linux Foundation’s Core Infrastructure Initiative, which was formed in the wake of the "Heartbleed Bug".
Press Articles and Reports
-
Information Week: Linux Foundation Funds NTP (http://www.informationweek.com/cloud/infrastructure-as-a-service/linux-foundation-funds-ntps-father-time/d/d-id/1321775)
-
ZD Net: Saving NTP (http://www.zdnet.com/article/saving-time-the-funding-of-ntp)
-
Center for Trustworthy Scientific Cyberinfrastructure, Year Three Report, Section 4.11 - Network Time Protocol (https://www.scholarworks.iu.edu/dspace/bitstream/handle/2022/20401/CTSC%20Year%203%20Report_v3ASC.pdf)
Contact Information
For more information, contacts are:
Mark Atwood
Project Manager pro-tem
NTP Security Project
<EMAIL_ADDRESS>and
John D. Bell
Evangelism, Outreach, and Training Coordinator
Internet Civil Engineering Institute (ICEI)
+1-567-343-0164
<EMAIL_ADDRESS>and
Emily Ratliff
Senior Director of Infrastructure Security
The Linux Foundation, Core Infrastructure Initiative (CII)
eratliff@linuxfoundation.org


url = "/tactics" layout = "default" title = "Tactics" [blogPost tacticsPost] slug = "{{ :slug }}" categoryPage = "tactics" [blogPosts tacticsPosts] categoryFilter = "tactics" postsPerPage = 50 noPostsMessage = "No posts found" sortOrder = "published_at" categoryPage = "tactics" postPage = "tactics" ==
The images below show a situation. This is an exercise in situational thinking. The Black circle dots represent the attacking team whilst the yellow circle dots represent the defending team. On the right hand side of the images there are buttons which can be found. Each button represents a player (Black Dot) that can be found on the image of that section, when you press a button a pop up will apear giving you some advice on why you should or shouldn't pass the ball to the collerating player.


Step 2: Deployment¶
Debugging¶
If the doctr-versions-menu
command behaves unexpectedly, add the --debug
flag as follows:
doctr deploy --command="doctr-versions-menu --debug" --no-require-master --build-tags "$DEPLOY_DIR"
Make sure to include the debug output when reporting bugs.
Default assumptions¶
You should not have to customize doctr-versions-menu
provided you stick to the following sensible assumptions:
Releases should be tagged as e.g.
v0.1.0
and deployed to a folder of the same name. That is, a lower case letterv
followed by a PEP 440-compatible version identifier.The
master
branch should be deployed to a foldermaster
.Any other branch for which documentation is to be deployed should go in a folder matching the branch name.
By default, the index.html
file will forward to the documentation of the
latest public release (excluding pre-releases such as v1.0.0-rc1
), or to
master
if there have been no public releases. There is no support for an RTD-style
“latest”/”stable” folder. This is by design: deep-linking to “latest” documentation
is a bad practice, as such links easily become invalid when a new version is
released.
Download links¶
By default, doctr-versions-menu
looks for a file _downloads
inside each
folder on the gh-pages
branch. Each line in this file should have the
markdown-like format [label]: url
, e.g.
[html]: https://dl.bintray.com/goerz/doctr_versions_menu/doctr_versions_menu-v0.1.0.zip
These links will be shown in the versions menu in a section “Downloads”, using the label as the link text. See Documentation Artifacts for further information on how to build and upload the underlying files.
Customization¶
Command line options¶
Custom warning messages¶
By default, the Doctr Versions Menu plugin injects warnings in the rendered HTML files, within the following types of folders:
an ‘outdated’ warning for
<releases>
older than the latest public release (--latest
)an ‘unreleased’ warning for
<branches>
(anything that is not a PEP 440-conforming release) or<local-releases>
(typically not used)a ‘prereleased’ warning for anything considered a pre-release by PEP 440, e.g.
v1.0.0-rc1
Which folders are included in the above three categories can be modified via the --warning
option.
This options receives two arguments, a “warning label” string (the above
‘outdated’, ‘unreleased’, or ‘prereleased’), and a
folder specification for the
folders to which the warning should apply. An empty specification would disable
the warning, e.g.
doctr-versions-menu --warning prereleased ''
to disable the warning message on pre-releases.
It is also possible to define entirely new warning labels using --warning
. For example,
doctr-versions-menu --warning post '<post-releases>'
would define a warning ‘post’ for all post-releases.
The information about which folders should display which warnings is stored
internally in the resulting versions.json
file, in a dict ‘warnings’ the
maps folder names to a list of warning labels.
To actually show the warning, the doctr-versions-menu.js_t
template would
have to be modified to pick up on the ‘post’ label, see the instructions for
the Customization of the doctr_versions_menu
Sphinx
extension.
In the config file, the above options may be configured as e.g.:
warning = '''[
('post', '<post-releases>'),
('prereleased', ''),
]'''
The default settings (with the default --latest
) correspond to:
warning = '''[
('outdated', '(<releases> < (<public-releases>)[-1])'),
('unreleased', '<branches>, <local-releases>'),
('prereleased', '
'),
]'''
Note the triple-quotes required for a multi-line entry.
Customizing index.html
¶
By default, doctr-versions-menu
generates an index.html
file in the
root of the gh-pages
branch that redirects to the current “default folder”.
This is the folder for the most current public release (--latest
), or
master
if no public release exists.
The generated index.html
file can be customized by placing an
index.html_t
Jinja template into the root of the gh-pages
branch.
This template will be rendered receiving a dict version_data
containing the
data in versions.json
(see the versions.json Structure).
The default template is
<!DOCTYPE html>

<meta http-equiv="Refresh" content="0; url={{ version_data['latest'] | default('master', true) }}" />

Go to the default documentation.

Alternatively, if you want a completely static index.html
, you could also
just add that file by hand and use --no-write-index-html
(that is, write_index_html=False
in the doctr-versions-menu.conf configuration file).
Maintenance on gh-pages
¶
Unless --no-write-index-html
is given, running
doctr-versions-menu
will generate a script versions.py
in the root of
the gh-pages
branch that may be used to regenerate the versions.json
file. This script is intended for manual maintenance on the gh-pages
branch, that is, outside of the normal automatic Doctr-deployment through
Travis. For example, you may occasionally want to remove folders for outdated
branches or pre-releases from the gh-pages
branch, or update existing
download links.
After any such change, run the versions.py
script before committing and
pushing the gh-pages
branch.
Remember that each folder on the gh-pages
branch generally contains its own
doctr-versions-menu.js
script. Switching a project to a new major version
of doctr-versions-menu
, if that version changes the internal data structure of
versions.json
, may require updating the doctr-versions-menu.js
script
in existing folders by hand.


Fig. formation shoot tomato vitro root mutant Plant capacity double genes mutants auxin gene regeneration competence explants cotyledons plants MT- {border-style form plant roots .±. Arabidopsis Cell analysis genetic px;} #ffffff;} -day-old branching dgt high induction observed used controlling organ cells leaves presented different lateral leaf meristem Koornneef acquisition development developmental epistatic phenotype pro sensitivity RIM Table accumulation cell expression increased low probably shoots transcript Peres RT-PCR SIM described homeobox number organs Rg, compared lsRg present seedlings specific using (A) Che Christianson DELLA GUS allele axillary containing expressed levels medium reduced reduction sections single B), Lima Micro-Tom TKn cytokinin pathway petiole px; signalling staining tissue {font-family (B) A), Biology However, JXB KNOX LEP Physiology SIM, Shoot Skoog Sugimoto Supplementary Van affecting alterations assume common control diageotropica effect font-size formation. gibberellin growth mass natural organogenesis parenchyma presence response shoots, young (C) A). Bassel C). DR::GUS Development Developmental Experimental Figs Pino able acid adventitious associated available background bud competence, dgtRg, does double; explant important improve improving increase mutants. plants. proRg tissues vitro, (), (see Botany Branching CIM Carvalho Cary Chen Gallois Genetics Hay Knotted Miller PROCERA RNA Rg. Science Sinha Solanum São Tomato Warnick auxin-to-cytokinin balance border-color border-width cotyledonary cultivated culture cultured dgtRg distinct endogenous fates followed given greenhouse grown hormonal indeterminate index influence key maintained model molecular necessary new non-branching pro, procera proposed protein related represent respectively. showed showing solid; stage synthetic tips variation vegetative {font-style {font-weight Bar Figueira GAMyb-like Hake Jasinski Mouse Petri Planta Regeneration Research Root STM Society Student’s TKn), Warnick, action alters basis bold callus case characterization commercial competent conducted consistent constitutive courier; data dgt, dry em; enhanced fact factor formed fresh gene, genotypes hand, higher incubated indicate initial interactions known length ls. main maize mean mutant, mutant. mutation noteworthy novel obtained organs. phase possible prepared previously pt;} regulation requirement reverse scion seed seeds shown significantly sodium species spongy stem step studies suggest suggesting suggests tested thaliana transcription transgenic upstream


BNO
TITLE
WRITER
REGDATE
[[${board.bno}]]
[[${board.title}]]
[[${board.writer}]]
[[${#dates.format(board.regdate, 'yyyy-MM-dd')}]]


A Visual Introduction to Machine Learning
In machine learning, computers apply statistical learning techniques to automatically identify patterns in data. These techniques can be used to make highly accurate predictions.
Keep scrolling. Using a data set about homes, we will create a machine learning model to distinguish homes in New York from homes in San Francisco.
scroll
First, some intuition
Let’s say you had to determine whether a home is in San Francisco or in New York. In machine learning terms, categorizing data points is a classification task.
Since San Francisco is relatively hilly, the elevation of a home may be a good way to distinguish the two cities.
Based on the home-elevation data to the right, you could argue that a home above 240 ft should be classified as one in San Francisco.
Adding nuance
Adding another dimension allows for more nuance. For example, New York apartments can be extremely expensive per square foot.
So visualizing elevation and price per square foot in a scatterplot helps us distinguish lower-elevation homes.
The data suggests that, among homes at or below 240 ft, those that cost more than $1776 per square foot are in New York City.
Dimensions in a data set are called features, predictors, or variables.
Drawing boundaries
You can visualize your elevation (>242 ft) and price per square foot (>$1776) observations as the boundaries of regions in your scatterplot. Homes plotted in the green and blue regions would be in San Francisco and New York, respectively.
Identifying boundaries in data using math is the essence of statistical learning.
Of course, you’ll need additional information to distinguish homes with lower elevations and lower per-square-foot prices.
The dataset we are using to create the model has 7 different dimensions. Creating a model is also known as training a model.
On the right, we are visualizing the variables in a scatterplot matrix to show the relationships between each pair of dimensions.
There are clearly patterns in the data, but the boundaries for delineating them are not obvious.
And now, machine learning
Finding patterns in data is where machine learning comes in. Machine learning methods use statistical learning to identify boundaries.
One example of a machine learning method is a decision tree. Decision trees look at one variable at a time and are a reasonably accessible (though rudimentary) machine learning method.
Finding better boundaries
Let's revisit the 240-ft elevation boundary proposed previously to see how we can improve upon our intuition.
Clearly, this requires a different perspective.
By transforming our visualization into a histogram, we can better see how frequently homes appear at each elevation.
While the highest home in New York is ~240 ft, the majority of them seem to have far lower elevations.
Your first fork
A decision tree uses if-then statements to define patterns in data.
For example, if a home's elevation is above some number, then the home is probably in San Francisco.
In machine learning, these statements are called forks, and they split the data into two branches based on some value.
That value between the branches is called a split point. Homes to the left of that point get categorized in one way, while those to the right are categorized in another. A split point is the decision tree's version of a boundary.
Tradeoffs
Picking a split point has tradeoffs. Our initial split (~240 ft) incorrectly classifies some San Francisco homes as New York ones.
Look at that large slice of green in the left pie chart, those are all the San Francisco homes that are misclassified. These are called false negatives.
However, a split point meant to capture every San Francisco home will include many New York homes as well. These are called false positives.
The best split
At the best split, the results of each branch should be as homogeneous (or pure) as possible. There are several mathematical methods you can choose between to calculate the best split.
As we see here, even the best split on a single feature does not fully separate the San Francisco homes from the New York ones.
Recursion
To add another split point, the algorithm repeats the process above on the subsets of data. This repetition is called recursion, and it is a concept that appears frequently in training models.
The histograms to the left show the distribution of each subset, repeated for each variable.
The best split will vary based which branch of the tree you are looking at.
For lower elevation homes, price per square foot is, at X dollars per sqft, is the best variable for the next if-then statement. For higher elevation homes, it is price, at Y dollars.
Growing a tree
Additional forks will add new information that can increase a tree's prediction accuracy.
Splitting one layer deeper, the tree's accuracy improves to 84%.
Adding several more layers, we get to 96%.
You could even continue to add branches until the tree's predictions are 100% accurate, so that at the end of every branch, the homes are purely in San Francisco or purely in New York.
These ultimate branches of the tree are called leaf nodes. Our decision tree models will classify the homes in each leaf node according to which class of homes is in the majority.
Making predictions
The newly-trained decision tree model determines whether a home is in San Francisco or New York by running each data point through the branches.
Here you can see the data that was used to train the tree flow through the tree.
This data is called training data because it was used to train the model.
Because we grew the tree until it was 100% accurate, this tree maps each training data point perfectly to which city it is in.
Reality check
Of course, what matters more is how the tree performs on previously-unseen data.
To test the tree's performance on new data, we need to apply it to data points that it has never seen before. This previously unused data is called test data.
Ideally, the tree should perform similarly on both known and unknown data.
So this one is less than ideal.
These errors are due to overfitting. Our model has learned to treat every detail in the training data as important, even details that turned out to be irrelevant.
Overfitting is part of a fundamental concept in machine learning that we’ll explain in our next post.
Recap
- Machine learning identifies patterns using statistical learning and computers by unearthing boundaries in data sets. You can use it to make predictions.
- One method for making predictions is called a decision trees, which uses a series of if-then statements to identify boundaries and define patterns in the data
- Overfitting happens when some boundaries are based on on distinctions that don't make a difference. You can see if a model overfits by having test data flow through the model.
Coming up next
In our next post, we will explore overfitting, and how it relates to a fundamental trade-off in machine learning.
Questions? Thoughts? We would love to hear from you. Tweet us at @r2d3us or email us at<EMAIL_ADDRESS>Want to be notified when the next post is released?
Follow us on Twitter...
Machine learning explained in interactive visualizations (part 1) http://t.co/g75lLydMH9 #d3js #machinelearning
— r2d3.us (@r2d3us) July 27, 2015
… or keep in touch with email!
Footnotes
- Machine learning concepts have arisen across disciplines (computer science, statistics, engineering, psychology, etc), thus the different nomenclature.
- To learn more about calculating the optimal split, search for 'gini index' or 'cross entropy'.
- One reason computers are so good at applying statistical learning techniques is that they're able to do repetitive tasks, very quickly and without getting bored.
- The algorithm described here is greedy, because it takes a top-down approach to splitting the data. In other words, it is looking for the variable that makes each subset the most homogeneous at that moment.
- Hover over the dots to see the path it took in the tree.
- Spoiler alert: It's the bias/variance tradeoff!


Example 1:
ngForm handling Enter keypress & button click using ngSubmit
User Name:
User Age:
Example 2:
ngForm handling Enter keypress & button click with only ng-click on Submit button
User Name:
User Age:
Example 3:
"div" handling Enter keypress
User Name:
User Age:


This demo is in the process of moving! When it is available, you will be able to find it on our samples page by searching for keyboard navigation.


USD/CAD price news & forecast: Canadian dollar spikes to 1.3300 handle, over 1-month tops
by Yohay ElamUSD/CAD spikes to 1.3300 handle, over 1-month tops
The USD/CAD pair jumped to over one-month tops during the early European session on Wednesday, with bulls now looking to extend the momentum further beyond the 1.3300 handle. Read more...
USD/CAD
| Overview | |
|---|---|
| Today last price | 1.3296 |
| Today Daily Change | 0.0025 |
| Today Daily Change % | 0.19 |
| Today daily open | 1.3271 |
| Trends | |
|---|---|
| Daily SMA20 | 1.3169 |
| Daily SMA50 | 1.3212 |
| Daily SMA100 | 1.3203 |
| Daily SMA200 | 1.3276 |
| Levels | |
|---|---|
| Previous Daily High | 1.3274 |
| Previous Daily Low | 1.3156 |
| Previous Weekly High | 1.3272 |
| Previous Weekly Low | 1.3212 |
| Previous Monthly High | 1.3349 |
| Previous Monthly Low | 1.3042 |
| Daily Fibonacci 38.2% | 1.3228 |
| Daily Fibonacci 61.8% | 1.3201 |
| Daily Pivot Point S1 | 1.3193 |
| Daily Pivot Point S2 | 1.3115 |
| Daily Pivot Point S3 | 1.3075 |
| Daily Pivot Point R1 | 1.3311 |
| Daily Pivot Point R2 | 1.3351 |
| Daily Pivot Point R3 | 1.3429 |
USD/CAD Analysis: breakout occurs
Upside risks have dominated the US Dollar against the Canadian Dollar since yesterday's trading session. The currency pair has appreciated by 98 basis points since the previous session. A breakout occurred through the upper boundary of a descending channel pattern at 1.3220 on Tuesday. Read more...


In-class assignments¶
Lecture 22¶
In your IPython Notebook, write a block of code that implements a protein molecular calculator by summing the weights of the individual amino acids in a protein sequence.
Lecture 20¶
Write a python Hello World script¶
Log in to the cluster, write a program called hello_world.py
that when run prints the text Hello World!
to the terminal. You should be able to call this script from anywhere like:
hello_world.py
Lectures 13-14¶
Working with a graphical ftp client¶
Use Cyberduck to move data files to and from the cluster. You can find instructions for connecting to AWS with Cyberduck here.
QIIME Illumina Overview tutorial¶
We’ll work through the QIIME Illumina overview tutorial in class. This will be related to the next homework assignment, so I encourage you to continue working on this outside of class if we don’t finish up. You can find the QIIME Illumina Overview Tutorial here.
Lectures 10-11¶
Today’s exercises are all to be performed in the QIIME AWS instance. You’ll get a public DNS entry to connect in class.
Writing a first shell script¶
Set your PATH
environment variable to contain a new scripts
directory under your directory. Create a new file called my_script.sh
with nano
and enter the following text:
#!/bin/bash
echo "Below are contents of the directory:"
pwd
ls -al
echo "The time is currently:"
date
Now change the permissions on this file to give it execute permissions:
chmod u+x my_script.sh
This exercise is derived from Practical Computing for Biologists
Lecture 9¶
Today’s exercises are all to be performed in the QIIME AWS instance. You’ll get a public DNS entry to connect in class.
Using grep
¶
Under the qiime_software
directory, there is a gg_otus_4feb2011
directory, and another directory under that called rep_set
which contains several fasta files. How do you perform a search with grep
? How do you get grep
to print the number of lines in a file rather than the lines which match a pattern? How many sequence records are in the 97
, 88
, and 73
variants of those files?
Using curl
and grep
¶
Download the EMP minimal mapping file, directly into your AWS instance, from here
using curl
- you’ll need to unzip that file with gunzip
to get started. You can read about the file format here.
How do you invert a search with grep
? How many lines contain information on human-associated samples? How many lines contain information on non-human-associated samples?
Lecture 5¶
These exercises will make use a text editor, such as TextWrangler or jEdit.
Lecture 4¶
Reformat sequence headers in a fasta file¶
Start with this file
Rewrite each identifier as the portion of the identifier preceding the . character, followed by an underscore, followed by the genus name.
Reformat coordinates¶
Start with LatLong.txt
(source: Practical Computing for Biologists)
Make Lat/Long pairs tab-separated on a single line
Remove trailing N and E, and replace with leading +
Remove trailing S and W, and replace with leading -


Fill in the blank.
\dfrac{NUM_1}{DEN_1}
____ \dfrac{NUM_2}{DEN_2}
SOLUTION
<
>
=
It is easier to compare these two fractions when they have the same denominator.
Their smallest common denominator is the LCM of DEN_1 and DEN_2.
\lcm(DEN_1, DEN_2) = LCM
The first fraction BECOMES_1 \dfrac{NUM_1 * F1}{LCM}
.
The second fraction BECOMES_2 \dfrac{NUM_2 * F2}{LCM}
.
We see that \dfrac{NUM_1 * F1}{LCM} SOLUTION \dfrac{NUM_2 * F2}{LCM}
.


KSI brands Justin Bieber 'delusional as f***' after Logan Paul fight
KSI has reflected on his triumphant win against Logan Paul – and he had some strong words for Justin Bieber.
It’s kicking off.
The Love Yourself singer was left red-faced after going full force into supporting the American YouTuber, later even refusing to believe the end result.
However, KSI – real name Olajide ‘JJ’ Olatunji – isn’t bothered about Biebs. In fact, the situation appears to have cracked him up.
‘A lot of influential people were hoping for my downfall, including Justin Bieber and, well, it sucks to be him right now. Get mad,’ the YouTube star giggled.
‘Justin Bieber was in the ring while I was in the ring as well. He was on the opposite side though, because man’s afraid of me. Literally I was mouthing to him: “Logan didn’t win. Logan didn’t win.”‘
 Justin Bieber was just sarcastically smiling…my man was so triggered.’
Advertisement Advertisement
Pulling up a screenshot of Justin Bieber’s Instagram post, which declared ‘REALITY..LOGAN PAUL CHAMPION’, the YouTuber added: ‘Clearly he’s not living in reality and is delusional as f*** so Justin Bieber – let me confirm this reality.
‘Here is my boxing record, and here is Logan’s.
Ouch.

KSI defeated Logan against the odds, making sure to have a party in Las Vegas like no other following the win.
However, the Paul brothers haven’t stayed quiet following the result (is anyone surprised?) with Logan claiming he’s going to contest the Commission.
Meanwhile, his younger brother Jake shared a video of himself breaking down in tears following the loss, getting into a heated argument with heavyweight champ Shannon.
With KSI fans urging the brothers to simply ‘take the L’, it seems all is a lot calmer on the British side of the pond.
Got a showbiz story?
If you've got a story, video or pictures get in touch with the Metro.co.uk Entertainment team by emailing us<EMAIL_ADDRESS>calling 020 3615 2145 or by visiting our Submit Stuff page - we'd love to hear from you.


Add Measurement
Value Number:
Value Text:
Batch Id:
Process:
Measurement Type:
Time:
Synch Status:
Save


Let sea-discoverers to new worlds have gone, Let maps to others, worlds on worlds have shown, Let us possess one world, each hath one, and is one.


| Path: | README |
| Last Update: | Thu Aug 20 10:28:22 -0300 2009 |
GraphMap is a Ruby On Rails plugin that provides a few view helpers to represent information in maps.
You can get the code using Git:
git clone gitosis@safernet.org.br:graph_map
DataMap is a Ruby library which GraphMap depends on. It‘s attached as a submodule, so inside the graph_map directory, do the following:
git submodule init git submodule update
Yet inside the graph_map directory, you can run ‘rake’ to run its tests.
Now at the root of your Rails application, run
rake graph_map:install
to perform some actions the will let you use the plugin.
And so, you‘re done.
Some things can be set up using the hash GraphMapConfig.config, see its keys below. Possible settings are (both methods get and set are available for each key):
root: The root of your Rails application. Default: RAILS_ROOT output: Output file name. Default: graph_map maps_dir: Where the maps are saved. Default: /public/graph_maps maps_path: Where to look for maps to show in the browser. Default: /graph_maps js: Where to look for javascripts. Default: /javascripts default_output_format: Image format to use when none is sent. Default: png
This plugin provides some view helpers. To use some of them, simply call
<%= <view_helper_name> <parameters> %>
inside some view of your Rails application. Below it‘s listed what helpers are available and what are their possible parameters.
For example, you could do the following inside a view:
<%= plot_svg_map { "es" => 200, "en" => 150, "br" => 200, "ar" => 25 } :map => { :title => 'This is my map', :output => 'map_filename_without_extension', :tag => 'iframe' }, :html => { :width => 900, :height => 500, :class => 'map-class' } %> <%= plot_bitmap_map { "es" => 200, "en" => 150, "br" => 200, "ar" => 25 } :map => { :title => 'This is my map', :output => 'map_filename_without_extension', :quality => 75, :size => '800x600>', :format => 'jpg' }, :html => { :class => 'map-class', :alt => 'This is a map' } %> <%= link_to_map 'Download my map!', { "es" => 200, "en" => 150, "br" => 200, "ar" => 25 }, :map => { :title => 'My map', :format => 'pdf', :output => 'map' }, :html => { :class => 'pdf-link' } %> <%= plot_table { "br" => 100, "ar" => 50 }, { :flag => 'Flag', :country => 'Country name', :value => 'Number of...', :total => 'Sum' }, { :width => '100%' } %>
plot_svg_map: Plots a map in SVG format. Returns an HTML tag with an SVG map inside it.
plot_svg_map(data = {}, options = { :html => {}, :map => {} }) data: Hash whose pairs are in format '<country code> => <value>'. Example: { "us" => 125, "es" => 254 } Default: {} options: Can be HTML options or map options. html: Options for the tag that will be rendered. Any HTML attribute can be passed. Example: { :style => 'border: 0', :id => 'my_map', :class => 'some class' } Default: {} Possible values: Any HTML attribute map: Options for the map. Some of them are the same as the ones accepted by the DataMap lib. Example: { :tag => 'embed', :title => 'Title of my map' } Default: {} Possible values: tag: Which tag will be rendered to show the SVG map. If some strange value is passed, an exception will be raised. Possible values: 'embed', 'iframe' and 'object' Default: 'iframe' use_js: If the map must be dynamic or not. Dynamic means that when the mouse is over a country, this country is highlighted and a tooltip is shown, with the country name, flag and value. Possible values: true or false Default: true range_function: What function will be used to make the values ranges. An exception will be raised if the function name is not recognized. Possible values: 'log' or 'linear' Default: 'log' log_base: What is the base of the log function, if the chosen range function is log. Possible values: Any integer Default: 10 title: Title of the map. Possible values: Any string Default: No title legend: If the legend must be shown or not. Possible values: true or false Default: true palette: What color palette will be used. Possible values: The path for a color palette file. This file must be a plain text file with each line being a color in RGB format. See the default one as an example. Default: WorldMap.default_palette (data_map/palettes/tango.color) lang: In what language the country names will be shown. Possible values: A string representing a language. The directory data_map/lang/<language> must be present, with the proper files inside it. See data_map/lang/en for example. Default: WorldMap.default_language (en) template: The SVG template that will be used. Possible values: The path for an SVG file. Default: WorldMap.default_template (default_world_map.svg) output: The output filename (without the .svg extension). Possible values: Any string Default: GraphMapConfig.output
plot_bitmap_map: Plots a map in some bitmap format. Returns a img tag with the map.
plot_bitmap_map(data = {}, options = { :html => {}, :map => {} }) data: Hash whose pairs are in format '<country code> => <value>'. Example: { "us" => 125, "es" => 254 } Default: {} options: Can be HTML options or map options. html: Options for the img tag that will be rendered. Any HTML attribute can be passed. Example: { :style => 'border: 0', :id => 'my_map', :class => 'some class' } Default: {} Possible values: Any HTML attribute map: Options for the map. Some of them are the same as the ones accepted by the DataMap lib. Example: { :title => 'Title of my map', :format => 'pdf' } Default: {} Possible values: format: Which format will be used to render the map. Possible values: The ones accepted by RMagick Default: GraphMapConfig.default_output_format range_function: What function will be used to make the values ranges. An exception will be raised if the function name is not recognized. Possible values: 'log' or 'linear' Default: 'log' log_base: What is the base of the log function, if the chosen range function is log. Possible values: Any integer Default: 10 title: Title of the map. Possible values: Any string Default: No title legend: If the legend must be shown or not. Possible values: true or false Default: true palette: What color palette will be used. Possible values: The path for a color palette file. This file must be a plain text file with each line being a color in RGB format. See the default one as an example. Default: WorldMap.default_palette (data_map/palettes/tango.color) lang: In what language the country names will be shown. Possible values: A string representing a language. The directory data_map/lang/<language> must be present, with the proper files inside it. See data_map/lang/en for example. Default: WorldMap.default_language (en) template: The SVG template that will be used. Possible values: The path for an SVG file. Default: WorldMap.default_template (default_world_map.svg) output: The output filename (without the file extension). Possible values: Any string Default: GraphMapConfig.output quality: Quality of the image. Possible values: Any integer from 0 to 100 Default: 100 size: Dimensions of the image. Possible values: A size string accepted by RMagick Default: Size of the chosen template
link_to_map: Links to a map. Returns an <a> tag pointing to the map.
link_to_map(link_text = 'Download', data = {}, options = { :html => {}, :map => {} }) The 'data' and 'options' parameters works as the same as above. If the options[:map][:format] parameter is set as 'svg', an SVG map will be generated, so use the same map options as plot_svg_map. Else, a bitmap map will be generated, so use the same map options as plot_bitmap_map. link_text: The text link. Possible values: Any string Default: 'Download'
plot_table: Plots a table showing each value by country. Returns a <table> tag. The first column are the flags, then the country names
and finally the values. The countries column and the values column are sortable (by Javascript). The sum of all the values is shown in the last row of the table. The country name depends on the language set in DataMap. plot_table(data = {}, header = {}, html_options = {}) data: Hash whose pairs are in format '<country code> => <value>'. Example: { "us" => 125, "es" => 254 } Default: {} header: How the columns will be named. The keys are: flag: How to call the flags column. Default: 'Flag' country: How to call the countries column. Default: 'Country' value: How to call the values column. Default: 'Value' country: How to call the total row. Default: 'Total' html_options: Options for the img tag that will be rendered. Any HTML attribute can be passed. Example: { :style => 'border: 0', :id => 'my_map', :class => 'some class' } Default: {} Possible values: Any HTML attribute
At some time, possibly the number of files stored in your graph_maps directory will be really big. To avoid this, you can generate some maps with the same name (this way it will be always overwritten). Thus, you can clear the cache at any time by running
rake graph_map:clear
It will erase all content in your graph_maps folder.
Check the RDoc documentation. See DataMap documentation for more details that are not covered here.
Take a look at MIT-LICENSE.
The DataMap dependencies: RMagick and Color.
For Debian users, it‘s enough to get the following packages: libcolor-tools-ruby librmagick-ruby
If you find any mistake in this documentation or bug in the plugin, please contact me at<EMAIL_ADDRESS>Copyright (c) 2009 Caio SBA <caiosba@gmail.com>


Importance
Classes
Teachers
Certifications
About TSA
Competition
Leadership
Community Service
BETA TSA
Tech & Engineering Education
arrow_drop_down
Chapter 513
arrow_drop_down
About
2017 Prompt


You can create a range slider bar using minimal markup. The slider's value will be assigned to any elements inside the data-slider div and as the value of the data-slider attribute. Requires the file range-slider.scss.
range-slider.scss


{{ define "title" }} Signin {{ end }} {{ define "body" }}
{{ .ErrMsg }}
Want to sign in with a one time link?
Don't have an account? Sign up!


We appreciate that you took the time to send us your story with maine coon. Your comment has been sent to the webmaster.
Back to Maine Coon Cat Blog.


| {{dnaPurification.id}} | {{dnaPurification.created_at | utDate | date:'MM/dd/yy'}} | {{dnaPurification.User.fullName}} | {{dnaPurification.Sample.description}} | {{dnaPurification.cultureVolume}} | {{dnaPurification.elutionVolume}} | {{dnaPurification.concentration}} | {{dnaPurification.dnaYield}} | {{dnaPurification.chain}} | {{dnaPurification.expectedDate | utDate | date:'MM/dd/yy'}} | {{dnaPurification.status}} | {{dnaPurification.completionDate | utDate | date:'MM/dd/yy'}} | {{dnaPurification.comments}} | {{dnaPurification.notes}} | {{dnaPurification.updated_at | utDate | date:'MM/dd/yy'}} |
|


About the program
The purpose of this simulator is to experiment with cellular automata (analogs of Game of Life), living on the Minkowski plane: an analog of the Euclidean plane governed by the pseudo-euclidean distance (note the minus sign):
$$d((x_1,y_1),(x_2,y_2)) = (x_1-x_2)^2-(y_1-y_2)^2.$$World
Lattice matrix
Cellular automata in this simulator live on a lattice of regularly spaced points with a certain rotational symmetry. In the Euclidean plane, there are two such lattices: square lattice having 4-fold rotational symmetry and hexagonal lattice having 6-fold symmetry. However, in the Minkowski plane there is infinite number of such lattices.
It might be not obvious, but such lattices (both Euclidean and Minkowskian) are identified by the integer matrices of determinant 1, also known as the elements of the special linear group \(SL(2,\mathbb{Z})\).
$$M=\left(\begin{matrix}a && b\\c && d\end{matrix}\right), |M|=ad-bc=1,$$ $$a,b,c,d \in \mathbb{Z}.$$This, however, is not a one-to-one relation, since different matrices could define the same lattice. Particularly, congruent matrices generate the same lattice. The trace of the matrix (the sum of its diagonal elements \(a+c\)) determines lattice type: if \(|tr(M)|<2\), then the lattice is Euclidean, and if \(|tr(M)|>2\) then it is Minkowskian. The case when \(tr(M)=\pm 2\) is degenerate and not supported by the simulator.
In the simulator, Lattice matrix field defines the lattice. It must contain four integers separated by spaces, in row-major order: "a b c d".
Some matrices and their lattices:
| Matrix | Lattice |
|---|---|
| \(\left(\begin{matrix}0 && -1\\1 && 0\end{matrix}\right)\) | Euclidean square lattice |
| \(\left(\begin{matrix}1 && 1\\-1 && 0\end{matrix}\right)\) | Euclidean hexagonal lattice |
| \(\left(\begin{matrix}2 && 1\\1 && 1\end{matrix}\right)\) | The simplest Minkowskian lattice |
Neighbors
In cellular automata, cell state is affected by the states of its neighbors. In this simulator, all cells at the certain distance are considered neighbors. It is also possible to specify several distances. Instead of specifying the distance directly (which proved to be inconvenient), a list of sample neighbor coordinates is given. The format of the list is "x1 y1;x2 y2;..."
Cell coordinates are not obvious, to see them enable Draw tool and hover the mouse above the grid. Text in the bottom left corner of the view displays cell coordinates. Alternatively, use Cue tool and select some cell. Cell coordinates are displayed at the same place
Interface
Some tools on the top toolbar are grouped in pairs. When using mouse, left mouse button activates the selected tool, and right mouse button activates its pair.
Editing cells
Enable Draw tool and use mouse do toggle cells. When using custom rules with more than two states, additional buttons allowing to select state are shown. Draw tool hotkey is D.
Running the automaton
To run the simulation for one step, use the Step button. Its hotkey is N. If the calculation takes long time, a button allowing to cancel the simulation appears.
To undo the last simulation step, press Z.
To run the simulation continuously, click Go button.
Random configuration
To generate random configuration of cells, use controls in the "Randomize" section of the left panel. Enter the size of the random block, and the density of "alive" cells, then click Random fill. The hotkey is A.
Using cue mark
Use Cue mark tool to select a cell and see its neighbors. Neighbor cells are shown with empty circles; curves of equal distance are shown by dashed lines. Cue cool hotkey is U. Alternatively, use Draw tool and right mouse button to put cue mark. To remove cue mark, put it on the same place, or click Hide button on the top toolbar.
Copying and pasting
To copy a configuration of cells, activate Copy tool on the top toolbar, and drag mouse over it. Selected configuration of cells is put into the "buffer" text area on the left control panel. Hotkey is C.
To paste a copied configuration, Paste tool and click where to paste it. Hotkey is P.
Navigation: pan, zoom, rotate
Use Pan tool to drag the view using the mouse. Alternatively, hold Ctrl key and drag mouse - this method works when any other tool is active
Use Rotate tool or drag mouse left and right holding Shift to rotate the view around the center. When Euclidean lattice matrix is used, this tool does a familiar Euclidean rotation. When the matrix is Minkowskian, it does pseudo-rotation (squeeze mapping).
To zoom the view in and out, either use mouse wheel or buttons Zoom In/Out on the left control panel.
"Navigator" section of the left control panel shows current position and rotation of the view. Rotation angle is in degrees; when Minkowskian lattice is active, rotation angle could be arbitrarily high or low: unlike Euclidean rotation, pseudo-rotation does not loops after 360°.
Adjusting the display
Several buttons on the left panel control the appearance of the view:
- C - show lines, connecting neighbor cells. In Minkowskian lattices, neighbors are not obvious.
- E - show empty cells, i. e. the cells whose state is 0.
- X - show cross in the view center.
- N - show numbers instead of colored dots.
The slider above them controls the size of the colored circle of the cell image.
Exporting data
There are two ways to export data from the simulator. The first one is to use Copy tool and then copy the text from the "Buffer" text area. This text can be stored, then pasted back to the buffer.
The other way is to export current simulator state to the URL using the Save to url... button. It generates an URL that encodes current world state, rule and view position.
Custom rules
Custom rules allow to run custom automata with any number of states and user-defined transition rules. They need not be totalistic. To enable custom rule, click Custom rule... button and edit the code. Click Basic sample / Complex sample buttons to load sample code.
Custom rule is defined by a JavaScript object having following fields:
next
: required, its type isfunction(state, sumNeighbors) -> newState
. Calculates new state of a cell from its current state and (generalized) sum of the neighbors.states
: optional integer, number of states in the automaton. Default value is 2.foldInitial
: optional, any type. Initial values for calculating generalized sum of neighbors. Default value is 0.fold
: optional,function(sum, state) -> updatedSum
. Function to calculate generalized sum of the neighbors. Adds one neighbor state to the sum. Default is plain summation:function(sum,s){return sum+s;}
.
Note that the order of neighbor summation is not defined. Also, only cells with nonzero state are summed.
Shortcuts overview
| Editing | |
| N | Next step |
| Z | Undo last step |
| G | Run/stop simulation |
| A | Random fill |
| E | Clear the world |
| Mouse tools | |
| D | Enable drawing mode |
| U | Enable cue mark mode |
| M | Enable panning mode |
| R | Enable rotation mode |
| C | Enable select and copy mode |
| P | Enable paste mode |
| Viewing and navigation | |
| Q | Hide cue mark if shown |
| H | Havigate to origin |
| [ | Zoom in |
| ] | Zoom out |
References
- Mind-blowing "Dichronauts" novel by Greg Egan that inspired this piece of software.
- Sources at the Github.


Monitoring: Turning Noise into Signal
The humble text based log file has been with us for a very long time and it often still suffices for simple tasks which will require little genuine support.  Consequently the volume of diagnostic data has grown too as we struggle to make sense of what’s going on inside our “black boxes”.
Moving a system from the relative comfort of an internal data centre out onto the cloud also brings with it a host of new experiences which will add to the general ambient noise as networks routes go up and down and servers come and go on a regular basis, and I’m not talking about continuous deployment here either. Okay, it’s not quite that unstable at the smaller end of the spectrum but there are many more reasons why your service does not appear to be behaving itself over-and-above the mistakes that we might make ourselves.
With that growing volume of diagnostic data we need to remain smart about how we handle it if we are to understand it. What might initially seem like just a bunch of random failures may in fact be something much easier to explain if only we chose to drill into it and categorise it further. We refactor our functional code to help us make more sense of the design and the same is true for our diagnostic code too – we need to continually refine it as we learn more about the problems we encounter so that they are easier to monitor and triage in future. Ultimately you do not want to be woken in the middle of the night because you cannot distinguish between a failure on your side of the fence and a known limitation of an upstream system which you cannot control.
Hence this article is about the journey you might take after starting with a big ball of monitoring data so that you can start to make sense of it, both inside and outside your team, by improving the code so that what was once potentially just noise can be turned into more useful signals.
The most basic form of log file is just a simple collection of lines of freeform text. These generally form a narrative which tries to tell a story about what’s going on inside, e.g.
2018-02-16 08:14:01.100 9876 INF Fetching orders
2018-02-16 08:14:01.123 9876 ERR Failed to fetch orders for ‘1234’
2018-02-16 08:14:01.145 4225 PRF Orders request took 10 ms
With each message being a blank canvas it often falls to the individual author to tell their part of the story in their own way and that naturally leads to differences in style, just as we might find stylistic differences in the code as each author stamps their own identity on it through brace placement and vocabulary. Consequently we find that different delimiters are used to surround text parameters (or maybe none at all), dates might be formatted locally or in UTC, primitive values output with different units, and spelling & grammar of variable quality too.
This melting pot of text might have some “character” but it’s a royal pain to parse automatically and make sense of at any non-trivial scale. Once you start needing to aggregate and slice-and-dice this data so it can be used to provide overviews of the system health and generate alerts when the proverbial muck hits the fan, then freeform text no longer cuts it – a clear, consistent format needs to be adopted which is machine readable and aggregatable. The alternative is a life spent trying to concoct ever more complex (and unmaintainable) regular expressions to match the dizzying array of similar messages; or not, which is the usual outcome. Machine learning may provide some hope in the future for an otherwise lost cause but until that day comes we need a simpler solution.
Enter stage left – structured logging. Instead of trying to use our natural language, such as English prose, to describe our situation we start to think about it in less emotional terms. Log messages stop being a flat representation of what has passed and instead become richer “diagnostic” events which can be reacted to programmatically, both within the process as well as outside it [1]. In its simplest guise instead of writing the traditional message “File ‘Xxx’ loaded in 10 ms” you instead log, say, a FileLoaded diagnostic event with a property DurationInMs set to 10 and a Path property set to the file name, e.g.
log.Info($“File ‘{filename}’ loaded in {duration} ms”);
now becomes,
log.Write(new FileLoaded{ DurationInMs=duration, Path=filename });
As you can imagine all the usual programming tricks are still available to hide some of the complexity and keep the logging noise down in the code to ensure the functional aspects still shine through, e.g. free functions or extension methods for the most common scenarios:
log.FileLoaded(filename, duration);
In C# you can leverage the “using” mechanism to wrap up the timing and event writing with even less noise thereby giving you some really simple instrumentation [2]:
using(journal.FileLoad(filename))
{
. . .
}
(To try and make it clearer that it’s not a traditional log file I like to rename the eventing façade something like “journal” and use terms like “record” to get away from past traditions.)
This change in mind-set opens up a new world of possibilities about how you write your monitoring and instrumentation code, and how and where you aggregate it. For example although modern logging search services are fairly powerful, even if you have used correlation IDs [3] to relate your events within a process, finding and summing them can still be arduous, especially if the entire IT department is sending all their logs to, say, the same Splunk instance. Tools like Splunk allow you to “join” log event streams (much like a SQL join) but they have limits to keep processing costs down which often means your joins are truncated unless your queries are highly selective. Consequently you may choose to perform some local aggregation in-process and send out “summary” events too to ease the burden on the monitoring side. For example a REST API might generate a summary event that includes the overall timing for handling a request along with a breakdown of the time spent in database calls, outbound to other external services, heavy computations, etc.
With all diagnostic events being richly typed (if you choose to go that far) and flowing through a single stream it’s easy to build in in-process feedback loops as well to monitor and react to local conditions. For example the Circuit Breaker pattern which Michael Nygard introduced in his seminal book [4] is a technique for dealing with temporary failures by shutting off the value when things go awry. One way of implementing this is to continuously analyse the diagnostic stream and, if you see a continuous batch of failure events, flick the switch and turn yourself off for a bit. This can be combined with another classic design pattern – Leaky Bucket Counter [5] – to slowly turn the tap back on. This elevates the diagnostic event stream from a purely non-functional concern to a more functional one which brings with it greater responsibility. (Logging is often treated as a second class citizen due to its non-functional nature but reliable systems elevate non-functional concerns to ensure they get the attention they truly deserve.)
Although you can use much richer types to represent each event you need to bear in mind that the goal is to try and aggregate similar kinds of events across multiple processes to get a more holistic feel for what is happening. Consequently there is still a trade-off between being too loose or too rich and therefore being no better off than we were before. You need just enough structure to ensure a good degree of consistency and to make it easy to add similar classes of events while also ensuring your events are easy to consume in the monitoring system. At its most basic an event might simply be a name and an optional value, e.g.
class Event
{
public string Name { get; }
public double? Value { get; }
}
Naming in software development is always a difficult problem and once again it rears its ugly head here too. You can slice and dice the problem of naming events and their properties however you like but if you want them to be easy to consume in the monitoring tool then you’ll want to adopt a scheme that works well with the query facilities on offer. For example, using a namespace style approach which reads narrower from left-to-right, just like code, is a common option, such as “component.operation.duration” for the time taken to perform a specific operation. This could then be translated into a monitoring query like “database.*.duration” to get an overview for the performance of the database. Once you can aggregate key components over a time window you have the opportunity to automatically measure trends in performance which is a commonly missed critical source of system behaviour. (For a short tale about how performance can slowly decrease over time and then suddenly drop off a cliff see [6].)
What makes aggregation simpler is to adopt a convention for things which should be aggregated, so for example append “.duration” for timings and, say, “.count” for occurrences of interesting things. This helps reduce (though not eliminate) the chances of the classic mistake of treating “1” as “1 ms” instead of “1 failure” which might make your performance look better or worse than expected. If you’re going to be using different units for the same dimension, such as timings, you’ll either need to include that as a property or make it clear in the name so that you can correctly sum them, e.g. “.duration-ms” vs “.duration-ns”. With the events now being rendered automatically by code instead of manually formatted by hand you stand a better chance of rescuing the situation if you decide that your precision is too low for one particular kind of event. The Prometheus documentation is a good source of inspiration for categorising and naming events [7].
If you’re wondering about the basic logging attributes you’ve grown to know and love, such as the timestamp and process ID, well you can use a base class to capture those, e.g.
class EventBase
{
public DateTime Timestamp { get; }
public string Name { get; }
public int ProcessId { get; }
}
class DatabaseQueried : EventBase
{
public int Duration { get; }
public DatabaseQueried(string operation int duration)
: this($“database.{operation}.duration”)
{
Duration = duration;
}
}
// Client code.
journal.DatabaseQueried(“find-customer”, duration);
However before you go sticking all your favourite extra fields in every event, just ask yourself whether you really need them every time or is that only a choice you’ve taken because of the inherent limitations of the traditional method of logging? A fixed width layout was always preferable for a classic log file; maybe it’s not so important now?
One immediate concern I had when first adopting this approach was that I was so used to reading classic log file narratives that I felt they would be much harder to consume on a development & debugging level rather than a production monitoring basis. For example I’ve become used to doing some basic system level testing after making certain kinds of changes to ensure that the non-functional aspects are well represented, i.e. is any new instrumentation as useful as it could be? Given the usual modes of failure do we have recorded what we would generally expect if we have a failure in the new area of functionality? If it involves any I/O have we timed it so that we can detect problems and visualise trends? This has usually involved reading the event stream manually and playing with classic command line and log viewing tools.
Hence, one question you may have had right from the get-go is about what the event stream looks like outside the process. It depends on the logging infrastructure and destination sinks but the simplest choice might be to just render them as JSON as it has a fairly simple format (less of an “angle-bracket” tax than XML) and JSON writers are often performant enough for most logging uses. Hence instead of lines of freeform text you might have lines such as this instead:
{ Time:"2018-02-16T08:14:01.123", Name:"orders.http-error.count",
...}
{ Time:"2018-02-16T08:14:01.123", Name:"orders.request.duration-ms",
...}
An event stream dumped as raw JSON is clearly not quite as easy to read as plain text. At least, it’s perhaps not as easy to read at first, but you soon realise that what really matters is whether you can easily relate the event that you see to where it occurs in the code and what it’s telling you about the operation that happened. Surprisingly I got used to this style very quickly; I almost began to feel like Cypher in The Matrix – I stopped seeing the curly braces and quote marks and just saw “HTTP client error”, “token refreshed”, “database queried”, etc. Your mileage will of course vary considerably, but just give it a little time. Apparently there’s more wisdom in the old adage “omit needless words” than I had quite appreciated.
The other good thing about a decent JSON serializer is that you can usually control the order of serialization so you can ensure that the most important attributes get written first (e.g. Timestamp and Name) and lesser ones later which will reduce the burden somewhat when eyeballing the log file.
Another benefit which a more event focused approach gives you is a little more discipline around sticking to true operational events rather than mixing in “debug” or “trace” level messages, which are often of little concern to first level support but which might be useful to a developer doing a full-on post mortem later. This duality of narratives (monitoring vs debugging) is oft debated and what I’ve personally come to favour is slightly richer events where the “headline” is aimed at operational use and designed to be aggregated as before but attached to it might be a “Debug” property with additional context. For example an “http.request-failure.count” event might have the response status code as a core property but the response headers and first N bytes of the payload attached to separate “Debug.Headers” and “Debug.Content” properties should the need arise.
This brings me back to my earlier comment about attaching too much data to every event. Whilst process IDs might be useful to the ops team, thread IDs are probably less so, and therefore consider where and when you really need them. As for deciding in the code what severity an event should be, that probably disappears too as the interpretation shifts from being considered in isolation to being considered at scale, probably in the context of high availability where more transient failures are tolerated. That isn’t to say that failing to read a fundamental configuration setting like a database connection string is not pretty serious, but just that failure without recovery is what we’re looking for in the short term, not necessarily failure in its own right which is more about trends - let the event state the facts and allow the monitoring to decide if it’s noteworthy or not.
There is one other reason I have seen for overly verbose trace-level logging and that is due to overly complex code. When the code is difficult to reason about, and a problem shows up which is difficult to reproduce, it is natural to resort to “printf” style debugging by adding extra logging to show which code paths are taken and why. Simple code that is well written and easy to reason about is also easier to diagnose without resorting to such heavy handed tactics.
Consequently when writing code we should ask ourselves what we would need to know to be able to determine what significant paths might be taken. If the design does not lend itself to being comprehensible from certain key inputs then we should ask ourselves whether it is too complex and needs to be refactored. Other techniques like Design by Contract may add a little more code here-and-there but this allows us to fail earlier when it’s easier to diagnose and therefore can avoid us logging excessively to track the same problem down later when the problem finally manifests itself, e.g. a latent NullReferenceException.
One half of the equation is about having the means in play to represent diagnostic events in a way that can easily be queried and aggregated, the other is about improving the quality of events as you learn more about the system’s failure modes. This was rammed home early one Saturday morning when I got a support call about a weekend job failure and the diagnostic message was simply:
ERROR: Failed to calibrate currency pair!
Tracking down which of the many currency combinations was at fault would have been made so much easier if the author had included in the message the pair of currencies in question that had failed. Whilst debugging the issue I discovered the information was already available at the code site where the message was logged and so I naturally added it right away to help those on support in the future. (Reflecting on and improving exception messages is a related topic I’ve tackled before [8].)
In an ideal world we would know all the ways that our system can fail and we could code accordingly, but life is just not that fair. Networks like the Internet are awash with services, proxies, caches, load balancers, etc. from different vendors that will behave in different ways when they fail or return an error. Even fairly simple protocols like HTTP are open to different interpretations by different teams, and that’s before you bring in the need to work around broken behaviours on supposedly transparent services because they are trying to be “clever”. The on-premise world does not necessarily fare any better, you still get plenty of bizarre server, network and storage problems to deal with [9].
Consequently although we can predict certain types of failure, such as a network route being unavailable, we may not know all the ways that it will manifest or what diagnostic information we might need to diagnose it. Waking someone up in the middle of the night every time a network request fails is not going to win you any favours and so we need to analyse our errors and drill in to see if we can categorise them further so that the support team only springs into action when there is actually something to do. During development it’s also possible that we can unearth “impedance mismatches” between various components in our system that on the face of it presents as occasional failures but could lead to significant problems once we reach production scale.
What follows are a couple of examples of how non-catastrophic failures can be diagnosed and either resolved or vastly improved upon to ensure the signal-to-noise ratio remains as high as possible. Each step was a small code change, pushed quickly into production, to gather more information about the situation each time it cropped up. Naturally an approach like continuous delivery really helps to keep the feedback loop short between learning and reacting to the new diagnostic information.
Soon after bringing the initial version of an adapter to bridge the on-premise and cloud-hosted worlds of a service together, the team started seeing occasional errors in the HTTP 5xx range. The general rule is that these are server-side issues and therefore our logic used the classic exponential back-off to ride it out, eventually. What was odd was that the subsequent request always went through fine so whatever the transient condition was it always cleared immediately.
Our first-order approach to diagnostic logging in that component only logged the HTTP status code on failure at this point and so we decided to immediately start capturing more information, such as the HTTP response headers, because this error didn’t appear to resonate from the service at the other end. One always needs to be wary of dumping untrusted input even into a logging system and so we were careful to limit what new information we captured.
What this showed was that the HTTP response was coming from Akamai, the company providing some basic security for the underlying service, such as API throttling. Despite only asking for a JSON response we actually got back an HTML content type which raised some interesting questions about our current response handling code. We also decided to include the first half KB of the response content into a “Debug” section of the logging event so that we could inspect it further when it showed up.
The payload included what appeared to be a standard HTML error response from Akamai that also included an “error reference” which the support team could look up and see what the underlying issue was. We now felt that we had enough heuristics available to crudely parse this Akamai error response correctly into an event of its own type and expose the error reference directly as a property so that we could quickly look them up with Akamai’s own diagnostic tools.
As it turned out, in this instance, the error was due to different idle socket timeouts between Akamai and the AWS load balancer which in itself took some effort to discover. At least we now knew for sure that certain 5xx errors were definitely not from the service and therefore there was no need to go looking elsewhere. Also, unless someone had changed the Akamai configuration, which was very rare, we could say with a high degree of certainty that the error really was transient and out of our control. Consequently the monitoring dashboard would now show a reduction in generic HTTP error responses in favour of the more specific Akamai ones.
The second example looks at a wider reaching code change in reaction to learning more about the way a system dependency reported certain types of errors.
Initially our message handler only had very basic error handling – enough to know to retry on a 5xx HTTP response but to triage a request if it got a 4xx one. In HTTP the 4xx response codes generally mean the client needs to take some corrective action, such as fixing its request format (400), authenticating itself (401), or checking if the right resource was being requested (404). We never saw a 404 but we did see occasional 400s which was worrying as we didn’t know what we could have done wrong to create a malformed request for such a simple query.
Luckily, by adding a little extra diagnostic data, we discovered the upstream service provided a richer error payload, in JSON, that we could attach to a “Debug” property of the, then simple, “http.client-error.count” event, along with any response headers, just like earlier. From doing this we discovered that the “service specific” error code was tantamount to a “not found” error, which you’d usually report with a 404 HTTP status code. The service was only providing a simple lookup and therefore we could attach the identifier we were looking up in our request onto a new, richer diagnostic event designed to show which IDs were unfortunately not correctly mapped.
This not only enhanced the monitoring side of the equation but it also meant that we could use a much richer exception type to convey this unusual condition to the outer layers of our code. It turned out that these missing mappings were never going to be addressed any time soon and therefore pushing the request onto a “slow retry” queue was a complete waste of time and we might as well drop them on the floor until further notice.
Aside from a reduced number of “poisoned” messages to deal with on our side, by including the information directly in the monitoring data this also meant the other team which owned the upstream service could query our logs and find out how bad the problem was and whether it was getting better or worse over time. We could also easily see if any particular IDs cropped up regularly and ask if they could be fixed manually. A better choice of HTTP status code in the first place would have made life slightly simpler but we would have still needed to understand the problem to ensure that we removed the unnecessary noise and reacted accordingly; either way we couldn’t have left it as it was.
The freeform text log file has been with us for a long time and still continues to serve us well in the right situations. However as the size and complexity of our systems grows we need something more “grown-up” that can sensibly cater for the needs of the operations team as well as those developing it (even if they are one and the same). These two forces are often at odds with each other, not out of malice but out of ignorance and so we should redress the balance by putting their diagnostic needs on an equal par with our own. Their hands are effectively tied and so those who work with the code need to be the ones to cater for them as well.
Moving towards a more structured approach to logging brings with it an improvement in monitoring as we start to consider what each diagnostic event means, not in isolation but in the context of the system as a whole. By making it easy to aggregate related events across time, by component, or by subsystem we can get a more holistic feel for what is going on under the hood. Then, taking these same data points over a much longer period of time we can get a feel for what the trends are in the system as it grows so that we are not caught off guard when those cyclic peaks come around second or third time.
Finally we also need to accept that “there are more things in heaven and earth, Horatio, than are dreamt of in your philosophy”. We don’t know all the ways our system will fail but by continuing to invest in refining the way we capture and report errors we stand a better chance of ensuring the signal stands out from the noise.
[1] Test Smell: Logging is also a feature, Steve Freeman,
http://www.mockobjects.com/2007/04/test-smell-logging-is-also-feature.html
[2] Simple Instrumentation, Chris Oldwood, Overload #116,
https://accu.org/index.php/journals/1843
[3] Causality – Relating Distributed Diagnostic Contexts, Chris
Oldwood, Overload #114,
https://accu.org/index.php/journals/1870
[4] Release It! Design and Deploy Production-Ready Software,
Michael T. Nygard,
https://pragprog.com/book/mnee/release-it
[5] Leaky bucket, Wikipedia,
https://en.wikipedia.org/wiki/Leaky_bucket
[6] Single Points of Failure - The SAN, Chris Oldwood,
http://chrisoldwood.blogspot.co.uk/2014/09/single-points-of-failure-san.html
[7] Metric and Label Naming, Prometheus,
https://prometheus.io/docs/practices/naming
[8] Terse Exception Messages, Chris Oldwood, Overload #127,
https://accu.org/index.php/journals/2110
[9] The Cost of Not Designing the Database Schema, Chris
Oldwood,
http://chrisoldwood.blogspot.co.uk/2015/12/the-cost-of-not-designing-database.html
Chris Oldwood
25 March 2018
Chris is a freelance programmer who started out as a bedroom coder in the 80’s writing assembler on 8-bit micros. These days it's enterprise grade technology in plush corporate offices. He also commentates on the Godmanchester duck race and can be easily distracted via<EMAIL_ADDRESS>or @chrisoldwood.


Administration QuickStart¶
This document will cover installation and administration points of Singularity for multi-tenant HPC resources and will not cover usage of the command line tools, container usage, or example use cases.
Installation¶
There are two common ways to install Singularity, from source code and
via binary packages. This document will explain the process of
installation from source, and it will depend on your build host to have
the appropriate development tools and packages installed. For Red Hat
and derivatives, you should install the following yum
group to ensure you
have an appropriately setup build server:
$ sudo yum groupinstall "Development Tools"
Downloading the source¶
You can download the source code either from the latest stable tarball release or via the GitHub master repository. Here is an example downloading and preparing the latest development code from GitHub:
$ mkdir ~/git
$ cd ~/git
$ git clone https://github.com/singularityware/singularity.git
$ cd singularity
$ ./autogen.sh
Once you have downloaded the source, the following installation procedures will assume you are running from the root of the source directory.
Source Installation¶
The following example demonstrates how to install Singularity into /usr/local
.
You can install Singularity into any directory of your choosing, but
you must ensure that the location you select supports programs running
as SUID
. It is common for people to disable SUID
with the mount option nosuid
for
various network mounted file systems. To ensure proper support, it is
easiest to make sure you install Singularity to a local file system.
Assuming that /usr/local
is a local file system:
$ ./configure --prefix=/usr/local --sysconfdir=/etc
$ make
$ sudo make install
Note
The make install
above must be run as root to have Singularity properly
installed. Failure to install as root will cause Singularity to not
function properly or have limited functionality when run by a non-root
user.
Also note that when you configure, squashfs-tools
is not required, however it is
required for full functionality. You will see this message after the
configuration:
mksquashfs from squash-tools is required for full functionality
If you choose not to install squashfs-tools
, you will hit an error when your users try
a pull from Docker Hub, for example.
Prefix in special places (–localstatedir)¶
As with most autotools-based build scripts, you are able to supply the --prefix
argument to the configure script to change where Singularity will be
installed. Care must be taken when this path is not a local filesystem
or has atypical permissions. The local state directories used by
Singularity at runtime will also be placed under the supplied --prefix
and this
will cause malfunction if the tree is read-only. You may also
experience issues if this directory is shared between several
hosts/nodes that might run Singularity simultaneously.
In such cases, you should specify the --localstatedir
variable in addition to --prefix
. This
will override the prefix, instead placing the local state directories
within the path explicitly provided. Ideally this should be within the
local filesystem, specific to only a single host or node.
For example, the Makefile contains this variable by default:
CONTAINER_OVERLAY = ${prefix}/var/singularity/mnt/overlay
By supplying the configure argument --localstatedir=/some/other/place
Singularity will instead be built
with the following. Note that ${prefix}/var
that has been replaced by the supplied
value:
CONTAINER_OVERLAY = /some/other/place/singularity/mnt/overlay
In the case of cluster nodes, you will need to ensure the following
directories are created on all nodes, with root:root
ownership and 0755
permissions:
${localstatedir}/singularity/mnt
${localstatedir}/singularity/mnt/container
${localstatedir}/singularity/mnt/final
${localstatedir}/singularity/mnt/overlay
${localstatedir}/singularity/mnt/session
Singularity will fail to execute without these directories. They are
normally created by the install make target; when using a local
directory for --localstatedir
these will only be created on the node make
is run on.
Building an RPM directly from the source¶
Singularity includes all of the necessary bits to properly create an RPM package directly from the source tree, and you can create an RPM by doing the following:
$ ./configure
$ make dist
$ rpmbuild -ta singularity-*.tar.gz
Near the bottom of the build output you will see several lines like:
...
Wrote: /home/gmk/rpmbuild/SRPMS/singularity-2.3.el7.centos.src.rpm
Wrote: /home/gmk/rpmbuild/RPMS/x86_64/singularity-2.3.el7.centos.x86_64.rpm
Wrote: /home/gmk/rpmbuild/RPMS/x86_64/singularity-devel-2.3.el7.centos.x86_64.rpm
Wrote: /home/gmk/rpmbuild/RPMS/x86_64/singularity-debuginfo-2.3.el7.centos.x86_64.rpm
...
You will want to identify the appropriate path to the binary RPM that
you wish to install, in the above example the package we want to install
is singularity-2.3.el7.centos.x86_64.rpm
, and you should install it with the following command:
$ sudo yum install /home/gmk/rpmbuild/RPMS/x86_64/singularity-2.3.el7.centos.x86_64.rpm
Note
If you want to have the binary RPM install the files to an alternative location, you should define the environment variable ‘PREFIX’ (below) to suit your needs, and use the following command to build:
$ PREFIX=/opt/singularity
$ rpmbuild -ta --define="_prefix $PREFIX" --define "_sysconfdir $PREFIX/etc" --define "_defaultdocdir $PREFIX/share" singularity-*.tar.gz
We recommend you look at our security admin guide to get further information about container privileges and mounting.


Transportation
Getting to town
Sea-Tac International Airport (SEA)
Sound Transit light rail runs from the airport to Pioneer Square
Getting around town
Parking: Free parking all day on Sundays
Sea-Tac International Airport (SEA)
Sound Transit light rail runs from the airport to Pioneer Square
Parking: Free parking all day on Sundays