as per the title, really - I've been quietly beavering away (on and off) for the last year - the fruits of my labour are here.
I'll gradually update with more details, but feel free to ask questions/ gasp at the expense*...
*delete as appropriate
Mike
Re: Another solar immersion controller *sigh*
Wow, that's some project! Can you explain the numbers shown in the display please. I would have expected Solar power to be the sum of House power and Immersion power. And are there really three CTs in there? I presume that you're not measuring voltage, just currents. Are you using the current-only sketch in some form?
So many questions (sorry)
Re: Another solar immersion controller *sigh*
Hi, it's nice to have some questions rather than just typing more and more details into the project page...
The sketch is based on Glyn's 2_Channel_NonInvasive, but updated for 3 CT's - one on the House tail (import/ export), one on the Solar line (generation) and one on the Immersion circuit. The 'normal' control is done using a PID (actually only PI) algorithm based on a target export figure (30W), but when I want an Immersion 'one hour boost' I change the setpoint to be 2500W immersion and use the other CT to achieve it. The PID controller allows easy gain tuning for fast response with stability.
The display photo was taken during an immersion 'boost' (top red LED on), so the readings are Solar 30W (generation nearly over), Immersion 2500W, House 2708W - i.e. my other house loads at the time must have been (2708+30) - 2500 = 238W.
The temperature is copper surface 300mm from the top of the water cylinder - I plan to use this with the immersion thermostat as a back-stop because I find them rather unreliable and I like the idea of using the immersion to get the water 'just' warm enough for my morning shower but then let it cook up through the day on 'free export' power...
I'll post the code up once I get a github logon ;-)
Cheers,
Mike
Re: Another solar immersion controller *sigh*
You will find that the phase angle controller will produce harmonics and emissions out of spec for emc. Not a worry for DIY unless you listen to old fashioned radio at 198kHz ( am radio 2). If the filter is a standard one or two stage it will not do anything except very high frequency clean up beacuse it uses a common mode transformer. What you need is to add a large inductor(250uH) in series with the phase angle controller you can make one quite easily the cores are only a few quid. Alternatively UA sell a phase angle filter which incorporates the inductor for about £30. This reduces the harmonics and emc emissions to class A level (industrial) which is more than enough for DIY. Unfortunately they don't do one for domestic class B if you want to be squeaky legal. I have done conducted emission tests on standard filters and Choke combinations as well as UA filter . Graphs available if you are interested.
Re: Another solar immersion controller *sigh*
If you want to heat all of your tank rather than going to the expense of fitting a new tank you could just add a pump, check valve and stat to circulate the water I have done this for the last year and it works well.
Re: Another solar immersion controller *sigh*
I think it depends on the quality of the EMC filter. When running my Mk1 version in phase-angle mode using a Carlo Gavazzi AC controller, I wasn't able to detect any disturbance on the radio.
Circulating the water in the tank to encourage more take-up of available power makes good sense. Wth Mk2's logic, this could be achieved without requiring a second thermostat, and all using free power. As soon as the primary stat opens, the energy bucket's level will increase to the maximum, and this can cause the secondary pump to turn on. If that pump draws more surplus power than is available, the bucket's level will start to fall. Dropping below some convenient threshold could cause the circulation pump to turn off. This approach should be applicable to most schemes which divert surplus power.
I've often thought of having a pumped return pipe from the kitchen. After a few seconds' worth of pumping, hot water could then be made to appear straught away, rather than having to run off half a bowlful of cold. A temperature sensor & display would be a nice touch.
Re: Another solar immersion controller *sigh*
Or, if it has the room you can get a 36in heater to replace a standard 27in one, that's what I did only £20. I do my "topping up" between 8.00 and 9.00 PM (its only needed to do so once) and have the tank stat set to a gnats below what I get on a good day from the solar, I also have the immersion stat set to maximum, the idea being in the morning the water has dropped a few degrees so is more suitable to use in the shower.
Re: Another solar immersion controller *sigh*
Thanks for the replies guys!
I have a Schaffner FN660-20 2 phase filter with inductor that I thought would be appropriate - what do you think? It looks as if it should go in series on the feed from the PAC to the immersion load.
The 36" heater is a great idea, so is the recirculating pump. The funny thing is, I already have a re-circ pump on the circuit to my extension where the shower is to help get the hot water there (bungalow with about 20m pipe run)- I'd been cursing it as a major cause of water heat loss, where in reality what I was seeing was the mixing of hot and cold water in the tank!
I realised it this morning, when I saw the water temp 300mm from top of tank drop suddenly 10 mins before my shower - this was the pump programmer switching on the recirc. The graph shows the 1 hour immersion 'boost' at 4am, then at 4:50 the recirc pump came on mixing the hot and cold water, then at 5am I had my shower! Rather Big Brother-esque that the whole world can see this data - when I'm at work, I can see when the kettle goes on at home...
The good news in all this is that I can schedule the recirc pump to come on for 10mins in the afternoon so my immersion can do some more work on a sunny day - result!
Re: Another solar immersion controller *sigh*
When you want to "boost" water with your immersion, I wonder why you limit the power to 2500W. Couldn't you just put it on full whack and save using the third CT? Or am I missing something?
Re: Another solar immersion controller *sigh*
Good question! I really wanted the third CT so I could monitor the total kWh I send to the immersion. Once I'd decided that, it was easy to switch the PID controller setpoint to hold it at a desired 2500W. I picked 2500W as I've had to replace burnt out 3kW immersion heaters in the past and guessed they might be a bit more durable at 2500W...


--- title: A dedication to Ramanujan author: "Janani Ravi & Arjun Krishnan" date: '2016-04-15' slug: ramanujan categories: - Books-Blogs tags: - math-modeling - India # - review # - movie - books summary: "... one of the greatest mathematicians of all time!" math: true ---
Authors: Janani Ravi, Arjun Krishnan
Srinivasa Ramanujan has been noted as one of the classic ‘genius-mathematicians’ of all time! Hailing from the same state/country as him, Arjun Krishnan and I have always looked up to him & his elegant math as a source of inspiration right through our formative years!
Recent movie that has drastically increased Ramanujan’s international visibility The Man Who Knew Infinity by Matthew Brown.
We watched this beautiful play in the University of Virginia campus during our grad school. Such unmistakable passion for math, that’s almost art-like or was it art!
Here is a great companion book to better understand the Man and little bit of the Math!
Without the help of the following people (and many more we haven’t mentioned here), Ramanujan would have remained an unknown & obsolete genius!
Hardy was an English mathematician who brought Ramanujan to Cambridge, encouraged his professional growth and helped publish his many theorems and elegant solutions.
The director of the new Ramanujan movie, The Man Who Knew Infinity
Princeton number theorist, winner of the Field’s medal, and Associate Producer of the movie.
Ken Ono is this very accomplished mathematician who’s central research questions revolve around the work of Ramanujan. He is also a fantastic and sincere communicator, teacher and author so much so that you can read/listen to almost anything that he has to say.
Famous applied mathematician and author of superb books & articles on mathematics.
December 22, 2012, marks the 125th anniversary of the birth of legendary Indian mathematician Srinivasa Ramanujan. An intuitive mathematical genius, Ramanujan’s discoveries have influenced several areas of mathematics, but he is probably most famous for his contributions to number theory and infinite series, among them fascinating formulas that can be used to calculate digits of pi in unusual ways.


Indexed DB Tutorial
About
IndexedDB
An easy way to store your data in the browser
By Novan Adams
Database Operations
Add a name to this database
Add Name
Search for a Name
Search Name
Edit a Name
Delete Name


Before Change
//!/usr/bin/env python
from setuptools import setup, find_packages
setup(name="tflearn",
version="0.2.2",
description="Deep Learning Library featuring a higher-level API for "
"Tensorflow",
author="TFLearn contributors",
<EMAIL_ADDRESS>url="https://github.com/tflearn/tflearn",
download_url="https://github.com/tflearn/tflearn/tarball/0.2.2",
license="MIT",
packages=find_packages(),
install_requires=[
"numpy",
"six",
"Pillow"
],
test_suite="tests",
classifiers=[
"Programming Language :: Python",
"Operating System :: OS Independent",
"Intended Audience :: Developers"
"Intended Audience :: Science/Research",
"Topic :: Scientific/Engineering :: Artificial Intelligence"
],
keywords=[
"TFLearn",
"TensorFlow",
"Deep Learning",
"Machine Learning",
"Neural Networks",
"AI"
]
)
After Change
from setuptools.command.test import test as TestCommand
// explicitly config
test_args = [
"--cov-report=term",
"--cov-report=html",
"--cov=tflearn",
"tests"
]
class PyTest(TestCommand):
user_options = [("pytest-args=", "a", "Arguments to pass to py.test")]


f_str
objects are intended to be used within the function
set_format_strings
. The f_str
object carries information that
powers a significant amount of layer processing. The format_string
parameter is capable of controlling the display of a data point and decimal
precision. The variables provided in ...
control which data points are
used to populate the string formatted output.
f_str(format_string, ..., empty = c(.overall = ""))
| format_string | The desired display format. X's indicate digits. On the left, the number of x's indicates the integer length. On the right, the number of x's controls decimal precision and rounding. Variables are inferred by any separation of the 'x' values other than a decimal. |
|---|---|
| ... | The variables to be formatted using the format specified in
|
| empty | The string to display when the numeric data is not available. For desc layers, an unnamed character vector will populate within the provided format string, set to the same width as the fitted numbers. Use a single element character vector, with the element named '.overall' to instead replace the whole string. |
A f_str
object
Format strings are one of the most powerful components of 'Tplyr'. Traditionally, converting numeric values into strings for presentation can consume a good deal of time. Values and decimals need to align between rows, rounding before trimming is sometimes forgotten - it can become a tedious mess that is realistically not an important part of the analysis being performed. 'Tplyr' makes this process as simple as we can, while still allowing flexibility to the user.
Tplyr provides both manual and automatic decimal precision formatting. The
display of the numbers in the resulting data frame is controlled by the
format_string
parameter. For manual precision, just like dummy
values may be presented on your mocks, integer and decimal precision is
specified by the user providing a string of 'x's for how you'd like your
numbers formatted. If you'd like 2 integers with 3 decimal places, you
specify your string as 'xx.xxx'. 'Tplyr' does the work to get the numbers
in the right place.
To take this a step further, automatic decimal precision can also be obtained based on the collected precision within the data. When creating tables where results vary by some parameter, different results may call for different degrees of precision. To use automatic precision, use a single 'a' on either the integer and decimal side. If you'd like to use increased precision (i.e. you'd like mean to be collected precision +1), use 'a+1'. So if you'd like both integer and and decimal precision to be based on the data as collected, you can use a format like 'a.a' - or for collected+1 decimal precision, 'a.a+1'. You can mix and match this with manual formats as well, making format strings such as 'xx.a+1'.
If you want two numbers on the same line, you provide two sets of x's. For example, if you're presenting a value like "mean (sd)" - you could provide the string 'xx.xx (xx.xxx)', or perhaps 'a.a+1 (a.a+2). Note that you're able to provide different integer lengths and different decimal precision for the two values. Each format string is independent and relates only to the format specified.
The other parameters of the f_str
call specify what values should
fill the x's. f_str
objects are used slightly differently between
different layers. When declaring a format string within a count layer,
f_str
expects to see the values n
and (if desired)
pct
, which specifies the formatting for your n's and percent values.
But in descriptive statistic layers, f_str
parameters refer to the
names of the summaries being performed, either by built in defaults, or
custom summaries declared using set_custom_summaries
. See
set_format_strings
for some more notes about layers specific
implementation.
Count and shift layers frequencies and percentages can be specified with 'n' and 'pct' respectively.


§431:14-105 Policy revisions that alter coverage. (a) Any policy revisions that alter coverage in any manner shall be filed with the commissioner and shall include an analysis of the impact of each revision on rates.
(b) A filing shall consist of either:
(1) An electronic copy of the filing; or
(2) Two printed copies of the filing.
The commissioner may also request a printed version of an electronic filing to be submitted pursuant to paragraph (1).
(c) After review by the commissioner, the commissioner shall determine whether a rate filing for the policy revision must be submitted in accordance with section 431:14-104. [L 1987, c 349, §9; am L 2005, c 132, §4; am L 2009, c 77, §11]


{% extends 'maps/base.html' %} {% load maps_extras %} {% block title %}{{ 'Welcome' | titlify }}{% endblock %} {% block bodyclass %}page home{% endblock %} {% block content %}
The Platform Co-op Directory is a place where you can search for and connect with co-operatives and other members of the co-operative community.


---
layout: layouts/post.njk
title: "An Illustrated (and Musical) Guide to Map, Reduce, and Filter Array Methods"
permalink: /map-reduce-filter/
date: '2019-03-26'
comments: true
tags:
- Houdini
- Paint API
- Browser API
- CSS
- Exprimental
- Future
description: "Map, reduce, and filter are three very useful array methods in JavaScript that give developers a ton of power. This post makes them a little easier to understand."
link-out: https://css-tricks.com/an-illustrated-and-musical-guide-to-map-reduce-and-filter-array-methods/
---
If you are not redirected, please visit this think!


Toggle navigation
Home
About
Services
Contact
Admin test
Passenger SignUp
Bus Owner SignUp
Online Bus Ticketing
Book a Bus Ticket for your long journey with ease.
Login


Shall I compare thee to a summer's day?

Thou art more lovely and more temperate:

Rough winds do shake the darling buds of May,

And summer's lease hath all too short a date:

Sometime too hot the eye of heaven shines,

And often is his gold complexion dimm'd;

And every fair from fair sometime declines,

By chance, or nature's changing course untrimm'd;

But thy eternal summer shall not fade,

Nor lose possession of that fair thou ow'st,

Nor shall death brag thou wander'st in his shade,

When in eternal lines to time thou grow'st;

So long as men can breathe, or eyes can see,

So long lives this, and this gives life to thee.


SAP to return 1.5 billion euros to shareholders in 2020
FRANKFURT (Reuters) - German business software group SAP (SAPG.DE) said on Monday it would return an extra 1.5 billion euros ($1.7 billion) to shareholders in 2020, with the enhanced payout made possible by its strong financial and operational performance.
SAP, Europe’s most valuable technology firm, said after U.S. market hours that its supervisory board had approved a management proposal for extra payouts either through share repurchases or a special dividend.
The announcement comes ahead of SAP’s capital markets day on Nov. 12 in New York at which some investors - including U.S. activist fund Elliott that declared a stake in SAP earlier this year - had hoped for a far larger buyback.
SAP said, however, that the extra capital return to shareholders - which comes on top of its policy of paying out 40% of after-tax profits in dividends - would be made with the company’s long-term prospects in mind.
“The Supervisory Board and Executive Board are confident that SAP’s strategy of investment in innovation and profitable growth, together with disciplined capital return, will maximize shareholder value for the long term,” it said in a statement.
The capital markets day will be the first major outing for new co-CEOs Jennifer Morgan and Christian Klein. The duo have taken over from long-time boss Bill McDermott, who is leaving to head up ServiceNow (NOW.N).


Class WriteContext
Context object provided during write operations.
Inheritance
System.Object
WriteContext
Namespace: Cesil
Assembly: Cesil.dll
Syntax
public sealed class WriteContext : ValueType, IEquatable<WriteContext>
Properties
Column
Column being written.
If HasColumn == false, or Mode != WriteColumn this will throw.
Declaration
public ColumnIdentifier Column { get; }
Property Value
Context
The object, if any, provided to the call to CreateWriter or
CreateAsyncWriter that produced the writer which is
performing the writer operation which is described
by this context.
Declaration
public object Context { get; }
Property Value
| Type |
Description |
| System.Object |
|
HasColumn
Whether or not Column is available.
Declaration
public bool HasColumn { get; }
Property Value
| Type |
Description |
| System.Boolean |
|
HasRowNumber
Whether or not RowNumber is available.
Declaration
public bool HasRowNumber { get; }
Property Value
| Type |
Description |
| System.Boolean |
|
Mode
What, precisely, a writer is doing.
Declaration
public WriteContextMode Mode { get; }
Property Value
Options
Options used to create writer. Useful for accessing
shared configurations, like MemoryPool(char).
Declaration
public Options Options { get; }
Property Value
RowNumber
The index of the row being written (0-based).
If HasRowNumber == false, or Mode is DiscoveringColumns this will throw.
Declaration
public int RowNumber { get; }
Property Value
| Type |
Description |
| System.Int32 |
|
Methods
Equals(WriteContext)
Returns true if this object equals the given WriteContext.
Declaration
public bool Equals(WriteContext context)
Parameters
Returns
| Type |
Description |
| System.Boolean |
|
Equals(Object)
Returns true if this object equals the given WriteContext.
Declaration
public override bool Equals(object obj)
Parameters
| Type |
Name |
Description |
| System.Object |
obj |
|
Returns
| Type |
Description |
| System.Boolean |
|
GetHashCode()
Returns a stable hash for this WriteContext.
Declaration
public override int GetHashCode()
Returns
| Type |
Description |
| System.Int32 |
|
ToString()
Returns a string representation of this WriteContext.
Declaration
public override string ToString()
Returns
| Type |
Description |
| System.String |
|
Operators
Equality(WriteContext, WriteContext)
Compare two WriteContexts for equality
Declaration
public static bool operator ==(WriteContext a, WriteContext b)
Parameters
Returns
| Type |
Description |
| System.Boolean |
|
Inequality(WriteContext, WriteContext)
Compare two WriteContexts for inequality
Declaration
public static bool operator !=(WriteContext a, WriteContext b)
Parameters
Returns
| Type |
Description |
| System.Boolean |
|
Implements
System.IEquatable<>


Overview
Package
Class
Use
Tree
Deprecated
Index
Help
PREV LETTER
NEXT LETTER
FRAMES
NO FRAMES
All Classes
A
C
D
E
F
G
I
L
M
N
P
R
S
T
U
W
P
PieChart
- Class in
teachersV3.jFreeChart
Contains the implementation for jFreeChart (Pie chart).
PieChart(String, String, Student)
- Constructor for class teachersV3.jFreeChart.
PieChart
Build the PieChart for the student (marks).
PieChart(String, String, Promotion)
- Constructor for class teachersV3.jFreeChart.
PieChart
Build the PieChart for the Promotion (average of students).
pList
- Static variable in class teachersV2.io.
FileReadService
pList
- Static variable in class teachersV3.io.
FileReadService
List of Promotions.
Professor
- Class in
teachersV1
The professor can put marks to students depending of the promotion, the id of the student and if he wants to add/modify a mark.
Professor(String, String)
- Constructor for class teachersV1.
Professor
The constructor with parameters.
professor
- Static variable in class teachersV2.io.
FileRead
Professor
- Class in
teachersV2
The professor can put marks to students depending of the promotion, the id of the student and if he wants to add/modify a mark.
Professor(String, String)
- Constructor for class teachersV2.
Professor
The constructor with parameters.
professor
- Static variable in class teachersV3.io.
FileRead
The temporary attribute for the student.
Professor
- Class in
teachersV3
The professor can put marks to students depending of the promotion, the id of the student and if he wants to add/modify a mark.
Professor(String, String)
- Constructor for class teachersV3.
Professor
The constructor with parameters.
profList
- Static variable in class teachersV2.io.
FileReadService
profList
- Static variable in class teachersV3.io.
FileReadService
List of Professors.
profListFiles
- Static variable in class teachersV2.io.
FileRead
profListFiles
- Static variable in class teachersV3.io.
FileRead
The Real List for all files which contains all professors.
Promotion
- Class in
teachersV1
The Promotion permits to sort students in promotions to know where they belong to.
Promotion(String)
- Constructor for class teachersV1.
Promotion
The constructor with a parameter.
promotion
- Static variable in class teachersV2.io.
FileRead
Promotion
- Class in
teachersV2
The Promotion permits to sort students in promotions to know where they belong to.
Promotion(String)
- Constructor for class teachersV2.
Promotion
The constructor with a parameter.
promotion
- Static variable in class teachersV3.io.
FileRead
The temporary attribute for the Promotion.
Promotion
- Class in
teachersV3
The Promotion permits to sort students in promotions to know where they belong to.
Promotion(String)
- Constructor for class teachersV3.
Promotion
The constructor with a parameter.
Overview
Package
Class
Use
Tree
Deprecated
Index
Help
PREV LETTER
NEXT LETTER
FRAMES
NO FRAMES
All Classes
A
C
D
E
F
G
I
L
M
N
P
R
S
T
U
W


Benjamin Valbret
Questions? - Feel free to contact me.


Delivery Location:
Size:
Toppings: (1 slip GPL each)
Specialized Toppings: (2 slips GPL each)
Cheeses: (2 slips GPL each)
*we are not respolsible for any bio-neural gel packs that are damaged due to proximity to Schplict*


| ${value}m Radius | 3.5 |
| ${value} Electrical Burn Damage over ${duration} Second(s) | min: [9.0, 14.2, 17.5, 21.1, 24.9, 28.9, 33.1, 37.6, 42.4, 47.4, 52.8, 58.4] Electrical Burn Damage over min: 4.0 Second(s) |
| ${value}% Slower Movement over ${duration} Second(s) | min: [9.0, 14.2, 17.5, 21.1, 24.9, 28.9, 33.1, 37.6, 42.4, 47.4, 52.8, 58.4]% Slower Movement over min: 4.0 Second(s) |
| +${value}% Damage to Demon | 8 12 16 20 24 28 32 36 40 44 48 52 |
| MaxLevel | 8 |
| UltimateLevel | 12 |


The SJSU Career Center assists employers in gaining a Competitive Edge by identifying, attracting, and recruiting the top talent of Silicon Valley for a global marketplace. We provide an array of services that can save you money while hiring skilled employees.
SJSU Alumni Association You’re already a proud Spartan. Now it’s time to become a member of the SJSU Alumni Association. As a member, you'll have the opportunity to network with graduates throughout the country who represent the ideals of San José State. Strengthen the value of your degree, enjoy fantastic money-saving...


hi! I am a final year student of masters in computer science in specilization in Artificial intelligent in Central university of Rajasthan.
CEO - Founder & developers
Lorem ipsum dolor sit amet, mutat paulo simul per no, assum fastidii vituperata eam no.
Want to work with us or just want to say hello? Don't hesitate to get in touch!
View from google map


This year’s conference theme “Shaping the Energy of Tomorrow” focuses on globally emerging energy trends and markets while simultaneously providing insight on the interplay of different energy sectors. Discussion topics include: The feasibility of alternative energy sources in today’s economy, promising technologies impacting energy-related issues and the problems associated with government led and free market approaches to alternative energy sources.
The Queen’s Global Energy Conference is head on Feburary 26-28, 2016 at Queen’s University in Kingston Ontario.
n our inaugural year, QGEC plans on recruiting 80 of the best and brightest individuals, and providing them with the opportunity to gain unique insight into the diverse energy sectors. What sets QGEC apart from other conferences is our focus on energy integration, and how this will impact the future of the energy industry. It is our goal to work with partners to engage delegates and create a forum in which they can collaborate to shape the future of the energy industry.
Educate our delegates on globally emerging energy trends in the diverse energy sectors.
Facilitate the engagement of delegates with industry leaders to create a forum in which partners, speakers, and delegates can comfortably interact.
Motivate our delegates to become leaders in the future of global energy.
Contact him at:
<EMAIL_ADDRESS>Contact him at:
<EMAIL_ADDRESS>Why come to QGEC?
Energy is one of the largest sectors in today’s market having direct impacts on global economies. The Queen’s Global Energy Conference aims to provide students with the unique opportunity to learn about globally emerging trends and markets in today’s diverse energy sectors. In our inaugural year, QGEC will be hosting professionals from across the industry to discuss the future of global energy.
The motivation behind QGEC was to eliminate the niche and provide a floor where all sectors of the energy industry are equally represented. As a result, our delegates will be able to understand the interplay of all sectors of the energy industry and its connection with the future of energy as a whole.
QGEC primarily aims to educate delegates on trends and technologies in a variety of energy sectors, while providing insight into how they affect the energy industry as a whole. Our conference facilitates this through our speakers. These leaders at the forefront of their respective fields have the potential to inspire future leaders by engaging them in talks while providing a better understanding of what the future of energy entails. For more information about speaking opportunities, please contact our speaker’s team
If you want to get in touch, don't be afraid to email us.


User:
Hello
Bot:
Good morning! How can I help you?
User:
What's the weather today?
Bot:
Cloudy, +9°C in Geneva, Switzerland
is typing ...


Click on a Point to edit its parameters or create a new Point below. Points are values observed by the operator that are recorded
into this data collection system.
No "Points" available, click the button below to add one. Points are values observed by the operator that are recorded into
this data collection system.
Click on a "Group" to modify its "Data Points." Groups typically represent systems such as "Prefiltration" or "Wellfield."
Data points are the parameters that are measured by an operator.
No "Groups" available, click the button below to add one. Groups typically represent systems such as "Prefiltration" or "Wellfield."
Welcome to the data portal admin site. Here you can add locations where data colleciton is performed.
Click the button to create a new "location" for data collection. A location is, generally, a plant.
Click on one of the "locations" to view and edit its "groups". A location is, generally, a plant. The groups are its subsystems.


WITNESS LIST Ways & Means Committee April 20, 2006-10:00A HB 3 For: Allaway, Bill (Texas Taxpayers and Research Assn. (TTARA)) Hawkins, John (Texas Hospital Assn.) Kennedy, Dr. Paul (Texas Dental Association) Norman, Scott (Texas Association of Builders) Reagan, Karen (Texas Federation of Drug Stores) Stagner, Steve (Texas Council of Engineering Companies) Vasek, Heather (Texas Association for Home Care, Inc.) Zamrazil, Kristie (Texas Pharmacy Association) Against: Fuller, Hayes (Texas Association of Defense Counsel) On: Barton, Karey (Texas Tax Reform Commission) Courtney, Chuck (Texas Retailers Association) Fainter Jr., John W. (Association of Electric Companies of Texas, Inc.) Harvey, Jay (Self and Texas Trial Lawyers Assoc.) Kuntz, Steve (Law Firm Legislative Coalition) Lebas, James (Tx Tax Reform Commission) Palmer, David C. (International Council of Shopping Centers) Rosenbaum, Glen (Law Firm Legislative Coalition) HB 4 On: Craig, Mike (Tx Dept. of Transportation) O'Ryan, Anne (AAA Texas) HB 5 For: Curtis, Larriann (Texas PTA) Hornaday, Jon R. (American Cancer Society) Khera, Amit (American Heart Association) Lavine, Dick (Center for Public Policy Priorities) Tully, Ron (Council of Independant Tobacco Mfgs. of America (CITMA)) Against: Moore, Jason (Permian Basin Citizen Watchdogs) Orzecliowski Ph.D., William (R.J. Reynolds Tobacco Co.) Ross, Shon (Self) Venable, Peggy (Americans for Prosperity) On: Edwards, Kimberly (Texas Public Finance Authority) Porras, Judith M. (Texas Public Finance) Ratcliff, Joe (Texas Grocery & Convenience Association) HB 5 - Committee Substitute (Villarreal) Against: Curtis, Larriann (Texas PTA) Headrick, Carter (Self and Campaign for Tobacco Free Kids) Hornaday, Jon R. (American Cancer Society) Khera, Dr. Amit (TX - American Heart Association) Tully, Ron (Council of Independant Tobacco Mfgs. of America)


| I l@ve RuBoard |
The basics. Here's the solution we coded up for this exercise, along with some interactive tests. The __add__ overload has to appear only once, in the superclass. Notice that you get an error for expressions where a class instance appears on the right of a +; to fix this, use __radd__ methods also (an advanced topic we skipped; see other Python books and/or Python reference manuals for more details). You could also write the add method to take just two arguments, as shown in the chapter's examples.
% cat adder.py class Adder: def add(self, x, y): print 'not implemented!' def __init__(self, start=[]): self.data = start def __add__(self, other): return self.add(self.data, other) # or in subclasses--return type? class ListAdder(Adder): def add(self, x, y): return x + y class DictAdder(Adder): def add(self, x, y): new = {} for k in x.keys(): new[k] = x[k] for k in y.keys(): new[k] = y[k] return new % python >>> from adder import * >>> x = Adder() >>> x.add(1, 2) not implemented! >>> x = ListAdder() >>> x.add([1], [2]) [1, 2] >>> x = DictAdder() >>> x.add({1:1}, {2:2}) {1: 1, 2: 2} >>> x = Adder([1]) >>> x + [2] not implemented! >>> >>> x = ListAdder([1]) >>> x + [2] [1, 2] >>> [2] + x Traceback (innermost last): File "<stdin>", line 1, in ? TypeError: __add__ nor __radd__ defined for these operands
Operator overloading. Here's what we came up with for this one. It uses a few operator overload methods we didn't say much about, but they should be straightforward to understand. Copying the initial value in the constructor is important, because it may be mutable; you don't want to change or have a reference to an object that's possibly shared somewhere outside the class. The routes method __getattr__ calls to the wrapped list:
% cat mylist.py class MyList: def __init__(self, start): #self.wrapped = start[:] # copy start: no side effects self.wrapped = [] # make sure it's a list here for x in start: self.wrapped.append(x) def __add__(self, other): return MyList(self.wrapped + other) def __mul__(self, time): return MyList(self.wrapped * time) def __getitem__(self, offset): return self.wrapped[offset] def __len__(self): return len(self.wrapped) def __getslice__(self, low, high): return MyList(self.wrapped[low:high]) def append(self, node): self.wrapped.append(node) def __getattr__(self, name): # other members--sort/reverse/etc. return getattr(self.wrapped, name) def __repr__(self): return `self.wrapped` if __name__ == '__main__': x = MyList('spam') print x print x[2] print x[1:] print x + ['eggs'] print x * 3 x.append('a') x.sort() for c in x: print c, % python mylist.py ['s', 'p', 'a', 'm'] a ['p', 'a', 'm'] ['s', 'p', 'a', 'm', 'eggs'] ['s', 'p', 'a', 'm', 's', 'p', 'a', 'm', 's', 'p', 'a', 'm'] a a m p s
Subclassing. Our solution appears below. Your solution should appear similar.
% cat mysub.py from mylist import MyList class MyListSub(MyList): calls = 0 # shared by instances def __init__(self, start): self.adds = 0 # varies in each instance MyList.__init__(self, start) def __add__(self, other): MyListSub.calls = MyListSub.calls + 1 # class-wide counter self.adds = self.adds + 1 # per instance counts return MyList.__add__(self, other) def stats(self): return self.calls, self.adds # all adds, my adds if __name__ == '__main__': x = MyListSub('spam') y = MyListSub('foo') print x[2] print x[1:] print x + ['eggs'] print x + ['toast'] print y + ['bar'] print x.stats() % python mysub.py a ['p', 'a', 'm'] ['s', 'p', 'a', 'm', 'eggs'] ['s', 'p', 'a', 'm', 'toast'] ['f', 'o', 'o', 'bar'] (3, 2)
Metaclass methods. We worked through this exercise as follows. Notice that operators try to fetch attributes through _ _getattr__ too; you need to return a value to make them work.
>>> class Meta: ... def __getattr__(self, name): print 'get', name ... def __setattr__(self, name, value): print 'set', name, value ... >>> x = Meta() >>> x.append get append >>> x.spam = "pork" set spam pork >>> >>> x + 2 get __coerce__ Traceback (innermost last): File "<stdin>", line 1, in ? TypeError: call of non-function >>> >>> x[1] get __getitem__ Traceback (innermost last): File "<stdin>", line 1, in ? TypeError: call of non-function >>> x[1:5] get __len__ Traceback (innermost last): File "<stdin>", line 1, in ? TypeError: call of non-function
Set objects. Here's the sort of interaction you should get; comments explain which methods are called.
% python >>> from set import Set >>> x = Set([1,2,3,4]) # runs __init__ >>> y = Set([3,4,5]) >>> x & y # __and__, intersect, then __repr__ Set:[3, 4] >>> x | y # __or__, union, then __repr__ Set:[1, 2, 3, 4, 5] >>> z = Set("hello") # __init__ removes duplicates >>> z[0], z[-1] # __getitem__ ('h', 'o') >>> for c in z: print c, # __getitem__ ... h e l o >>> len(z), z # __len__, __repr__ (4, Set:['h', 'e', 'l', 'o']) >>> z & "mello", z | "mello" (Set:['e', 'l', 'o'], Set:['h', 'e', 'l', 'o', 'm'])
Our solution to the multiple-operand extension subclass looks like the class below. It needs only to replace two methods in the original set. The class's documentation string explains how it works:
from set import Set class MultiSet(Set): """ inherits all Set names, but extends intersect and union to support multiple operands; note that "self" is still the first argument (stored in the *args argument now); also note that the inherited & and | operators call the new methods here with 2 arguments, but processing more than 2 requires a method call, not an expression: """ def intersect(self, *others): res = [] for x in self: # scan first sequence for other in others: # for all other args if x not in other: break # item in each one? else: # no: break out of loop res.append(x) # yes: add item to end return Set(res) def union(*args): # self is args[0] res = [] for seq in args: # for all args for x in seq: # for all nodes if not x in res: res.append(x) # add new items to result return Set(res)
Assuming the new set is stored in a module called multiset.py, your interaction with the extension will be something along these lines; note that you can intersect by using & or calling intersect, but must call intersect for three or more operands; & is a binary (two-sided) operator:
>>> from multiset import * >>> x = MultiSet([1,2,3,4]) >>> y = MultiSet([3,4,5]) >>> z = MultiSet([0,1,2]) >>> x & y, x | y # 2 operands (Set:[3, 4], Set:[1, 2, 3, 4, 5]) >>> x.intersect(y, z) # 3 operands Set:[] >>> x.union(y, z) Set:[1, 2, 3, 4, 5, 0] >>> x.intersect([1,2,3], [2,3,4], [1,2,3]) # 4 operands Set:[2, 3] >>> x.union(range(10)) # non-MultiSets work too Set:[1, 2, 3, 4, 0, 5, 6, 7, 8, 9]
Class tree links. Here's the way we extended the Lister class and a rerun of the test to show its format. To display class attributes too, you'd need to do something like what the attrnames method currently does, but recursively, at each class reached by climbing __bases__ links.
class Lister: def __repr__(self): return ("<Instance of %s(%s), address %s:\n%s>" % (self.__class__.__name__, # my class's name self.supers(), # my class's supers id(self), # my address self.attrnames()) ) # name=value list def attrnames(self):Unchanged...
def supers(self): result = "" first = 1 for super in self.__class__.__bases__: # one level up from class if not first: result = result + ", " first = 0 result = result + super.__name__ return result C:\python\examples> python testmixin.py <Instance of Sub(Super, Lister), address 7841200: name data3=42 name data2=eggs name data1=spam >
Composition. Our solution is below, with comments from the description mixed in with the code. This is one case where it's probably easier to express a problem in Python than it is in English:
class Lunch: def __init__(self): # make/embed Customer and Employee self.cust = Customer() self.empl = Employee() def order(self, foodName): # start a Customer order simulation self.cust.placeOrder(foodName, self.empl) def result(self): # ask the Customer what kind of Food it has self.cust.printFood() class Customer: def __init__(self): # initialize my food to None self.food = None def placeOrder(self, foodName, employee): # place order with an Employee self.food = employee.takeOrder(foodName) def printFood(self): # print the name of my food print self.food.name class Employee: def takeOrder(self, foodName): # return a Food, with requested name return Food(foodName) class Food: def __init__(self, name): # store food name self.name = name if __name__ == '__main__': x = Lunch() x.order('burritos') x.result() x.order('pizza') x.result() % python lunch.py burritos pizza
| I l@ve RuBoard |


| Interface | Description |
|---|---|
| ExtraParameterProvider | |
| Param.UnnamedParametric |
Tags can implement this to support nested param tags without the name attribute.
|
| UrlProvider |
Implemntations of this interface can be used to build a URL
|
| UrlRenderer |
Implementations of this interface are responsible for rendering/creating URLs for a specific
environment (e.g.
|
| Class | Description |
|---|---|
| ActionComponent | |
| ActionError |
Render action errors if they exists the specific layout of the rendering depends on
the theme itself.
|
| ActionMessage |
Render action messages if they exists, specific rendering layout depends on the
theme itself.
|
| Anchor | |
| AppendIterator | |
| Bean | |
| Checkbox |
Renders an HTML input element of type checkbox, populated by the specified property from the ValueStack.
|
| CheckboxList |
Creates a series of checkboxes from a list.
|
| ClosingUIBean |
ClosingUIBean is the standard superclass for UI components such as div etc.
|
| ComboBox | |
| Component |
Base class to extend for UI components.
|
| ComponentUrlProvider |
Default implementation of UrlProvider
|
| ContextBean |
Base class for control and data tags
|
| Date |
Format Date object in different ways.
|
| DateTextField | |
| Debug | |
| DoubleListUIBean |
DoubleListUIBean is the standard superclass of all Struts double list handling components.
|
| DoubleSelect |
Renders two HTML select elements with second one changing displayed values depending on selected entry of first one.
|
| Else | |
| ElseIf | |
| FieldError |
Render field errors if they exists.
|
| File |
Renders an HTML file input element.
|
| Form | |
| Form.FieldVisitorValidatorWrapper |
Wrap field validator, add visitor's field prefix to the field name.
|
| FormButton |
FormButton.
|
| GenericUIBean | |
| Head |
Renders parts of the HEAD section for an HTML file.
|
| Hidden |
Renders an HTML input element of type hidden, populated by the specified property from the ValueStack.
|
| I18n |
Gets a resource bundle and place it on the value stack.
|
| If | |
| Include | |
| InputTransferSelect | |
| IteratorComponent | |
| Label | |
| ListUIBean |
DoubleListUIBean is the standard superclass of all Struts list handling components.
|
| MergeIterator | |
| Number | |
| OptGroup | |
| OptionTransferSelect | |
| Param | |
| Password | |
| Property | |
| Push | |
| Radio | |
| Reset |
Render a reset button.
|
| Select |
Render an HTML input tag of type select.
|
| ServletUrlRenderer |
Implementation of the
UrlRenderer interface that creates URLs suitable in a servlet environment. |
| Set | |
| Submit |
Render a submit button.
|
| Text | |
| TextArea | |
| TextField | |
| Token | |
| UIBean |
UIBean is the standard superclass of all Struts UI components.
|
| UpDownSelect | |
| URL |
Copyright © 2000–2020 Apache Software Foundation. All rights reserved.


Welcome to the NHK
Twenty-two-year-old college dropout Tatsuhiro Satou has been a hikikomori for almost four years now. In his isolation, he has come to believe in many obscure conspiracy theories, but there is one in particular which he holds unshakable faith in: the theory that the evil conspirator behind his shut-in NEET (Not in Employment, Education or Training) status is the Nihon Hikikomori Kyokai (NHK) - an evil and secret organization dedicated to fostering the spread of hikikomori culture.
More info
| Media | Item | Details | Availability | |
|---|---|---|---|---|
| DVD | Welcome to the NHK - Disc 1 | Region 2 | Available | request |
| Manga | Welcome to the NHK - Vol 1 | Requested | request | |
| Manga | Welcome to the NHK - Vol 3 | On Loan | request |


import numpy as np
= 0
start = []
y = 1000
n
for i in range(n):
= np.random.choice([-1, 1], p = [0.5, 0.5])
step += step
start y.append(start)
Introduction to Markov Chains
Everything that will happen in the future only depends on what is happening right now.
A Markov chain is a random process with the Markov property. A random process or often called stochastic property is a mathematical object defined as a collection of random variables. A Markov chain has either discrete state space (set of possible values of the random variables) or discrete index set (often representing time) - given the fact, many variations for a Markov chain exists. Usually the term “Markov chain” is reserved for a process with a discrete set of times, that is a Discrete Time Markov chain (DTMC).
To develop better intuition about Markov chain, the simpler version of it is to model a basic random walk.
Random Walk 101
The situation: we will walk 1000 steps. At each step, we flip a fair coin and move 1 step to the right if it is head and one step to the left if it is tail.
From scratch
import matplotlib.pyplot as plt
plt.plot(y)#plt.xlabel('Number of steps')
#plt.ylabel(r'$S_{n}$')
Using financial data
Python code coming from this post
A Monte-Carlo simulation of a random-walk of a stock price does assume that the returns follow a normal distribution. A second assumption is that the past volatility of returns will continue (or be very similar) in the future. This is of course not totally the case.
Getting data and feature engineeriing
Getting data using the yfinance package.
import yfinance as yf
import pandas as pd
= yf.download("SBUX", start = "2005-01-01")
yo
"../../raw_data/sbux.csv")
yo.to_csv(
'Adj Close'][-1]
yo[
yo.info()
We have imported SBUX stock price data and stored them on disk. We’ll retrieve it using pandas and construct our returns and volatility variables.
import pandas as pd
= pd.read_csv("../../raw_data/sbux.csv")
sbux
sbux.tail()
# get the daily returns and then filter on the last 2 years of trading.
# calculate volatiliy on these last years (not realistic of course)
= sbux['adjClose'].pct_change()
daily_returns #sbux = sbux.tail(505)
= daily_returns.std()
daily_volat
print(daily_volat)
sbux.info()
0.01987813423096984
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5607 entries, 0 to 5606
Data columns (total 13 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 date 5607 non-null object
1 open 5607 non-null float64
2 high 5607 non-null float64
3 low 5607 non-null float64
4 close 5607 non-null float64
5 adjClose 5607 non-null float64
6 volume 5607 non-null int64
7 unadjustedVolume 5607 non-null int64
8 change 5607 non-null float64
9 changePercent 5607 non-null float64
10 vwap 5607 non-null float64
11 label 5607 non-null object
12 changeOverTime 5607 non-null float64
dtypes: float64(9), int64(2), object(2)
memory usage: 569.6+ KB
Single simulation
import numpy as np
import matplotlib.pyplot as plt
= 252
look_back = 0
count = []
price_list = sbux['adjClose'].iloc[-1]
last_price
= last_price * (1 + np.random.normal(0, daily_volat))
price
price_list.append(price)
for y in range(look_back):
if count == 251:
break
= price_list[count] * (1 + np.random.normal(0, daily_volat))
price
price_list.append(price) += 1
count
plt.plot(price_list)
plt.show()
#price_list
An here would be another single simulation. It will of course look vastly different although it is build from the same normal distribution with same mean \(\mu = 0\) and sd \(\sigma = 0\).
Now we can re-use that code if we want to create 100’s of these simulations.
= 100
num_of_simulations = 252
model_ahead
= pd.DataFrame()
df = []
last_price_list
for x in range(num_of_simulations):
= 0
count = []
price_list = sbux.iloc[-1]['adjClose']
last_price = last_price * (1 + np.random.normal(0, daily_volat))
price
price_list.append(price)
for y in range(model_ahead):
if count == 251:
break
= price_list[count] * (1 + np.random.normal(0, daily_volat))
price
price_list.append(price) += 1
count
= price_list
df[x] -1])
last_price_list.append(price_list[
= plt.figure()
fig "Monte Carlo Simulation for SBUX")
fig.suptitle(
plt.plot(df)"Day")
plt.xlabel("Price")
plt.ylabel( plt.show()
With just 10 simulated random-walks on SBUX given the last 17 years of volatility, we can see that price could range between $40 to around $140 dollars over the next 252 trading days (one year).
Analysis of our MC simulation
print("Expected price: ", round(np.mean(last_price_list), 2))
print("Quantile (5%): ", np.percentile(last_price_list, 5))
print("Quantile (95%): ", np.percentile(last_price_list, 95))
Expected price: 4.34
Quantile (5%): 2.5041396468319865
Quantile (95%): 6.967420228662214
=10)
plt.hist(last_price_list, bins plt.show()
Markov Chain 101
The main concept to deal with in a markov chain is a transition matrix.
Transition Matrix
In a transition matrix, the rows are you starting state and columns are your end of state.
So with the below matrix, the probability to go from state A to state A is 0.8 and probability to go from state A to state D is 0.2. In this sense, all the rows of a transition matrix should always add up to 1.
= [0.1, 0.4, 0.3, 0.2, 0]
state_A = [0.0, 0.5, 0.5, 0.0, 0]
state_B = [0.0, 0.0, 1.0, 0.0, 0]
state_C = [0.0, 0.0, 0.0, 0.0, 1.0]
state_D = [0.0, 0.0, 0.0, 0.5, 0.5]
state_E
= [state_A, state_B, state_C, state_D, state_E] transition_matrix
We could also create a function to check if a transition matrix is indeed a properly formatted transition matrix to model a markov chain.
def check_markov_chain(m):
for i in range(0, len(m)):
if sum(m[i]) != 1:
return False
print("This is a correctly formatted transition matrix")
check_markov_chain(transition_matrix)
This is a correctly formatted transition matrix


Design solutions with a human touch including print design, branding, signage, logo design, and web work. Masterful typography and impactful layouts.
Illustrative and concept work dealing in the imaginative and entirely fantastical. Highly informed character and creature design, nostalgia-inducing children's book illustrations, and rendering ranging from realistic to specifically stylized.
Ability to create work in pencil, pastel, chalk, charcoal, marker, ink, watercolor, acrylic, oil, Photoshop, Illustrator, and InDesign. Stylistic tendencies include: impressionistic, detail-oriented, warm, clean, and nostalgic.
Haley has pursued a rigorous and technical artistic education her entire life, from specialized courses to an accredited private art school education. During her college career at Rocky Mountain College of Art + Design, she pursued graphic design, art education, and illustration majors; graduating with a BFA in Illustration.
Haley's freelance design and illustration clients range from personal commissions to authors to PR firms to major retail malls.


Hello Guest! YOU'RE ALMOST DONE NOW COMPLETE THIS QUICK SURVEY
LIFE INSURANCE
There are some things we think about but never do. Turn your thoughts into action with Insuranceline’s Life Insurance cover PLUS receive a Bonus $100 gift card^ after your first 2 months’ cover. Click here to find out more.
Qualifying criteria apply. Insurance issued by TAL Life Limited. Please read the PDS.


lorem ipsum dolor sitameet mamet tamet wawkakwa tamet malsaosnma ajwknsalndlksna alsndlknqwilan lknlanslkdn aksdnlkandlnaslkdn anlk nslkadnlkanlkns sndalksndlkasn constectur polodong talamat padasak lololo woqwq lamet tamet sitamet mawalam tollllleemad dadadak lamkadah
Bootstrap is a powerful front-end framework for faster and easier web development. The Bootstrap tutorial section will help you learn the techniques of Bootstrap so that you can create web your own website with much less efforts.
The references section outlines all the standard HTML tags and CSS properties along with other useful references such as color names and values, symbols and character entities, web safe fonts, language codes, HTTP messages and much more.


Import XML
Click the buttons below to import XML.
Import Companies
Import Projects
Import Employees
All XML files are imported.


{% if missingOverview or missingSummary or missingAuthor %}
Help needed for "{{page.title}}"!
The following is missing from this set of questions:
{% if missingOverview %}Overview: e.g.
overview example...{% endif %} {% if missingAuthor %}
Author: The name of the author, team e.g.
author: John Doe{% endif %} {% if missingSummary %}
Summary: Couple of good words here
- question: What is your favourite sport? summary: Cuz I wanna know...{% endif %} {% if page.link %}
Tip! You may be able to find the missing information at the source URL:
{{page.link}}.
It would be great if you could contribute and update this set of questions with anything that's missing!


Dont see an option to remap this page? Then chances are this page is not directly supported or does not contain SRG names.
Even if this page is not supported it may still be possible to remap if it is a plane text page. Press the button bellow to try.


Basically, just run make
. This will start config.sh
if there is no file config.h yet and then proceed compiling the
sources. After successful compilation you should find the executable
gwc
in the src
directory.
The shell script config.sh creates the header file config.h and is automagically executed when running make. Set CC, CPP, CFLAGS, LDFLAGS (or any other variables you use for your compiler) before running config.sh so that it uses the same settings as the make afterwards. Otherwise, config.sh might create a wrong config.h. If you want to use -Werror with GCC, make sure you don't use it while running config.sh or this step will fail. If you use TenDRA you should add -Xa to CFLAGS and you might have to add -I/usr/include as well.
Run ./config --help for a list of available options. The defaults should be fine however.
If config.sh fails or doesn't work properly, see config_test.log. It contains all test programs followed compiler output i.e., warnings, errors, other diagnostic messages. If you want to change the compiler or compiler settings run make clobber remove all object files and config.h. After that run make again to re-create config.h and to re-build gwc.
If you get any compiler warnings, make sure they are harmless. If you don't know what they mean or assume they indicate a bug, inform the author(s), please.
If your operating system has support for kqueue(), gwc will use this instead of poll() to gain better performance. On systems with a Linux kernel 2.6 or newer, there's typically support for epoll which has the same benefits and will be used if detected. You can override this by invoking config.sh with --use-poll as argument.
There's no hardcoded location for the configuration file. You have to pass the absolute pathname to gwc using the command-line option -f:
gwc -f /path/to/crab.conf
The process runs as a daemon i.e., detaches from the terminal (if started from one) and redirects stderr to the file as configured by log_main for the main process and to log_dns for the co-process which resolves hostnames. Checks of other GWebCaches are logged to log_checks. These files are opened in append-mode so they will continuously grow until you move them aside (see below for SIGHUP).
The cached peers and URLs are read from and saved to peer_cache and good_url_cache respectively bad_url_cache approx. every 15 minutes. They're not saved on exit mainly to prevent clearing the cache accidently. The configured pathnames for these must point to regular files, not symbolic links or anything else.
Any client with the client ID GNUT is immediately disconnected without handling the request at all as this is a worm and not the deprecated Gnutella servent Gnut.
Updates from peers with a port below 1024 are discarded. There don't seem to be any of those but an attacker might use this to keep a server on the same machine or the proxy busy.
The net request is supported regardless of the support_v2 setting. However gwc doesn't support serving more than one network at a time. Use the setting network_id if you want to use gwc for something else than the Gnutella network. If you don't use the network_id setting, any requests with a net parameter other than gnutella will be rejected appropriately. The value is treated case-insensitive.
Anonymous requests are disallowed by default.
Peers are currently reported for a maximum of 20 minutes. GWCs are validated every 5 hours. This can be adjusted at compile-time in gwc.h. Keep in mind that you should also adjust the cache sizes then. Failing caches are checked again after this time: 1 hour * 2^(number of checks). This way they will be contacted less and less if they keep on failing but have enough time to recover without getting flooded.
HTTP redirects are not followed not just because it makes things simpler but also because all redirects either lead to 404 sites or already cached URLs currently and there's little hope this will ever change. Also, since this could cause unwanted dupes and/or traffic for uninvolved third parties, following redirects is assumed to be a security issue for Gnutella Web Caches.
See example.conf for more information. It contains all possible configuration options along with descriptive comments.
GhostWhiteCrab is at first a GWebCache. The option support_gwc is therefore true by default. There are several options that are only effective when the GWebCache support is active:
This is the most important option and configure the URL that will be used by your GWebCache. A valid example looks like this:
location http://myhost.example.org:8000/path
In this example, myhost.example.org has to be the hostname of your server. You cannot use a mere IP address instead of fully qualified domain name. If you do not specify a port number (like 8000 in the example), the default HTTP port 80 is used. The path is not very important but restricted to a combination of lowercase letters, digits, slashs, dots, underscores, hyphens and tildes. A simple slash as path is fine. GhostWhiteCrab will not accept any variants (missing trailing slash, URL-encoded characters etc.) of this URL in requests. This is a very important feature and not a bug.
This option must be set to the absolute path of the file that will be used to store URLs of GWebCaches that failed. This file must be read- and writable by the GWebCache user.
This option must be set to the absolute path of the file that will be used to store URLs of GWebCaches that are currently known to be working. This file must be read- and writable by the GWebCache user.
The following are not necessary in most cases and a therefore optional. They might be required under certain circumstances or useful for fine-tuning:
By default newly submitted URLs are not added to list of known good URLs but only added the list of bad URLs with a magic number of retries (32). Thus, only URLs that are already known as bad or good are managed that is verified and propagated. This a precaution against hostile parties that might try to inject bad URLs. If you consider this paranoid or too much work to add new URLs manually, set this option to true.
This is disabled by default. If enabled, the HTTP Server header contains the name of the operating system as a comment. The name is retrieved with the uname() system call and is normally the same what uname -s shows.
By default one TCP listening socket is bound to the IPv4 address "<IP_ADDRESS>". To restrict access to a certain network interface, you can set this the appropriate IPv4 address. If "listen6_address" is specified, there is no TCP socket created for IPv4 by default.
By default one TCP listening socket is bound to the IPv6 address "::". To restrict access to a certain network interface, you can set this the appropriate IPv6 address.
By default the port is derived from the option location. However, if you use port forwarding (NAT) e.g., to forward incoming traffic from TCP port 80 to port 8080, use this in your configuration file:
listen_port 8080
so that gwc listens on the correct port.
Use this to adjust the backlog parameter passed to the listen() system call. The default is 32. If the frequency of incoming connections is very high, it is recommended to increase the value. Most operating systems limit this to 128 or 256 internally.
The amount of peer addresses to return for a hostfile request. The default is 20.
The amount of URLs to return for an urlfile request. The default is 10. This should not be lower than 4. Otherwise, other GhostWhiteCrab GWebCaches will always consider your URL a bad one.
The number of bytes accepted for a HTTP request including headers. The default is 1024. Some web browsers might send a huge amount of header data. If you experience problems when connecting with your browser, try to increase the size.
This is set to 40 by default which means that GhostWhiteCrab attempts to return 40 percent of peers of the same vendor for a hostfile request. It is meant to help clients that are not well represented finding peers of their own kind to reduce connection problems in case of compatibility issues.
The default is false. If enabled, HTTP responses contain a X-Remote-IP header which tells the client its IP addresses. This can help clients behind NAT routers to quickly discover their own IP address.
The default is false. If enabled, Crab sends adds a "From:" header to HTTP requests when verifying submitted URLs. The content of this header is either the URL of your GWebCache or - if configured - your contact address. Crab does not aggressively crawl URLs but some webmasters might still feel abused. Thus, it's polite to provide contact information. Of course, it might just be an invitation for spam. However, the contact information is free form, it can be an URL, email address, phone number etc.
This is used together with send_from_header. The value should be a short text without control characters. The value can also be used in templates under the identifier &crab.contact.address;.
The number of seconds after which an idle connection is closed. The default is 10 seconds but that might be too long on heavily loaded servers. Some clients simply do not properly disconnect or keep the connection open for no reason. This will waste a lot of resources and the process might also run out of file descriptors. However, be careful because a too low value might make it impossible for clients with slow connections to get a response. GhostWhite does not support persistent HTTP connections as this makes little sense for a GWebCache and will always close the connection after all data has been transmitted.
This is similar to idle_timeout but sets the absolute number of seconds after which a connection is closed even if there is pending data. The default is 15 seconds.
This option can be used to limit the number of accept() calls per second that is incoming connections. The default is zero which means no limit is applied. Setting this to an acceptable value limits the amount of outbound traffic that can be generated. However if you set it too low, the server becomes seemingly dead.
This option is normally only available on Linux systems. It is ignored on supports which do not support it. The default value is 20 seconds. In case of problems, it can be disabled by setting the value to zero. See the man page tcp(7) for more information. In a nutshell, a low value heavily reduces the amount sockets in use because there are often many incoming connections which just timeout or hangup without sending any request. On FreeBSD crab will use an HTTP accept() filter which has basically the same effect but requires no configuration.
The default is 600 (10 minutes). This the is the time in seconds an IP address stays locked after making 3 requests. You should not set this higher than 3600 (1 hour) for the Gnutella network. You can save a lot of memory by reducing this value but it will allow higher request rates and may cause more traffic. Setting it to zero will disable locking altogether.
In order to prevent a flood of connection attempts (possibly to the same host each time) for URL verifications, each attempt is delayed by a number of seconds. The default value is 30.
This limits the number of bytes read from a server when verifying an URL. The default value is 4096.
The default is true which means a GWebCache must report a minimum amount of known good URLs and not too many known bad URLs to be considered good itself. If set to false, a single valid URL is sufficient to mark an URL as good.
If set to false, duplicate URLs returned by other GWebCaches are tolerated. By default returning a duplicate (usually several variants of the same URL) will cause the URL to be marked as bad. If GWebCaches return duplicates (especially variants), much more clients might connect to the given URL and more frequently which might overload that server.
The next options are mostly for testing or debugging purposes.
If you set this option to true, clients do not need to send a client ID in requests. While this might be useful for testing purposes, it's usually badly written software or a hostile party that does not send such an ID. Such requests are logged to the log_alert file.
GhostWhiteCrab is usally run as a daemon i.e., it forks itself and disconnects from the terminal etc. If this not desired, this option can be set to true to disable that behaviour.
Normally, a process that crashes is terminated. If this option is set to true, the process will be stopped in case of a fatal signal so that you can attach a debugger to it.
When set to true, all HTTP headers of requests are dumped to log_main.
GhostWhiteCrab can also serve as an UDP Hostcache (UHC) for the Gnutella network. You can enable UHC support and GWebCache support or both. By default GWebCache mode is enabled and UHC mode is disabled. In a nutshell, an UDP Hostcache works like a GWebCache but uses UDP instead of TCP and Gnutella packets instead of HTTP. Therefore, it uses less resources and traffic. Further, an UHC doesn't depend on updates by Gnutella peers but discovers peers itself. See the specifications for details.
If you want to run an UDP Hostcache only but no GWebCache, set support_gwc to false.
UHC support is enabled by setting support_uhc to true. If you enable the UDP Hostcache support, the option network_id must be either be unset or set to gnutella as this a very Gnutella-specific feature. The default is false.
This option configures the port to be used for the UDP socket. Make sure this UDP port is not blocked by your firewall. The port must be above 1023 whether the process runs as root or not.
If you have assigned a public DNS name to the server, you can use this option to tell the clients about it. This allows clients to contact your UHC by hostname instead of an IP address.
By default one UDP socket will be bound to the IPv4 address "<IP_ADDRESS>" If you want to restrict access to a specific network interface, set this to the appropriate IP address. If "uhc_bind6_address" is specified, no UDP socket is bound to "<IP_ADDRESS>" by default.
By default one UDP socket will be bound to the IPv6 address "::". If you want to restrict access to a specific network interface, set this to the appropriate IPv6 address. If this option is used, you might have to set "uhc_bind_address" explicitely to an IPv4 address too.
This limits the number of peer addresses that will put into a pong. The current default is 50. Keep in mind that UDP packets larger than about 500 bytes can be problematic due to fragmentation. A single peer address is 6 bytes large.
This limits the overall amount of pongs that will be sent per second. The current default is 10.
The amount of time in milliseconds to wait for a pong reply. Replies to pings after this timeout are discarded. The current default is 60000, thus one minute.
The default is 600 (10 minutes). This the is the time in seconds an IP address stays locked after making 3 requests. You can save memory by reducing this value but it will allow higher request rates and may cause more traffic. Setting it to zero will disable locking altogether.
If set to a non-zero value, the UDP socket will initialized using the socket option SO_RCVBUF. Normally, the default value used by your operating system should be fine. However if you experience packet loss due to overflowing buffers, increasing this value might help. See also the manpage for setsockopt(2).
If set to a non-zero value, the UDP socket will initialized using the socket option SO_SNDBUF. This is the analogous of uhc_rcvbuf_size but for the send buffer.
Make sure that the gwc user cannot write to the configuration file, the templates, the address filter or the directories in which these files are. gwc should only be able to write to the log and the cache files. This is easy to accomplish and absolutely necessary for basic security.
The support for running under a different UID, GID and chroot()ing to a specified directory is present but not thoroughly tested yet. Use it with extreme caution! It should be unnecessary in almost all cases. You can use a packet filter, a firewall or a NAT server to redirect ports below 1024 to another port above so that you don't have to run gwc with privileges. See also listen_port.
Never ever set the set-UID-bit for gwc! A set-GID-bit shouldn't be necessary in any case either but Crab won't complain about this.
It's recommended to compile gwc statically to make it immune against potential exploits based on dynamic linking i.e., certain race-conditions, LD_LIBRARY_PATH etc. However, depending on your operating system static linking might not be possible.
Use a dedicated account - if possible on its own partition - for gwc, I mean that!
Use a compiler which checks the stack for overflows e.g., GCC-SSP which is a modified GCC with stack-smashing protection.
Use systrace, a FreeBSD jail or other security tools to limit the potential damage which could be caused by bugs. Such tools are also useful to discover bugs which would not be found easily otherwise.
There are currently two templates supported base_template and data_template. The first is used for requests to the GWC URI without any query. The second is used for data requests i.e., if the query contains ?data=key whereas key must match the configured data_key. If data_key is not set (this is the default), the key is ignored and all data requests are served. The template format is simple:
<HTTP response code and message> <HTTP headers> <One empty line> <The document data>
The HTTP response line and the HTTP headers are optional. You probably want to use a Content-Type header if the document data is anything else but text/plain with ASCII or ISO-8859-1 encoded data. See the examples directory for ready-to-use templates. Note that you can use templates to redirect requests to a different URL:
HTTP/1.1 302 Moved Temporarily Location: http://www.example.org/some/path/
The response line and the headers are not strictly checked. Thus, it's your responsibility to use valid HTTP. However, you don't have to terminate lines with CRLF, that is \r or \n are sufficient as well. Spaces will be stripped from the end of the line and a single \n will be replaced by \r\n when sending the response.
There are several private entities which will be replaced by the actual
contents when sending the document. For example, you can add the version
information to your document by using the entity
&crab.user.agent;. This is mostly interesting for customizing
a status page. Whether you use XHTML, HTML, plain text or anything else
is completely up to. Just make sure you use an appropriate Content-Type
header. The default is text/plain. See src/template.c
or the examples for available entities. Their names should be more or
less self-explanatory. You can also rather easily extend the set of
entities (if you have moderate C skills that is). Templates are only read
at startup time. The entities are replaced with their actual values at
run-time so that the processing overhead is pretty low.
Note: You cannot use entities in the HTTP headers or the response line as of yet. Consider using a HTTP redirect to a different web host with static data in the base_template if you simply want a fancy web site for human visitors.
It is highly recommended to create a dedicated user crab along with a dedicated group crab. The user crab should be the only user in this group and it shouldn't be in any other group.
The following shows a typical simple setup. At first, the directory/file structure:
mode owner group pathname drwxr-x--T root crab /home/crab drwxrwx--T root crab /home/crab/cores -rw-r----- root crab /home/crab/crab.conf -rw-r----- root crab /home/crab/hostiles.txt drwxrwx--T root crab /home/crab/db -rw-r----- crab crab /home/crab/db/peer_cache -rw-r----- crab crab /home/crab/db/urls.bad -rw-r----- crab crab /home/crab/db/urls.good drwxrwx--T root crab /home/crab/log -rw-r----- crab crab /home/crab/log/access.log -rw-r----- crab crab /home/crab/log/alert.log -rw-r----- crab crab /home/crab/log/checks.log -rw-r----- crab crab /home/crab/log/dns.log -rw-r----- crab crab /home/crab/log/main.log -rw-r----- crab crab /home/crab/log/uhc.log drwxr-x--T root crab /home/crab/templates -r--r----- root crab /home/crab/templates/base.template -r--r----- root crab /home/crab/templates/data.template
In the example above, you can actually replace root by any other user.
However it is best if Crab cannot modify any files or directories
that it is not supposed to. This is simply a defense in the unfortunate case
of exploitable bugs. The sticky bit is set just in case
as
well. Note that the user crab has no write-access to its home
directory.
To use the above layout, the corresponding options would be configured as shown here:
log_access /home/crab/log/access.log log_alert /home/crab/log/alert.log log_checks /home/crab/log/checks.log log_dns /home/crab/log/dns.log log_main /home/crab/log/main.log log_uhc /home/crab/log/uhc.log
If you are not interested in some of those logs, you can configure /dev/null as so that no diskspace gets wasted e.g.:
log_dns /dev/null
Note that all paths must point to different files. You may set all of them to /dev/null, though. The log files will be generated by crab if they don't exist. Thus you don't need to generate them in advance.
If you have to start Crab as root - because you want to use the chroot feature or because you want to use a privileged port (below 1024), you must also add these options:
user crab group crab
Otherwise, Crab will terminate after reading the options file. The option group is actually redundant if crab is the primary and only group the user crab belongs to.
Further you probably want to set up a cron job to start Crab automagically after a reboot and hourly just in case it crashed. The TCP port should be sufficient as lock to prevent running multiple Crabs accidently. As the log files grow indefinitely and can become quite large, you should also add a cron job to remove or archive them on a daily basis. Crab will reopen all log files if it receives a SIGHUP signal. Use this after (re)moving the log files.
The templates are not necessary but they are useful to get some information about the current status of Crab or to provide an informational page possibly with contact information so that people can notify you if they notice any problems with your cache. The example templates are ready to use. Just copy them to their target location and configure the proper paths:
data_template ~crab/db/data.template base_template ~crab/db/base.template
If you want to prevent that everyone can see the status information using the data request, you can set data_key to a secret string. Keep in mind that all transmissions are unencrypted, so that everyone who can read the traffic will see this key.
Causes gwc to re-open the log file assigned to stderr. This allows to archive the log files without having to restart gwc. For example, like this:
stamp="`date '+%Y-%m-%d'`" mv access.log "access-${stamp}.log" mv gwc.log "gwc-${stamp}.log" mv dns.log "dns-${stamp}.log" mv checks.log "checks-${stamp}.log" pkill -HUP -u gwc gwc
The signal will not cause the process to re-read any configuration or status files.
The following files are used by gwc. There are no default paths or filenames, they have to be configured. If you really don't care about any of those, set the path to /dev/null or comment them out if they're optional.
First, the log files. Note that only log_alert and log_access contain backslash escaped data. Other log files may contain any possible characters and you should be careful to not send the contents straight to your terminal but use a filter such as less or more.
Log messages from the main process. These are mostly debug messages. The default is "/dev/null".
HTTP access log messages using the Combined Log Format
of Apache. A typical message looks
like this:
<IP_ADDRESS> - - [11/Nov/2004:18:04:20 +0000] "GET / HTTP/1.1" 200 304 - "Mozilla/5.0"
The first token is the address of the remote client. The next two tokens are never set exist only for compatibility reasons. Third comes the timestamp in brackets; the timestamp is hardcoded to GMT. The fifth token is the received request, followed by the returned HTTP response code, the content-length of the reply, the HTTP Referer header. The last token is the HTTP User-Agent header as sent by the client. The data sent by the client is backslash escaped if necessary and truncated if the internal limit is exceeded.
Logs diverse alert messages. Each message consists of four
tokens. The IP address of the remote client, a timestamp, the client
agent ID and the alert message. The alert message is either Hostile
,
Scan
, Trash
or the GWebCache error message that was returned
to the client. Hostile
refers to connection attempts from address
that are listed in address_filter. Scan
means the remote
client connected but send no data at all - not a single byte. Trash
means the client send something but it did not look like a HTTP request.
The GWebCache error messages are hopefully self-explanatory. If
any of the tokens is empty, a single dash will be printed instead. The
client agent ID is backslash escaped if it contains any strange
characters.
Log messages regarding checks of other GWebCaches. This is merely a raw dump of connection attempts to GWebCaches and their replies.
Log messages from the hostname resolver. The default is "/dev/null".
This is similar to log_access but concerns the UDP hostcache. Incoming UDP PING and PONG messages are logged here. This log file can get pretty huge very quickly.
Second, the files that represent the database
to preserve the state
between restarts.
All currently known peers; re-written every ~15 min.
All currently known bad URLs; re-written every ~15 min.
All currently known good URLs; re-written every ~15 min.
Last, the files that contains static data and are only read but never written by Crab:
The template which is used for GWC URI when no query was provided. If this option isn't used an empty document is returned.
The template to be used for data requests. If this option isn't used an empty document is returned.
A file with IPv4 addresses respectively address ranges to block. Valid formats are:
address # single host e.g., <IP_ADDRESS> address/netmask # a network e.g., <IP_ADDRESS>/<IP_ADDRESS> address/prefixlen # a network CIDR notation e.g., <IP_ADDRESS>/24
It's usually better to block access from/to addresses with a firewall or packet filter. In any case, it is highly recommended to block hostile parties. If all GWebCaches block them, they'll have a hard time to get connected. Further hostile parties often aggressively poll GWebCaches for fresh addresses and therefore seem to be ubiquitous.
There are not supposed to be any dependencies except a more or less POSIX-compliant operating system with installed system C header files, a C compiler and the make tool. config.sh requires a Bourne Shell but the contents of config.h can be configured by hand if absolutely necessary. config.sh utilizes a few standard Unix tools e.g., sed. Future versions might require zlib. The UHC support does not strictly require zlib but it's recommended to save some traffic and for full compatibility.
External resources:
Author: Christian Biere <christianbiere at gmx dot de>
Last-Edited: 2007-01-14


Take control of snake and eat so many viruses as you can. Fight with viruses and do not let them take control! The snake will help you
The game will end if your snake collides with herself or with gameboard boundaries.
You will earn +1 point for every eaten virus.
There are 3 levels implemented. 1 level is the slowest and 3 level is the fastest. Choose Level 2 for balanced speed


?HERBARIO
DEL
COLEGIO
de
Lft
SALLE
ntn
vedado-habana
Familia------------------------------
Nœm.-19^9À
n.Viotoi—n
-
Bros.
Le¡nÈ
"
F
Col
01447608
Ludwigia
sect.
Oligospermum
ã
.
i.
T.P.
Ramamoorthy
1980
Determined
by
Missouri
Botanical
Garden
Ludwigia
peploides
(HBK.)
Raven
subsp.
peploides
Determined
by
Elsa
Zardini
1986
Missouri
Botanical
Garden
01447608
Legend -
Level of confidence that token is an accurately-transcribed word
extremely low
very low
low
undetermined
medium
high
very high


[Name]
[Channel Partner Name]
Phone:
[Phone Number]
Email:
[Email-ID]
Copy Source Code
[Name]
[Channel Partner Name]
Phone:
[Phone Number]
Email:
[Email-ID]
Close
Copy


5. Background of MPI-1.1, MPI-1.2, and MPI-2.0
Up: Contents
Next: Background of MPI-1.3 and MPI-2.1
Previous: Background of MPI-1.0
Beginning in March 1995, the MPI Forum began meeting to
consider corrections and extensions to the original MPI Standard
document [22]. The first product of these deliberations was
Version 1.1 of the MPI specification, released in June of 1995
[23]
(see
http://www.mpi-forum.org for official MPI document releases).
At that time, effort focused
in
five areas.
1. Further corrections and clarifications for the MPI-1.1 document.
2. Additions to MPI-1.1 that do not significantly change its types of
functionality (new datatype constructors, language interoperability, etc.).
3. Completely new types of functionality (dynamic processes, one-sided
communication, parallel I/O, etc.) that are what everyone thinks of as
`` MPI-2 functionality.''
4. Bindings for Fortran 90 and C++.
MPI-2 specifies
C++ bindings
for both MPI-1 and MPI-2 functions, and extensions to the Fortran 77
binding of MPI-1 and MPI-2 to handle Fortran 90 issues.
5. Discussions of areas in which the MPI process and framework seem likely
to be useful, but where more discussion and experience are needed before
standardization (e.g.,
zero-copy
semantics on shared-memory machines, real-time
specifications).
Corrections and clarifications (items of type 1 in the above list)
were
collected in Chapter 3 of the MPI-2 document:
``Version 1.2 of MPI.''
That
chapter also contains the function for
identifying the version number. Additions to MPI-1.1 (items of
types 2, 3, and 4 in the above list) are in the
remaining chapters of the MPI-2 document,
and constitute the specification for
MPI-2.
Items of type 5 in the
above list have been moved to a separate document, the `` MPI Journal
of Development'' (JOD), and are not part of the MPI-2 Standard.
This structure makes it easy for users and implementors to understand
what level of MPI compliance a given implementation has:
- MPI-1 compliance will mean compliance with MPI-1.3.
This is a useful level of compliance. It means that the
implementation conforms to the clarifications of MPI-1.1 function
behavior given in Chapter 3 of the MPI-2 document. Some
implementations may require changes to be MPI-1 compliant.
- MPI-2 compliance will mean compliance with all of
MPI-2.1.
- The MPI Journal of Development is not part of the MPI Standard.
It is to be emphasized that forward compatibility is preserved. That is,
a valid MPI-1.1 program is both a valid
MPI-1.3
program and a valid
MPI-2.1
program, and a valid
MPI-1.3
program is a valid
MPI-2.1
program.
Up: Contents
Next: Background of MPI-1.3 and MPI-2.1
Previous: Background of MPI-1.0
Return to MPI-3.1 Standard Index
Return to MPI Forum Home Page
(Unofficial) MPI-3.1 of June 4, 2015
HTML Generated on June 4, 2015


Chapter 6 Coding practices
by Kunal Mishra, Jade Benjamin-Chung, and Stephanie Djajadi
6.1 Organizing scripts
Just as your data “flows” through your project, data should flow naturally through a script. Very generally, you want to:
- describe the work completed in the script in a comment header
- source your configuration file (
0-config.R
) - load all your data
- do all your analysis/computation
- save your data.
Each of these sections should be “chunked together” using comments. See this file for a good example of how to cleanly organize a file in a way that follows this “flow” and functionally separate pieces of code that are doing different things.
6.2 Documenting your code
6.2.1 File headers
Every file in a project should have a header that allows it to be interpreted on its own. It should include the name of the project and a short description for what this file (among the many in your project) does specifically. You may optionally wish to include the inputs and outputs of the script as well, though the next section makes this significantly less necessary.
################################################################################
# @Organization - Example Organization
# @Project - Example Project
# @Description - This file is responsible for [...]
################################################################################
6.2.2 Sections and subsections
Rstudio (v1.4 or more recent) supports the use of Sections and Subsections. You can easily navigate through longer scripts using the navigation pane in RStudio, as shown on the right below.
# Section -------
## Subsection -------
### Sub-subsection -------
6.2.3 Code folding
Consider using RStudio’s code folding feature to collapse and expand different sections of your code. Any comment line with at least four trailing dashes (-), equal signs (=), or pound signs (#) automatically creates a code section. For example:
6.2.5 Function documentation
Every function you write must include a header to document its purpose, inputs, and outputs. For any reproducible workflows, they are essential, because R is dynamically typed. This means, you can pass a string
into an argument that is meant to be a data.table
, or a list
into an argument meant for a tibble
. It is the responsibility of a function’s author to document what each argument is meant to do and its basic type. This is an example for documenting a function (inspired by JavaDocs and R’s Plumber API docs):
##############################################
##############################################
# Documentation: calc_fluseas_mean
# Usage: calc_fluseas_mean(data, yname)
# Description: Make a dataframe with rows for flu season and site
# and the number of patients with an outcome, the total patients,
# and the percent of patients with the outcome
# Args/Options:
# data: a data frame with variables flu_season, site, studyID, and yname
# yname: a string for the outcome name
# silent: a boolean specifying whether the function shouldn't output anything to the console (DEFAULT: TRUE)
# Returns: the dataframe as described above
# Output: prints the data frame described above if silent is not True
calc_fluseas_mean = function(data, yname, silent = TRUE) {
### function code here
}
The header tells you what the function does, its various inputs, and how you might go about using the function to do what you want. Also notice that all optional arguments (i.e. ones with pre-specified defaults) follow arguments that require user input.
Note: As someone trying to call a function, it is possible to access a function’s documentation (and internal code) by
CMD-Left-Click
ing the function’s name in RStudioNote: Depending on how important your function is, the complexity of your function code, and the complexity of different types of data in your project, you can also add “type-checking” to your function with the
assertthat::assert_that()
function. You can, for example,assert_that(is.data.frame(statistical_input))
, which will ensure that collaborators or reviewers of your project attempting to use your function are using it in the way that it is intended by calling it with (at the minimum) the correct type of arguments. You can extend this to ensure that certain assumptions regarding the inputs are fulfilled as well (i.e. thattime_column
,location_column
,value_column
, andpopulation_column
all exist within thestatistical_input
tibble).
6.3 Object naming
Generally we recommend using nouns for objects and verbs for functions. This is because functions are performing actions, while objects are not.
Try to make your variable names both more expressive and more explicit. Being a bit more verbose is useful and easy in the age of autocompletion! For example, instead of naming a variable vaxcov_1718
, try naming it vaccination_coverage_2017_18
. Similarly, flu_res
could be named absentee_flu_residuals
, making your code more readable and explicit.
- For more help, check out Be Expressive: How to Give Your Variables Better Names
We recommend you use Snake_Case.
Base R allows
.
in variable names and functions (such asread.csv()
), but this goes against best practices for variable naming in many other coding languages. For consistency’s sake,snake_case
has been adopted across languages, and modern packages and functions typically use it (i.e.readr::read_csv()
). As a very general rule of thumb, if a package you’re using doesn’t usesnake_case
, there may be an updated version or more modern package that does, bringing with it the variety of performance improvements and bug fixes inherent in more mature and modern software.Note: you may also see
camelCase
throughout the R code you come across. This is okay but not ideal – try to stay consistent across all your code withsnake_case
.Note: again, its also worth noting there’s nothing inherently wrong with using
.
in variable names, just that it goes against style best practices that are cropping up in data science, so its worth getting rid of these bad habits now.
6.4 Function calls
In a function call, use “named arguments” and put each argument on a separate line to make your code more readable.
Here’s an example of what not to do when calling the function a function calc_fluseas_mean
(defined above):
mean_Y = calc_fluseas_mean(flu_data, "maari_yn", FALSE)
And here it is again using the best practices we’ve outlined:
mean_Y = calc_fluseas_mean(
data = flu_data,
yname = "maari_yn",
silent = FALSE
)
6.5 The here package
The here
package is one great R package that helps multiple collaborators deal with the mess that is working directories within an R project structure. Let’s say we have an R project at the path /home/oski/Some-R-Project
. My collaborator might clone the repository and work with it at some other path, such as /home/bear/R-Code/Some-R-Project
. Dealing with working directories and paths explicitly can be a very large pain, and as you might imagine, setting up a Config with paths requires those paths to flexibly work for all contributors to a project. This is where the here
package comes in and this a great vignette describing it.
6.6 Reading/Saving Data
6.6.1 .RDS
vs .RData
Files
One of the most common ways to load and save data in Base R is with the load()
and save()
functions to serialize multiple objects in a single .RData
file. The biggest problems with this practice include an inability to control the names of things getting loaded in, the inherent confusion this creates in understanding older code, and the inability to load individual elements of a saved file. For this, we recommend using the RDS format to save R objects.
- Note: if you have many related R objects you would have otherwise saved all together using the
save
function, the functional equivalent withRDS
would be to create a (named) list containing each of these objects, and saving it.
6.6.2 CSVs
Once again, the readr
package as part of the Tidvyerse is great, with a much faster read_csv()
than Base R’s read.csv()
. For massive CSVs (> 5 GB), you’ll find data.table::fread()
to be the fastest CSV reader in any data science language out there. For writing CSVs, readr::write_csv()
and data.table::fwrite()
outclass Base R’s write.csv()
by a significant margin as well.
6.7 Integrating Box and Dropbox
Box and Dropbox are cloud-based file sharing systems that are useful when dealing with large files. When our scripts generate large output files, the files can slow down the workflow if they are pushed to GitHub. This makes collaboration difficult when not everyone has a copy of the file, unless we decide to duplicate files and share them manually. The files might also take up a lot of local storage. Box and Dropbox help us avoid these issues by automatically storing the files, reading data, and writing data back to the cloud.
Box and Dropbox are separate platforms, but we can use either one to store and share files. To use them, we can install the packages that have been created to integrate Box and Dropbox into R. The set-up instructions are detailed below.
Make sure to authenticate before reading and writing from either Box or Dropbox. The authentication commands should go in the configuration file; it only needs to be done once. This will prompt you to give your login credentials for Box and Dropbox and will allow your application to access your shared folders.
6.7.1 Box
Follow the instructions in this section to use the boxr
package. Note that there are a few setup steps that need to be done on the box website before you can use the boxr
package, explained here in the section “Creating an Interactive App.” This gets the authentication keys that must be put in box.
Once that is done, add the authentication keys to your code in the configuration file, with box_auth(client_id = "<your_client_id>", client_secret = "<your_client_secret_id>")
. It is also important to set the default working directory so that the code can reference the correct folder in box: box_setwd(<folder_id>)
. The folder ID is the sequence of digits at the end of the URL.
Further details can be found here.
6.7.2 Dropbox
Follow the instructions at this link to use the rdrop2
package. Similar to the boxr
package, you must authenticate before reading and writing from Dropbox, which can be done by adding drop_auth()
to the configuration file.
Saving the authentication token is not required, although it may be useful if you plan on using Dropbox frequently. To do so, save the token with the following commands. Tokens are valid until they are manually revoked.
# first time only
# save the output of drop_auth to an RDS file
token <- drop_auth()
# this token only has to be generated once, it is valid until revoked
saveRDS(token, "/path/to/tokenfile.RDS")
# all future usages
# to use a stored token, provide the rdstoken argument
drop_auth(rdstoken = "/path/to/tokenfile.RDS")
6.8 Tidyverse
Throughout this document there have been references to the Tidyverse, but this section is to explicitly show you how to transform your Base R tendencies to Tidyverse (or Data.Table, Tidyverse’s performance-optimized competitor). For most of our work that does not utilize very large datasets, we recommend that you code in Tidyverse rather than Base R. Tidyverse is quickly becoming the gold standard in R data analysis and modern data science packages and code should use Tidyverse style and packages unless there’s a significant reason not to (i.e. big data pipelines that would benefit from Data.Table’s performance optimizations).
The package author has published a great textbook on R for Data Science, which leans heavily on many Tidyverse packages and may be worth checking out.
The following list is not exhaustive, but is a compact overview to begin to translate Base R into something better:
| Base R | Better Style, Performance, and Utility |
|---|---|
| _ | _ |
read.csv() |
readr::read_csv() or data.table::fread() |
write.csv() |
readr::write_csv() or data.table::fwrite() |
readRDS |
readr::read_rds() |
saveRDS() |
readr::write_rds() |
| _ | _ |
data.frame() |
tibble::tibble() or data.table::data.table() |
rbind() |
dplyr::bind_rows() |
cbind() |
dplyr::bind_cols() |
df$some_column |
df %>% dplyr::pull(some_column) |
df$some_column = ... |
df %>% dplyr::mutate(some_column = ...) |
df[get_rows_condition,] |
df %>% dplyr::filter(get_rows_condition) |
df[,c(col1, col2)] |
df %>% dplyr::select(col1, col2) |
merge(df1, df2, by = ..., all.x = ..., all.y = ...) |
df1 %>% dplyr::left_join(df2, by = ...) or dplyr::full_join or dplyr::inner_join or dplyr::right_join |
| _ | _ |
str() |
dplyr::glimpse() |
grep(pattern, x) |
stringr::str_which(string, pattern) |
gsub(pattern, replacement, x) |
stringr::str_replace(string, pattern, replacement) |
ifelse(test_expression, yes, no) |
if_else(condition, true, false) |
Nested: ifelse(test_expression1, yes1, ifelse(test_expression2, yes2, ifelse(test_expression3, yes3, no))) |
case_when(test_expression1 ~ yes1, test_expression2 ~ yes2, test_expression3 ~ yes3, TRUE ~ no) |
proc.time() |
tictoc::tic() and tictoc::toc() |
stopifnot() |
assertthat::assert_that() or assertthat::see_if() or assertthat::validate_that() |
For a more extensive set of syntactical translations to Tidyverse, you can check out this document.
Working with Tidyverse within functions can be somewhat of a pain due to non-standard evaluation (NSE) semantics. If you’re an avid function writer, we’d recommend checking out the following resources:
- Tidy Eval in 5 Minutes (video)
- Tidy Evaluation (e-book)
- Data Frame Columns as Arguments to Dplyr Functions (blog)
- Standard Evaluation for *_join (stackoverflow)
- Programming with dplyr (package vignette)
6.9 Coding with R and Python
If you’re using both R and Python, you may wish to check out the Feather package for exchanging data between the two languages extremely quickly.
6.10 Reviewing Code
Before publishing new changes, it is important to ensure that the code has been tested and well-documented. GitHub makes it possible to document all of these changes in a pull request. Pull requests can be used to describe changes in a branch that are ready to be merged with the base branch (more information in the GitHub section). Github allows users to create a pull request template in a repository to standardize and customize the information in a pull request. When you add a pull request template to your repository, everyone will automatically see the template’s contents in the pull request body.
6.10.1 Creating a Pull Request Template
Follow the instructions below to add a pull request template to a repository. More details can be found at this GitHub link.
- On GitHub, navigate to the main page of the repository.
- Above the file list, click
Create new file
. - Name the file
pull_request_template.md
. GitHub will not recognize this as the template if it is named anything else. The file must be on themaster
branch.- To store the file in a hidden directory instead of the main directory, name the file
.github/pull_request_template.md
.
- To store the file in a hidden directory instead of the main directory, name the file
- In the body of the new file, add your pull request template. This could include:
- A summary of the changes proposed in the pull request
- How the change has been tested
- @mentions of the person or team responsible for reviewing proposed changes
Here is an example pull request template.
# Description
## Summary of change
Please include a summary of the change, including any new functions added and example usage.
## Link to Spec
Please include a link to the Trello card or Google document with details of the task.
## Who should review the pull request?
@ ...
6.2.4 Comments in the body of your script
Commenting your code is an important part of reproducibility and helps document your code for the future. When things change or break, you’ll be thankful for comments. There’s no need to comment excessively or unnecessarily, but a comment describing what a large or complex chunk of code does is always helpful. See this file for an example of how to comment your code and notice that comments are always in the form of:
# This is a comment -- first letter is capitalized and spaced away from the pound sign