Markov switching autoregression models¶
This notebook provides an example of the use of Markov switching models in statsmodels to replicate a number of results presented in Kim and Nelson (1999). It applies the Hamilton (1989) filter the Kim (1994) smoother.
This is tested against the Markov-switching models from E-views 8, which can be found at http://www.eviews.com/EViews8/ev8ecswitch_n.html#MarkovAR or the Markov-switching models of Stata 14 which can be found at http://www.stata.com/manuals14/tsmswitch.pdf.
[1]:
%matplotlib inline
import numpy as np
import pandas as pd
import statsmodels.api as sm
import matplotlib.pyplot as plt
import requests
from io import BytesIO
# NBER recessions
from pandas_datareader.data import DataReader
from datetime import datetime
usrec = DataReader('USREC', 'fred', start=datetime(1947, 1, 1), end=datetime(2013, 4, 1))
/home/travis/miniconda/envs/statsmodels-test/lib/python3.7/site-packages/pandas_datareader/compat/__init__.py:7: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
from pandas.util.testing import assert_frame_equal
Hamilton (1989) switching model of GNP¶
This replicates Hamilton’s (1989) seminal paper introducing Markov-switching models. The model is an autoregressive model of order 4 in which the mean of the process switches between two regimes. It can be written:
Each period, the regime transitions according to the following matrix of transition probabilities:
where \(p_{ij}\) is the probability of transitioning from regime \(i\), to regime \(j\).
The model class is MarkovAutoregression
in the time-series part of statsmodels
. In order to create the model, we must specify the number of regimes with k_regimes=2
, and the order of the autoregression with order=4
. The default model also includes switching autoregressive coefficients, so here we also need to specify switching_ar=False
to avoid that.
After creation, the model is fit
via maximum likelihood estimation. Under the hood, good starting parameters are found using a number of steps of the expectation maximization (EM) algorithm, and a quasi-Newton (BFGS) algorithm is applied to quickly find the maximum.
[2]:
# Get the RGNP data to replicate Hamilton
dta = pd.read_stata('https://www.stata-press.com/data/r14/rgnp.dta').iloc[1:]
dta.index = pd.DatetimeIndex(dta.date, freq='QS')
dta_hamilton = dta.rgnp
# Plot the data
dta_hamilton.plot(title='Growth rate of Real GNP', figsize=(12,3))
# Fit the model
mod_hamilton = sm.tsa.MarkovAutoregression(dta_hamilton, k_regimes=2, order=4, switching_ar=False)
res_hamilton = mod_hamilton.fit()
[3]:
res_hamilton.summary()
[3]:
| Dep. Variable: | rgnp | No. Observations: | 131 |
|---|---|---|---|
| Model: | MarkovAutoregression | Log Likelihood | -181.263 |
| Date: | Fri, 21 Feb 2020 | AIC | 380.527 |
| Time: | 13:53:23 | BIC | 406.404 |
| Sample: | 04-01-1951 | HQIC | 391.042 |
| - 10-01-1984 | |||
| Covariance Type: | approx |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| const | -0.3588 | 0.265 | -1.356 | 0.175 | -0.877 | 0.160 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| const | 1.1635 | 0.075 | 15.614 | 0.000 | 1.017 | 1.310 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| sigma2 | 0.5914 | 0.103 | 5.761 | 0.000 | 0.390 | 0.793 |
| ar.L1 | 0.0135 | 0.120 | 0.112 | 0.911 | -0.222 | 0.249 |
| ar.L2 | -0.0575 | 0.138 | -0.418 | 0.676 | -0.327 | 0.212 |
| ar.L3 | -0.2470 | 0.107 | -2.310 | 0.021 | -0.457 | -0.037 |
| ar.L4 | -0.2129 | 0.111 | -1.926 | 0.054 | -0.430 | 0.004 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| p[0->0] | 0.7547 | 0.097 | 7.819 | 0.000 | 0.565 | 0.944 |
| p[1->0] | 0.0959 | 0.038 | 2.542 | 0.011 | 0.022 | 0.170 |
Warnings:
[1] Covariance matrix calculated using numerical (complex-step) differentiation.
We plot the filtered and smoothed probabilities of a recession. Filtered refers to an estimate of the probability at time \(t\) based on data up to and including time \(t\) (but excluding time \(t+1, ..., T\)). Smoothed refers to an estimate of the probability at time \(t\) using all the data in the sample.
For reference, the shaded periods represent the NBER recessions.
[4]:
fig, axes = plt.subplots(2, figsize=(7,7))
ax = axes[0]
ax.plot(res_hamilton.filtered_marginal_probabilities[0])
ax.fill_between(usrec.index, 0, 1, where=usrec['USREC'].values, color='k', alpha=0.1)
ax.set_xlim(dta_hamilton.index[4], dta_hamilton.index[-1])
ax.set(title='Filtered probability of recession')
ax = axes[1]
ax.plot(res_hamilton.smoothed_marginal_probabilities[0])
ax.fill_between(usrec.index, 0, 1, where=usrec['USREC'].values, color='k', alpha=0.1)
ax.set_xlim(dta_hamilton.index[4], dta_hamilton.index[-1])
ax.set(title='Smoothed probability of recession')
fig.tight_layout()
From the estimated transition matrix we can calculate the expected duration of a recession versus an expansion.
[5]:
print(res_hamilton.expected_durations)
[ 4.07604747 10.42589381]
In this case, it is expected that a recession will last about one year (4 quarters) and an expansion about two and a half years.
Kim, Nelson, and Startz (1998) Three-state Variance Switching¶
This model demonstrates estimation with regime heteroskedasticity (switching of variances) and no mean effect. The dataset can be reached at http://econ.korea.ac.kr/~cjkim/MARKOV/data/ew_excs.prn.
The model in question is:
Since there is no autoregressive component, this model can be fit using the MarkovRegression
class. Since there is no mean effect, we specify trend='nc'
. There are hypothesized to be three regimes for the switching variances, so we specify k_regimes=3
and switching_variance=True
(by default, the variance is assumed to be the same across regimes).
[6]:
# Get the dataset
ew_excs = requests.get('http://econ.korea.ac.kr/~cjkim/MARKOV/data/ew_excs.prn').content
raw = pd.read_table(BytesIO(ew_excs), header=None, skipfooter=1, engine='python')
raw.index = pd.date_range('1926-01-01', '1995-12-01', freq='MS')
dta_kns = raw.loc[:'1986'] - raw.loc[:'1986'].mean()
# Plot the dataset
dta_kns[0].plot(title='Excess returns', figsize=(12, 3))
# Fit the model
mod_kns = sm.tsa.MarkovRegression(dta_kns, k_regimes=3, trend='nc', switching_variance=True)
res_kns = mod_kns.fit()
[7]:
res_kns.summary()
/home/travis/build/statsmodels/statsmodels/statsmodels/base/model.py:1354: RuntimeWarning: invalid value encountered in sqrt
bse_ = np.sqrt(np.diag(self.cov_params()))
/home/travis/miniconda/envs/statsmodels-test/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py:903: RuntimeWarning: invalid value encountered in greater
return (a < x) & (x < b)
/home/travis/miniconda/envs/statsmodels-test/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py:903: RuntimeWarning: invalid value encountered in less
return (a < x) & (x < b)
/home/travis/miniconda/envs/statsmodels-test/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py:1912: RuntimeWarning: invalid value encountered in less_equal
cond2 = cond0 & (x <= _a)
[7]:
| Dep. Variable: | 0 | No. Observations: | 732 |
|---|---|---|---|
| Model: | MarkovRegression | Log Likelihood | 1001.895 |
| Date: | Fri, 21 Feb 2020 | AIC | -1985.790 |
| Time: | 13:53:28 | BIC | -1944.428 |
| Sample: | 01-01-1926 | HQIC | -1969.834 |
| - 12-01-1986 | |||
| Covariance Type: | approx |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| sigma2 | 0.0012 | 0.000 | 7.136 | 0.000 | 0.001 | 0.002 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| sigma2 | 0.0040 | 0.000 | 8.489 | 0.000 | 0.003 | 0.005 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| sigma2 | 0.0311 | 0.006 | 5.461 | 0.000 | 0.020 | 0.042 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| p[0->0] | 0.9747 | 0.000 | 7857.416 | 0.000 | 0.974 | 0.975 |
| p[1->0] | 0.0195 | 0.010 | 1.949 | 0.051 | -0.000 | 0.039 |
| p[2->0] | 2.354e-08 | nan | nan | nan | nan | nan |
| p[0->1] | 0.0253 | 3.97e-05 | 637.835 | 0.000 | 0.025 | 0.025 |
| p[1->1] | 0.9688 | 0.013 | 75.528 | 0.000 | 0.944 | 0.994 |
| p[2->1] | 0.0493 | 0.032 | 1.551 | 0.121 | -0.013 | 0.112 |
Warnings:
[1] Covariance matrix calculated using numerical (complex-step) differentiation.
Below we plot the probabilities of being in each of the regimes; only in a few periods is a high-variance regime probable.
[8]:
fig, axes = plt.subplots(3, figsize=(10,7))
ax = axes[0]
ax.plot(res_kns.smoothed_marginal_probabilities[0])
ax.set(title='Smoothed probability of a low-variance regime for stock returns')
ax = axes[1]
ax.plot(res_kns.smoothed_marginal_probabilities[1])
ax.set(title='Smoothed probability of a medium-variance regime for stock returns')
ax = axes[2]
ax.plot(res_kns.smoothed_marginal_probabilities[2])
ax.set(title='Smoothed probability of a high-variance regime for stock returns')
fig.tight_layout()
Filardo (1994) Time-Varying Transition Probabilities¶
This model demonstrates estimation with time-varying transition probabilities. The dataset can be reached at http://econ.korea.ac.kr/~cjkim/MARKOV/data/filardo.prn.
In the above models we have assumed that the transition probabilities are constant across time. Here we allow the probabilities to change with the state of the economy. Otherwise, the model is the same Markov autoregression of Hamilton (1989).
Each period, the regime now transitions according to the following matrix of time-varying transition probabilities:
where \(p_{ij,t}\) is the probability of transitioning from regime \(i\), to regime \(j\) in period \(t\), and is defined to be:
Instead of estimating the transition probabilities as part of maximum likelihood, the regression coefficients \(\beta_{ij}\) are estimated. These coefficients relate the transition probabilities to a vector of pre-determined or exogenous regressors \(x_{t-1}\).
[9]:
# Get the dataset
filardo = requests.get('http://econ.korea.ac.kr/~cjkim/MARKOV/data/filardo.prn').content
dta_filardo = pd.read_table(BytesIO(filardo), sep=' +', header=None, skipfooter=1, engine='python')
dta_filardo.columns = ['month', 'ip', 'leading']
dta_filardo.index = pd.date_range('1948-01-01', '1991-04-01', freq='MS')
dta_filardo['dlip'] = np.log(dta_filardo['ip']).diff()*100
# Deflated pre-1960 observations by ratio of std. devs.
# See hmt_tvp.opt or Filardo (1994) p. 302
std_ratio = dta_filardo['dlip']['1960-01-01':].std() / dta_filardo['dlip'][:'1959-12-01'].std()
dta_filardo['dlip'][:'1959-12-01'] = dta_filardo['dlip'][:'1959-12-01'] * std_ratio
dta_filardo['dlleading'] = np.log(dta_filardo['leading']).diff()*100
dta_filardo['dmdlleading'] = dta_filardo['dlleading'] - dta_filardo['dlleading'].mean()
# Plot the data
dta_filardo['dlip'].plot(title='Standardized growth rate of industrial production', figsize=(13,3))
plt.figure()
dta_filardo['dmdlleading'].plot(title='Leading indicator', figsize=(13,3));
The time-varying transition probabilities are specified by the exog_tvtp
parameter.
Here we demonstrate another feature of model fitting - the use of a random search for MLE starting parameters. Because Markov switching models are often characterized by many local maxima of the likelihood function, performing an initial optimization step can be helpful to find the best parameters.
Below, we specify that 20 random perturbations from the starting parameter vector are examined and the best one used as the actual starting parameters. Because of the random nature of the search, we seed the random number generator beforehand to allow replication of the result.
[10]:
mod_filardo = sm.tsa.MarkovAutoregression(
dta_filardo.iloc[2:]['dlip'], k_regimes=2, order=4, switching_ar=False,
exog_tvtp=sm.add_constant(dta_filardo.iloc[1:-1]['dmdlleading']))
np.random.seed(12345)
res_filardo = mod_filardo.fit(search_reps=20)
[11]:
res_filardo.summary()
[11]:
| Dep. Variable: | dlip | No. Observations: | 514 |
|---|---|---|---|
| Model: | MarkovAutoregression | Log Likelihood | -586.572 |
| Date: | Fri, 21 Feb 2020 | AIC | 1195.144 |
| Time: | 13:53:42 | BIC | 1241.808 |
| Sample: | 03-01-1948 | HQIC | 1213.433 |
| - 04-01-1991 | |||
| Covariance Type: | approx |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| const | -0.8659 | 0.153 | -5.658 | 0.000 | -1.166 | -0.566 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| const | 0.5173 | 0.077 | 6.706 | 0.000 | 0.366 | 0.668 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| sigma2 | 0.4844 | 0.037 | 13.172 | 0.000 | 0.412 | 0.556 |
| ar.L1 | 0.1895 | 0.050 | 3.761 | 0.000 | 0.091 | 0.288 |
| ar.L2 | 0.0793 | 0.051 | 1.552 | 0.121 | -0.021 | 0.180 |
| ar.L3 | 0.1109 | 0.052 | 2.136 | 0.033 | 0.009 | 0.213 |
| ar.L4 | 0.1223 | 0.051 | 2.418 | 0.016 | 0.023 | 0.221 |
| coef | std err | z | P>|z| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| p[0->0].tvtp0 | 1.6494 | 0.446 | 3.702 | 0.000 | 0.776 | 2.523 |
| p[1->0].tvtp0 | -4.3595 | 0.747 | -5.833 | 0.000 | -5.824 | -2.895 |
| p[0->0].tvtp1 | -0.9945 | 0.566 | -1.758 | 0.079 | -2.103 | 0.114 |
| p[1->0].tvtp1 | -1.7702 | 0.508 | -3.484 | 0.000 | -2.766 | -0.775 |
Warnings:
[1] Covariance matrix calculated using numerical (complex-step) differentiation.
Below we plot the smoothed probability of the economy operating in a low-production state, and again include the NBER recessions for comparison.
[12]:
fig, ax = plt.subplots(figsize=(12,3))
ax.plot(res_filardo.smoothed_marginal_probabilities[0])
ax.fill_between(usrec.index, 0, 1, where=usrec['USREC'].values, color='gray', alpha=0.2)
ax.set_xlim(dta_filardo.index[6], dta_filardo.index[-1])
ax.set(title='Smoothed probability of a low-production state');
Using the time-varying transition probabilities, we can see how the expected duration of a low-production state changes over time:
[13]:
res_filardo.expected_durations[0].plot(
title='Expected duration of a low-production state', figsize=(12,3));
During recessions, the expected duration of a low-production state is much higher than in an expansion.


Online Pandemic Fun
Online Boardgames
- Code Names
- Cards for Quarantine
- Battleship
- Regular Video Games
Social Video Conferencing
- Zoom
- Google Hangouts
- Happy Hours
- Coffee Dates
Video Entertainment
- Rewatch Sporting Events
- Marble Racing
- Netflix
- Instructional videos to learn something new


Individuals should not leave their purses or wallets unattended while shopping. Especially during the Holiday Season, criminals will be looking for the opportunity to victimize individuals that do not take precautions to minimize their risk of theft.
Issues surrounding these types of crimes have become high-profile, particularly those surrounding hacking, copyright infringement, unwarranted mass-surveillance, sextortion, child pornography, and child grooming.
Passwords are usually stolen during what's called a phishing attempt where the hacker gives the user a website or form that the user thinks is the real login page for whatever site they want the password for.
The exhibit also brings lesser-known fraudsters to light like Clarence Hatry. Financial historians generally agree that the infamous stock market crash of 1929 actually began with a chain of events surrounding the bankruptcy of this flamboyant English financier. On Sept. 20, 1929, it was revealed that Hatry had forged $67 million in municipal bonds to cover his debts. As a consequence, many of his shareholders had to fortify themselves by selling their American holdings, creating panic in the market. Less than 30 days later, the stock market crashed, causing the worse financial crisis in U.S. history. Hatry served 15 years in prison for his crimes.
In the late 1980s, Ivan Boesky’s name became synonymous with insider trading on Wall Street. After being caught, Boesky informed on his fellow traders, including Michael Milken, and received a three-and-a-half year prison sentence and a fine of $100 million.
WorldCom, Inc., headquartered in Clinton, Mississippi, fell victim to an $11 billion accounting scandal that resulted in the company filing bankruptcy in 2002 — the largest bankruptcy in U.S. history at the time. Its CEO, Bernard Ebbers, was convicted of fraud and conspiracy and sentenced to twenty-five years in prison.
Adelphia Corporation was founded in 1972 by John Rigas and quickly became one of the nation’s largest providers of cable television services. However, according to prosecutors, Rigas and his family used the public company as their “private piggy bank,” looting the organization of hundreds of millions of dollars and causing shareholder losses of more than $60 billion. In 2005, Rigas, his two sons, and two other executives were sentenced on several fraud charges to lengthy prison terms.
The truth?
describes your situation.
Don’t send anything.
There are three things in the world that deserve no mercy, hypocrisy, fraud, and tyranny.


{% extends "base.html" %} {% load staticfiles %} {% block content %}
This is a template for a simple marketing or informational website. It includes a large callout called a jumbotron and three supporting pieces of content. Use it as a starting point to create something more unique.


SocNet
Home
Directory
Active Chats
My posts
New post
Add user
Logout
Problems while saving data.
Successfully saved data.
User Data
First Name
Last Name
E-mail
Password
Gender
Country
Role
Regular
Admin
Submit


--- layout: tweet ---
RT @philliphaydon: WCF Best Practice: Don't use WCF
2/1/2013, 6:41:43 PM
Favs: 0
Retweets: 0


On {{ the_date }} vLab deleted the following VMs:
These VMs were randomly choosen, and were deleted due to the soft-quota grace period expiring.


--- title: Panel title test layout: default ---
Alcohol is permitted and must be served by your selected caterer
exquisiteaffaircatering.com
cuisineunlimited.com
culinarycrafts.com
gooddaycatering.com
havepartywilltravel.com
lecroissantcatering.com
theblendedtable.com
utahfoodservices.com
EMAIL: orangerie@redbutte.utah.edu


"Hello! Revellers"
fesoroxx season, wild sound 2021 coming soon!!!
Fesoroxx Wild Sound 2021 is going to be most exciting and the most thrilling concert experience you've ever had, with sought after live bands from all over the world.
2021.05.21(FRI) ~ 23(SUN)
@ The O2 Arena – London, UK
Main Program
Acts
View our list of performing acts for the coming festivals.
Conference
The digital experience will feature screenings and exhibitions.
Participate
Find out how you can make the Fesoroxx 2021 experience better.
Exhibitions
Top creatives from all over the world will come together for this online expo.
Organizers
Sebastian Adewale
Event plannerLorem ipsum dolor sit amet consectetur adipisicing elit. Maxime, mollitia. Tenetur quisquam nulla.
Isabella Paisley
Production ManagerLorem ipsum dolor sit amet consectetur adipisicing elit. Maxime, mollitia. Tenetur quisquam nulla.


Remote IO Interface¶
This section presentes the communication structure and setup of the Motion-Lab. The laboratory consist of two Stewart Platforms (E-Motion 8000 and 1500), an industrial robot (COMAU Smart 5 NJ 110-3.0), two Motion Reference Units (MRU), a laser tracker (Leica AT960), a motion capture system (Qualisys Oqus 700+), and an embedded PC (Beckhoff CX2040) working as a central control unit.
Structure and Setup¶
A total of 7 PC’s, are currently installed on the Motion-lab:
- Two motion PC’s which are connected to the Stewart Platforms
- Linux platform responsible for the industrial COMAU robot
- Motion Capture PC, connected to the Qualisys motion capture system
- The embedded PC (Beckhoff CX2040), which is a combination of an industrial PC and a hardware PLC
- Real-Time Target PC
- Host PC, used as the user interface
Ethernet¶
A schematic of the communication structure is illustrated in the figure below, where the lowest level represents the installed hardware components.
As the figure shows, different connections use different communication protocols,
where each component has been assigned one or more static IP addresses, and communication ports.
All of the systems have been assigned IP addresses on the form: 192.168.90.xx
,
where the two last digits xx
, are determined by the row and column number, respectively.
This is illustrated in the figure by the red dashed lines.
It should be noted, that some of the components has several communication interfaces,
hence multiple IP addresses.
Two network switches are installed in the system.
“Internet Switch” is used for internet connection, where the components are connected with dynamic IP addresses.
The “Rack Switch” is used for internal connection between the components,
where the components are assigned static IP addresses, on the form: <IP_ADDRESS>xx
where the two last digits,
correlates to direct connection.
(e.g. Host PC has an IP Address: <IP_ADDRESS>
for connection to the Beckhoff embedded PC/PLC,
and IP Address: <IP_ADDRESS>
for the “Rack Switch” connection.)
The table shown below lists all of the connections for each component, with the respected protocol, IP address and port.
| Component: | Protocol: | IP address: | Port: | Connection Description: |
|---|---|---|---|---|
| Stewart Platform - EM 8000 | SERCOS | <IP_ADDRESS> | Motion PC 1 | |
| Stewart Platform - EM 1500 | SERCOS | <IP_ADDRESS> | Motion PC 2 | |
| COMAU Robot | POWERLINK | <IP_ADDRESS> | COMAU PC | |
| Ethernet | <IP_ADDRESS> | Rack Switch | ||
| MRU 1 | UDP | <IP_ADDRESS> | 50013 | Beckhoff PC/PLC |
| MRU 2 | UDP | <IP_ADDRESS> | 50014 | Beckhoff PC/PLC |
| Laser Tracker - Leica AT960 | EtherCAT | <IP_ADDRESS> | Beckhoff PC/PLC | |
| Motion PC 1 | SERCOS | <IP_ADDRESS> | EM 8000 | |
| UDP | <IP_ADDRESS> | 50030 | Beckhoff PC/PLC | |
| Ethernet | <IP_ADDRESS> | Rack switch | ||
| Motion PC 2 | SERCOS | <IP_ADDRESS> | EM 1500 | |
| UDP | <IP_ADDRESS> | 50031 | Beckhoff PC/PLC | |
| Ethernet | <IP_ADDRESS> | Rack switch | ||
| COMAU PC | POWERLINK | <IP_ADDRESS> | COMAU Robot | |
| UDP | <IP_ADDRESS> | 50032 | Rack Switch | |
| Ethernet | Internet Switch | |||
| Beckhoff Embedded PC | UDP | <IP_ADDRESS> | 50040 | Motion PC 1 |
| UDP | <IP_ADDRESS> | 50041 | Motion PC 2 | |
| UDP | <IP_ADDRESS> | 50042 | Rack Switch | |
| UDP | <IP_ADDRESS> | 50043 | MRU 1 | |
| UDP | <IP_ADDRESS> | 50044 | MRU 2 | |
| EtherCAT | <IP_ADDRESS> | 50045 | Leica AT960 | |
| UDP | <IP_ADDRESS> | 50050 | Host PC | |
| Ethernet | <IP_ADDRESS> | Rack switch | ||
| Host PC | UDP | <IP_ADDRESS> | 50060 | Beckhoff PC/PLC |
| Ethernet | <IP_ADDRESS> | Rack Switch | ||
| Ethernet | Internet Switch |
Analog and Digital I/O¶
In addition to the ethernet communication, the central control unit (Beckhoff CX2040) have analog- and digital I/O-modules. This allows for easy connection to external devices. The figure below illustrates the installed modules and the connection configuration of the installed analog and digital devices.
The table below lists the installed modules, with the related specification, and the connected devices.
| Module: | Type: | Range: | Connection: | Device/Component: |
|---|---|---|---|---|
| EL4134 | Analog Output | -10 .. +10 V | Out 1 | Device 1 |
| Out 2 | Device 2 | |||
| Out 3 | Device 3 | |||
| Out 4 | Device 4 | |||
| EL3104 | Analog Input | -10 .. +10 V | In 1 | Device 1 |
| In 2 | Device 2 | |||
| In 3 | Device 3 | |||
| In 4 | Device 4 | |||
| In 5 | Device 5 | |||
| In 6 | Device 6 | |||
| In 7 | Device 7 | |||
| In 8 | Device 8 |
KVM Switch¶
All the presented PC’s are connected to a KVM switch, with the exception of the Beckhoff PC/PLC (which is controlled by a remote desktop connection).
The KVM switch allows for control of all the installed PC’s from the master keyboard.
To activate the desired computer, use the command combination: CTRL + F12 + # + ENTER
, where #
is the port number.
Available systems are listed in the table below, with the related port number.
| System: | Port Number: |
|---|---|
| External Connection | 1 |
| Host PC | 2 |
| Motion Capture PC: Qualisys | 3 |
| Motion PC: EM 8000 | 4 |
| Motion PC: EM 1500 | 5 |
| Linux PC: COMAU | 6 |
| Spare connection (not in use) | 7 |
| Spare connection (not in use) | 8 |
Remote UDP Interface¶
The motion-lab can be operated using the dedicated remote UDP interface. Using this iterface, the essential control and feedback signals can be utilized.
#pragma once
#include <vector>
#include <iostream>
// Xbox data
struct XboxData {
float leftStickX;
float leftStickY;
float rightStickX;
float rightStickY;
float leftTrigger;
float rightTrigger;
bool buttonA = false;
bool buttonB = false;
};


|
Name
|
Type
|
Description
|
| binaryData | xsd:base64Binary |
The host sub specification data in Binary for wire efficiency.
Since vSphere API 6.7
|
| createdTime | xsd:dateTime |
Time at which the host sub specification was created.
|
| data | xsd:byte[] |
The host sub specification data
|
| name | xsd:string |
The full name of the host sub specification. The format of this member
variable is:
CompanyName_ProductName_SubSpecName.
Thus, name conflict is avoided by containing the
company name , product name , and sub
specification name in this full name.
|
|
Properties inherited from DynamicData |
| None |


Red Cross donation bins stolen from gas station parking lot in Riverside
by ABC7.com staffRIVERSIDE, Calif. (KABC) -- Riverside police are trying to track down the three people who stole two Red Cross donation bins.
The bins were stolen on Thursday from a gas station parking lot at Van Buren Boulevard and Wells Avenue.
The suspects were driving a newer model Toyota Tundra that was pulling a black trailer.
Two suspects could be seen getting out of the vehicle and lifting the bins into the trailer.
The driver and one of the passengers were described as 25-year-old male adults. The third suspect was described as a male in his 20s.
The bins are worth $1,500 each.
Anyone with information is asked to contact detectives at<PHONE_NUMBER>.


 It's barely social engineering. It's more like natural selection.
Finally, back at the office after a short hiatus. I think letting a week sink in after starting the GORM project gave me the time to really digest what I learned. Kind of like when you write a paper and let a few days pass so you can have fresh eyes to look over your work. I really like I am grasping the concept of Object Relational Mapping. I basically finished up the extract, transform, load (ETL) work yesterday.
I basically exported the appeals MS Access database into a text file, separated by tabs. Then took the struct tables I created earlier before the baby and mapped each column to their respective attribute in the struct. Even the polymorphic address table is working perfectly. I have never even thought that polymorphism is possible in relational databases. Basically, I have two types of addresses to store for each appeal, the appellant and their optional representative. It uses a combination of two additional fields that I named resident ID and Type. GORM takes care of the rest when I explicitly tell it that the Person's Address has a polymorphic relationship with an address. The type lets me know if it is a representative or an appellant and the ID that is stored is actually the ID of the person it is storing information about. Super cool.
I reduced the number of columns I would need to have in the person and representative table by forcing them to store their information in the same table called addresses. GORM takes care of the joining for me and as the web designer I am glad I do not have to think about complex SQL statements in order to glue all this information together. I defined it once in the structs and GORM takes care of the complexity. This is exactly what I was trying to solve when I began adding some of the 80+ attributes to an appeal struct in December. I really need this code to look and feel maintainable.
The other thing that I accomplished with GORM is once the data was in a slice of appeals cases I was able to iterate each item in the slice and save it to the Linux MSSQL database I have been running as a development database. I even had benchmark tests run like when I run the seeddb function I create a begin time saved as a Unix timestamp and when the loading finishes I calculate how long in minutes and seconds it took the function to run. Turns out to load 3,200 appeals and over 6,000 outreach notes it took one minute and fifty-one seconds.
The only minor issue I have with GORM is really not an issue at all. It is the lazy-loading
it does when I pull records from the database. It is really meant as a performance boost
because really, when are you really going to need all 80+ columns loaded into a single
data structure? I kind of wish there was an eager-loading flag I could turn on if I did
indeed need all that data injected to the appeal object. Because the only way to do that
is to call a function call Preload("table_name")
and chain it on to the db object
you are currently using. It forces the programmer to have to know all the relationships
in order to get some meaningful data. You can't just say use this appeal object, but only
return the slice of hearing dates for a particular appeal without having to know that those
hearing dates are tied to some action which in turn is tied to some appeal case.
Even though the Doc was sick this week, we still managed to come together as a group online and start fleshing out some of the features needed to fulfill some requirements. I was stuck on the security requirement my group put down as a functional requirement. It didn't make a whole lot of sense because it was so vague. I remember spending an extraodinary amount of time in the session arguing (in a civil manor) why all the other's suggestions suck. This is the same day I finished chapter 3 of the book Why Software Sucks which completely deals with security in applications.
My thought process is that the Doc mentioned briefly that he didn't want to over complicate anything with a logon process with a username and password. I suggested that in order to make sure that the volunteer responsible is the one input data we would have the volunteer also input some information about them self on top of the numbers they were submitting. This could be as simple as a last name or initials or first initial + last name or email address. Something they would remember in a heartbeat because it belongs to them in some form or fashion. What I got in response sounded exactly like the programmers David Platt was berating in his book. I got responses like my suggestion is not unique enough. The volunteers need something along the lines of WIT emails last name + first initial + some number (i.e. walshb8, hayesj2, coxt1 ...). My response was just it wasn't user friendly enough and way too complicated for no added reason. Like the code camp would have 8 sessions in total for the day, and worse case that would mean 8 different volunteers counting attendance. The odds of two volunteers having the same last name or even the same first initial would be very slim.
One other student suggested something that made me laugh and my wife thought I was crazy. They said that we should create a unique IDs that cannot be easily guessed. I posed the question, "Do you want to be the one who supports all the volunteers who forgot the id?" Then I laughed to myself when he suggested that we would either give each volunteer a piece of paper with their ID on it or just email them. My response was that if we are giving the volunteers paper we might as well get them a pen so they can write the attendance down and just submit it at the end physically. It would take zero code to implement and achieve the same goal as spending an entire course developing an app that paper and pen could achieve faster and more securely.
The person got defensive and his counter was this is easy stuff to do. I thought I am actually dealing with a one of those pesky programmers David talked about that thinks his way is perfect because it solves all the cases in his head perfectly. Anyways, the email thing brought up a good point. Email addresses are inherently unique and I would suppose that most people who have any social media account know their email like the back of their hand. The organizers would have a list of people who will be volunteering and most likely the main form of communication will be email so why not make all the volunteers submit their email before they submit their numbers and that way everything is guaranteed to be unique and somewhat prevents anyone from submitting garbage numbers.
Nope, that is a security hazard apparently. When I tried to challenge why that is the case I didn't get an explanation back which goes to show you that this particular team member was really disconnected from his users. Maybe he was afraid that having things tied to email exposes it somehow to everyone? I don't know how it would unless you programmed the system to do that by default which is a silly idea in the first place. The more and more I puzzle myself over this single requirement dilemma, the more it just makes me want to create a Google Form that points to a single Google sheet document and collects all this information for us. Again, zero effort from a programming perspective and it would accomplish the goals for this project easily.
I wanted to briefly explain the quote a bit more. I remember going over the social engineering section of the text and when he got to user writing passwords down on a sticky note I just had to include this brilliant quote from one of my all-time favorite television shows. They even had an episode where they had user test and give feedback on the application. The main character ended up making a great app with the UI/UX in mind of a programmer and had zero design choices to make the interface simpler to understand. They even through in a clippy joke in the episode the reminds me of David's anger for the late Microsoft mascot.


Glossary¶
This document is currently a work-in-progress QIIME 2 terminology glossary.
- Action
- A general term for a method or visualizer.
- Artifact
- Data that can be used as input to a QIIME method or visualizer, or that can be generated as output from a QIIME method. Artifacts typically have the file extension
.qza
. - Method
- An action that takes some combination of artifacts and parameters as input, and produces one or more artifacts as output. These output artifacts could subsequently be used as input to other QIIME 2 methods or visualizers. Methods can produce intermediate or terminal outputs in a QIIME analysis.
- Parameter
- A primitive (i.e., non-artifact) input to an action. For example, strings, integers, and booleans are primitives. Primitives are never output from an action.
- Pipeline
- A combination of actions. This is not yet implemented.
- Plugin
- A plugin provides microbiome (i.e. domain-specific) analysis functionality that is accessible to users through a variety of interfaces built around the QIIME 2 framework. Plugins can be developed and distributed by anyone. In more technical terms, a plugin is a Python 3 package that instantiates a
qiime.plugin.Plugin
object, and registers actions, data formats, and/or semantic types that become discoverable in the QIIME 2 framework. - Result
- A general term for an artifact or visualization. A result is produced by a method, visualizer, or pipeline.
- Visualization
- Data that can be generated as output from a QIIME visualizer. Visualizations typically have the file extension
.qzv
. - Visualizer
- An action that takes some combination of artifacts and parameters as input, and produces exactly one visualization as output. Output visualizations, by definition, cannot be used as input to other QIIME 2 methods or visualizers. Visualizers can only produce terminal output in a QIIME analysis.


To obtain an access_token
, required to access certain endpoints of the API, the standard OAuth 2.0 flow should be followed. For details see RFC 6749.
A list of all available operations for this endpoint.
Parameter should be passed as part of the url.
| Parameter | Type | Optional | Default | Description |
client_id |
String | undefined |
Your client_id |
|
request_uri |
String | undefined |
Callback url as set in your account settings. Used to return the client back to you and as an additional check. | |
response_type |
String | undefined |
Should be set to code |
Will return an authorization_code
as part of the callback url which will be valid for 10 minutes before expiring. The authorization_code
can only be used once, a new one must be generated should you desire to re authenticate.
authorization_code
.Parameter should be passed as a application/json
format as part of the request-body.
| Parameter | Type | Optional | Default | Description |
client_id |
String | undefined |
Your client_id |
|
client_secret |
String | undefined |
Your client_secret |
|
redirect_uri |
String | undefined |
Callback url as set in your account settings. | |
grant_type |
String | undefined |
Should be set to authorization_code |
|
code |
String | undefined |
authorization_code from the /authorize operation |
Important
The client_id
and client_secret
can be provided as part of an Authorization Basic header (recommended).
Will return an access_token
as well as further relevant information
{
value: "{access_token}",
clientId: "{client_id}",
userId: "{user_id}",
expires:<PHONE_NUMBER>243
}
refresh_token
.Parameter should be passed as a application/json
format as part of the request-body.
| Parameter | Type | Optional | Default | Description |
client_id |
String | undefined |
Your client_id |
|
client_secret |
String | undefined |
Your client_secret |
|
refresh_token |
String | undefined |
refresh_token returned with the original access_token request |
|
grant_type |
String | undefined |
Should be set to refresh_token |
Will return an access_token
as well as further relevant information
{
value: "{access_token}",
clientId: "{client_id}",
userId: "{user_id}",
expires:<PHONE_NUMBER>243
}
access_token
Parameter should be passed as part of the url.
| Parameter | Type | Optional | Default | Description |
token |
String | undefined |
Your access_token |
Will return an access_token
as well as further relevant information
{
value: "{access_token}",
clientId: "{client_id}",
userId: "{user_id}",
expires:<PHONE_NUMBER>243,
valid: true
}


LENT 5 WEEK
FIRST READING 2 Kings
4:18-21.32-37
[The son of the Shunammite]
went out one day to his father among the reapers. And he said to his father,
"Oh, my head, my head!" The father said to his servant, "Carry
him to his mother." And when he had lifted him, and brought him to his
mother, the child sat on her lap till noon, and then he died. And she went up
and laid him on the bed of the man of God, and shut the door upon him, and went
out. When Elisha came into the house, he saw the
child lying dead on his bed.   Then he got up again, and walked once to and fro in the
house, and went up, and stretched himself upon him; the child sneezed seven
times, and the child opened his eyes. Then he summoned Gehazi
and said, "Call this Shunammite." So he
called her. And when she came to him, he said, "Take up your son."
She came and fell at his feet, bowing to the ground; then she took up her son
and went out.
The word of the Lord
RESPONSORIAL PSALM Psalm 17
R. When I awake, O Lord, I shall be filled with the
vision of your presence.
O LORD, hear a cause that is just, pay heed to my
cry. Turn your ear to my prayer: no deceit is on my lips.
R. When I awake, O Lord, I shall be filled with the
vision of your presence.
To you I call; for you will surely heed me, O God.
Turn your ear to me; hear my words. Display your merciful love. By your right
hand you deliver from their foes those who put their trust in you.
R. When I awake, O Lord, I shall be filled with the
vision of your presence.
Guard me as the apple of your eye. Hide me in the
shadow of your wings. As for me, in justice I shall behold your face; when I
awake I shall be filled with the vision of your presence.
R. When I awake, O Lord, I shall be filled with the
vision of your presence.
VERSE BEFORE THE GOSPEL John 11:25,26
R. Glory and praise to you, O Christ.

GOSPEL
John 11:1-45
At that time: A certain man was ill, Lazarus of Bethany,
the village of Mary and her sister Martha. It was Mary who
anointed the Lord with ointment and wiped his feet with her hair, whose brother
Lazarus was ill. So the sisters sent to him, saying, "Lord, he whom you
love is ill." But when Jesus heard it he said, "This illness is not
unto death; it is for the glory of God, so that the Son of God may be glorified
by means of it." Now Jesus loved Martha and her sister and Lazarus. So
when he heard that he was ill, he stayed two days longer in the place where he
was. Then after this he said to the disciples, "Let us go into Judea again." The disciples said to him,
"Rabbi, the Jews were but now seeking to stone you, and are you going
there again?" Jesus answered, "Are there not twelve hours in the day?
If any one walks in the day, he does not stumble, because he sees the light of
this world. But if any one walks in the night, he stumbles, because the light
is not in him." Thus he spoke, and then he said to them, "Our friend
Lazarus has fallen asleep, but I go to awake him out of sleep." The
disciples said to him, "Lord, if he has fallen asleep, he will
recover." Now Jesus had spoken of his death, but they thought that he
meant taking rest in sleep. Then Jesus told them plainly, "Lazarus is
dead; and for your sake I am glad that I was not there, so that you may believe.
But let us go to him." Thomas, called the Twin, said to his fellow
disciples, "Let us also go, that we may die with him." Now when Jesus
came, he found that Lazarus had already been in the tomb four days. Bethany was near Jerusalem,
about two miles off, and many of the Jews had come to Martha and Mary to
console them concerning their brother. When Martha heard that Jesus was coming,
she went and met him, while Mary sat in the house. Martha said to Jesus,
"Lord, if you had been here, my brother would not have died. And even now
I know that whatever you ask from God, God will give you." Jesus said to
her, "Your brother will rise again." Martha said to him, "I know
that he will rise again in the resurrection at the last day."
Do you believe this?" She said to him, "Yes, Lord; I believe that you
are the Christ, the Son of God, he who is coming into the world." When she
had said this, she went and called her sister Mary, saying quietly, "The
Teacher is here and is calling for you." And when she heard it, she rose
quickly and went to him. Now Jesus had not yet come to the village, but was still
in the place where Martha had met him. When the Jews
who were with her in the house, consoling her, saw Mary rise quickly and go
out, they followed her, supposing that she was going to the tomb to weep there.
Then Mary, when she came where Jesus was and saw him, fell at his feet, saying
to him, "Lord, if you had been here, my brother would not have died."
When Jesus saw her weeping, and the Jews who came with her also weeping, he was
deeply moved in spirit and troubled; and he said, "Where have you laid
him?" They said to him, "Lord, come and see." Jesus wept. So the
Jews said, "See how he loved him!" But some of them said, "Could
not he who opened the eyes of the blind man have kept this man from
dying?" Then Jesus, deeply moved again, came to the tomb; it was a cave,
and a stone lay upon it. Jesus said, "Take away the stone." Martha,
the sister of the dead man, said to him, "Lord, by this time there will be
an odour, for he has been dead four days." Jesus
said to her, "Did I not tell you that if you would believe you would see
the glory of God?" So they took away the stone. And Jesus lifted up his
eyes and said, "Father, I thank you that you have heard me. I knew that
you hear me always, but I have said this on account of the people standing by,
that they may believe that you have sent me." When he had said this, he
cried with a loud voice, "Lazarus, come out."
Jesus said to them, "Unbind him, and let him go." Many of the Jews
therefore, who had come with Mary and had seen what he did, believed in him.
The Gospel of the Lord


Tata Power raises Rs 1,500 cr via NCDs
The company has issued 15,000 NCDs worth Rs 1,500 crore on November 21, 2019, Tata Power said in a BSE filing.
by PTITata Power on November 21 raised Rs 1,500 crore by issuing unsecured, non-cumulative, redeemable, taxable, listed, rated, non-convertible debentures (NCDs) on private placement basis.
The company has issued 15,000 NCDs worth Rs 1,500 crore on November 21, 2019, Tata Power said in a BSE filing.
The NCDs would be listed on WDM (wholesale debt market) segment of NSE, it added.


| 031:014 Khan | : |
And We have enjoined on man (to be dutiful and good) to his parents.  To Me is the eventual coming. |
| 031:014 Pickthal | : |
And We have enjoined upon man concerning his partners - His mother beareth him in weakness upon weakness, and his weaning is in two years - Give thanks unto Me and unto thy parents. Unto Me is the journeying. |
| 031:014 Rashad | : |
We enjoined the human being to honor his parents.  It takes two years (of intensive care) until weaning. You shall be appreciative of Me, and of your parents. To Me is the ultimate destiny. |
| 031:014 Sarwar | : |
(Concerning his parents), We advised the man, whose mother bears him with great pain and breast-feeds him for two years, to give thanks to Me first and then to them, to Me all things proceed. |
| 031:014 Shakir | : |
And We have enjoined man in respect of his parents-- his mother bears him with faintings upon faintings and his weaning takes two years-- saying: Be grateful to Me and to both your parents; to Me is the eventual coming. |
| 031:014 Sherali | : |
And we have enjoined on man to be good to his parents -his mother bears him in weakness upon weakness, and his weaning takes two years - and said, 'Give thanks to ME and thy parents. Unto ME is the final return; |
| 031:014 Yusufali | : |
And We have enjoined on man (to be good) to his parents: in travail upon travail did his mother bear him, and in years twain was his weaning: (hear the command), "Show gratitude to Me and to thy parents: to Me is (thy final) Goal. |
Concepts/Themes Covered: أركان الإسلام > التوحيد > توحيد الله تعالي > أوامره Pillars of Islam> Islamic> Oneness of Allah> His Order الإيمان > الله جلّ جلاله > شكره جلّ وعلا Faith> Allah, Glory to his Majesty> To Express Thanks to Allah الإنسان والعلاقات الاجتماعية > الأسرة > حق الوالدين Man and The Social Relations>The Family>The Parents' Right الإنسان والعلاقات الاجتماعية > الأسرة > الحمل والرضاع Man and The Social Relations>The Family>Pregnancy and Babies Nursing
This tool uses eight English parallel translations for the Holy Quran imported from Mr. Clay Smith. You can also visit Qurany Concepts for browsing the Quran themes or topics. If you want to search the Holy Quran for keywords, please visit Qurany Keywords.


Deals and offers
Hygiene equipments
04
Days
12
Hours
58
Min
02
Sec
Quick view
-20%
Name article
$140
$198
Free shipping
+
−


--- layout: post date: 2012-09-06 time: "09:00:00" authors: ["Greg Wilson"] title: How Quickly Do Workshops Fill Up? tags: ["Bootcamps"] ---
I had to compile some data on signups anyway, so here's a graph showing cumulative registration per workshop over time (counting from the day registration opened) for past workshops:
and for upcoming workshops (excluding two that just opened up):
Doing a bit more analysis, this spring's workshops filled up a lot faster than the ones we're running this fall. I think the main reason is that those ones were organized and advertised while school was on, while the upcoming series have been set up over the summer. We'll repeat the analysis in a few months to see if that hypothesis holds.


Vivek is currently serving as Business Intelligence Director and Member of Board of Management (BOM) at VPBank, one of the fastest growing and most profitable non-state owned banks in Vietnam. He has 12+ years of experience building & managing high performing decision analytics teams resulting in substantial financial impact. Vivek has expertise in key drivers of data-driven organization namely: data governance, data management, BI and analytics with a keen eye on deriving maximum value out of data. Under his leadership, VPBank became an industry pioneer with two major firsts - implementing a comprehensive Data Governance mechanism and setting up a BI Competency Center (BICC).
The Data Governance program of the bank garnered international recognition after receiving TDWI (The Data Warehousing Institute, USA) “Best Practices Award” in 2015. Since its launch, critical data elements for new to bank customers have seen 48% improvement in quality. BICC, in addition to bringing millions of dollars of additional operating income for the bank, was ranked no. 1 support unit under “Excellence in Internal Customer Service” category for 4 consecutive years (2014-2017) since its inception.
Serving on the bank’s Board of Management, Vivek has been instrumental in putting BI & analytics into business executives’ agenda. He was awarded a letter of appreciation from the Chairman of VPBank’s Board of Directors for outstanding contribution to bank’s success 2 years in a row. In Feb 2018, Vivek was awarded "VPBank Transformation Star - Gold Award" by the Board for substantial effort and contribution to the bank's transformation (2012-2017).
Vivek is a top rated speaker at international and regional conferences. His data-driven approach and framework for organizations was published in Journal of Digital Banking, Henry Stewart Publications, UK and APAC CIOoutlook Magazine, USA.
Vivek lives in Hanoi with his wife and three daughters.
Honors and Awards
- Vivek Bhanot awarded "VPBank Transformation Star - Gold Award" by the Board of Directors for substantial effort and contribution to the bank's transformation (2012-2017)
- BICC ranked no. 1 in the VPBank for excellence in internal customer service for 4 consecutive years (2014, 2015, 2016, 2017)
- VPBank won The Data Warehousing Institute (TDWI) 2015 Best Practices Award for Enterprise Data Management at their annual conference in San Diego, USA
- BICC recognized by Retail Banking Division of VPBank for exemplary BI/Analytics service
Speaking engagements
- BankTech Asia Conference, Kuala Lumpur, Malaysia (26-27 May, 2015)
- Back Office Forum Asia, Kuala Lumpur, Malaysia (18-19 Nov, 2015)
- The Future of Digital Banking Conference, Singapore (15-16 Mar, 2016)
- The Asian Banker Summit, Hanoi, Vietnam (10-12 May, 2016)
- BankTech Asia Conference, Jakarta, Indonesia (24-25 Aug, 2016)
- BankTech Asia Conference, Colombo, Sri Lanka (21-22 Mar, 2017)
- Next BigTech Asia Conference/Workshop, Kuala Lumpur, Malaysia (12-16 Mar, 2018)
- Human Resources Development Fund (HRDF) Conference, Kuala Lumpur, Malaysia (19 Mar, 2018)
Publications
- How to transform your bank into a data-driven organisation?
- Data Governance - Doing it right!
- Data-driven organization: Looking beyond technology


College Election System
Login
Register
College Election System
______________________________
Login to vote your favorite party!!
For Admin login click
here.
Welcome Login Here !
Username
Password
Remember username
Not a member Register
here!!


We got infrastructure!
- enhance frictionless markets
- exploit turn-key synergies
- integrate cutting-edge content
- expedite front-end infomediaries
- mesh out-of-the-box convergence
- empower B2B architectures
- monetize collaborative users
- matrix global users
- cultivate interactive initiatives
- extend intuitive eyeballs
- drive bricks-and-clicks experiences


aaa
todoList
Choose Categroies
0
All Tasks
0
In Progress Tasks
0
Completed Tasks
0
Archived Tasks
Task Name
Date
Description
Options
Add Task
Delete All
Task Name:
Date:
Description:


=== Compiling === Output: Linking ./tmp/codeworld/MMGame.jsexe (Main) Errors: Exit Code: 0 === Running ===


Fingerstache flexitarian street art 8-bit waistcoat. Distillery hexagon disrupt edison bulbche.
Fingerstache flexitarian street art 8-bit waistcoat. Distillery hexagon disrupt edison bulbche.
Fingerstache flexitarian street art 8-bit waistcoat. Distillery hexagon disrupt edison bulbche.


Explosion at Dippin’ Dots production facility injures 4
PADUCAH, Ky. - Authorities say an explosion at the Dippin’ Dots production facility in Kentucky has injured four people.
Dippin’ Dots CEO Scott Fischer says the explosion occurred in a production area around 10 p.m. Friday evening. Fischer said in a release that it would be premature to comment on the cause of the blast.
“At this moment what is most important to us is our employees, especially our four team members who were injured last night,” Fischer said.
Paducah city spokeswoman Pam Spencer said the plant had a nitrogen leak, but it was unclear whether that leak caused the explosion or happened afterward. Nitrogen is used to flash-freeze Dippin’ Dots novelty ice cream.
All four people injured were hospitalized. WPSD-TV reported that one of the injured workers was released from the hospital early Saturday morning.
At one point employees were not being allowed inside the building after the incident because the oxygen level was too low, the power was shut off and smoke remained inside
Along with firefighters, personnel from Atmos Energy, Jackson Purchase Energy Corporation and Paducah Water were at the scene. Dippin’ Dots is headquartered in Paducah, a western Kentucky city near the Illinois state line.
___
Information from: WPSD-TV, http://www.wpsdlocal6.com


How to install Disney+ as a Windows 10 app
Source: Daniel Rubino/Windows Central
Disney+ is upon us, and while there are plenty of apps to watch the service on, there is no dedicated app for Windows 10. But that's OK because thanks to the power of modern web browser standards and Progressive Web Apps (PWAs) you can get one-click access and start streaming.
All you need is your favorite browser, like the new Microsoft Edge based on Chromium or Google Chrome.
A magical streaming service
Disney+
The new home for anything Disney's light touches
Disney, Star Wars, National Geographic, Marvel, Disney Channel, and Fox programs, are all finally here for you to savor on all your devices.
What is a PWA?
Source: Windows CentralStreaming Disney+ as a PWA feels just like a native app for Windows 10.
A Progressive Web App (PWA) is when a site has been effectively designed to be "installed" as an app to a device. With a minimal border, a dedicated icon for the Windows 10 Start menu that can be pinned to the Task Bar, and one-click run ability, the website itself becomes the app.
All you need to know about Progressive Web Apps and Microsoft
With Disney+ installed as a PWA on your Windows 10 desktop PC or laptop, you can access the service without having to launch your web browser.
How to install Disney+ as a PWA
Source: Windows Central
Open the new Edge browser (beta, developer, or canary branch) or Google Chrome to get started. (Opera, Firefox, and the older version of Edge do not currently support the installable PWA function.)
- Navigate to www.disneyplus.com.
- Log into your Disney+ account.
- Select the browser menu '…' in the upper right corner in Edge. Source: Windows Central
- Navigate to Apps and select Install this site as an app. Source: Windows Central
- Name the app Disney+ and hit Install. Source: Windows Central
The Disney+ "app" will now show in the Windows 10 Start menu like any other app. From there, you can right-click on it to pin to the Start menu or send it to the Task Bar for quick access. Right-clicking on the icon also lets you uninstall the "app" if you no longer need it.
You now have a Disney+ Windows app!
Source: Windows Central
PWAs represent the future for a lot of website experiences on Windows 10 and even mobile phones. The ability to have one-click access to Disney+ as an app ready to stream your favorite shows and movies is much better than using the web browser.
Make sure you install the Disney+ PWA on all your laptops and PCs where you will want to watch iy. And don't forget, you can follow the same steps above for Netflix, Hulu, and others too.
A magical streaming service
Disney+
The new home for anything Disney's light touches
Disney, Star Wars, National Geographic, Marvel, Disney Channel, and Fox programs, are all finally here for you to savor on all your devices.
Related: Disney+ Xbox One app now available to download
We may earn a commission for purchases using our links. Learn more.


I’ve started working with Zookeeper. Since I’ve started doing that I’ve put together this blog post. It’s aim is to provide a structured approach to learning Zookeeper and researching the elements that make its features tick. Along the way I have a few call outs to people that have also provided excellent talks, material, or contributions to learning about Zookeeper along the way. With that, let’s get started.
Zookeeper is a consensus system written based on ideas presented via consensus algorithms. The idea is key value stores that keep all of their data in memory for read heavy workloads. The qualities in this context present a system that is highly consistent, with intent for access from distributed systems to data that won’t be lost.
Start Learning
The starting point should be a complete read of the Apache Zookeeper Project Home Page.
At this point I took an administrators’ angle on determining what I needed to know and do next. I knew that my situation would meet the basic assumptions of reliability around Zookeeper; First is that only a minority of servers in a deployment would fail at a particular time or become inaccessible from a crash, partition, or related issue, and second is that deployed machines would have correctly operating clocks, storage, and network components that perform consistently.
I had also made an assumption that I would need 2 x F + 1 machines in order to maintain data consistency. The F here is the number of failed or inaccessible machines. This meant that if I wanted to have 2 failures, I’d need at least 5 machines. For a failure of up to 3 machines, that would be 7 machines. Pretty easy, just a little simple math.
The other thing I was curious about, especially on a single machine, would be Zookeeper’s overall overhead. Would it come into contention with the services that are already running? Would it be ok to put Zookeeper on the machines that run the micro-services that Zookeeper is providing information to? Well, Zookeeper does indeed content with other application for CPU, network, memory, and storage. For this reason I have to balance the deployment of Zookeeper in relation to the other applications, as my server loads may not be super high, and thus I’d be able to have Zookeeper on some of the servers that have actual other services deployed. But YMMV depending on your services you’ve got deployed.
While I was thinking through how I’d build out the architecture for my implementation of Zookeeper I came upon a very important note in the documentation,
“ZooKeeper’s transaction log must be on a dedicated device. (A dedicated partition is not enough.) ZooKeeper writes the log sequentially, without seeking Sharing your log device with other processes can cause seeks and contention, which in turn can cause multi-second delays.
Do not put ZooKeeper in a situation that can cause a swap. In order for ZooKeeper to function with any sort of timeliness, it simply cannot be allowed to swap. Therefore, make certain that the maximum heap size given to ZooKeeper is not bigger than the amount of real memory available to ZooKeeper. For more on this, see Things to Avoid below.”
After reading up on the following documentation it seemed like a good time to do a test deployment:
BEGIN BUG DESCRIPTION - 1ST DOCKER ATTEMPT
NOTE: If you just want to get to the Zookeeper installation & setup and skip this issue, GOTO here.
My first go was to pull up a clean Ubuntu docker image and prep it as a container. Then start installing the necessary parts of Zookeeper. These steps consisted of the following. I made a video for it (see toward the bottom of this entry), so you can actually see the flow and I also wrote the commands I’m tying in bash below. Then you can pick your preferred use.
docker-machine start fusion-fire
Docker machine starts my virtual machine on OS-X that runs the Docker daemon, which I’ve named fusion-fire, thus the command above. Then after that I pulled down an Ubuntu image, started a container from the image, connecting to the container and all set for installation.
docker pull ubuntu
docker run -it ubuntu
To install the Zookeeper server and begin execution I then issued the following.
sudo apt-get update
sudo apt-get -Y install default-jdk
While this was executing I also ran into a situation where the Java Development Kit was hanging on getting the certificates put into place.
The last dozen or so #Ubuntu #docker images I've tried to run "sudo apt-get install default-jre" on have all ended up spiking the CPU.
— Λdrøn (@Adron) January 29, 2016
I began looking into this problem, and found currently on Ubuntu 14.04, running sudo-apt-get update and then running the install will trigger the bug. Two other points of reference are here, here, and there are other postings and issues related to the issue, just google. So what I did at that point to fix this issue was the following.
First I forcefully killed the docker container by just restarting the whole docker VM.
docker-machine stop fusion-fire
docker-machine start fusion-fire
Once that stopped I then started the virtual machine again.
sudo apt-get -y install default-jre
When it started I ran sudo apt-get install again. At that point apt-get attempted to recover but the install kept getting stuck on registering the certificates. So I gave up on this avenue for now. Hopefully a future Docker & Linux Kernal fixes the problem. So instead I went out and just spooled up some AWS instances for now, I’ll update this blog entry with a “Part II: Docker is Zookeeper Fixed” when the Java + Linux Kernal + Docker issue is remedied, until then, here’s the installation process on the AWS instances.
END BUG DESCRIPTION - AWS Instance Zookeeper Installation
Once this was setup I started 5 nano instances for Zookeeper (nano, since it’s just a test example for learning) and then logged in using broadcast with iTerm 2. From there each instance had the following commands executed.
sudo apt-get update
sudo apt-get install -y default-jdk
cd /opt/
sudo mkdir zookeeper
cd zookeeper/
sudo wget http://mirror.tcpdiag.net/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
sudo tar -zxvf zookeeper-3.4.6.tar.gz
cd conf/
sudo nano zoo.cfg
NOTE: Nano is the text editor I used above for “sudo nano zoo.cfg”. If you don’t have it available just install it with “sudo apt-get install nano”.
In that zoo.cfg I entered the following. For the IPs I actually used the AWS private IPs for the config file example below.
tickTime=2000 dataDir=/var/zookeeper clientPort=2181 initLimit=5 syncLimit=2 server.1=<IP_ADDRESS>:2888:3888 server.2=<IP_ADDRESS>:2888:3888 server.3=<IP_ADDRESS>:2888:3888 server.4=<IP_ADDRESS>:2888:3888 server.5=<IP_ADDRESS>:2888:3888
Now I started the service using the zkServer.sh script file.
sudo /opt/zookeeper/zookeeper-3.4.6/bin/zkServer.sh start-foreground
When I booted up I ran into an error about the myid file, so I added the file with a sequential number for the byid in the /var/zookeeper directory.
sudo nano /var/zookeeper/myid
In each of the files I added a number, respectively 1 through 5 for the id of each and saved those files. Upon attempting to start the zookeeper service with the following command I finally got to see the various nodes in the ensemble gain access to each other and start working. Which, I gotta admit, was a pretty damn cool feeling.
After all that fussing it seemed good to note, especially since they’re hard to find in the documentation (which is kind of a bit hard to use), here are some of the switches for zkServer.sh.
start start-foreground (super useful for debugging) stop restart status upgrade print-cmd
Once this is done, restart the service but this time instead of using the start-foreground command, just use the start command and that will start the service and return the bash back to you to issues commands or whatnot. An easy way to test out Zookeeper now that it is running is to use the Zookeeper CLI. This is the zkCli.sh shell script (or zkcli.bat file if you’re running it on windows - which I’d strongly suggest NOT to do).
Ok, that’s it for this entry. More to come in the near future. Cheers!
Excellent Additional References
- In Search of an Understandable Consensus Algorithm (Extended Version) - Diego Ongaro & John Ousterhout @ Stanford University
- Paxos Made Simple by Leslie Lamport
- Camille Fournier (@skamille) provided an excellent talk about Zookeeper (Consensus Systems) on InfoQ. In this talk Camille also mentions the The Chubby Lock Service for Loosely Coupled Distributed Systems.
- Zookeeper for the Skeptical Architect by Camille Fournier - RICON East 2013 Talks about why Riak, Cassandra, and even the company Camille works at (Rent the Runway) doesn’t use Zookeeper. An interesting talk from the regard of why not to use it in some cases.


WITNESS LIST Title Insurance (Interim) Committee September 9, 1998-9:00A Title Insurance On: Carter, Robert (Tex. Dept. of Insurance), Austin Hester, Stephen A. (Chicago Title Ins. Co.), Dallas Registering, but not testifying: On: Hopson, Merritt L. (Tex. Dept. of Insurance), Austin


All HDFS commands are invoked by the bin/hdfs script. Running the hdfs script without any arguments prints the description for all commands.
Usage: hdfs [SHELL_OPTIONS] COMMAND [GENERIC_OPTIONS] [COMMAND_OPTIONS]
Hadoop has an option parsing framework that employs parsing generic options as well as running classes.
| COMMAND_OPTIONS | Description |
|---|---|
| --config --loglevel |
The common set of shell options. These are documented on the Commands Manual page. |
| GENERIC_OPTIONS | The common set of options supported by multiple commands. See the Hadoop Commands Manual for more information. |
| COMMAND COMMAND_OPTIONS | Various commands with their options are described in the following sections. The commands have been grouped into User Commands and Administration Commands. |
Commands useful for users of a hadoop cluster.
Usage: hdfs classpath
Prints the class path needed to get the Hadoop jar and the required libraries
Usage: hdfs dfs [COMMAND [COMMAND_OPTIONS]]
Run a filesystem command on the file system supported in Hadoop. The various COMMAND_OPTIONS can be found at File System Shell Guide.
Usage: hdfs fetchdt [--webservice <namenode_http_addr>] <path>
| COMMAND_OPTION | Description |
|---|---|
| --webservice https_address | use http protocol instead of RPC |
| fileName | File name to store the token into. |
Gets Delegation Token from a NameNode. See fetchdt for more info.
Usage:
hdfs fsck <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-blocks [-locations | -racks]]] [-includeSnapshots] [-storagepolicies] [-blockId <blk_Id>]
| COMMAND_OPTION | Description |
|---|---|
| path | Start checking from this path. |
| -delete | Delete corrupted files. |
| -files | Print out files being checked. |
| -files -blocks | Print out the block report |
| -files -blocks -locations | Print out locations for every block. |
| -files -blocks -racks | Print out network topology for data-node locations. |
| -includeSnapshots | Include snapshot data if the given path indicates a snapshottable directory or there are snapshottable directories under it. |
| -list-corruptfileblocks | Print out list of missing blocks and files they belong to. |
| -move | Move corrupted files to /lost+found. |
| -openforwrite | Print out files opened for write. |
| -storagepolicies | Print out storage policy summary for the blocks. |
| -blockId | Print out information about the block. |
Runs the HDFS filesystem checking utility. See fsck for more info.
Usage:
hdfs getconf -namenodes hdfs getconf -secondaryNameNodes hdfs getconf -backupNodes hdfs getconf -includeFile hdfs getconf -excludeFile hdfs getconf -nnRpcAddresses hdfs getconf -confKey [key]
| COMMAND_OPTION | Description |
|---|---|
| -namenodes | gets list of namenodes in the cluster. |
| -secondaryNameNodes | gets list of secondary namenodes in the cluster. |
| -backupNodes | gets list of backup nodes in the cluster. |
| -includeFile | gets the include file path that defines the datanodes that can join the cluster. |
| -excludeFile | gets the exclude file path that defines the datanodes that need to decommissioned. |
| -nnRpcAddresses | gets the namenode rpc addresses |
| -confKey [key] | gets a specific key from the configuration |
Gets configuration information from the configuration directory, post-processing.
Usage: hdfs lsSnapshottableDir [-help]
| COMMAND_OPTION | Description |
|---|---|
| -help | print help |
Get the list of snapshottable directories. When this is run as a super user, it returns all snapshottable directories. Otherwise it returns those directories that are owned by the current user.
Usage: hdfs jmxget [-localVM ConnectorURL | -port port | -server mbeanserver | -service service]
| COMMAND_OPTION | Description |
|---|---|
| -help | print help |
| -localVM ConnectorURL | connect to the VM on the same machine |
| -port mbean server port | specify mbean server port, if missing it will try to connect to MBean Server in the same VM |
| -service | specify jmx service, either DataNode or NameNode, the default |
Dump JMX information from a service.
Usage: hdfs oev [OPTIONS] -i INPUT_FILE -o OUTPUT_FILE
| COMMAND_OPTION | Description |
|---|---|
| -i,--inputFile arg | edits file to process, xml (case insensitive) extension means XML format, any other filename means binary format |
| -o,--outputFile arg | Name of output file. If the specified file exists, it will be overwritten, format of the file is determined by -p option |
| COMMAND_OPTION | Description |
|---|---|
| -f,--fix-txids | Renumber the transaction IDs in the input, so that there are no gaps or invalid transaction IDs. |
| -h,--help | Display usage information and exit |
| -r,--ecover | When reading binary edit logs, use recovery mode. This will give you the chance to skip corrupt parts of the edit log. |
| -p,--processor arg | Select which type of processor to apply against image file, currently supported processors are: binary (native binary format that Hadoop uses), xml (default, XML format), stats (prints statistics about edits file) |
| -v,--verbose | More verbose output, prints the input and output filenames, for processors that write to a file, also output to screen. On large image files this will dramatically increase processing time (default is false). |
Hadoop offline edits viewer.
Usage: hdfs oiv [OPTIONS] -i INPUT_FILE
| COMMAND_OPTION | Description |
|---|---|
| -i,--inputFile arg | edits file to process, xml (case insensitive) extension means XML format, any other filename means binary format |
| COMMAND_OPTION | Description |
|---|---|
| -h,--help | Display usage information and exit |
| -o,--outputFile arg | Name of output file. If the specified file exists, it will be overwritten, format of the file is determined by -p option |
| -p,--processor arg | Select which type of processor to apply against image file, currently supported processors are: binary (native binary format that Hadoop uses), xml (default, XML format), stats (prints statistics about edits file) |
Hadoop Offline Image Viewer for newer image files.
Usage: hdfs oiv_legacy [OPTIONS] -i INPUT_FILE -o OUTPUT_FILE
| COMMAND_OPTION | Description |
|---|---|
| -h,--help | Display usage information and exit |
| -i,--inputFile arg | edits file to process, xml (case insensitive) extension means XML format, any other filename means binary format |
| -o,--outputFile arg | Name of output file. If the specified file exists, it will be overwritten, format of the file is determined by -p option |
Hadoop offline image viewer for older versions of Hadoop.
Usage: hdfs snapshotDiff <path> <fromSnapshot> <toSnapshot>
Determine the difference between HDFS snapshots. See the HDFS Snapshot Documentation for more information.
Commands useful for administrators of a hadoop cluster.
Usage:
hdfs balancer [-threshold <threshold>] [-policy <policy>] [-exclude [-f <hosts-file> | <comma-separated list of hosts>]] [-include [-f <hosts-file> | <comma-separated list of hosts>]] [-idleiterations <idleiterations>]
| COMMAND_OPTION | Description |
|---|---|
| -policy <policy> | datanode (default): Cluster is balanced if each datanode is balanced. blockpool: Cluster is balanced if each block pool in each datanode is balanced. |
| -threshold <threshold> | Percentage of disk capacity. This overwrites the default threshold. |
| -exclude -f <hosts-file> | <comma-separated list of hosts> | Excludes the specified datanodes from being balanced by the balancer. |
| -include -f <hosts-file> | <comma-separated list of hosts> | Includes only the specified datanodes to be balanced by the balancer. |
| -idleiterations <iterations> | Maximum number of idle iterations before exit. This overwrites the default idleiterations(5). |
Runs a cluster balancing utility. An administrator can simply press Ctrl-C to stop the rebalancing process. See Balancer for more details.
Note that the blockpool policy is more strict than the datanode policy.
Usage: hdfs cacheadmin -addDirective -path <path> -pool <pool-name> [-force] [-replication <replication>] [-ttl <time-to-live>]
See the HDFS Cache Administration Documentation for more information.
Usage:
hdfs crypto -createZone -keyName <keyName> -path <path> hdfs crypto -help <command-name> hdfs crypto -listZones
See the HDFS Transparent Encryption Documentation for more information.
Usage: hdfs datanode [-regular | -rollback | -rollingupgrace rollback]
| COMMAND_OPTION | Description |
|---|---|
| -regular | Normal datanode startup (default). |
| -rollback | Rollback the datanode to the previous version. This should be used after stopping the datanode and distributing the old hadoop version. |
| -rollingupgrade rollback | Rollback a rolling upgrade operation. |
Runs a HDFS datanode.
Usage:
hdfs dfsadmin [GENERIC_OPTIONS] [-report [-live] [-dead] [-decommissioning]] [-safemode enter | leave | get | wait] [-saveNamespace] [-rollEdits] [-restoreFailedStorage true |false |check] [-refreshNodes] [-setQuota <quota> <dirname>...<dirname>] [-clrQuota <dirname>...<dirname>] [-setSpaceQuota <quota> <dirname>...<dirname>] [-clrSpaceQuota <dirname>...<dirname>] [-setStoragePolicy <path> <policyName>] [-getStoragePolicy <path>] [-finalizeUpgrade] [-rollingUpgrade [<query> |<prepare> |<finalize>]] [-metasave filename] [-refreshServiceAcl] [-refreshUserToGroupsMappings] [-refreshSuperUserGroupsConfiguration] [-refreshCallQueue] [-refresh <host:ipc_port> <key> [arg1..argn]] [-reconfig <datanode |...> <host:ipc_port> <start |status>] [-printTopology] [-refreshNamenodes datanodehost:port] [-deleteBlockPool datanode-host:port blockpoolId [force]] [-setBalancerBandwidth <bandwidth in bytes per second>] [-allowSnapshot <snapshotDir>] [-disallowSnapshot <snapshotDir>] [-fetchImage <local directory>] [-shutdownDatanode <datanode_host:ipc_port> [upgrade]] [-getDatanodeInfo <datanode_host:ipc_port>] [-triggerBlockReport [-incremental] <datanode_host:ipc_port>] [-help [cmd]]
| COMMAND_OPTION | Description |
|---|---|
| -report [-live] [-dead] [-decommissioning] | Reports basic filesystem information and statistics. Optional flags may be used to filter the list of displayed DataNodes. |
| -safemode enter|leave|get|wait | Safe mode maintenance command. Safe mode is a Namenode state in which it 1. does not accept changes to the name space (read-only) 2. does not replicate or delete blocks. Safe mode is entered automatically at Namenode startup, and leaves safe mode automatically when the configured minimum percentage of blocks satisfies the minimum replication condition. Safe mode can also be entered manually, but then it can only be turned off manually as well. |
| -saveNamespace | Save current namespace into storage directories and reset edits log. Requires safe mode. |
| -rollEdits | Rolls the edit log on the active NameNode. |
| -restoreFailedStorage true|false|check | This option will turn on/off automatic attempt to restore failed storage replicas. If a failed storage becomes available again the system will attempt to restore edits and/or fsimage during checkpoint. ‘check’ option will return current setting. |
| -refreshNodes | Re-read the hosts and exclude files to update the set of Datanodes that are allowed to connect to the Namenode and those that should be decommissioned or recommissioned. |
| -setQuota <quota> <dirname>…<dirname> | See HDFS Quotas Guide for the detail. |
| -clrQuota <dirname>…<dirname> | See HDFS Quotas Guide for the detail. |
| -setSpaceQuota <quota> <dirname>…<dirname> | See HDFS Quotas Guide for the detail. |
| -clrSpaceQuota <dirname>…<dirname> | See HDFS Quotas Guide for the detail. |
| -setStoragePolicy <path> <policyName> | Set a storage policy to a file or a directory. |
| -getStoragePolicy <path> | Get the storage policy of a file or a directory. |
| -finalizeUpgrade | Finalize upgrade of HDFS. Datanodes delete their previous version working directories, followed by Namenode doing the same. This completes the upgrade process. |
| -rollingUpgrade [<query>|<prepare>|<finalize>] | See Rolling Upgrade document for the detail. |
| -metasave filename | Save Namenode’s primary data structures to filename in the directory specified by hadoop.log.dir property. filename is overwritten if it exists. filename will contain one line for each of the following 1. Datanodes heart beating with Namenode 2. Blocks waiting to be replicated 3. Blocks currently being replicated 4. Blocks waiting to be deleted |
| -refreshServiceAcl | Reload the service-level authorization policy file. |
| -refreshUserToGroupsMappings | Refresh user-to-groups mappings. |
| -refreshSuperUserGroupsConfiguration | Refresh superuser proxy groups mappings |
| -refreshCallQueue | Reload the call queue from config. |
| -refresh <host:ipc_port> <key> [arg1..argn] | Triggers a runtime-refresh of the resource specified by <key> on <host:ipc_port>. All other args after are sent to the host. |
| -reconfig <datanode |…> <host:ipc_port> <start|status> | Start reconfiguration or get the status of an ongoing reconfiguration. The second parameter specifies the node type. Currently, only reloading DataNode’s configuration is supported. |
| -printTopology | Print a tree of the racks and their nodes as reported by the Namenode |
| -refreshNamenodes datanodehost:port | For the given datanode, reloads the configuration files, stops serving the removed block-pools and starts serving new block-pools. |
| -deleteBlockPool datanode-host:port blockpoolId [force] | If force is passed, block pool directory for the given blockpool id on the given datanode is deleted along with its contents, otherwise the directory is deleted only if it is empty. The command will fail if datanode is still serving the block pool. Refer to refreshNamenodes to shutdown a block pool service on a datanode. |
| -setBalancerBandwidth <bandwidth in bytes per second> | Changes the network bandwidth used by each datanode during HDFS block balancing. <bandwidth> is the maximum number of bytes per second that will be used by each datanode. This value overrides the dfs.balance.bandwidthPerSec parameter. NOTE: The new value is not persistent on the DataNode. |
| -allowSnapshot <snapshotDir> | Allowing snapshots of a directory to be created. If the operation completes successfully, the directory becomes snapshottable. See the HDFS Snapshot Documentation for more information. |
| -disallowSnapshot <snapshotDir> | Disallowing snapshots of a directory to be created. All snapshots of the directory must be deleted before disallowing snapshots. See the HDFS Snapshot Documentation for more information. |
| -fetchImage <local directory> | Downloads the most recent fsimage from the NameNode and saves it in the specified local directory. |
| -shutdownDatanode <datanode_host:ipc_port> [upgrade] | Submit a shutdown request for the given datanode. See Rolling Upgrade document for the detail. |
| -getDatanodeInfo <datanode_host:ipc_port> | Get the information about the given datanode. See Rolling Upgrade document for the detail. |
| -triggerBlockReport [-incremental] <datanode_host:ipc_port> | Trigger a block report for the given datanode. If ‘incremental’ is specified, it will be otherwise, it will be a full block report. |
| -help [cmd] | Displays help for the given command or all commands if none is specified. |
Runs a HDFS dfsadmin client.
Usage:
hdfs haadmin -checkHealth <serviceId> hdfs haadmin -failover [--forcefence] [--forceactive] <serviceId> <serviceId> hdfs haadmin -getServiceState <serviceId> hdfs haadmin -help <command> hdfs haadmin -transitionToActive <serviceId> [--forceactive] hdfs haadmin -transitionToStandby <serviceId>
| COMMAND_OPTION | Description |
|---|---|
| -checkHealth | check the health of the given NameNode |
| -failover | initiate a failover between two NameNodes |
| -getServiceState | determine whether the given NameNode is Active or Standby |
| -transitionToActive | transition the state of the given NameNode to Active (Warning: No fencing is done) |
| -transitionToStandby | transition the state of the given NameNode to Standby (Warning: No fencing is done) |
See HDFS HA with NFS or HDFS HA with QJM for more information on this command.
Usage: hdfs journalnode
This comamnd starts a journalnode for use with HDFS HA with QJM.
Usage: hdfs mover [-p <files/dirs> | -f <local file name>]
| COMMAND_OPTION | Description |
|---|---|
| -f <local file> | Specify a local file containing a list of HDFS files/dirs to migrate. |
| -p <files/dirs> | Specify a space separated list of HDFS files/dirs to migrate. |
Runs the data migration utility. See Mover for more details.
Note that, when both -p and -f options are omitted, the default path is the root directory.
Usage:
hdfs namenode [-backup] | [-checkpoint] | [-format [-clusterid cid ] [-force] [-nonInteractive] ] | [-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] | [-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] | [-rollback] | [-rollingUpgrade <downgrade |rollback> ] | [-finalize] | [-importCheckpoint] | [-initializeSharedEdits] | [-bootstrapStandby] | [-recover [-force] ] | [-metadataVersion ]
| COMMAND_OPTION | Description |
|---|---|
| -backup | Start backup node. |
| -checkpoint | Start checkpoint node. |
| -format [-clusterid cid] [-force] [-nonInteractive] | Formats the specified NameNode. It starts the NameNode, formats it and then shut it down. -force option formats if the name directory exists. -nonInteractive option aborts if the name directory exists, unless -force option is specified. |
| -upgrade [-clusterid cid] [-renameReserved <k-v pairs>] | Namenode should be started with upgrade option after the distribution of new Hadoop version. |
| -upgradeOnly [-clusterid cid] [-renameReserved <k-v pairs>] | Upgrade the specified NameNode and then shutdown it. |
| -rollback | Rollback the NameNode to the previous version. This should be used after stopping the cluster and distributing the old Hadoop version. |
| -rollingUpgrade <downgrade|rollback|started> | See Rolling Upgrade document for the detail. |
| -finalize | Finalize will remove the previous state of the files system. Recent upgrade will become permanent. Rollback option will not be available anymore. After finalization it shuts the NameNode down. |
| -importCheckpoint | Loads image from a checkpoint directory and save it into the current one. Checkpoint dir is read from property fs.checkpoint.dir |
| -initializeSharedEdits | Format a new shared edits dir and copy in enough edit log segments so that the standby NameNode can start up. |
| -bootstrapStandby | Allows the standby NameNode’s storage directories to be bootstrapped by copying the latest namespace snapshot from the active NameNode. This is used when first configuring an HA cluster. |
| -recover [-force] | Recover lost metadata on a corrupt filesystem. See HDFS User Guide for the detail. |
| -metadataVersion | Verify that configured directories exist, then print the metadata versions of the software and the image. |
Runs the namenode. More info about the upgrade, rollback and finalize is at Upgrade Rollback.
Usage: hdfs secondarynamenode [-checkpoint [force]] | [-format] | [-geteditsize]
| COMMAND_OPTION | Description |
|---|---|
| -checkpoint [force] | Checkpoints the SecondaryNameNode if EditLog size >= fs.checkpoint.size. If force is used, checkpoint irrespective of EditLog size. |
| -format | Format the local storage during startup. |
| -geteditsize | Prints the number of uncheckpointed transactions on the NameNode. |
Runs the HDFS secondary namenode. See Secondary Namenode for more info.
Usage: hdfs storagepolicies
Lists out all storage policies. See the HDFS Storage Policy Documentation for more information.
Usage: hdfs zkfc [-formatZK [-force] [-nonInteractive]]
| COMMAND_OPTION | Description |
|---|---|
| -formatZK | Format the Zookeeper instance |
| -h | Display help |
This comamnd starts a Zookeeper Failover Controller process for use with HDFS HA with QJM.
Useful commands to help administrators debug HDFS issues, like validating block files and calling recoverLease.
Usage: hdfs debug verify [-meta <metadata-file>] [-block <block-file>]
| COMMAND_OPTION | Description |
|---|---|
| -block block-file | Optional parameter to specify the absolute path for the block file on the local file system of the data node. |
| -meta metadata-file | Absolute path for the metadata file on the local file system of the data node. |
Verify HDFS metadata and block files. If a block file is specified, we will verify that the checksums in the metadata file match the block file.
Usage: hdfs debug recoverLease [-path <path>] [-retries <num-retries>]
| COMMAND_OPTION | Description |
|---|---|
| [-path path] | HDFS path for which to recover the lease. |
| [-retries num-retries] | Number of times the client will retry calling recoverLease. The default number of retries is 1. |
Recover the lease on the specified path. The path must reside on an HDFS filesystem. The default number of retries is 1.


|
||||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES |
See:
Description
| Class Summary | |
| AccessLogValve | Implementation of the Valve interface that generates a web server access log with the detailed line contents matching a configurable pattern. |
| CertificatesValve | Implementation of a Valve that deals with SSL client certificates, as follows: If this request was not received on an SSL socket, simply pass it on unmodified. |
| Constants | Manifest constants for the org.apache.catalina.valves
package. |
| Constants.AccessLog | |
| ErrorDispatcherValve | Implementation of a Valve that handles the error dispatch (that is, will forward to the appropriate error page if necessary). |
| ErrorReportValve | Implementation of a Valve that outputs HTML error pages. |
| RemoteAddrValve | Concrete implementation of RequestFilterValve that filters
based on the string representation of the remote client's IP address. |
| RemoteHostValve | Concrete implementation of RequestFilterValve that filters
based on the string representation of the remote client's IP address. |
| RequestDumperValve | Implementation of a Valve that logs interesting contents from the specified Request (before processing) and the corresponding Response (after processing). |
| RequestFilterValve | Implementation of a Valve that performs filtering based on comparing the appropriate request property (selected based on which subclass you choose to configure into your Container's pipeline) against a set of regular expressions configured for this Valve. |
| ValveBase | Convenience base class for implementations of the Valve interface. |
This package contains a variety of small Valve implementations that do
not warrant being packaged separately. In addition, there is a convenience
base class (ValveBase
) that supports the usual mechanisms for
including custom Valves into the corresponding Pipeline.
Other packages that include Valves include
org.apache.tomcat.logger
and
org.apache.tomcat.security
.
|
||||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES |


Served with Raw Jelly Beans
What's your spaghetti policy?
This is ham soaked in rum, it's loaded with booze


Multi-talented architect and software developer. Experience in hybrid mobile development, front-end development, back-end development with the serverless framework and hardware/electronics design
Work history
5-2019 - Present
Chief Technology Officer
Next-E, Monterrey, Nuevo Leon
- Architect and develop a scalable RESTFUL API using JavaScript, Python and the serverless framework.
- Developed a flutter mobile app that communicates with custom Developed hardware through BLE.
- Ensure best practices (code review and testing) are followed.
- Test out and incorporate cutting edge technologies to increase productivity and the efficiency of our processes.
5-2018 - 8-2018
Engineering Practicum Intern
Google, LLC, Sunnyvale, CA
- Created an advanced search feature for a launch assistant application used in Google Cloud Platform.
- Created a UI component and the corresponding client-server integration for filtering on date. Both single date and date ranges were supported, and shortcuts were included.
- Enhanced server functionality by moving to an AST query structure and refactoring existing code to add support for complex filtering criteria.
- Got familiarized with Agile software development and unit testing.
Education
8-2016 - Present
Bachelor of Science: Digital Systems and Robotics
Instituto Tecnológico Y De Estudios Superiores De Monterrey
8-2013 - 5-2015
Mechatronics Technician
CBTis 271 - Ciudad Victoria, Tamaulipas
Projects
DO>
A to-do list app written in Flutter.
- Designed the UI for the app using Adobe XD and built it with flutter.
- Followed the Bloc architecture pattern to build a scalable and testable app.
- Used firestore, storage buckets and firebase functions by firebase as database, media storage and serverless functions respectively.
- The code is open source and available on Github.
EEE LARC open 2018
An autonomous robot that simulates a maritime port
- Designed a custom pcb using smd components which included and embedded Atmega microcontroller running the arduino bootloader.
- The robot won first International place at Robocop Brazil 2018.


august 29, 2015
Thus far in my meager programming career, this is how regular expressions have appeared to me:
/\%p.(d?\)~*//.w\%b
...baaasically gibberish. I know understanding them will be important though, so that's why I've chosen to make regexes the topic of my tech blog this week. Please don't be surprised if information here is lacking or blatantly incorrect.
In the most basic sense, regexes search for patterns in text. Imagine a really powerful find-and-replace tool. Something cool that I just learned: once you master the regex syntax, it's generally the same across programming languages. Actually, you can even use them outside of programming in many text-editors. So they're a nifty tool to have at your disposal.
 The reason that regexes sometimes look like gibberish is they employ several special characters that take on new meanings within an expression. Basic code characters like: $ . | * + ? /
don't behave the same in the context of a regular expression. In order to use one of these characters without its special regex meaning (say, if we wanted to search for a literal question mark, rather than apply the question mark's meaning to our expression), we have to use a backslash (\
) to escape the character.
Right. Before we can really jump into examples, we need to lay down some basics. Here's a handy syntax chart. (This is not an exhaustive list, but I've linked the image to the full cheat sheet for future reference.)
Keep in mind that we are searching for patterns. So when we write a regular expression, we're essentially defining the search parameters and telling the system what matches we want returned. Here's a super basic example:
We can search the string "12 cats, 8 dogs, 14 elephants" for the letter "d" by simply writing:
/d/
==> "12 cats, 8 dogs, 14 elephants"
Adding a backslash to escape that "d" changes the entire meaning of the regex:
/\d/
==> "12 cats, 8 dogs, 14 elephants"
Now, instead of searching for the letter "d", we're searching for numerical digits, because the letter d has a special meaning within the confines of a regular expression.
That was a very simplified example, but as you can imagine, this gets really complicated, really fast. Of course you can stack different types of regex symbols (anchors, character classes, quantifiers, etc) together to create super complex and specific searches (that don't look all that different from my gibberish example!) and pair them with substitution methods to replace those patterns with new material. Regexes are also a powerful tool for developers to check the validity of user inputs. For example: have you ever wondered about the inner workings of the dreaded password complexity tests?
Take a look at this regex that checks a password string based on its strength criteria like uppercase, special case, and length:
^(?=.*[A-Z].*[A-Z])(?=.*[!@#$&*])(?=.*[0-9].*[0-9])(?=.*[a-z].*[a-z].*[a-z]).{8}$
Here it is broken down:
^ |
Start anchor |
(?=.*[A-Z].*[A-Z]) |
Ensure string has two uppercase letters. |
(?=.*[!@#$&*]) |
Ensure string has one special case letter. |
(?=.*[0-9].*[0-9]) |
Ensure string has two digits. |
(?=.*[a-z].*[a-z].*[a-z]) |
Ensure string has three lowercase letters. |
.{8} |
Ensure string is of length 8. |
$ |
End anchor. |
Regular expressions definitely need to be practiced and experimented with (sort of like learning a new language). We've barely even scratched the surface of this powerful tool, but I'm looking forward to finding excuses to play around with them more.


| Song A1 | Song A2 | Song B1 | Song B2 | Artist 1 | Artist 2 | Music Cat | Archived | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ${jbselection.A1Song} | ${jbselection.A2Song} | ${jbselection.B1Song} | ${jbselection.B2Song} | ${jbselection.Artist1} | ${jbselection.Artist2} | ${jbselection.MusicCategory} | ${jbselection.Archived} |


{{{edit-button 'writing_web.html' 'section'}}} {{#markdown}} ## Web content: creating and editing
This page will tell you who creates content, where you must go to create or edit content, and how to use the editors.
Content on the EMBL-EBI website is entered in one of two places: a programatically accesible content database and the www.ebi.ac.uk website itself.
The content database focuses on descriptions of people and resources, which may appear in many places on the website, e.g.:
Login at content.ebi.ac.uk/login with your Shibboleth credentials
The www.ebi.ac.uk website contains mainly static and not reusable content, e.g.:
Login at www.ebi.ac.uk/login with your Shibboleth credentials
[Resides on Content database]
All information about people - whether they are staff or visiting speakers - lives in the content database.
Updating information about staff is fairly self-explanatory, but the biography field should be approached carefully. People often participate in events, which will have an event web page. That page will pull photos and bios from the content database, and display them all together. This presents a good opportunity to check bios for consistency, in terms of grammar, length and the order in which work experience is presented. For an example, see the BioBeat 2015 event page.
[Resides on Content database]
Information about resources is displayed in many different places on the website, for example in result lists. Filling in and updating information about a resource in the content database is fairly self explanatory.
The "Popular" box determines whether a resource goes to the top of a list on various pages. The Cluster Heads determine whether a resource is "popular".
Be sure to enter in three contacts if you can - these are the people the web team or others might contact if something does not seem right, or has gone out of date.
Resource descriptions also include links to modules in Train Online - do update these when you create new online courses.
The "maintainer" is the person who will actually be updating these descriptions.
[Resides on Content database]
Team descriptions show up on GTL pages, among other places. Updating this content is fairly self-explanatory.
[Resides on www.ebi.ac.uk webiste]
Service teams are warmly encouraged to enter service updates, like new releases and spotlight features, into the EMBL-EBI website. Using this feature showcases our diverse services in one place (and one RSS feed) and makes it easier to send content directly to the @EMBLEBI Twitter feed. It is also a good way to alert External Relations of important developments, which may require further promotion.
If you do not create service news often, it is a good idea to sit with someone in External Relations who can walk you through it. There are plenty of things to watch out for with this content type, for example the Summary field is for text that shows up on lists, but this is not open by default. Stuff like that.
Questions to address before you get started:
Things to look out for with this content type:
External Relations is very happy to help you put together service news items! E-mail<EMAIL_ADDRESS>[Resides on www.ebi.ac.uk webiste]
If you do not add new pages often to the EMBL-EBI website, it is best to sit with someone in the External Relations or Web Development team, as there are many quirks and pitfalls to watch out for. For example, click Save often - and look out for that button shifting about.
After you create a piece of content, be sure to Save the Panel. Be careful here - the Save box shifts about.
Style the boxes by selecting the cog on the right of each small 'pane'.
To style a box as "Intro" (grey, with a big, underlined title), edit the content. For the pane title itself, put <none> in the box. In the WYSIWYG write the title and style it <h2>. Save it. In the panel view, click the cog on the right of the box and select the Intro style.
If you are making sidebar content, use Shortcuts-transparent.
{{/markdown}}


This plugin allows notifications to be sent through Pushover. The targets are user Pushover User Keys or Group Keys (useful if you want to deliver the message to more than one user).
Remember to configure your API Token in System Configuration.