Page maintained by Ke-Sen Huang.
If you have additions or changes, send an e-mail.
Note that when possible I link to the page containing the link to the actual PDF or PS of the preprint.
I prefer this as it gives some context to the paper and avoids possible copyright problems with direct linking.
Thus you may need to search on the page to find the actual document.
Changelog
Radiance Transfer
- Precomputed Radiance Transfer for Real-time Indirect Lighting using A Spectral Mesh Basis
-
Rui Wang,
Jiajun Zhu,
Greg Humphreys
- Large-Scale Data Management for PRT-Based Real-Time Rendering of Dynamically Skinned Models
-
Wei-wen Feng,
Liang Peng,
Yuntao Jia,
Yizhou Yu
- Precomputed Radiance Transfer for Dynamic Scenes Taking into Account Light Interreflection
-
Kei Iwasaki,
Yoshinori Dobashi,
Fujiichi Yoshimoto,
Tomoyuki Nishita
Shadows
- Pixel-Correct Shadow Maps with Temporal Reprojection and Shadow Test Confidence
-
Daniel Scherzer,
Stefan Jeschke,
Michael Wimmer
- Convolution Shadow Maps
-
Thomas Annen,
Tom Mertens,
Philippe Bekaert,
Hans-Peter Seidel,
Jan Kautz
- Interactive Illumination with Coherent Shadow Maps
-
Tobias Ritschel,
Thorsten Grosch,
Jan Kautz,
Stefan Muller
Tracing Rays and Beams
- Ray Tracing Dynamic Scenes using Selective Restructuring
-
Sung-Eui Yoon,
Sean Curtis,
Dinesh Manocha
- A Real-time Beam Tracer with Application to Exact Soft Shadows
-
Ryan Overbeck,
Ravi Ramamoorthi,
William R. Mark
- Whitted Ray-Tracing for Dynamic Scenes using a Ray-Space Hierarchy on the GPU
-
David Roger,
Ulf Assarson,
Nicolas Holzschuch
Camera, Sound and Painting
- Instant Sound Scattering
-
Nicolas Tsingos,
Carsten Dachsbacher,
Sylvain Lefebvre,
Matteo Dellepiane
- General Linear Cameras with Finite Aperture
-
Andrew Adams,
Marc Levoy
- Scene Collages and Flexible Camera Arrays
-
Yoshikuni Nomura,
Li Zhang,
Shree Nayar
- Dynamic point distribution for stroke-based rendering
-
David Vanderhaeghe,
Pascal Barla,
Joëlle Thollot,
François Sillion
Materials I
- Efficient Rendering of Human Skin
-
Eugene d'Eon,
David Luebke,
Eric Enderton
- Dirty Glass: Rendering Contamination on Transparent Surfaces
-
Jinwei Gu,
Ravi Ramamoorthi,
Peter Belhumeur,
Shree Nayar
- Material Based Splashing of Water Drops
-
Kshitiz Garg,
Shree K. Nayar,
Gurunandan Krishnan
Materials II
- Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination
-
Wan-Chun Ma,
Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss,
Paul Debevec
- Microfacet Models for Refraction through Rough Surfaces
-
Bruce Walter,
Stephen R. Marschner,
Hongsong Li,
Kenneth E. Torrance
- Efficient Basis Decomposition for Scattered Reflectance Data
-
Pete Weistroffer,
Kristen Walcott,
Greg Humphreys,
Jason Lawrence
- Procedural Editing of Bidirectional Texture Functions
-
Gero Müller,
Ralf Sarlette,
Reinhard Klein
Scattering and Translucency
- Rendering Random Media Using Precomputed Scattering Solutions
-
Jonathan T. Moon,
Bruce Walter,
Stephen R. Marschner
- Rendering Translucent Materials using Photon Diffusion
-
Craig Donner,
Henrik Wann Jensen
- Physically Based Real-Time Translucency for Leaves
-
Ralf Habel,
Alexander Kusternig,
Michael Wimmer
- Soft shadows by Ray Tracing Multilayer Transparent Shadow Maps
-
Feng Xie,
Eric Tabellion,
Andrew Pearce
Illumination and Perception
- Incremental Instant Radiosity for Real-Time Indirect Illumination
-
Samuli Laine,
Hannu Saransaari,
Janne Kontkanen,
Jaakko Lehtinen,
Timo Aila
- Photorealistic Image Rendering with Population Monte Carlo Energy Redistribution
-
Yu-Chi Lai,
ShaoHua Fan,
Stephen Chenney,
Charles R. Dyer
- An Interactive Perceptual Rendering Pipeline using Contrast and Spatial Masking
-
George Drettakis,
Nicolas Bonneel,
Carsten Dachsbacher,
Sylvain Lefebvre,
Michael Schwarz,
Isabelle Viaud-Delmon
Images and Videos
- Natural Image Colorization (Project)
-
Qing Luan,
Fang Wen,
Daniel Cohen-Or,
Lin Liang,
Ying-qing Xu,
Heung-Yeung Shum
- High Dynamic Range Image Hallucination (TR)
-
Lvdi Wang,
Li-Yi Wei,
Kun Zhou,
Baining Guo,
Heung-Yeung Shum
- Using Photographs to Enhance Videos of a Static Scene
-
Pravin Bhat,
C. Lawrence Zitnick,
Noah Snavely,
Aseem Agarwala,
Maneesh Agrawala,
Michael Cohen,
Brian Curless,
Sing Bing Kang
Trees, Shells and Flows
- Compressed Random-Access Trees for Spatially Coherent Data
-
Sylvain Lefebvre,
Hugues Hoppe
- Interactive Smooth and Curved Shell Mapping
-
Stefan Jeschke,
Stephan Mantler,
Michael Wimmer
- Feature-Guided Dynamic Texture Synthesis on Continuous Flows
-
Rahul Narain,
Vivek Kwatra,
Huai-Ping Lee,
Theodore Kim,
Mark Carlson,
Ming Lin
kesen.huang@gmail.com


Symbol Index for "Location Structure Type"
Symbol Index for diagram
"Location Structure Type"
AC/Direction (Generalization Node)
Associative Category (Class)
classifier (Association)
Incident Involvement Type (Class)
Inter Subject Relationship Type (Class)
Location Structure (Class)
Location Structure Type (Class)
Phenomenon Qualifier Type (Class)
Subject Location Type (Class)
Temporal Link Type (Class)


Excerpt from "Leave"
She IM'd him to have a one-on-one
discussion of the sort that runs one way.
"I'll meet you in Crayon in five."
The building's smaller conference rooms
("larger" here means 12 x 12 or more)
seldom have conferences, whatever that
word might mean—he'd never seen one happen.
Mt Porte Crayon Meeting Room, its full name,
(the room names match the P1000 peaks
in West Virginia, 44 in all)
is nearest to the elevator hall
and has more privacy despite the glass
door and triple-paned windows. The view
is dense with bushes, oval waxy leaves,
and a sloping parking lot (its striping paint
bright and clean in the slow autumn, clumsy sun).
The room's east/west left/right untextured walls
are two shades less than avocado green.
The vinyl chairs complement with a thin,
dull algae hue. The two opposing high
thin walls, without pretense, are sand dune white.
Early per usual he sat and traced
the heat-shimmer edged shadows slanting
across the table—slight burled birch-veneered
—and dulled through the clanging, thin metal blinds
The Polycom unplugged and dusty, smelled
of fresh Saltines.
She entered harried, tight:
a magnet coiled with "Sorry! :)" pleasantries
(her office on this floor and one row down).
If middle management, you place your iPhone
near where your wrists will rest while talking slow.
The managers are taught a dialect
that has a sour-sounding lilt. It's meant
to sooth the employee. You also learn
to posture-check yourself ("for some, your back
might hurt at first") and "I-to-eye" contact.
He rubbed his bitten fingernails (one bled
a bit this morning on the way to work).
After the half-expected work-life bal-
ance talk, or rather "talking-to", she made
her point: he had to use his PTO.
This year, he would. She checked her notepad (yes,
despite what century we're living in
she takes her meeting notes in longhand, tall
wide letters she circles and underlines)
and found the page: I'm glad to hear it. Laurence. Yes—
(Law- or Laurence, pronounced the same but spelled
however the barista feels that day;
he gave up spelling out the name last year.)
the month of April rarely gets approved.
The IRM's the same as US Code
so in this case tradition matters most.
August? May, September can be nice.
The Sharepoint site would have the forms
and someone had the room booked after them.
He shifted, looked outside. A heavy lull
began to fill the room. She hemmed again.
She'd check the lifecycle (their certified
approach to IT service management,
called ITIL version 3, or "eye-till three")
was current, practical and common sense.
The data service RESTful API
was documented well and April c/sh-ould be fine.
About one week, including the 15th.
You'll need to take a laptop, check your mail,
be—business hours—available by phone.
So what're your (grabbing iPhone) plans?
Expedia and Travelocity
had zero flights from HGR to DBQ.
(On Travelocity that search returned
a message worded: "problem processing,"
etc. The debug console showed
a client-side request timeout and fail.
More googling, he found a route that worked
(the airport codes return the best results).
From IAD to MLI, with one
connecting stop at DTW.
The cheapest Delta flight from Apr 10
to 17. Reserved the rental car
with upgrade (special season package deal).
Four-door Corolla. Manual windows. White.
Few names of cars seem well thought out.
Corolla: petals of a flower form
a whorl within the sepals and enclose
the reproductive organs. Focusing,
he checked the inkjet, pulled up Google maps:
Maquoketa, the caves where Norris Brown
had felt the weight of equal stones.
Reprinted from <The Rotary Dial, August 2013.


{% extends 'base.html' %} {% block title %} Home {% endblock %} {% block content %}
Our server has unique pp formula for relax and autopilot. Good at waving your cursor or just smashing your keyboard? Join {{ appName() }} and get a fair reward for your skills!
You won't have to pay for osu!supporter to have a convenient gameplay experience in the client - osu!direct is available for everyone! Keep in mind supporting us still grants a huge set of cool perks but absense of it doesn't affect your game sessions!
We're written on top of cmyui's multipurpose library allowing for a fully asynchronous environment, both improving performance and the user's experience in the long term!


Here you'll find a test-sender page at Sender Test
And the receiver for your chromecast at Castronaut Receiver
Everything is served via an NodeJS server that handles meta-data, subtitle-conversion etc. Check out the repo for details.


{% load staticfiles %}
FarmBuddy
Farm3UDDY
In search of lands?
Not getting good price for your land?
Then you are at right place.
REGISTER NOW!
Log in
We help you to get the best deals.
Best customer support so that you don't get stuck.
Reliable network of farmers and landlords at one place.
REGISTER NOW!
Get land rate details per day so that you don't miss any best deal.
GET STATISTICS!
Farm3uddy
- AN INITIATIVE TO PUT FARMER AND LANDLORD CLOSER -


WITNESS LIST Economic Development Committee (Subcommittee on Subcommittee on House Bill 39) February 27, 1997-3:30P HB 39 For: Polland, Micheal (TLIA/TLROA) On: Davis, Will D. (TLIA/TLROA) Thompson, Jay (TLIA/TLROA)


Devang is a recent graduate from the Master of Telecommunications and Information Security program at the University of Victoria. Devang has over 3.5 years of experience in the Telecommunications and Cyber Security Industry, and is skilled in wireless communications and cloud computing technologies. Before joining the MTIS graduate program at UVic, Devang also completed a Post-Graduate programin Embedded Systems Design, fueling his passion for IoT and Cybersecurity. Along with his dream to make the world a smarter place, he also believes in value of good home-cooking and enjoys his time behind the stovetopas much as his time behind his computer screen.
Software engineer primarily responsible for feature development and testing of Service Capability Exposure Function (SCEF) – A control plane Network Function (NF) introduced by 3GPP in Release 13.
Associate consultant for a cyber risk and services team, responsible for ensuring Identity and Access Management (IAM) service availability and enhancements.
Part-time sales associate responsible for customer interaction and providing them with a tailored solution based on their needs and requirements.


Full-Stack Web Developer with background running live events that helped develop extensive multi-tasking skills, as well as the ability to understand and execute the bigger picture. The love I have for technology and desire for continuous growth drove me towards web development. The ever advancing tools and languages fascinate and excite me for the possibilities of the future. I attended the University of Denver for Web Development and expanded my knowledge of React, MongoDB, and Node.js allowing my visions to become a reality. My live events skills combined with my web development skills make me a valuable developer by entwining an engineering mindset with a drive for creative innovation, allowing me to consistently push boundaries and redefine what is possible.
ProjectsFull Stack React Application that brings skiers and snowboarders together.
React Application where users play a memory game.
Full Stack Application that scrapes news articles and also allows users to save and leave comments on each article.
Full Stack RPG where a cat travels through space conquering exoplanets discovered by NASA.
Full Stack Application utilizing Node.js, Express, MySQL, Handlebars, and a custom ORM in an MVC design pattern.
Full Stack Application that takes in user results from surveys and compares them to other users.
A Cannabis Exploration Website.
Node.js application utilizing MySQL to create a command line storefront with multiple interfaces.
Command line application that takes in multiple commands and returns relevant data.
Application that generates GIFs based on user input.


| CONTENTS |
Making copies of critical files in case the originals become inaccessible is called backing them up or making backups. Backups are insurance. They are time and effort you spend protecting yourself from things that might never happen. Your hard drive might never crash, but what vital things would you lose if it did?
Exactly what "making a backup" means varies depending on your circumstances. All of the following examples are ways to make backups applicable to some specific environment:
Copying some files onto another disk on the same machine, so that if one hard drive dies you still have a copy. (A more sophisticated and automatic way of doing this, which you may have heard about, is called Redundant Array of Inexpensive Disks or RAID.)
Making a compressed tar file and copying it to another machine, so that if one machine crashes you still have a copy.
Writing copies of your files to a Zip drive, CD-RW, or DVD-RW.
tarring (Section 38.2) files to a tape.
Nightly automatic backups of everything that's changed that day (called an incremental backup) to multiple tapes, with copies of the tapes stored in offsite secure storage.
If you are just trying to protect your files on your personal machine, simply making sure that critical files have copies on multiple physical disks or occasionally copying files onto another machine or removable storage is probably sufficient. If you're administering a machine that has multiple users, regular backups are almost certainly a necessity. If those users are doing business-critical tasks, very regular backups and off-site copies are a requirement to protect the investment of time involved.
DJPH
When many Unix users think of file archives, on tape or in an archive file, they think of the tar utility. There are other ways to make archives and handle tapes including dump and dd. This article summarizes articles about tar in this chapter and others.
Although tar is a tape archiver, one of its common uses is making an archive file on disk (Section 39.2). Because tar "pads" its archives with NUL characters, on-disk tar archive files can be much bigger than the size of the individual files put together. Both to fix that and generally to save space, tar files are often compressed. The GNU tar (Section 39.3) can compress files while storing them and uncompress them while reading them, automatically. If you don't have GNU tar, you may need to uncompress an archive manually. Note that a compressed tar archive can take less disk space (Section 15.7) than compressing individual small files.
Because tar keeps most of a file's inode information, it can make a more complete copy (Section 10.13) of a file or directory tree than utilities such as cp.
Yes, we do have articles about archives on tape. Section 38.3 has enough information to make your own archive, although you might need the details from Section 38.5, too. After you've made an archive, you'll probably want to restore it, at least as a test to be sure your archive is okay. Section 38.6 explains how.
If there isn't a tape drive on your computer, read Section 38.7 about using a drive on another computer.
tar copies a directory tree, recursively, from top to bottom. What if you don't want to archive everything? You can back up just some files by combining ls -lt and find. Some versions of tar have options for including or excluding certain files and directories (Section 39.3).
JP
As someone who has been an end user and a system administrator, I strongly believe that every user should understand the importance of backups.
|
Accidents and oversights happen. Tapes can be damaged, lost, or mislabeled. Assume that your system administrator is top-notch. The best administrator can recover your lost data 99 percent of the time. There is still a small chance that the files you need might not be recovered. Can you afford to duplicate months of effort 1 percent of the time? No.
An experienced user learns to be pessimistic. Typically, this important perspective is learned the hard way. Perhaps a few hours are lost. Perhaps days. Sometimes months are lost.
Here are some common situations:
A user works on a file all day. At the end of the day, the file is deleted by accident. The system manager cannot recover the file. A day's work has been lost.
A programmer tries to clean up a project directory. Instead of typing rm *.o the programmer types rm * .o and the entire directory is lost.
A user deletes a file by accident. After a few days, the user asks the system administrator to recover the file. The incremental backup system has reused the only tape the missing file was on.
A large project is archived on a magnetic tape and deleted from the disk. A year later, some of the information is needed. The tape has a bad block at the beginning. The system manager must learn how to recover data from a bad tape. The attempt is often unsuccessful. The information is lost forever, and must be re-created at the cost of months of effort.
Someone breaks into a computer and alters or deletes crucial information.
A fire breaks out in the computer room. The disks and all of the backup tapes are lost.
Gulp! I scared myself.
Ah! I feel better now. As I was saying, being pessimistic has its advantages.
Making a backup is easy. Get a blank tape and put a label on it. Learn how to load it into the tape drive. Then do the following:
% cd % tar c .
Take the tape out. Write-protect the tape (usually, just slide the tab). That's all.
[Well, okay, not exactly. That would back up only your home directory to the default tape device (usually something like /dev/rmt0). You may want to back up more than just your home directory, the tape drive may not be at the default device, and you may not have permission to write to the tape drive by default. The rest of the chapter talks about variations on the theme. DJPH]
BB
Section 38.3 explains the minimal basics of using tar to make backups, but there are lots of variations that can be very useful.
To create a tar archive for copying to another disk or another machine:
% tar cvf 20020214-book.tar ./book
tar 's c option stands for create, v for verbose, and the f option for file. 20020214-book.tar is the new archive file to create, and ./book says to archive the directory book in the current directory. Once you have an archive, you might want to compress it to save space. gzip and bzip2 are your best bets. (I use bzip2 here largely because it tends to give better compression, but be aware that gzip is more widely available and thus may be safer for backups.) You can compress it once you've made it:
% ls -l 20020214-book.tar -rw-r--r-- 1 deb deb 19415040 Feb 14 23:15 20020214-book.tar % bzip2 20020214-book.tar % ls -l 20020214-book.tar.bz2 -rw-r--r-- 1 deb deb 4033775 Feb 14 23:15 20020214-book.tar.bz2
Or you can compress it as you make it. GNU tar supports gzip compression on the fly with the z or - -gzip options and bzip2 compression on the fly with the - -bzip2 option, or you can pipe into gzip or bzip2:
% tar czvf 20020214-book.tar.gz ./book % tar cvf 20020214-book.tar.bz2 --bzip2 ./book % tar cvf - ./book | bzip2 > 20020214-book.tar.bz2
Section 39.2 and Section 39.3 have more information on using tar.
You can get more protection from certain kinds of mishaps by using a version control system like RCS (Section 39.5) or CVS (Section 39.7) to save every version of a file you are updating frequently. While it doesn't protect you from disk crashes, a version control system provides the ability to back up to a previous version if something gets changed or deleted incorrectly.
DJPH
This article was written for Linux systems, but the advice applies everywhere. You may need to make some adjustments in the names of the tape drive devices and some filesystem directories, for instance. If you're making personal backups (of the files on your account, for instance), you can substitute your directory names for the system directories covered here, but the command names and techniques won't change.
As Section 38.3 says, the simplest way to make a backup is to use tar to archive all the files on the system or only those files in a set of specific directories. Before you do this, however, you need to decide what files to back up. Do you need to back up every file on the system? This is rarely necessary, especially if you have your original installation disks or CD-ROM. If you have made specific, important changes to the system, but everything else could simply be reinstalled in case of a problem, you could get by archiving only those files you have made changes to. Over time, however, it is difficult to keep track of such changes.
In general, you will be making changes to the system configuration files in /etc. There are other configuration files as well, and it can't hurt to archive directories such as /usr/local (where various packages generally get installed) and /usr/X11R6/lib/X11 (which contains the X Window System configuration files). You may want to do filtering on these directories and back up only the configuration files, since binaries in /usr/local and things like fonts in the X11 distribution can be reinstalled from their original packages easily enough.
You should also back up your kernel sources (if you have patched your kernel sources); these are found in /usr/src/linux (/usr/src/sys on *BSD). At the very least, you'll want to back up your kernel configuration file if you've built your own kernel; it's in /usr/src/linux/.config (or /usr/src/sys/platform/conf/KERNELNAME on *BSD).
It's a good idea to keep notes on what features of the system you've changed so you can make intelligent choices when making backups. If you're truly paranoid, go ahead and back up the whole system: that can't hurt, but the cost of backup media might.
Of course, you should also back up the home directories for each user on the system; these are generally found in /home. If you have your system configured to receive electronic mail, you might want to back up the incoming mail files for each user. Many people tend to keep old and "important" electronic mail in their incoming mail spool, and it's not difficult to accidentally corrupt one of these files through a mailer error or other mistake. These files are usually found in /var/spool/mail.
Assuming you know what files or directories to back up, you're ready to roll. The tar command can be used directly, as we saw in Section 39.2, to make a backup. For example, the command:
tar cvf /dev/rft0 /usr/src /etc /home
archives all of the files from /usr/src, /etc, and /home to /dev/rft0. /dev/rft0 is the first "floppy-tape" device that is, for the type of tape drive that hangs off of the floppy controller. Many popular tape drives for the PC use this interface. If you have a SCSI tape drive, the device names are /dev/st0, /dev/st1, and so on, based on the drive number. Those tape drives with another type of interface have their own device names; you can determine these by looking at the documentation for the device driver in the kernel.
You can then read the archive back from the tape using a command such as:
tar xvf /dev/rft0
This is exactly as if you were dealing with a tar file on disk, as in Section 39.2.
When you use the tape drive, the tape is seen as a stream that may be read from or written to in one direction only. Once tar is done, the tape device will be closed, and the tape will rewind (if you're using the default tape device; see below on how to prevent this). You don't create a filesystem on a tape, nor do you mount it or attempt to access the data on it as files. You simply treat the tape device itself as a single "file" to create or extract archives from.
Be sure your tapes are formatted before you use them if you are using a tape drive that needs it. This ensures that the beginning-of-tape marker and bad-blocks information has been written to the tape. At the time of this writing, no tools exist for formatting QIC-80 tapes (those used with floppy tape drivers) under Linux; you'll have to format tapes under MS-DOS or use preformatted tapes.
Creating one tar file per tape might be wasteful if the archive requires a fraction of the capacity of the tape. To place more than one file on a tape, you must first prevent the tape from rewinding after each use, and you must have a way to position the tape to the next "file marker," both for tar file creation and for extraction.
The way to do this is to use the nonrewinding tape devices, which are named /dev/nrft0, /dev/nrft1, and so on for floppy-tape drivers, and /dev/nrst0, /dev/nrst1, and so on for SCSI tapes. When this device is used for reading or writing, the tape will not be rewound when the device is closed (that is, once tar has completed). You can then use tar again to add another archive to the tape. The two tar files on the tape won't have anything to do with each other. Of course, if you later overwrite the first tar file, you may overwrite the second file or leave an undesirable gap between the first and second files (which may be interpreted as garbage). In general, don't attempt to replace just one file on a tape that has multiple files on it.
Using the nonrewinding tape device, you can add as many files to the tape as space permits. To rewind the tape after use, use the mt command. mt is a general-purpose command that performs a number of functions with the tape drive. For example, the command:
mt /dev/nrft0 rewind
rewinds the tape in the first floppy-tape device. (In this case, you can use the corresponding rewinding tape device as well; however, the tape will rewind just as a side effect of the tape device being closed.)
Similarly, the command:
mt /dev/nrft0 reten
retensions the tape by winding it to the end and then rewinding it.
When reading files on a multiple-file tape, you must use the nonrewinding tape device with tar and the mt command to position the tape to the appropriate file.
For example, to skip to the next file on the tape, use the command:
mt /dev/nrft0 fsf 1
This skips over one file on the tape. Similarly, to skip over two files, use:
mt /dev/nrft0 fsf 2
Be sure to use the appropriate nonrewinding tape device with mt. Note that this command does not move to "file number two" on the tape; it skips over the next two files based on the current tape position. Just use mt to rewind the tape if you're not sure where the tape is currently positioned. You can also skip back; see the mt manual page for a complete list of options.
You need to use mt every time you read a multifile tape. Using tar twice in succession to read two archive files usually won't work; this is because tar doesn't recognize the file marker placed on the tape between files. Once the first tar finishes, the tape is positioned at the beginning of the file marker. Using tar immediately will give you an error message, because tar will attempt to read the file marker. After reading one file from a tape, just use:
mt device fsf 1
to move to the next file.
Just as we saw in the last section, the command:
tar cvf /dev/fd0 /usr/src /etc /home
makes a backup of /usr/src, /etc, and /home to /dev/fd0, the first floppy device. You can then read the backup using a command such as:
tar xvf /dev/fd0
If we use /dev/hdd instead of /dev/fd0 (and our Zip drive is the slave drive on the second IDE controller), we'll be writing to and reading from a Zip disk instead of a floppy. (Your device name may vary depending on your OS.) Because floppies and Zip disks have a rather limited storage capacity, GNU tar allows you to create a "multivolume" archive. (This feature applies to tapes as well, but it is far more useful in the case of smaller media.) With this feature, tar prompts you to insert a new volume after reading or writing each disk. To use this feature, simply provide the M option to tar, as in:
tar cvMf /dev/fd0 /usr/src /etc /home
Be sure to label your disks well, and don't get them out of order when attempting to restore the archive.
One caveat of this feature is that it doesn't support the automatic gzip compression provided by the z option. However, there are various reasons why you may not want to compress your backups created with tar, as discussed later. At any rate, you can create your own multivolume backups using tar and gzip in conjunction with a program that reads and writes data to a sequence of disks (or tapes), prompting for each in succession. One such program is backflops, available on several Linux distributions and on the FTP archive sites. A do-it-yourself way to accomplish the same thing would be to write the backup archive to a disk file and use dd or a similar command to write the archive as individual chunks to each disk. If you're brave enough to try this, you can figure it out for yourself. [Aw, come on, guys, have a heart! (Psst, readers: look at the end of Section 21.9.) JP]
There are good arguments both for and against compression of tar archives when making backups. The overall problem is that neither tar nor gzip is particularly fault-tolerant, no matter how convenient they are. Although compression using gzip can greatly reduce the amount of backup media required to store an archive, compressing entire tar files as they are written to floppy or tape makes the backup prone to complete loss if one block of the archive is corrupted, say, through a media error (not uncommon in the case of floppies and tapes). Most compression algorithms, gzip included, depend on the coherency of data across many bytes to achieve compression. If any data within a compressed archive is corrupt, gunzip may not be able to uncompress the file at all, making it completely unreadable to tar. The same applies to bzip2. It may compress things better than gzip, but it has the same lack of fault-tolerance.
This is much worse than if the tar file were uncompressed on the tape. Although tar doesn't provide much protection against data corruption within an archive, if there is minimal corruption within a tar file, you can usually recover most of the archived files with little trouble, or at least those files up until the corruption occurs. Although far from perfect, it's better than losing your entire backup.
A better solution would be to use an archiving tool other than tar to make backups. There are several options available. cpio (Section 38.13) is an archiving utility that packs files together, much like tar. However, because of the simpler storage method used by cpio, it recovers cleanly from data corruption in an archive. (It still doesn't handle errors well on gzipped files.)
The best solution may be to use a tool such as afio. afio supports multivolume backups and is similar in some respects to cpio. However, afio includes compression and is more reliable because each individual file is compressed. This means that if data on an archive is corrupted, the damage can be isolated to individual files, instead of to the entire backup.
These tools should be available with your Linux distribution, as well as from all of the Internet-based Linux archives. A number of other backup utilities, with varying degrees of popularity and usability, have been developed or ported for Linux. If you're serious about backups, you should look into them.[1]
MW, MKD, and LK
When you create an archive, there are several ways to specify the directory. If the directory is under the current directory, you could type:
% tar c project
A similar way to specify the same directory is:
% tar c ./project
If you are currently in the directory you want archived, you can type:
% tar c .
Another way to archive the current directory is to type:
% tar c *
Here, the shell expands the asterisk (*) to the files in the current directory. However, it does not match files starting with a dot (.), which is why the previous technique is preferred.
This causes a problem when restoring a directory from a tar archive. You may not know whether an archive was created using . or the directory name.
I always check the names of the files before restoring an archive:
% tar t
If the archive loads the files into the current directory, I create a new directory, change to it, and extract the files.
If the archive restores the directory by name, then I restore the files into the current directory.
If you want to restore a single file, get the pathname of the file as tar knows it, using the t flag. You must specify the exact filename, because filename and ./filename are not the same to tar. You can combine these two steps into one command by using:
% tar xvf /dev/rst0 `tar tf /dev/rst0 | grep filename`
Note that this may run very slowly, though, as the entire tar file has to be read once (and the tape rewound) before any restoration can happen. Be careful: you may also get a lot more than you expected; for example, if you're looking for README using this technique, you'd also get README.Solaris and everything in the doc/READMEs directory, possibly overwriting files you wanted to keep.
Whenever you use tar to restore a directory, you must always specify some filename. If none is specified, no files are restored.
There is still the problem of restoring a directory whose pathname starts with a slash (/). Because tar restores a file to the pathname specified in the archive, you cannot change where the file will be restored. The danger is that either you may overwrite some existing files or you will not be able to restore the files because you don't have permission.
You can ask the system administrator to rename a directory and temporarily create a symbolic link pointing to a directory where you can restore the files. Other solutions exist, including editing the tar archive and creating a new directory structure with a C program executing the chroot(2) system call. Another solution is to use GNU tar (Section 39.3), which allows you to remap pathnames starting with slash (/).
But the best solution is never to create an archive of a directory that starts with slash (/) or tilde (~) (Section 31.11) (since the shell will expand ~ into an absolute path that starts with a /).
To restore a directory from a remote host, use the following command:
rsh Section 1.21
% rsh -n host dd if=/dev/rst0 bs=20b | tar xvBfb - 20 files
This runs dd on the remote host, reading from /dev/rst0 with a blocksize of twenty blocks, and pipes it to a local tar. It is difficult to read fixed-size blocks over a network. This is why tar uses the B flag to force it to read from the pipe until a block is completely filled. Some versions of tar, including GNU tar, handle remote drives automatically (Section 38.8).
BB
If your computer doesn't have a tape drive connected, creating tar (Section 38.2) backup files is slightly more complicated. If you have an account on a machine with a tape drive, and the directory is mounted via NFS (Section 1.21), you can just rlogin (Section 1.21) to the other machine and use tar to back up your directory.
If the directory is not NFS mounted, or it is mounted but you have permission problems accessing your own files, you can use tar, rsh (Section 1.21), and dd (Section 21.6) to solve this dilemma. The syntax is confusing, but if you forget, you can use man tar (Section 2.1) to refresh your memory. The command to dump the current directory to a tape in a remote machine called zephyrus is:
% tar cvfb - 20 . | rsh zephyrus dd of=/dev/rmt0 obs=20b
Here, the output file of tar is -, which tar interprets as standard input if tar is reading an archive or standard output if tar is creating an archive.
The dd command copies data from standard input to the device /dev/rmt0.
This example assumes you can use rsh without requiring a password. You can add your current machine's name to the remote .rhosts file (Section 1.21) if you get a Password: prompt when you use rlogin to access this machine. You also can use ssh, which is generally more secure than rsh, and the ssh-agent utility to allow logins without a password.
BB
If you're using GNU tar, you can probably ignore the tips in Section 38.7 about using a tape drive on a remote system. GNU tar makes it easy to access a remote drive via rsh or a similar command like ssh.
When referring to a local host, the GNU tar f option takes a plain filename like foo.tar or a device name like /dev/rmt0. If you put a colon (:) before that name, though, you can prepend a remote hostname and, optionally, a username. For example, to get a table of contents of the tape on the drive /dev/rmt8 on the remote host server2, logging into server2 as yourself, type:
% tar tf server2:/dev/rmt8
To specify a different username than the one on your local host, add it with an @ before the hostname. (This assumes you're allowed to connect to the remote host without a password because there's a .rhosts file on the remote system, for instance.) For example, to connect to server2 as heather and extract the files reports/products.sgml and reports/services.sgml from /dev/rmt8:
{ } Section 28.4
% tar xf heather@server2:/dev/rmt8 reports/{products,services}.sgml
By default, GNU tar uses rsh, remsh, or nsh to access the remote machine, though that can be changed when tar is built and installed on your host. If you want another access command, like ssh, you can set that with the - -rsh-command option. The next example gets the contents of the archive on the drive /dev/rmt8 from the host capannole.it using ssh. Note that tar doesn't check your search path (Section 27.6) for the rsh-command; you have to give its absolute pathname (which you can get with a command like which (Section 2.6)):
% tar -x --file=capannole.it:/dev/rmt8 --rsh-command=/usr/bin/ssh
On the other hand, if you need to use a local filename with a colon in it, add the - -force-local option.
JP
As I was working on this book, I was constantly editing lots of random files all through a directory tree. I archived some of the files in a revision control system (Section 39.4), but those archives, as well as the nonarchived files, still would be vulnerable if my disk crashed. (And naturally, close to a deadline, one hard disk started making whining noises...)
The answer I came up with was easy to use and simple to set up. It's a script named ptbk, and this article explains it. To run the script, I just type its name. It searches my directory tree for files that have been modified since the last time I ran ptbk. Those files are copied into a dated compressed tar archive and copied to a remote system using scp. The process looks like this:
$ ptbk upt/upt3_changes.html upt/BOOKFILES upt/art/0548.sgm upt/art/1420.sgm upt/art/1430.sgm upt/art/0524.sgm upt/BOOKIDS upt/ulpt3_table Now copying this file to bserver: -rw-rw-r-- 1 jpeek 323740 Jan 3 23:08 /tmp/upt-200101032308.tgz upt-200101032308.tgz | 316 KB | 63.2 kB/s | ETA: 00:00:00 | 100%
The script actually doesn't copy all of the files in my directory tree. I've set up a tar exclude file that makes the script skip some files that don't need backing up. For instance, it skips any filename that starts with a comma (,). Here's the file, named ptbk.exclude:
upt/ptbk.exclude upt/tarfiles upt/gmatlogs upt/drv-jpeek-jpeek.ps upt/drv-jpeek.3l upt/BOOKFILES~ upt/ch*.ps.gz upt/ch*.ps upt/,* upt/art/,*
After the script makes the tar file, it touches a timestamp file named ptbk.last. The next time the script runs, it uses find -newer (Section 9.8) to get only the files that have been modified since the timestamp file was touched.
The script uses scp and ssh-agent to copy the archive without asking for a password. You could hack it to use another method. For instance, it could copy using rcp (Section 1.21) or simply copy the file to another system with cp via an NFS-mounted filesystem (Section 1.21).
This doesn't take the place of regular backups, if only because re-creating days' worth of work from the little individual archives would be tedious. But this system makes it painless to take snapshots, as often as I want, by typing a four-letter command. Here's the ptbk script:
|| Section 35.14, '...' Section 28.14
#!/bin/sh # ptbk - back up latest UPT changes, scp to $remhost dirbase=upt dir=$HOME/$dirbase timestamp=$dir/ptbk.last # the last time this script was run exclude=$dir/ptbk.exclude # file with (wildcard) pathnames to skip remhost=bserver # hostname to copy the files to remdir=tmp/upt_bak/. # remote directory (relative to $HOME) cd $dir/.. || exit # Go to parent directory of $dir datestr=`date '+%Y%m%d%H%M'` outfile=/tmp/upt-$datestr.tgz # Don't send vim recovery files (.*.swp): tar czvlf $outfile -X $exclude \ `find $dirbase -type f -newer $timestamp ! -name '.*.swp' -print` mv -f $timestamp $dir/,ptbk.last echo "Timestamp file for $0. Don't modify." > $timestamp echo "Now copying this file to $remhost:" ls -l $outfile scp $outfile ${remhost}:${remdir}
If the copy fails (because the remote machine is down, for instance), I have to either copy the archive somewhere else or wait and remember to copy the archive later. If you have an unreliable connection, you might want to modify the script to touch the timestamp file only if the copy succeeds at the possible cost of losing a data file that was modified while the previous archive was (not?) being transferred to the remote host.
JP
When extracting files from a tar archive, it's handy to be able to use wildcards. You have to protect them (Section 27.12) from the shell, so that they are passed directly to tar.
In general, tar can't do wildcard matching on the filenames within an archive. There's a terribly ugly hack that you can use to select the files you want anyway. Try a command like this:
'...' Section 28.14
% tar xvf /dev/rst0 `tar tf /dev/rst0 | egrep 'lib/(foo|bar)'`
What you're doing here is using tar twice. tar t will print the names of all the files on the tape. The pattern supplied to egrep (Section 13.4) selects the pathnames containg lib/foo or lib/bar, and the resulting filenames are passed to the first tar command, which actually extracts the files from the archive. Note that these patterns are regular expressions, not wildcards (Section 32.2).
Here's another subtle but important point. Because the regular expression patterns in the example above are not anchored with ^ or $ characters (Section 32.4), they can match anywhere in the file pathnames. So lib/(foo|bar) would match a pathname like lib/foo as well as a pathname like /usr/lib/glib/foo.h.
Go to http://examples.oreilly.com/upt3 for more information on: GNU tar
One of the many improvements in GNU tar is that it understands wildcards in the names of files within an archive. (Remember that because you want tar, not the shell, to see these wildcards, you have to quote (Section 27.12) the filename arguments that have wildcards.)
Unlike the examples in the previous section, GNU tar uses wildcards, not regular expressions (Section 32.2). Unlike shells, the wildcard expressions in GNU tar can match across slashes (/) in pathnames.
Here's a demonstration of using wildcards: we want to extract all subdirectories named editor. Command 1 shows how you'd do it in non-GNU tar: list the exact pathnames of the subdirectories. Notice that tar extracts the directory and any subdirectories too. Command 2 shows the easy way to do the job with GNU tar: make a wildcard expression that ends with a slash and the directory name. As before, tar extracts the directory and any subdirectories. What if you want to extract anything with the string editor in its name including individual files? Make a wildcard pattern without the slash and a filename surrounded by stars, as in command 3. Finally, command 4 shows an example of how (different than in shells) a wildcard can match across the / characters in pathnames.
There's more about wildcard matching in the GNU tar info page (but not its manpage).
One subtle (but sometimes important!) difference between GNU tar and other versions comes when you're trying to extract a file whose name contains a wildcard character. You'll probably have to type a backslash (\) before that name. Also, because the shell may also try to interpret that wildcard character or at least the backslash you also may need to quote the backslashed expression! (Section 27.18 shows a situation like this one, with multiple layers of backslash interpretation.)
Here's an example. We're archiving the system binary file named [ (or test) (Section 35.26):
5$ which [ /usr/bin/[ 6$ cd /tmp 7$ tar cvf tartest.tar /usr/bin/[ tar: Removing leading `/' from member names usr/bin/[ 8$ tar xvf tartest.tar usr/bin/[ tar: usr/bin/[: Not found in archive tar: Error exit delayed from previous errors 9$ tar xvf tartest.tar usr/bin/\[ tar: usr/bin/[: Not found in archive tar: Error exit delayed from previous errors 10$ tar xvf tartest.tar 'usr/bin/\[' usr/bin/[
Storing the file in the archive, in command 7, is simple. The shell doesn't have anything to expand; tar simply stores the name as-is because wildcards make sense only when tar is extracting from an archive. In command 8, though, when we try to extract the file into the current directory, tar says it isn't there because it's now treating [ as a wildcard pattern. Adding a single backslash in command 9 doesn't help because the shell strips it off before tar sees it. We need to put quotes around the backslash, as in command 10, to make tar match the actual name.
JP and TOR
One problem with most versions of tar: they can't change a file's pathname when restoring. Let's say that you put your home directory in an archive (tape or otherwise) with a command like this:
% tar c /home/mike
What will these files be named when you restore them, either on your own system or on some other system? They will have exactly the same pathnames they had originally. So if /home/mike already exists, it will be destroyed. There's no way to tell tar that it should be careful about overwriting files; there's no way to tell tar to put the files in some other directory when it takes them off the tape, etc.  If you use relative paths (Section 31.2) (for example, tar c .), you can restore the files in any directory you want.
This means that you should:
Avoid using absolute paths when you create an archive (see below).
Use tar t to see what files are on the tape before restoring the archive.
Use GNU tar. It strips the leading / by default when creating archives. (You can give it the -P option to make it store absolute pathnames.)
Rather than giving a command like tar c /home/mike, do something like:
% cd / % tar c home/mike
Or, even more elegant, use -C on the tar command line:
% tar c -C /home/mike .
This command tells tar to cd to the directory /home/mike before creating an archive of . (the current directory). If you want to archive several directories, you can use several -C options:
% tar c -C /home/mike ./docs -C /home/susan ./test
This command archives mike's docs directory and susan's test directory. [Note that it uses the subdirectory names, as we did in the second-previous example. When the files are extracted, they'll be restored to separate subdirectories, instead of all being mixed into the same . (current) directory. JP]
ML
tar's command line is one of Unix's little mysteries. It's difficult to associate arguments with options. Let's say you want to specify the block size (b), the output file ( f ), and an "exclude" file (X ). Where do you put all this information? It's easy enough to stick the option letters into a lump and put them into a command (tar cXbf). But where do you put the block size, the name of the exclude file, and so on?
List any arguments that you need after the block of key letters. You must place the arguments in the same order as the key letters, as shown in Figure 38-1.
In this command, keepout goes with the X option, 20 goes with the b option, and archive.shar goes with the f option. If we put the options in a different order, we also have to put the arguments in a different order (see Figure 38-2).
Note that the files you want to put on the tape (or the files you want to extract from the tape) always go at the end of the command. These are not arguments to c or X; they are part of the command itself.
The dump command and a few others work the same way.
GNU tar understands this traditional syntax as well as two syntaxes with separate options. For instance, the command line above could also be written in either of the following ways with GNU tar:
% tar -c -b 20 -X keepout -f archive.tar *.txt % tar --create --block-size=20 --exclude-from=keepout \ --file=archive.tar *.txt
ML
There was a time when people used to debate whether BSD tar (Section 38.2, Section 39.2) (tape archiver) or System V cpio (copy in/out) was the better file archive and backup program. At this point, though, no one ships out cpio archives over the Net (Section 1.21). tar is widespread, and there are free versions available, including GNU tar (Section 39.3).
There's still a good reason to use cpio: it's better at recovering backups from partially damaged media. If a block of your tape or disk archive goes bad, cpio can probably recover all files except the one with the bad block. A tar archive may not fare as well. Though we don't give it much air time in this book, here are a few cpio basics:
To write out an archive, use the -o option and redirect output either to a tape device or to an archive file. The list of files to be archived is often specified with find (Section 9.1), but it can be generated in other ways cpio expects a list of filenames on its standard input. For example:
% find . -name "*.old" -print | cpio -ocBv > /dev/rst8
or:
% find . -print | cpio -ocBv > mydir.cpio
To read an archive in, use the -i option and redirect input from the file or tape drive containing the archive. The -d option is often important; it tells cpio to create directories as needed when copying files in. You can restore all files from the archive or specify a filename pattern (with wildcards quoted to protect them from the shell) to select only some of the files. For example, the following command restores from a tape drive all C source files:
% cpio -icdv "*.c" < /dev/rst8
Subdirectories are created if needed (-d), and cpio will be verbose (-v), announcing the name of each file that it successfully reads in.
To copy an archive to another directory, use the -p option, followed by the name of the destination directory. (On some versions of cpio, this top-level destination directory must already exist.) For example, you could use the following command to copy the contents of the current directory (including all subdirectories) to another directory:
% find . -depth -print | cpio -pd newdir
There are lots of other options for things like resetting file access times or ownership or changing the blocking factor on the tape. See your friendly neighborhood manual page for details. Notice that options are typically "squashed together" into an option string rather than written out as separate options.
TOR and JP
This book mostly focuses on tools like tar, because that's what we expect most of you to use most of the time. However, there are other tools that are very important for large-scale backups that it's good to know at least a little about.
dump is an old Unix standby and a complete if somewhat arcane tool for backing up file systems. It is extremely useful for system administrators and personal machines, and it is available as part of the operating system on nearly any Unix. For industrial-strength backups, no simple solution beats dump it is the most reliable tool for ensuring data consistency and stability. It's also a pain to use, so generally system administrators end up writing scripts around it to make it easier, or using a system like Amanda (see below).
The Advanced Maryland Automatic Network Disk Archiver, known as Amanda, is a free system for performing regular backups of one or more network-connected machines. Information on Amanda is generally available at http://www.amanda.org. Amanda uses tar or dump to do the actual work of backing up files; its job is to coordinate backups of multiple filesystems to one or more network-accessible tape drives on a regular basis.
Note also that full-scale backup processes need to address things such as tape lifetimes, electronic and physical security of backed-up data, off-site storage, incremental backup schemes and the like. Should you be in a position to need to set up such a process, read one of the good books on the subject we might recommend O'Reilly's Unix Backup and Recovery.
DJPH
[1] Of course, this section was written after the author took the first backup of his Linux system in nearly four years of use!
| CONTENTS |


Some writings about some things I've learnt
Human Skills Blog Posts
This blog is a brief overview of what EQ (Emotional Intelligence) is, how it differs from IQ and how important it is.
A quick and very basic overview of Neuroplasticity. This is such a interesting topic and if understood even on the mot basic level it can be hugely beneficial to anyone on a learning journey, which is all of us right? This post also touches on the Growth Mindset and how this works into Neuroplasticity. This is a very brief overview and the post includes some links and resources for anyone wanting to know more!
Discussions about a time Ive been stuck on a problem and what techniques were used to overcome this issue. Also some common problem solving techniques that have been proven to be helpful in coding.
More like a journal post then a blog post this is a collection of my reflections about learning about core human skills topics. Topics like the ego, self awareness, empathy and values.
This ones quite a lengthy read so if you decide to read be warned. This ones about my core values, who I am and some factors that have influenced me.


new pastetrends switch-image A GUEST SEP 30TH, 2015 122 NEVER Not a member of Pastebin yet? Sign Up, it unlocks many cool features! rawdownloadreport 0.65 KB


Name:
Age:
Email:
How did you discover Microverse?:
Did you have any programming knowledge prior to joining Microverse?:
What motivated you to join Microverse?:
How did you prepare for the course?:
What do you like most about Microverse?:
Comment


The Camera API allows a user to pick a photo from their photo album or take a picture. On iOS, this uses UIImagePickerController
, and on Android this
API sends an intent which will be handled by the core Camera app by default.
iOS requires the following usage description be added and filled out for your app in Info.plist
:
Name: Privacy - Camera Usage Description
Key: NSCameraUsageDescription
Read about Setting iOS Permissions in the iOS Guide for more information on setting iOS permissions in Xcode
This API requires the following permissions be added to your AndroidManifest.xml
:
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
The first permission is for Camera access, and the storage permissions are for reading/saving photo files.
Read about Setting Android Permissions in the Android Guide for more information on setting Android permissions.
Additionally, because the Camera API launches a separate Activity to handle taking the photo, you should listen for appRestoredResult
in the App
plugin
to handle any camera data that was sent in the case your app was terminated by the operating system while the Activity was running.
import { Plugins, CameraResultType } from '@capacitor/core';
const { Camera } = Plugins;
async takePicture() {
const image = await Camera.getPhoto({
quality: 90,
allowEditing: true,
resultType: CameraResultType.Uri
});
// image.webPath will contain a path that can be set as an image src. You can access
// the original file using image.path, which can be passed to the Filesystem API to
// read the raw data of the image, if desired (or pass resultType: CameraResultType.Base64 to getPhoto)
var imageUrl = image.webPath;
// can be set to the src of an image now
imageElement.src = imageUrl;
}
By default, Angular (>= 2.x) won't trust dynamic image urls. To trust the URL, inject DomSanitizer
and make sure to allow the
image URL to be trusted.
This example also demonstrates how to set the default source of the image. By default, getPhoto
will prompt the user to take a picture or select a photo from their existing photos.
import { Component } from '@angular/core';
import { DomSanitizer, SafeResourceUrl } from '@angular/platform-browser';
import { IonicPage, NavController, NavParams } from 'ionic-angular';
import { Plugins, CameraResultType, CameraSource } from '@capacitor/core';
@IonicPage()
@Component({
selector: 'page-camera',
templateUrl: 'camera.html',
})
export class CameraPage {
image: SafeResourceUrl;
constructor(public navCtrl: NavController, public navParams: NavParams, private zone: NgZone, private sanitizer: DomSanitizer) {
}
async takePicture() {
const { Camera } = Plugins;
const image = await Camera.getPhoto({
quality: 90,
allowEditing: true,
resultType: CameraResultType.Base64,
source: CameraSource.Camera
});
// Example of using the Base64 return type. It's recommended to use CameraResultType.Uri
// instead for performance reasons when showing large, or a large amount of images.
this.image = this.sanitizer.bypassSecurityTrustResourceUrl(image && (image.base64Data));
}
}
Component template:
<img [src]="image" />
<button (click)="takePicture()" ion-button color="primary">Take Picture</button>


In this puzzle, you just have to compute the next position of your character, depending on the input. It does not really deserve to be in the medium puzzles, as the solution is just a switch statement.
W, H = [int(i) for i in input().split()] rooms = [] for i in range(H): rooms.append([int(i) for i in input().split()]) exit = int(input()) while 1: XI, YI, POS = input().split() XI, YI = int(XI), int(YI) if rooms[YI][XI] in (1, 3, 7, 8, 9, 12, 13): YI+=1 elif rooms[YI][XI] in (2, 6): XI+= 1 if POS == 'LEFT' else -1 elif rooms[YI][XI] == 4: if POS == 'TOP': XI-=1 else: YI+=1 elif rooms[YI][XI] == 5: if POS == 'TOP': XI+=1 else: YI+=1 elif rooms[YI][XI] == 10: XI-=1 elif rooms[YI][XI] == 11: XI+=1 print("{} {}".format(XI, YI))


Monolithic ALE FEM - FSI Turek-Hron Benchmark
Learn More
Monolithic ALE FEM - Wave propagation in a compliant vessel
Learn more
Coupled Optimal Control Problems in MHD
Download the source code in zip format!
Download the source code in tar.gz format!
Browse the source code on github!


This is made by following this Tutorial and using Phaser.js For controlls, use ←↑→ keys on your keyboard


All Files
(54.48%
covered at
0.57
hits/line)
23 files in total.
145 relevant lines.
79 lines covered and
66 lines missed
Controllers
(46.0%
covered at
0.49
hits/line)
8 files in total.
100 relevant lines.
46 lines covered and
54 lines missed
Models
(68.57%
covered at
0.71
hits/line)
7 files in total.
35 relevant lines.
24 lines covered and
11 lines missed
Mailers
(100.0%
covered at
0.0
hits/line)
0 files in total.
0.0 relevant lines.
0.0 lines covered and
0.0 lines missed
| File |
% covered |
Lines |
Relevant Lines |
Lines covered |
Lines missed |
Avg. Hits / Line |
Helpers
(90.0%
covered at
0.91
hits/line)
8 files in total.
10 relevant lines.
9 lines covered and
1 lines missed
Jobs
(100.0%
covered at
0.0
hits/line)
0 files in total.
0.0 relevant lines.
0.0 lines covered and
0.0 lines missed
| File |
% covered |
Lines |
Relevant Lines |
Lines covered |
Lines missed |
Avg. Hits / Line |
Libraries
(100.0%
covered at
0.0
hits/line)
0 files in total.
0.0 relevant lines.
0.0 lines covered and
0.0 lines missed
| File |
% covered |
Lines |
Relevant Lines |
Lines covered |
Lines missed |
Avg. Hits / Line |
-
1
class ApplicationController < ActionController::Base
-
# Prevent CSRF attacks by raising an exception.
-
# For APIs, you may want to use :null_session instead.
-
1
protect_from_forgery with: :exception
-
-
-
1
protect_from_forgery with: :exception
-
-
1
def current_user
-
@current_user ||= User.find(session[:user_id]) if session[:user_id]
-
end
-
-
1
helper_method :current_user
-
-
1
def authorize
-
redirect_to '/login' unless current_user
-
end
-
-
1
def sing_in?
-
current_user.present?
-
end
-
-
end
-
1
class CategoriesController < ApplicationController
-
1
def new
-
@category = Category.new
-
end
-
-
1
def create
-
@category = Category.new(category_params)
-
if @category.save
-
redirect_to '/events'
-
else
-
redirect_to '/signup'
-
end
-
end
-
-
1
def category_params
-
params.require(:category).permit(:name)
-
end
-
-
end
-
1
class EventsController < ApplicationController
-
1
before_action :authorize
-
1
def index
-
@events = Event.all
-
@tickets = TicketType.all
-
-
if params[:search]
-
@events = Event.search(params[:search]).order("created_at DESC")
-
else
-
@events = Event.all.order('created_at DESC')
-
end
-
end
-
-
1
def new
-
@event = Event.new
-
@venues=Venue.all
-
# @event.categories.build
-
@event.build_venue
-
end
-
-
1
def create
-
@event = Event.new(params_event)
-
if @event.save
-
redirect_to new_event_ticket_type_path(@event)
-
else
-
redirect_to '/signup'
-
end
-
end
-
-
1
def show
-
@event = Event.find(params[:id])
-
end
-
-
1
private
-
-
1
def params_event
-
params.require(:event).permit(
-
:starts_at,:name,:ends_at,:user_id,:hero_image_url,
-
:extended_html_description,:category_id,:venue_id,:region_id,
-
venues_attributes: [:id, :name,:full_adress])
-
end
-
end
-
1
class RegionsController < ApplicationController
-
1
def new
-
@region = Region.new
-
-
end
-
-
1
def create
-
@region = Region.new(region_params)
-
if @region.save
-
redirect_to '/events'
-
else
-
redirect_to '/signup'
-
end
-
end
-
-
1
def region_params
-
params.require(:region).permit(:name)
-
end
-
-
end
-
1
class SessionsController < ApplicationController
-
1
def new
-
end
-
-
1
def create
-
1
user = User.find_by_email(params[:email])
-
# If the user exists AND the password entered is correct.
-
1
if user && user.authenticate(params[:password])
-
# Save the user id inside the browser cookie. This is how we keep the user
-
# logged in when they navigate around our website.
-
session[:user_id] = user.id
-
redirect_to '/events'
-
else
-
# If user's login doesn't work, send them back to the login form.
-
1
flash[:error] = 'Email and password do not match'
-
1
redirect_to '/login'
-
-
end
-
end
-
-
1
def destroy
-
session[:user_id] = nil
-
redirect_to '/login'
-
end
-
end
-
1
class TicketTypesController < ApplicationController
-
1
def new
-
@event = Event.find(params[:event_id])
-
@ticket_type= TicketType.new
-
end
-
-
1
def create
-
@event = Event.find(params[:event_id])
-
@ticket = TicketType.new(ticket_params)
-
if @ticket.save
-
redirect_to event_path(@event)
-
else
-
redirect_to '/signup'
-
end
-
end
-
-
1
def show
-
@event = Event.find(params[:event_id])
-
end
-
-
-
1
def ticket_params
-
params.require(:ticket_type).permit(:name,:price,:max_quantity, :event_id)
-
end
-
end
-
1
class UsersController < ApplicationController
-
1
def new
-
end
-
-
1
def create
-
user = User.new(user_params)
-
if user.save
-
session[:user_id] = user.id
-
redirect_to '/'
-
else
-
redirect_to '/signup'
-
end
-
end
-
-
1
private
-
-
1
def user_params
-
params.require(:user).permit(:name, :email, :password, :password_confirmation)
-
end
-
end
-
1
class VenuesController < ApplicationController
-
1
before_action :authorize
-
1
def new
-
@regions = Region.all
-
-
end
-
-
1
def create
-
@venue = Venue.new(venue_params)
-
if @venue.save
-
redirect_to '/events'
-
else
-
redirect_to '/signup'
-
end
-
end
-
-
1
def venue_params
-
params.require(:venue).permit(:name,:full_address,:region_id, :user_d)
-
end
-
-
end
-
1
module ApplicationHelper
-
end
-
1
module CategoriesHelper
-
end
-
1
module EventsHelper
-
1
def header_background_image_url
-
"https://az810058.vo.msecnd.net/site/global/Content/img/home-search-bg-0#{rand(6)}.jpg"
-
end
-
end
-
1
module RegionsHelper
-
end
-
1
module SessionsHelper
-
end
-
1
module TicketsHelper
-
end
-
1
module VenuesHelper
-
end
-
1
class ApplicationRecord < ActiveRecord::Base
-
1
self.abstract_class = true
-
end
-
class Category < ActiveRecord::Base
-
validates_inclusion_of :name, in: ['Entertainment', 'Learning', 'Everything Else']
-
validates_uniqueness_of :name
-
end
-
1
class Event < ActiveRecord::Base
-
1
belongs_to :venue
-
1
belongs_to :category
-
1
has_many :ticket_types
-
1
belongs_to :user
-
1
validates_presence_of :name, :starts_at #:extended_html_description, :venue, :category
-
#validates_uniqueness_of :name, uniqueness: {scope: [:venue, :starts_at]}
-
-
1
accepts_nested_attributes_for :venue
-
# accepts_nested_attributes_for :category
-
# accepts_nested_attributes_for :ticket_types
-
1
require 'date'
-
-
1
def self.incoming_event(begining)
-
1
begining < Date.today
-
end
-
-
1
def self.search(search)
-
where("name ILIKE ?", "%#{search}%")
-
end
-
-
1
def self.validate(event,ticket_type )
-
if event.ticket_type.present?
-
end
-
end
-
-
-
end
-
class Region < ActiveRecord::Base
-
validates_inclusion_of :name, in: ['Ho Chi Minh', 'Ha Noi', 'Binh Thuan', 'Da Nang', 'Lam Dong']
-
validates_uniqueness_of :name
-
end
-
1
class TicketType < ActiveRecord::Base
-
1
belongs_to :event
-
-
-
1
def self.available(max_qauntity,quantity)
-
max_quantity = max_quantity -quantity
-
end
-
end
-
1
class User < ApplicationRecord
-
1
has_secure_password
-
-
1
has_many :events
-
end
-
1
class Venue < ActiveRecord::Base
-
1
belongs_to :event
-
1
belongs_to :region
-
1
validates_uniqueness_of :name
-
end


| pgpool-II 4.0.3 Documentation | |||
|---|---|---|---|
| Prev | Up | Chapter 7. Configuration Examples | Next |
This section shows an example of streaming replication configuration using Pgpool-II. In this example, we use 3 Pgpool-II servers to manage PostgreSQL servers to create a robust cluster system and avoid the single point of failure or split brain.
We assume that all the Pgpool-II servers and the PostgreSQL servers are in the same subnet.
We use 3 servers with CentOS 7.4. Let these servers be server1, server2, server3. We install PostgreSQL and Pgpool-II on each server.
Note: The roles of Active, Standy, Primary, Standby are not fixed and may be changed by further operations.
Table 7-2. Hostname and IP address
| Hostname | IP Address | Virtual IP |
|---|---|---|
| server1 | <IP_ADDRESS> | <IP_ADDRESS> |
| server2 | <IP_ADDRESS> | |
| server3 | <IP_ADDRESS> |
Table 7-3. PostgreSQL version and Configuration
| Item | Value | Detail |
|---|---|---|
| PostgreSQL Version | 11.1 | - |
| port | 5432 | - |
| $PGDATA | /var/lib/pgsql/11/data | - |
| Archive mode | on | /var/lib/pgsql/archivedir |
| Start automatically | Disable | - |
Table 7-4. Pgpool-II version and Configuration
| Item | Value | Detail |
|---|---|---|
| Pgpool-II Version | 4.0.2 | - |
| port | 9999 | Pgpool-II accepts connections |
| 9898 | PCP process accepts connections | |
| 9000 | watchdog accepts connections | |
| 9694 | UDP port for receiving Watchdog's heartbeat signal | |
| Config file | /etc/pgpool-II/pgpool.conf | Pgpool-II config file |
| Pgpool-II start user | root | See Section 2.1.7 to startup Pgpool-II with non-root user |
| Running mode | streaming replication mode | - |
| Watchdog | on | Life check method: heartbeat |
| Start automatically | Disable | - |
In this example, we install Pgpool-II 4.0.2 and PostgreSQL 11.1 by using RPM packages.
Install PostgreSQL by using PostgreSQL YUM repository.
# yum install https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7-x86_64/pgdg-centos11-11-2.noarch.rpm # yum install postgresql11 postgresql11-libs postgresql11-devel postgresql11-server
Install Pgpool-II by using Pgpool-II YUM repository.
# yum install http://www.pgpool.net/yum/rpms/4.0/redhat/rhel-7-x86_64/pgpool-II-release-4.0-1.noarch.rpm # yum install pgpool-II-pg11-*
Before you start the configuration process, please check the following prerequisites.
Set up PostgreSQL streaming replication on the primary server. In this example, we use WAL archiving.
First, we create the directory /var/lib/pgsql/archivedir to store WAL segments on all servers.
[all servers]# su - postgres [all servers]$ mkdir /var/lib/pgsql/archivedir
Then we edit the configuration file $PGDATA/postgresql.conf on server1 (primary) as follows.
listen_addresses = '*' archive_mode = on archive_command = 'cp "%p" "/var/lib/pgsql/archivedir/%f"'
We use the online recovery functionality of Pgpool-II to setup standby server after the primary server is started.
Because of the security reasons, we create a user repl solely used for replication purpose, and a user pgpool for streaming replication delay check and health check of Pgpool-II.
Table 7-5. Users
| User Name | Passowrd | Detail |
|---|---|---|
| repl | repl | PostgreSQL replication user |
| pgpool | pgpool | Pgpool-II health check and replication delay check user |
| postgres | postgres | User running online recovery |
[server1]# psql -U postgres -p 5432 postgres=# SET password_encryption = 'scram-sha-256'; postgres=# CREATE ROLE pgpool WITH LOGIN; postgres=# CREATE ROLE repl WITH REPLICATION; postgres=# \password pgpool postgres=# \password repl postgres=# \password postgres
Assuming that all the Pgpool-II servers and the PostgreSQL servers are in the network of <IP_ADDRESS>/24, and edit pg_hba.conf to enable scram-sha-256 authentication method.
host all pgpool <IP_ADDRESS>/24 scram-sha-256 host all all <IP_ADDRESS>/0 scram-sha-256 host replication repl <IP_ADDRESS>/24 scram-sha-256
To use the failover and online recovery of Pgpool-II, the settings that allow SSH without passowrd to other servers are necessary.
To allow repl user without specifying password for streaming replication and online recovery, we create the .pgpass file in postgres user's home directory and change the permisson to 600 on each PostgreSQL server.
[all servers]# su - postgres [all servers]$ vi /var/lib/pgsql/.pgpass server1:5432:replication:repl:<repl user password> server2:5432:replication:repl:<repl user passowrd> server3:5432:replication:repl:<repl user passowrd> [all servers]$ chmod 600 /var/lib/pgsql/.pgpass
When connect to Pgpool-II and PostgreSQL servers, the target port must be accessible by enabling firewall management softwares. Following is an example for CentOS/RHEL7.
[all servers]# firewall-cmd --permanent --zone=public --add-service=postgresql [all servers]# firewall-cmd --permanent --zone=public --add-port=9999/tcp --add-port=9898/tcp --add-port=9000/tcp --add-port=9694/tcp [all servers]# firewall-cmd --permanent --zone=public --add-port=9999/udp --add-port=9898/udp --add-port=9000/udp --add-port=9694/udp [all servers]# firewall-cmd --reload
Here are the common settings on server1, server2 and server3.
When installing Pgpool-II from RPM, all the Pgpool-II configuration files are in /etc/pgpool-II. In this example, we copy the sample configuration file for streaming replicaton mode.
# cp /etc/pgpool-II/pgpool.conf.sample-stream /etc/pgpool-II/pgpool.conf
To allow Pgpool-II to accept all incoming connections, we set listen_addresses = '*'.
listen_addresses = '*'
Specifiy replication delay check user and password. In this example, we leave sr_check_user empty, and create the entry in pool_passwd. From Pgpool-II 4.0, if these parameters are left blank, Pgpool-II will first try to get the password for that specific user from sr_check_password file before using the empty password.
sr_check_user = 'pgpool' sr_check_password = ''
Enable health check so that pgpool-II performs failover. Also, if the network is unstable, the health check fails even though the backend is running properly, failover or degenerate operation may occur. In order to prevent such incorrect detection of health check, we set health_check_max_retries = 3. Specify health_check_user and health_check_password in the same way like sr_check_user and sr_check_password.
health_check_period = 5 # Health check period # Disabled (0) by default health_check_timeout = 30 # Health check timeout # 0 means no timeout health_check_user = 'pgpool' health_check_password = '' health_check_max_retries = 3
Specify the PostgreSQL backend informations. Multiple backends can be specified by adding a number at the end of the parameter name.
# - Backend Connection Settings - backend_hostname0 = 'server1' # Host name or IP address to connect to for backend 0 backend_port0 = 5432 # Port number for backend 0 backend_weight0 = 1 # Weight for backend 0 (only in load balancing mode) backend_data_directory0 = '/var/lib/pgsql/11/data' # Data directory for backend 0 backend_flag0 = 'ALLOW_TO_FAILOVER' # Controls various backend behavior # ALLOW_TO_FAILOVER or DISALLOW_TO_FAILOVER backend_hostname1 = 'server2' backend_port1 = 5432 backend_weight1 = 1 backend_data_directory1 = '/var/lib/pgsql/11/data' backend_flag1 = 'ALLOW_TO_FAILOVER' backend_hostname2 = 'server3' backend_port2 = 5432 backend_weight2 = 1 backend_data_directory2 = '/var/lib/pgsql/11/data' backend_flag2 = 'ALLOW_TO_FAILOVER'
Specify failover.sh script to be executed after failover in failover_command parameter. If we use 3 PostgreSQL servers, we need to specify follow_master_command to run after failover on the primary node failover. In case of two PostgreSQL servers, follow_master_command setting is not necessary.
failover_command = '/etc/pgpool-II/failover.sh %d %h %p %D %m %H %M %P %r %R' follow_master_command = '/etc/pgpool-II/follow_master.sh %d %h %p %D %m %M %H %P %r %R'
Create /etc/pgpool-II/failover.sh, and add execute permission.
# vi /etc/pgpool-II/failover.sh # vi /etc/pgpool-II/follow_master.sh # chmod +x /etc/pgpool-II/{failover.sh,follow_master.sh}
/etc/pgpool-II/failover.sh
#!/bin/bash # This script is run by failover_command. set -o xtrace exec > >(logger -i -p local1.info) 2>&1 # Special values: # %d = node id # %h = host name # %p = port number # %D = database cluster path # %m = new master node id # %H = hostname of the new master node # %M = old master node id # %P = old primary node id # %r = new master port number # %R = new master database cluster path # %% = '%' character FAILED_NODE_ID="$1" FAILED_NODE_HOST="$2" FAILED_NODE_PORT="$3" FAILED_NODE_PGDATA="$4" NEW_MASTER_NODE_ID="$5" NEW_MASTER_NODE_HOST="$6" OLD_MASTER_NODE_ID="$7" OLD_PRIMARY_NODE_ID="$8" NEW_MASTER_NODE_PORT="$9" NEW_MASTER_NODE_PGDATA="${10}" PGHOME=/usr/pgsql-11 logger -i -p local1.info failover.sh: start: failed_node_id=$FAILED_NODE_ID old_primary_node_id=$OLD_PRIMARY_NODE_ID \ failed_host=$FAILED_NODE_HOST new_master_host=$NEW_MASTER_NODE_HOST # If standby node is down, skip failover. if [ $FAILED_NODE_ID -ne $OLD_PRIMARY_NODE_ID ]; then logger -i -p local1.info failover.sh: Standby node is down. Skipping failover. exit 0 fi # Promote standby node. logger -i -p local1.info failover.sh: ssh: postgres@$NEW_MASTER_NODE_HOST pg_ctl promote if [ $UID -eq 0 ]; then su postgres -c "ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \ postgres@$NEW_MASTER_NODE_HOST ${PGHOME}/bin/pg_ctl -D ${NEW_MASTER_NODE_PGDATA} -w promote" else ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \ postgres@$NEW_MASTER_NODE_HOST ${PGHOME}/bin/pg_ctl -D ${NEW_MASTER_NODE_PGDATA} -w promote fi if [[ $? -ne 0 ]]; then logger -i -p local1.error failover.sh: new_master_host=$NEW_MASTER_NODE_HOST promote failed exit 1 fi logger -i -p local1.info failover.sh: end: new_master_node_id=$NEW_MASTER_NODE_ID started as the primary node exit 0
/etc/pgpool-II/follow_master.sh
#!/bin/bash # This script is run after failover_command to recover the slave from the new primary. set -o xtrace exec > >(logger -i -p local1.info) 2>&1 # special values: %d = node id # %h = host name # %p = port number # %D = database cluster path # %m = new master node id # %M = old master node id # %H = new master node host name # %P = old primary node id # %R = new master database cluster path # %r = new master port number # %% = '%' character FAILED_NODE_ID="$1" FAILED_NODE_HOST="$2" FAILED_NODE_PORT="$3" FAILED_NODE_PGDATA="$4" NEW_MASTER_NODE_ID="$5" OLD_MASTER_NODE_ID="$6" NEW_MASTER_NODE_HOST="$7" OLD_PRIMARY_NODE_ID="$8" NEW_MASTER_NODE_PORT="$9" NEW_MASTER_NODE_PGDATA="${10}" PGHOME=/usr/pgsql-11 ARCHIVEDIR=/var/lib/pgsql/archivedir REPL_USER=repl PCP_USER=pgpool PGPOOL_PATH=/usr/bin PCP_PORT=9898 # Recovery the slave from the new primary logger -i -p local1.info follow_master.sh: start: pg_basebackup for $FAILED_NODE_ID # Check the status of standby ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \ postgres@${FAILED_NODE_HOST} ${PGHOME}/bin/pg_ctl -w -D ${FAILED_NODE_PGDATA} status >/dev/null 2>&1 # If slave is running, recover the slave from the new primary. if [[ $? -eq 0 ]]; then # Execute pg_basebackup at slave ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null postgres@${FAILED_NODE_HOST} " ${PGHOME}/bin/pg_ctl -w -m f -D ${FAILED_NODE_PGDATA} stop rm -rf ${FAILED_NODE_PGDATA} ${PGHOME}/bin/pg_basebackup -h ${NEW_MASTER_NODE_HOST} -U ${REPL_USER} -p ${NEW_MASTER_NODE_PORT} -D ${FAILED_NODE_PGDATA} -X stream -R if [[ $? -ne 0 ]]; then logger -i -p local1.error follow_master.sh: end: pg_basebackup failed exit 1 fi rm -rf ${ARCHIVEDIR}/* cat >> ${FAILED_NODE_PGDATA}/recovery.conf << EOT restore_command = 'scp ${NEW_MASTER_NODE_HOST}:${ARCHIVEDIR}/%f %p' EOT $PGHOME/bin/pg_ctl -l /dev/null -w -D ${FAILED_NODE_PGDATA} start " if [[ $? -eq 0 ]]; then # Run pcp_attact_node to attach this slave to Pgpool-II. ${PGPOOL_PATH}/pcp_attach_node -w -h localhost -U ${PCP_USER} -p ${PCP_PORT} -n ${FAILED_NODE_ID} if [[ $? -ne 0 ]]; then logger -i -p local1.error follow_master.sh: end: pcp_attach_node failed exit 1 else logger -i -p local1.error follow_master.sh: end: follow master failed exit 1 fi else logger -i -p local1.info follow_master.sh: failed_nod_id=${FAILED_NODE_ID} is not running. skipping follow master command. exit 0 fi logger -i -p local1.info follow_master.sh: end: follow master is finished exit 0
Next, in order to perform online recovery with Pgpool-II we specify the PostgreSQL user name and online recovery command recovery_1st_stage. Because Supergroup privilege of PostgreSQL is required for online recovery, we specify postgres user to recovery_user. Then, we create recovery_1st_stage and pgpool_remote_start in database cluster directory of PostgreSQL primary server (server1), and add execute permission.
recovery_user = 'postgres' # Online recovery user recovery_password = '' # Online recovery password recovery_1st_stage_command = 'recovery_1st_stage'
[server1]# su - postgres [server1]$ vi /var/lib/pgsql/11/data/recovery_1st_stage [server1]$ vi /var/lib/pgsql/11/data/pgpool_remote_start [server1]$ chmod +x /var/lib/pgsql/11/data/{recovery_1st_stage,pgpool_remote_start}
/var/lib/pgsql/11/data/recovery_1st_stage
#!/bin/bash # This script is run by recovery_1st_stage to recovery the slave from the primary. set -o xtrace exec > >(logger -i -p local1.info) 2>&1 PRIMARY_NODE_PGDATA="$1" DEST_NODE_HOST="$2" DEST_NODE_PGDATA="$3" PRIMARY_NODE_PGPORT=$4 PRIMARY_NODE_HOST=$(hostname -s) PGHOME=/usr/pgsql-11 ARCHIVEDIR=/var/lib/pgsql/archivedir REPLUSER=repl logger -i -p local1.info online_recovery.sh: start: pg_basebackup for $DEST_NODE_HOST # Run pg_basebackup to recovery the slave from the primary ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null postgres@$DEST_NODE_HOST " rm -rf $DEST_NODE_PGDATA ${PGHOME}/bin/pg_basebackup -h $PRIMARY_NODE_HOST -U $REPLUSER -p $PRIMARY_NODE_PGPORT -D $DEST_NODE_PGDATA -X stream -R " if [[ $? -ne 0 ]]; then logger -i -p local1.error online_recovery.sh: end: pg_basebackup failed. online recovery failed. exit 1 fi ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null postgres@$DEST_NODE_HOST " rm -rf $ARCHIVEDIR/* cat >> $DEST_NODE_PGDATA/recovery.conf << EOT restore_command = 'scp $PRIMARY_NODE_HOST:$archivedir/%f %p' EOT " if [[ $? -ne 0 ]]; then logger -i -p local1.error online_recovery.sh: end: online recovery failed exit 1 else logger -i -p local1.info online_recovery.sh: end: online recovery is finished exit 0 fi
/var/lib/pgsql/11/data/pgpool_remote_start
#!/bin/bash # This script is run to start slave node after recovery. set -o xtrace exec > >(logger -i -p local1.info) 2>&1 PGHOME=/usr/pgsql-11 DEST_HOST="$1" DEST_HOST_PGDATA="$2" logger -i -p local1.info pgpool_remote_start: start: remote start PostgreSQL@$DEST_HOST # Start slave node ssh -T -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null postgres@$DEST_HOST $PGHOME/bin/pg_ctl -l /dev/null -w -D $DEST_HOST_PGDATA start if [[ $? -ne 0 ]]; then logger -i -p local1.info pgpool_remote_start: $DEST_HOST start failed. exit 1 fi logger -i -p local1.info pgpool_remote_start: end: $DEST_HOST PostgreSQL started successfully.
In order to use the online recovery functionality, the functions of
pgpool_recovery
, pgpool_remote_start
,
pgpool_switch_xlog
are required, so we need install
pgpool_recovery
on template1 of PostgreSQL server
server1.
[server1]# su - postgres [server1]$ psql template1 -c "CREATE EXTENSION pgpool_recovery"
Because in the section Before Starting, we already set PostgreSQL authentication method to scram-sha-256, it is necessary to set a client authentication by Pgpool-II to connect to backend nodes. Please note that only AES encrypted password or clear text passowrd can be specified in health_check_password, sr_check_password, wd_lifecheck_password, recovery_password in pgpool.conf. When installing with RPM, the Pgpool-II configuration file pool_hba.conf is in /etc/pgpool-II. By default, pool_hba authentication is disabled, and set enable_pool_hba = on to enable it.
enable_pool_hba = on
The format of pool_hba.conf file follows very closely PostgreSQL's pg_hba.conf format. Set pgpool and postgres user's authentication method to scram-sha-256.
host all pgpool <IP_ADDRESS>/0 scram-sha-256 host all postgres <IP_ADDRESS>/0 scram-sha-256
The default password file name for authentication is pool_passwd. To use scram-sha-256 authentication, the decryption key to decrypt the passwords is required. We create the .pgpoolkey file in root user's home directory.
[all servers]# echo 'some string' > ~/.pgpoolkey [all servers]# chmod 600 ~/.pgpoolkey
Execute command pg_enc -m -k /path/to/.pgpoolkey -u username -p to regist user name and AES encrypted password in file pool_passwd. If pool_passwd doesn't exist yet, it will be created in the same directory as pgpool.conf.
[all servers]# pg_enc -m -k /root/.pgpoolkey -u pgpool -p db password: [pgpool user's password] [all servers]# pg_enc -m -k /root/.pgpoolkey -u postgres -p db password: [postgres user's passowrd] # cat /etc/pgpool-II/pool_passwd pgpool:AESheq2ZMZjynddMWk5sKP/Rw== postgres:AESHs/pWL5rtXy2IwuzroHfqg==
Enable watchdog functionality on server1, server2, server3.
use_watchdog = on
Specify virtual IP address that accepts connections from clients on server1, server2, server3. Ensure that the IP address set to virtual IP isn't used yet.
delegate_IP = '<IP_ADDRESS>'
To bring up/down the virtual IP and send the ARP requests, we set if_up_cmd, if_down_cmd and arping_cmd. The network interface used in this example is "enp0s8".
if_up_cmd = 'ip addr add $_IP_$/24 dev enp0s8 label enp0s8:0' # startup delegate IP command if_down_cmd = 'ip addr del $_IP_$/24 dev enp0s8' # shutdown delegate IP command arping_cmd = 'arping -U $_IP_$ -w 1 -I enp0s8' # arping command
Set if_cmd_path and arping_path according to the command path.
if_cmd_path = '/sbin' # path to the directory where if_up/down_cmd exists arping_path = '/usr/sbin' # arping command path
Specify the hostname and port number of each Pgpool-II server.
server1
wd_hostname = 'server1' wd_port = 9000
server2
wd_hostname = 'server2' wd_port = 9000
server3
wd_hostname = 'server3' wd_port = 9000
Specify the hostname, Pgpool-II port number, and watchdog port number of monitored Pgpool-II servers on each Pgpool-II server.
server1
# - Other pgpool Connection Settings - other_pgpool_hostname0 = 'server2' # Host name or IP address to connect to for other pgpool 0 # (change requires restart) other_pgpool_port0 = 9999 # Port number for other pgpool 0 # (change requires restart) other_wd_port0 = 9000 # Port number for other watchdog 0 # (change requires restart) other_pgpool_hostname1 = 'server3' other_pgpool_port1 = 9999 other_wd_port1 = 9000
server2
# - Other pgpool Connection Settings - other_pgpool_hostname0 = 'server1' # Host name or IP address to connect to for other pgpool 0 # (change requires restart) other_pgpool_port0 = 9999 # Port number for other pgpool 0 # (change requires restart) other_wd_port0 = 9000 # Port number for other watchdog 0 # (change requires restart) other_pgpool_hostname1 = 'server3' other_pgpool_port1 = 9999 other_wd_port1 = 9000
server3
# - Other pgpool Connection Settings - other_pgpool_hostname0 = 'server1' # Host name or IP address to connect to for other pgpool 0 # (change requires restart) other_pgpool_port0 = 9999 # Port number for other pgpool 0 # (change requires restart) other_wd_port0 = 9000 # Port number for other watchdog 0 # (change requires restart) other_pgpool_hostname1 = 'server2' other_pgpool_port1 = 9999 other_wd_port1 = 9000
Specify the hostname and port number of destination for sending heartbeat signal on server1, server2, server3.
server1
heartbeat_destination0 = 'server2' # Host name or IP address of destination 0 # for sending heartbeat signal. # (change requires restart) heartbeat_destination_port0 = 9694 # Port number of destination 0 for sending # heartbeat signal. Usually this is the # same as wd_heartbeat_port. # (change requires restart) heartbeat_device0 = '' # Name of NIC device (such like 'eth0') # used for sending/receiving heartbeat # signal to/from destination 0. # This works only when this is not empty # and pgpool has root privilege. # (change requires restart) heartbeat_destination1 = 'server3' heartbeat_destination_port1 = 9694 heartbeat_device1 = ''
server2
heartbeat_destination0 = 'server1' # Host name or IP address of destination 0 # for sending heartbeat signal. # (change requires restart) heartbeat_destination_port0 = 9694 # Port number of destination 0 for sending # heartbeat signal. Usually this is the # same as wd_heartbeat_port. # (change requires restart) heartbeat_device0 = '' # Name of NIC device (such like 'eth0') # used for sending/receiving heartbeat # signal to/from destination 0. # This works only when this is not empty # and pgpool has root privilege. # (change requires restart) heartbeat_destination1 = 'server3' heartbeat_destination_port1 = 9694 heartbeat_device1 = ''
server3
heartbeat_destination0 = 'server1' # Host name or IP address of destination 0 # for sending heartbeat signal. # (change requires restart) heartbeat_destination_port0 = 9694 # Port number of destination 0 for sending # heartbeat signal. Usually this is the # same as wd_heartbeat_port. # (change requires restart) heartbeat_device0 = '' # Name of NIC device (such like 'eth0') # used for sending/receiving heartbeat # signal to/from destination 0. # This works only when this is not empty # and pgpool has root privilege. # (change requires restart) heartbeat_destination1 = 'server2' heartbeat_destination_port1 = 9694 heartbeat_device1 = ''
If you want to ignore the pgpool_status file at startup of Pgpool-II, add "- D" to the start option OPTS to /etc/sysconfig/pgpool.
[all servers]# vi /etc/sysconfig/pgpool ... OPTS=" -D -n"
In the example, we output Pgpool-II's log to syslog.
log_destination = 'syslog' # Where to log # Valid values are combinations of stderr, # and syslog. Default to stderr. syslog_facility = 'LOCAL1' # Syslog local facility. Default to LOCAL0
Create Pgpool-II log file.
[all servers]# mkdir /var/log/pgpool-II [all servers]# touch /var/log/pgpool-II/pgpool.log
Edit config file of syslog /etc/rsyslog.conf.
[all servers]# vi /etc/rsyslog.conf ... *.info;mail.none;authpriv.none;cron.none;LOCAL1.none /var/log/messages LOCAL1.* /var/log/pgpool-II/pgpool.log
Setting logrotate same as /var/log/messages.
[all servers]# vi /etc/logrotate.d/syslog ... /var/log/messages /var/log/pgpool-II/pgpool.log /var/log/secure
Restart rsyslog service.
[all servers]# systemctl restart rsyslog
Since user authentication is required to use the PCP command, specify user name and md5 encrypted password in pcp.conf. Here we create the encrypted password for pgpool user, and add "username:encrypted password" in /etc/pgpool-II/pcp.conf.
[all servers]# echo 'pgpool:'`pg_md5 PCP passowrd` >> /etc/pgpool-II/pcp.conf
Since follow_master_command script has to execute PCP command without entering the password, we create .pcppass in the home directory of Pgpool-II startup user (root user).
[all servers]# echo 'localhost:9898:pgpool:pgpool' > ~/.pcppass [all servers]# chmod 600 ~/.pcppass
The settings of Pgpool-II is completed.
Next we start Pgpool-II. Before starting Pgpool-II, please start PostgreSQL servers first. Also, when stopping PostgreSQL, it is necessary to stop Pgpool-II first.
Starting Pgpool-II
In section Before Starting, we already set the auto-start of Pgpool-II. To start Pgpool-II, restart the whole system or execute the following command.
# systemctl start pgpool.service
Stopping Pgpool-II
# systemctl stop pgpool.service
Let's start to use Pgpool-II. First, let's start Pgpool-II on server1, server2, server3 by using the following command.
# systemctl start pgpool.service
First, we should set up PostgreSQL standby server by using Pgpool-II online recovery functionality. Ensure that recovery_1st_stage and pgpool_remote_start scripts used by pcp_recovery_node command are in database cluster directory of PostgreSQL primary server (server1).
# pcp_recovery_node -h <IP_ADDRESS> -p 9898 -U pgpool -n 1 Password: pcp_recovery_node -- Command Successful # pcp_recovery_node -h <IP_ADDRESS> -p 9898 -U pgpool -n 2 Password: pcp_recovery_node -- Command Successful
After executing pcp_recovery_node command, vertify that server2 and server3 are started as PostgreSQL standby server.
# psql -h <IP_ADDRESS> -p 9999 -U pgpool postgres -c "show pool_nodes" node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change ---------+----------+------+--------+-----------+---------+------------+-------------------+-------------------+--------------------- 0 | server1 | 5432 | up | 0.333333 | primary | 0 | false | 0 | 2019-02-18 11:26:31 1 | server2 | 5432 | up | 0.333333 | standby | 0 | true | 0 | 2019-02-18 11:27:49 2 | server3 | 5432 | up | 0.333333 | standby | 0 | false | 0 | 2019-02-18 11:27:49
Confirm the watchdog status by using pcp_watchdog_info. The Pgpool-II server which is started first run as MASTER.
# pcp_watchdog_info -h <IP_ADDRESS> -p 9898 -U pgpool Password: 3 YES server1:9999 Linux server1 server1 server1:9999 Linux server1 server1 9999 9000 4 MASTER #The Pgpool-II server started first becames "MASTER". server2:9999 Linux server2 server2 9999 9000 7 STANDBY #run as standby server3:9999 Linux server3 server3 9999 9000 7 STANDBY #run as standby
Stop active server server1, then server2 or server3 will be promoted to active server. To stop server1, we can stop Pgpool-II service or shutdown the whole system. Here, we stop Pgpool-II service.
[server1]# systemctl stop pgpool.service # pcp_watchdog_info -p 9898 -h <IP_ADDRESS> -U pgpool Password: 3 YES server2:9999 Linux server2 server2 server2:9999 Linux server2 server2 9999 9000 4 MASTER #server2 is promoted to MASTER server1:9999 Linux server1 server1 9999 9000 10 SHUTDOWN #server1 is stopped server3:9999 Linux server3 server3 9999 9000 7 STANDBY #server3 runs as STANDBY
Start Pgpool-II (server1) which we have stopped again, and vertify that server1 runs as a standby.
[server1]# systemctl start pgpool.service [server1]# pcp_watchdog_info -p 9898 -h <IP_ADDRESS> -U pgpool Password: 3 YES server2:9999 Linux server2 server2 server2:9999 Linux server2 server2 9999 9000 4 MASTER server1:9999 Linux server1 server1 9999 9000 7 STANDBY server3:9999 Linux server3 server3 9999 9000 7 STANDBY
First, use psql to connect to PostgreSQL via virtual IP, and verify the backend informations.
# psql -h <IP_ADDRESS> -p 9999 -U pgpool postgres -c "show pool_nodes" node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change ---------+----------+------+--------+-----------+---------+------------+-------------------+-------------------+--------------------- 0 | server1 | 5432 | up | 0.333333 | primary | 0 | false | 0 | 2019-02-18 13:08:02 1 | server2 | 5432 | up | 0.333333 | standby | 0 | false | 0 | 2019-02-18 13:21:56 2 | server3 | 5432 | up | 0.333333 | standby | 0 | true | 0 | 2019-02-18 13:21:56
Next, stop primary PostgreSQL server server1, and verify automatic failover.
[server1]$ pg_ctl -D /var/lib/pgsql/11/data -m immediate stop
After stopping PostgreSQL on server1, failover occurs and PostgreSQL on server2 becomes new primary DB.
# psql -h <IP_ADDRESS> -p 9999 -U pgpool postgres -c "show pool_nodes" node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change ---------+----------+------+--------+-----------+---------+------------+-------------------+-------------------+--------------------- 0 | server1 | 5432 | down | 0.333333 | standby | 0 | false | 0 | 2019-02-18 13:22:25 1 | server2 | 5432 | up | 0.333333 | primary | 0 | true | 0 | 2019-02-18 13:22:25 2 | server3 | 5432 | up | 0.333333 | standby | 0 | false | 0 | 2019-02-18 13:22:28
server3 is running as standby of new primary server2.
[server3]# psql -h server3 -p 5432 -U pgpool postgres -c "select pg_is_in_recovery()" pg_is_in_recovery ------------------- t [server2]# su - postgres $ psql postgres=# select pg_is_in_recovery(); pg_is_in_recovery ------------------- f postgres=# select * from pg_stat_replication; -[ RECORD 1 ]----+------------------------------ pid | 11915 usesysid | 16385 usename | repl application_name | walreceiver client_addr | <IP_ADDRESS> client_hostname | client_port | 37834 backend_start | 2019-02-18 13:22:27.472038+09 backend_xmin | state | streaming sent_lsn | 0/8E000060 write_lsn | 0/8E000060 flush_lsn | 0/8E000060 replay_lsn | 0/8E000060 write_lag | flush_lag | replay_lag | sync_priority | 0 sync_state | async
Here, we use Pgpool-II online recovery functionality to restore server1 (old primary server) as a standby. Before restoring the old primary server, please ensure that recovery_1st_stage and pgpool_remote_start scripts exist in database cluster directory of current primary server server2.
# pcp_recovery_node -h <IP_ADDRESS> -p 9898 -U pgpool -n 0 Password: pcp_recovery_node -- Command Successful
Then verify that server1 is started as a standby.
# psql -h <IP_ADDRESS> -p 9999 -U pgpool postgres -c "show pool_nodes" node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | last_status_change ---------+----------+------+--------+-----------+---------+------------+-------------------+-------------------+--------------------- 0 | server1 | 5432 | up | 0.333333 | standby | 0 | false | 0 | 2019-02-18 13:27:44 1 | server2 | 5432 | up | 0.333333 | primary | 0 | false | 0 | 2019-02-18 13:22:25 2 | server3 | 5432 | up | 0.333333 | standby | 0 | true | 0 | 2019-02-18 13:22:28


Toggle navigation
Performance Task
Quiz
Question 2 of 13
Some unique question description
Next
© 2016 John Wiley & Sons, Performance SQA Team


Please unlock with your personal PIN
Could not unlock. Password wrong?
Connected to Keepass and ready to fill in passwords
Unlocked sucessfully, but can't connect to Keepass
Is Keepass running and the database unlocked?
Go to settings
Setup
AESKey:
Pin 1:
Pin 2:
Setup failed. Reason:


#include <stdafx.h>
this struct holds the object struct
Definition at line 29 of file stdafx.h.
flag if valid
Definition at line 40 of file stdafx.h.
number of this object
Definition at line 31 of file stdafx.h.
XL depth of this object
Definition at line 34 of file stdafx.h.
XL pixel of this object
Definition at line 33 of file stdafx.h.
XL position of this object
Definition at line 32 of file stdafx.h.
XM depth of this object
Definition at line 36 of file stdafx.h.
XM position of this object
Definition at line 35 of file stdafx.h.
XR depth of this object
Definition at line 39 of file stdafx.h.
XR pixel of this object
Definition at line 38 of file stdafx.h.
XR position of this object
Definition at line 37 of file stdafx.h.