Providing visibility of periodic changes to a database
<p>This is quite a general question, though Iâll give the specific use case for context.</p>

<p>I'm using a FileMaker Pro database to record personal bird observations. For each bird on the national list, I have extracted quite a lot of base data by website scraping in Python, for example conservation status, geographical range, scientific name and so on. In day-to-day use of the database, this base data remains fixed and unchanging. However, once a year or so I will want to re-scrape the base data to pick up the most recent published information on status, range, and even changes in scientific name (that happens).</p>

<p>I know there are options such as PyFilemaker or bBox which should allow me to write to the FileMaker database from Python, so the update mechanism itself shouldn't be a problem.</p>

<p>It would be rather dangerous simply to overwrite all of last yearâs base data with the newly scraped data, and I'm looking for general advice as to how best to provide visibility for the changes before manually importing them.  What I have in mind is to use pandas to generate a spreadsheet using the base data, and to highlight the changed cells.  Does that sound a sensible way of doing it?  I suspect that this may be a very standard requirement, and if anybody could help out with comments on an approach which is straightforward to implement in Python that would be most helpful. </p>

<p>This is not a standard requirement and there is no easy way of doing this. The best way to track changes is a Source Control system like git, but it is not applicable to FileMaker Pro as the files are binary.</p>

<p>You can try your approach, or you can try to add the new records in FileMaker instead of updating them and flag them as current or use only the last record</p>

<p>There are some amazing guys here, but you might want to take it to one of the FileMAker forums as the FIleMAker audience there is much larger then in SO</p>

python filemaker