Archive Dynamodb based on date/days
<p>I want to archive dynamodb table, keeping data only for 90 days. I have a field called recorded_on in the table which I can use to track 90days. Looked at Datapipeline and it seems little overkill with EMR since we don't need it. Any better ways to do this? </p>

<pre><code>1. Cronjob that will continue to run everyday and match recorded_on + 90days &gt; today's date and put those rows in s3 and delete those rows.

2. Separate cronjob to put data from s3 to redshift everyday.
</code></pre>

<p>You could create a scheduled Lambda function that runs daily (or at whatever interval you want) that performs the query and archives the items.  </p>

<p>Or, if you want that to scale and perform better, you could have the Lambda function perform the query and then write a message to an SNS topic for each item that needs to be archived and have another Lambda function trigger on that SNS topic and perform the archive operation.</p>

python amazon-s3 amazon-dynamodb boto3 amazon-data-pipeline