What to do if a query result is too big
<p>I'm using mysql connector of python3 to query from a mysql database. I have so many data that if I collect the recent 7 days' data, the size of the query will be over 10GB and therefore forced my python script to be killed. I think usually we can stream the result however I don't find a way to stream the query result in mysql conncetor. Is there any other way that I can solve the problem?</p>

<p>@Anthony Kong's comment is correct. To solve this issue, we can do <code>fetchmany</code> function from <a href="https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-fetchmany.html" rel="nofollow">https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-fetchmany.html</a></p>

<p>After digging into the code a little bit, I found that the <code>fetchmany</code> function calls many <code>fetchone</code> to achieve "fetchmany". So I decide to use fetchone at last. Also fetchone comes with an example on the document <a href="https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-fetchone.html" rel="nofollow">https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-fetchone.html</a></p>

python mysql mysql-connector