PyMongo Only include field in document if value is not a NaN
<p>I have a significant amount of data that needs to be inserted into my MongoDB database using PyMongo. The data I have is currently stored in flat files and is sparse (i.e. many of the individual values are NaN). In Mongo DB I would like to not insert fields if values are NaN but I'm not sure how to do that (I should point out I'm new to both MongoDB and Python).</p>

<p>My insert startement looks something like this</p>

<pre><code>            strategy.insert_many([
            {
                "strategyId": strategyInfo[stratIndex][ID],
                "strategyName": strategyInfo[stratIndex][NAME],
                "date": dates[i],
                "time": thisTime,
                "aum": stratAum[i],
                "return":0.0,
                "commission":0.0,
                "slippage":0.0,
                "basket":[{
                    "assetId": assets[m][ASSETID],
                    "order": orders[i, m],
                    "expiry": expiry[i, m],
                    "price": prices[i, m],
                    "ePrice": eprices[i, m]     &lt;&lt;&lt;Don't include line if eprices[i,m] is a NaN
                }
                    for m in range(len(assets))
                ]
            }
        ], False)
</code></pre>

<p>It's easy enough to check to see if one of my value's is NaN using <code>math.isnan()</code> but I can't figure out how to leave the entire field blank if that is the case.</p>

<blockquote>
  <p>It's easy enough to check to see if one of my value's is NaN usingÂ <code>math.isnan()</code>Â but I can't figure out how to leave the entire field blank if that is the case.</p>
</blockquote>

<p>Based on your example code, you can do the following instead:</p>



<pre><code># Create a strategy document. 
# This is inside of a loop where variable `i` is known, similar to your example. 
doc = {
  "strategyId": strategyInfo[stratIndex][ID],
  "strategyName": strategyInfo[stratIndex][NAME],
  "date": dates[i],
  "time": thisTime,
  "aum": stratAum[i],
  "return":0.0,
  "commission":0.0,
  "slippage":0.0
}
baskets = []
for m in range(len(assets)):
    basket = {
        "assetId": assets[m][ASSETID],
        "order": orders[i, m],
        "expiry": expiry[i, m],
        "price": prices[i, m],
    }
    if not math.isnan(eprice[i, m]):
        basket["ePrice"] = eprice[i, m]
    baskets.append(basket)

# You can also add a filter here to make sure `baskets` array is not null.
doc["basket"] = baskets
docs.append(doc)
</code></pre>

<p>Which essentially separating the making your document and the database insertion. </p>

<p>You can then use <a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert_many" rel="nofollow">insert_many()</a>:</p>



<pre><code>strategy.insert_many(docs, False)
</code></pre>

<p>You can also wrap the <code>insert_many</code> in a try/except to detect database insert error, which should be different error handling from your document creation error. </p>

python mongodb insert pymongo