Pymongo replace_one modified_count always 1 even if not changing anything
<p>Why and how can this work like this?</p>

<pre><code>item = db.test.find_one()
result = db.test.replace_one(item, item)
print(result.raw_result)
# Gives: {u'n': 1, u'nModified': 1, u'ok': 1, 'updatedExisting': True}
print(result.modified_count)
# Gives 1
</code></pre>

<p>when the equivalent in mongodb shell is always 0</p>

<pre><code>item = db.test.findOne()
db.test.replaceOne(item, item)
# Gives: {"acknowledged" : true, "matchedCount" : 1.0, "modifiedCount" : 0.0}
</code></pre>

<p>How can I get consistent results and properly detect when the replacement is actually changing the data?</p>

<p>This is because MongoDB stores documents in binary (<a href="http://bsonspec.org/" rel="nofollow">BSON</a>) format. Key-value pairs in a BSON document can have any order (except that _id is always first). 
Let's start with the <a href="https://docs.mongodb.com/manual/mongo/" rel="nofollow">mongo shell</a> first. The mongo shell preserves the key order when reading and writing data. 
For example: </p>

<p></p>

<pre><code>&gt; db.collection.insert({_id:1, a:2, b:3})
{ "_id" : 1, "a" : 2, "b" : 3 }
</code></pre>

<p>If you are performing <a href="https://docs.mongodb.com/manual/reference/method/db.collection.replaceOne/" rel="nofollow">replaceOne()</a> using this document value, it would avoid a modification because there's an existing BSON. </p>

<p></p>

<pre><code>&gt; var doc = db.collection.findOne()
&gt; db.collection.replaceOne(doc, doc)
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 0 }
</code></pre>

<p>However, if you change the ordering of the fields it would detect a modification</p>

<p></p>

<pre><code>&gt; var doc_2 = {_id:1, b:3, a:2}
&gt; db.collection.replaceOne(doc_2, doc_2)
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 1 }
</code></pre>

<p>Let's step into the Python world. <a href="https://api.mongodb.com/python/current/" rel="nofollow">PyMongo</a> represents BSON documents as Python dictionary by default, the order of keys in Python dictionary is not defined. Therefore, you cannot predict how it will be serialised to BSON. As per your example: </p>

<p></p>

<pre><code>&gt; doc = db.collection.find_one()
{u'_id': 1.0, u'a': 2.0, u'b': 3.0}

&gt; result = db.collection.replace_one(doc, doc)
&gt; result.raw_result
{u'n': 1, u'nModified': 1, u'ok': 1, 'updatedExisting': True}
</code></pre>

<p>If it matters for your use case, one workaround is to use <a href="http://api.mongodb.com/python/current/api/bson/son.html" rel="nofollow">bson.SON</a>. For example: </p>

<p></p>

<pre><code>&gt; from bson import CodecOptions, SON
&gt; opts=CodecOptions(document_class=SON)
&gt; collection_son = db.collection.with_options(codec_options=opts)
&gt; doc_2 = collection_son.find_one()
SON([(u'_id', 1.0), (u'a', 2.0), (u'b', 3.0)])

&gt; result = collection_son.replace_one(doc_2, doc_2)
{u'n': 1, u'nModified': 0, u'ok': 1, 'updatedExisting': True}
</code></pre>

<p>You can also observe that <code>bson.SON</code> is used in PyMongo (v3.3.0) i.e. <a href="https://github.com/mongodb/mongo-python-driver/blob/3.3.0/pymongo/collection.py#L697" rel="nofollow">_update() method</a>. See also related article: <a href="https://emptysqua.re/blog/pymongo-key-order/" rel="nofollow">PyMongo and Key Order in SubDocuments</a>.</p>

<p><strong>Update</strong> to answer an additional question: </p>

<p>As far as I know, there is no a 'standard' function to convert a nested dictionary to SON. Although you can write a custom <code>dict</code> to <code>SON</code> converter yourself, for example:</p>

<p></p>

<pre><code>def to_son(value):
     for k, v in value.iteritems():
         if isinstance(v, dict):
             value[k] = to_son(v)
         elif isinstance(v, list):
             value[k] = [to_son(x) for x in v]
     return bson.son.SON(value)
# Assuming the order of the dictionary is as you desired. 
to_son(a_nested_dict)
</code></pre>

<p>Or utilise bson as an intermediate format</p>

<p></p>

<pre><code>from bson import CodecOptions, SON, BSON
nested_bson = BSON.encode(a_nested_dict)
nested_son = BSON.decode(nested_bson, codec_options=CodecOptions(document_class=SON))
</code></pre>

<p>Once in <code>SON</code> format, you can convert back to Python dictionary using <a href="http://api.mongodb.com/python/current/api/bson/son.html#bson.son.SON.to_dict" rel="nofollow">SON.to_dict()</a></p>

python mongodb pymongo