Creating lmdb file in the right way
<p>I'm trying to create an lmdb file that contains all of my database images (in order to train CNN).</p>

<p>This is my 'test code', that I took from <a href="http://deepdish.io/2015/04/28/creating-lmdb-in-python/" rel="nofollow">here</a>:</p>

<pre><code>import numpy as np
import lmdb
import caffe
import cv2
import glob

N = 18

# Let's pretend this is interesting data
X = np.zeros((N, 1, 32, 32), dtype=np.uint8)
y = np.zeros(N, dtype=np.int64)

# We need to prepare the database for the size. We'll set it 10 times
# greater than what we theoretically need. There is little drawback to
# setting this too big. If you still run into problem after raising
# this, you might want to try saving fewer entries in a single
# transaction.
map_size = X.nbytes * 10


train_data = [img for img in glob.glob("/home/roishik/Desktop/Thesis/Code/cafe_cnn/third/code/train_images/*png")]
for i , img_path in enumerate(train_data):
    img = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
    X[i]=img
    y[i]=i%2


env = lmdb.open('train', map_size=map_size)
print X
print y
with env.begin(write=True) as txn:
    # txn is a Transaction object
    for i in range(N):
        datum = caffe.proto.caffe_pb2.Datum()
        datum.channels = X.shape[1]
        datum.height = X.shape[2]
        datum.width = X.shape[3]
        datum.data = X[i].tobytes()  # or .tostring() if numpy &lt; 1.9
        print 'a ' + str(X[i])        
        datum.label = int(y[i])
        print 'b ' + str(datum.label)
        str_id = '{:08}'.format(i)                

        txn.put(str_id.encode('ascii'), datum.SerializeToString())
</code></pre>

<p>As you can see I specified random binary labels (0 or 1, for even or odd, respectively). before I create much larger lmdb file I wanna make sure that I'm doing it the right way.</p>

<p>After creating this file I wanted to 'look into the file' and check if it's OK, but I couldn't. the file didn't  open properly using python, Access 2016, and .mdb reader (linux ubunto software). my problems are:</p>

<ol>
<li><p>I don't understand what this code is doing. what is <code>str_id</code>? what is <code>X[i].tobytes</code>? what does the last line do?</p></li>
<li><p>After I run the code, I got 2 files: 'data.mdb' and 'key.mdb'. what are those two? maybe those 2 files are the reason why I can't open the database?</p></li>
</ol>

<p><a href="http://i.stack.imgur.com/WcWCR.png" rel="nofollow"><img src="http://i.stack.imgur.com/WcWCR.png" alt="the two output files"></a></p>

<p>Thanks a lot, really appreciate your help!</p>

<p>You can use the <a href="https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblmdb/mdb_dump.c" rel="nofollow">mdb_dump</a> tool to inspect the contents of the database.</p>

python database neural-network caffe lmdb