Readlines function for an xlsx file works inproper
<p>The goal is  sentiment classification. The steps are to open 3 xlsx files, read them, process with gensim.doc2vec methods and classify with SGDClassificator. Just try to repeat <a href="https://districtdatalabs.silvrback.com/modern-methods-for-sentiment-analysis#disqus_thread" rel="nofollow">this code on doc2vec</a>. Python 2.7</p>

<pre><code>with open('C:/doc2v/trainpos.xlsx','r') as infile:
    pos_reviews = infile.readlines()
with open('C:/doc2v/trainneg.xlsx','r') as infile:
    neg_reviews = infile.readlines()
with open('C:/doc2v/unsup.xlsx','r') as infile:
    unsup_reviews = infile.readlines()
</code></pre>

<p>But it turned out that the resulting lists are not what they are expected to be:</p>

<pre><code>print 'length of pos_reviews is %s' % len(pos_reviews)
&gt;&gt;&gt; length of pos_reviews is 1
</code></pre>

<p>The files contain 18, 1221 and 2203 raws correspondingly. I thought that the lists will have the same number of elements.</p>

<p>The next step is to concatenate all the sentences. </p>

<pre><code>y = np.concatenate((np.ones(len(pos_reviews)), np.zeros(len(neg_reviews))))
x_train, x_test, y_train, y_test = train_test_split(np.concatenate((pos_reviews, neg_reviews)), y, test_size=0.2)
</code></pre>

<p>This leads to the situation when x-train, x-test are lists of sentences as they should be while </p>

<pre><code>y_train = [0.]
y_test = [1.]
</code></pre>

<p>After this division every sentence gets a label:</p>

<pre><code>def labelizeReviews(reviews, label_type):
labelized = []
for i,v in enumerate(reviews):
    label = '%s_%s'%(label_type,i)
    labelized.append(LabeledSentence(v, [label]))
return labelized
x_train = labelizeReviews(x_train, 'TRAIN')
x_test = labelizeReviews(x_test, 'TEST')
unsup_reviews = labelizeReviews(unsup_reviews, 'UNSUP')
</code></pre>

<p>As written in <a href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.concatenate.html" rel="nofollow">the numpy documentation</a>, the arrays should be equal in size. But when I reduce the bigger files to 18 lines, nothing changes.
As I searched on the forum noone has a similar error. I've broken my head what went wrong and how to fix it. Thanks for help!</p>

<p>Generally you can't read Microsoft Excel files as a text files using methods like <code>readlines</code> or <code>read</code>. You should convert files to another format before (good solution is .csv which can be readed by <a href="https://docs.python.org/2/library/csv.html" rel="nofollow">csv</a> module) or use a special python modules like <a href="https://pythonhosted.org/pyexcel/" rel="nofollow" title="pyexcel">pyexcel</a> and <a href="https://openpyxl.readthedocs.io/en/default/tutorial.html#loading-from-a-file" rel="nofollow" title="openpyxl">openpyxl</a> to read .xlsx files directly.</p>

python xlsx readlines doc2vec