Adding new values to empty nested lists
<p>This is related to <a href="http://stackoverflow.com/questions/6339235/how-to-append-to-the-end-of-an-empty-list">How to append to the end of an empty list?</a>, but I don't have enough reputation yet to comment there, so I posted a new question here.</p>

<p>I need to append terms onto an empty list of lists. I start with:</p>

<pre><code>Talks[eachFilename][TermVectors]=
      [['paragraph','1','text'],
       ['paragraph','2','text'],
       ['paragraph','3','text']]
</code></pre>

<p>I want to end with </p>

<pre><code>Talks[eachFilename][SomeTermsRemoved]=
      [['paragraph','text'],
       ['paragraph','2'],
       ['paragraph']]
</code></pre>

<p><code>Talks[eachFilename][SomeTermsRemoved]</code> starts empty. I can't specify that I want: </p>

<pre><code>Talks[eachFilename][SomeTermsRemoved][0][0]='paragraph'
Talks[eachFilename][SomeTermsRemoved][0][1]='text'
Talks[eachFilename][SomeTermsRemoved][1][0]='paragraph'
</code></pre>

<p>etc... (IndexError: list index out of range).  If I force populate the string  and then try to change it, I get a strings are immutable error.</p>

<p>So, how do I specify that I want <code>Talks[eachFilename][SomeTermsRemoved][0]</code> to be <code>['paragraph','text']</code>, and <code>Talks[eachFilename][SomeTermsRemoved][1]</code> to be <code>['paragraph','2']</code> etc?</p>

<p><code>.append</code> works, but only generates a single long column, not a set of lists.</p>

<p>To be more specific, I have a number of lists that are initialized inside a dict</p>

<pre><code>Talks = {}
Talks[eachFilename]= {}
Talks[eachFilename]['StartingText']=[]
Talks[eachFilename]['TermVectors']=[]
Talks[eachFilename]['TermVectorsNoStops']=[]
</code></pre>

<p><code>eachFilename</code> gets populated from a list of text files, e.g.:</p>

<pre><code>Talks[eachFilename]=['filename1','filename2']
</code></pre>

<p><code>StartingText</code> has several long lines of text (individual paragraphs)</p>

<pre><code>Talks[filename1][StartingText]=['This is paragraph one','paragraph two']
</code></pre>

<p>TermVectors are populated by the NLTK package with a list of terms, still grouped in the original paragraphs:</p>

<pre><code>Talks[filename1][TermVectors]=
     [['This','is','paragraph','one'],
      ['paragraph','two']]
</code></pre>

<p>I want to further manipulate the <code>TermVectors</code>, but keep the original paragraph list structure. This creates a list with 1 term per line:</p>

<pre><code>for eachFilename in Talks:
    for eachTerm in range( 0, len( Talks[eachFilename]['TermVectors'] ) ):
        for term in Talks[eachFilename]['TermVectors'][ eachTerm ]:
            if unicode(term) not in stop_words:
                Talks[eachFilename]['TermVectorsNoStops'].append( term )
</code></pre>

<p>Result (I lose my paragraph structure):</p>

<pre><code>Talks[filename1][TermVectorsNoStops]=
     [['This'],
      ['is'],
      ['paragraph'],
      ['one'],
      ['paragraph'],
      ['two']]
</code></pre>

<p>The errors you are reporting (strings immutable?) don't make any sense unless your list is actually not empty but already populated with strings. In any event, if you start with an empty list, then the simplest way to populate it is by appending:</p>

<pre><code>&gt;&gt;&gt; talks = {}
&gt;&gt;&gt; talks['each_file_name'] = {}
&gt;&gt;&gt; talks['each_file_name']['terms_removed'] = []
&gt;&gt;&gt; talks['each_file_name']['terms_removed'].append(['paragraph','text'])
&gt;&gt;&gt; talks['each_file_name']['terms_removed'].append(['paragraph','2'])
&gt;&gt;&gt; talks['each_file_name']['terms_removed'].append(['paragraph'])
&gt;&gt;&gt; talks
{'each_file_name': {'terms_removed': [['paragraph', 'text'], ['paragraph', '2'], ['paragraph']]}}
&gt;&gt;&gt; from pprint import pprint
&gt;&gt;&gt; pprint(talks)
{'each_file_name': {'terms_removed': [['paragraph', 'text'],
                                      ['paragraph', '2'],
                                      ['paragraph']]}}
</code></pre>

<p>If you have an empty list and try to assign to it by using indexing, it will throw an error:</p>

<pre><code>&gt;&gt;&gt; empty_list = []
&gt;&gt;&gt; empty_list[0] = 10
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
IndexError: list assignment index out of range
</code></pre>

<p>As an aside, code like this:</p>

<pre><code>for eachFilename in Talks:
    for eachTerm in range( 0, len( Talks[eachFilename]['TermVectors'] ) ):
        for term in Talks[eachFilename]['TermVectors'][ eachTerm ]:
            if unicode(term) not in stop_words:
                Talks[eachFilename]['TermVectorsNoStops'].append( term )
</code></pre>

<p>Is very far from proper Python style. Don't use <em>camelCase</em>, use <em>snake_case</em>. Don't capitalize variables. Also, in your mid-level for-loop, you use <code>for eachTerm in range(0, len(Talks[eachFilename]['TermVectors']</code>, but <code>eachTerm</code> is an <code>int</code>, so it makes more sense to use the standard <code>i</code> <code>j</code> or <code>k</code>. Even <code>idx</code>.</p>

<p>Anyway, there is no reason why that code should be turning this:</p>

<pre><code>Talks[filename1][TermVectors] =
     [['This','is','paragraph','one'],
      ['paragraph','two']] 
</code></pre>

<p>Into this:</p>

<pre><code>Talks[filename1][TermVectors] =
     [['This'],
      ['is'],
      ['paragraph'],
      ['one'],
      ['paragraph'],
      ['two']]
</code></pre>

<p>Here is a reproducible example (I've made this for you, BUT YOU SHOULD DO THIS YOURSELF BEFORE POSTING A QUESTION):</p>

<pre><code>&gt;&gt;&gt; pprint(talks)
{'file1': {'no_stops': [],
           'term_vectors': [['This', 'is', 'paragraph', 'one'],
                            ['paragraph', 'two']]},
 'file2': {'no_stops': [],
           'term_vectors': [['This', 'is', 'paragraph', 'three'],
                            ['paragraph', 'four']]}}
&gt;&gt;&gt; for file in talks:
...   for i in range(len(talks[file]['term_vectors'])):
...     for term in talks[file]['term_vectors'][i]:
...       if term not in stop_words:
...         talks[file]['no_stops'].append(term)
... 
&gt;&gt;&gt; pprint(file)
'file2'
&gt;&gt;&gt; pprint(talks)
{'file1': {'no_stops': ['This', 'paragraph', 'one', 'paragraph'],
           'term_vectors': [['This', 'is', 'paragraph', 'one'],
                            ['paragraph', 'two']]},
 'file2': {'no_stops': ['This', 'paragraph', 'paragraph', 'four'],
           'term_vectors': [['This', 'is', 'paragraph', 'three'],
                            ['paragraph', 'four']]}}
&gt;&gt;&gt; 
</code></pre>

<p>The more pythonic approach would be something like the following:</p>

<pre><code>&gt;&gt;&gt; pprint(talks)
{'file1': {'no_stops': [],
           'term_vectors': [['This', 'is', 'paragraph', 'one'],
                            ['paragraph', 'two']]},
 'file2': {'no_stops': [],
           'term_vectors': [['This', 'is', 'paragraph', 'three'],
                            ['paragraph', 'four']]}}
&gt;&gt;&gt; for file in talks.values():
...   file['no_stops'] = [[term for term in sub if term not in stop_words] for sub in file['term_vectors']]
... 
&gt;&gt;&gt; pprint(talks)
{'file1': {'no_stops': [['This', 'paragraph', 'one'], ['paragraph']],
           'term_vectors': [['This', 'is', 'paragraph', 'one'],
                            ['paragraph', 'two']]},
 'file2': {'no_stops': [['This', 'paragraph'], ['paragraph', 'four']],
           'term_vectors': [['This', 'is', 'paragraph', 'three'],
                            ['paragraph', 'four']]}}
&gt;&gt;&gt; 
</code></pre>

python list assign