Cleanest way to combine reduce and map in Python
<p>I'm doing a little deep learning, and I want to grab the values of all hidden layers.  So I end up writing functions like this:</p>

<pre><code>def forward_pass(x, ws, bs):
    activations = []
    u = x
    for w, b in zip(ws, bs):
        u = np.maximum(0, u.dot(w)+b)
        activations.append(u)
    return activations
</code></pre>

<p>If I didn't have to get the intermediate values, I'd use the much less verbose form:</p>

<pre><code>out = reduce(lambda u, (w, b): np.maximum(0, u.dot(w)+b), zip(ws, bs), x)
</code></pre>

<p>Bam.  All one line, nice and compact.  But I can't keep any of the intermediate values.</p>

<p>So, what is there any way to have my cake (nice compact one-liner) and eat it too (return intermediate values)?</p>

<p><strong>Edit:  My conclusions so far:</strong><br>
In Python 2.x, there is no clean one-liner for this.<br>
In Python 3, there is <code>itertools.accumulate</code>, but it is still not really clean because it doesn't accept an "initial" input, as <code>reduce</code> does.<br>
In Python 4, I hereby request a "map-reduce comprehension":  </p>

<pre><code>activations = [u=np.maximum(0, u.dot(w)+b) for w, b in zip(ws, bs) from u=x]  
</code></pre>

<p>Which would also give a useful job to that <code>from</code> keyword, which normally just sits around doing nothing after all the imports are done.</p>

<p>In Python 2.x, there is no clean one-liner for this.  </p>

<p>In Python 3, there is itertools.accumulate, but it is still not really clean because it doesn't accept an "initial" input, as reduce does.  </p>

<p>Here is a function that, while not as nice as a built-in comprehension syntax, does the job.</p>

<pre><code>def reducemap(func, sequence, initial=None, include_zeroth = False):
    """
    A version of reduce that also returns the intermediate values.
    :param func: A function of the form x_i_plus_1 = f(x_i, params_i)
        Where:
            x_i is the value passed through the reduce.
            params_i is the i'th element of sequence
            x_i_plus_i is the value that will be passed to the next step
    :param sequence: A list of parameters to feed at each step of the reduce.
    :param initial: Optionally, an initial value (else the first element of the sequence will be taken as the initial)
    :param include_zeroth: Include the initial value in the returned list.
    :return: A list of length: len(sequence), (or len(sequence)+1 if include_zeroth is True) containing the computed result of each iteration.
    """
    if initial is None:
        val = sequence[0]
        sequence = sequence[1:]
    else:
        val = initial
    results = [val] if include_zeroth else []
    for s in sequence:
        val = func(val, s)
        results.append(val)
    return results
</code></pre>

<p>Tests:</p>

<pre><code>assert reducemap(lambda a, b: a+b, [1, 2, -4, 3, 6, -7], initial=0) == [1, 3, -1, 2, 8, 1]
assert reducemap(lambda a, b: a+b, [1, 2, -4, 3, 6, -7]) == [3, -1, 2, 8, 1]
assert reducemap(lambda a, b: a+b, [1, 2, -4, 3, 6, -7], include_zeroth=True) == [1, 3, -1, 2, 8, 1]
</code></pre>

python numpy deep-learning reduce