Scrapy - send form data with multiple options
<p>I'm using scrapy to parse a website that has the following form:</p>

<pre><code>&lt;form id="form1"...&gt;
    &lt;select name="codes" multiple="multiple"...&gt;
        &lt;option value="0"&gt;Option one&lt;/option&gt;
        &lt;option value="1"&gt;Option two&lt;/option&gt;
        &lt;option value="2"&gt;Option three&lt;/option&gt;
        ....
    &lt;/select&gt;
&lt;/form&gt;
</code></pre>

<p>I'm filling and submitting the form with the following code:</p>

<pre><code>submit_form = FormRequest.from_response(response,
                                        formxpath="//form[@id='form1']",
                                        formdata={'codes': '0'},
                                        callback=self.parse_table)
yield submit_form
</code></pre>

<p>How can I submit multiple codes in the form data? I've tried:</p>

<pre><code>formdata={'codes': '["0", "1", "2"]'},
formdata={'codes': ['0', '1', '2']},
</code></pre>

<p>Without any luck.</p>

<p><strong>EDIT:</strong></p>

<p>The form has additional input controls, some of them hidden, that are correctly passed in the form. What I'm seeing after the form submit is like the server is returning to the same page with the form, when I'm expecting a new page with a table that actually has the data that I want to retrieve.</p>

<p>I don't know a lot about the server backend more than that it was built with .NET 2.0. It is a very old site from a goverment dependency.</p>

<p>Thanks.</p>

<p>to send form with multiple options, you should try passing formdata in following format</p>

<pre><code>formdata = {}
formdata['codes[]'] = ["0","1","2","3"]

yield scrapy.FormRequest.from_response(
            response=response,
            formid='UserLoginForm',
            formdata=formdata,
            callback=self.search_result,
        )
</code></pre>

<p>to verify that codes value is submitted in required format, here is the output of request_body</p>

<pre><code>codes%5B%5D=0&amp;codes%5B%5D=1&amp;codes%5B%5D=2&amp;codes%5B%5D=3
</code></pre>

<p>unquote</p>

<pre><code>codes[]=0&amp;codes[]=1&amp;codes[]=2&amp;codes[]=3
</code></pre>

<p>split on &amp;</p>

<pre><code>codes[]=0
codes[]=1
codes[]=2
codes[]=3
</code></pre>

python forms scrapy