Flask-WTF: CSRF token missing
<p>What seemed like a simple bug - a form submission that won't go through due to a "CSRF token missing" error - has turned into a day of hair pulling.  I have gone through every SO article related to Flask or Flask-WTF and missing CSRF tokens, and nothing seems to be helping.  </p>

<p>Here are the details:</p>

<p>Following <a href="http://stackoverflow.com/a/21501593/5578714" title="Martijin&#39;s guidelines">Martijin's guidelines</a> to an earlier question:</p>

<blockquote>
  <p>The Flask-WTF CSRF infrastructure rejects a token if:</p>
  
  <p>1) the token is missing. Not the case here, you can see the token in the form.</p>
</blockquote>

<p>The token is definitely present in my form, and being POST'ed successfully</p>

<blockquote>
  <p>2) it is too old (default expiration is set to 3600 seconds, or an hour).
  Set the TIME_LIMIT attribute on forms to override this. Probably not the 
  case here.</p>
</blockquote>

<p>Also OK for me - the token is well within the default expiration time </p>

<blockquote>
  <p>3) if no 'csrf_token' key is found in the current session. You can 
  apparently see the session token, so that's out too.</p>
</blockquote>

<p>In my case, session['csrf_token'] is properly set and seen by Flask</p>

<blockquote>
  <p>4) If the HMAC signature doesn't match; the signature is based on the
  random value set in the session under the 'csrf_token' key, the
  server-side secret, and the expiry timestamp in the token.</p>
</blockquote>

<p><strong>This is my problem</strong>.  The HMAC comparison between the submitted form's CSRF and the session CSRF fails.  And yet I don't know how to solve it.  I've been desperate enough (as with the other questioner) to dig into Flask-WTF code and set debugging messages to find out what's going on.  As best I can tell, it's working like this:</p>

<p>1) <code>generate_csrf_token()</code> in "form.py" (Flask-WTF) wants to generates a CSRF token.  So it calls:</p>

<p>2) <code>generate_csrf()</code> in "csrf.py".  That function generates a new session['csrf_token'] if one does not exist.  <strong>In my case, this always happens</strong> - although other session variables appear to persist between requests, my debugging shows that I <strong>never</strong> have a 'csrf_token' in my session at the start of a request.  Is this normal?</p>

<p>3) The generated token is returned and presumably incorporated into the form variable when I render hidden fields on the template.  (again, debugging shows that this token is present in the form and properly submitted and received)</p>

<p>4) Next, the form is submitted.</p>

<p>5) Now, <code>validate_csrf</code> in csrf.py is called.  But since another request has taken place, and generate_csrf() has generated a new session CSRF token, the two timestamps for the two tokens (in session and from the form) will not match.  And since the CSRF is made up in part by expiration dates, therefore validation fails.</p>

<p>I suspect the problem is in step #2, where a new token is being generated for every request.  But I have no clue why <strong>other</strong> variables in my session are persisting from request to request, but not "csrf_token".</p>

<p>There is no weirdness going on with SECRET_KEY or WTF_CSRF_SECRET_KEY either (they are properly set).</p>

<p>Anyone have any ideas?</p>

<p>I figured it out.  It appears to be a cookie/session limit (which probably beyond Flask's control) and a silent discarding of session variables when the limit is hit (which seems more like a bug).</p>

<p>Here's an example:</p>

<p><strong>templates/hello.html</strong></p>

<pre><code>&lt;p&gt;{{ message|safe }}&lt;/p&gt;
&lt;form name="loginform" method="POST"&gt;
  {{ form.hidden_tag() }}
  {{ form.submit_button() }}
&lt;/form&gt;
</code></pre>

<p><strong>myapp.py</strong></p>

<pre><code>from flask import Flask, make_response, render_template, session
from flask_restful import Resource, Api
from flask_wtf import csrf, Form
from wtforms import SubmitField

app = Flask(__name__)
app.secret_key = '5accdb11b2c10a78d7c92c5fa102ea77fcd50c2058b00f6e'
api = Api(app)

num_elements_to_generate = 500

class HelloForm(Form):
    submit_button = SubmitField('Submit This Form')

class Hello(Resource):
    def check_session(self):
        if session.get('big'):
            message = "session['big'] contains {} elements&lt;br&gt;".format(len(session['big']))
        else:
            message = "There is no session['big'] set&lt;br&gt;"
        message += "session['secret'] is {}&lt;br&gt;".format(session.get('secret'))
        message += "session['csrf_token'] is {}&lt;br&gt;".format(session.get('csrf_token'))
        return message

    def get(self):
        myform = HelloForm()
        session['big'] = list(range(num_elements_to_generate))
        session['secret'] = "A secret phrase!"
        csrf.generate_csrf()
        message = self.check_session()
        return make_response(render_template("hello.html", message=message, form=myform), 200, {'Content-Type': 'text/html'})

    def post(self):
        csrf.generate_csrf()
        message = self.check_session()
        return make_response("&lt;p&gt;This is the POST result page&lt;/p&gt;" + message, 200, {'Content-Type': 'text/html'})

api.add_resource(Hello, '/')

if __name__ == '__main__':
    app.run(debug=True)
</code></pre>

<p>Run this with <code>num_elements_to_generate</code> set to 500 and you'll get something like this:</p>

<pre><code>session['big'] contains 500 elements
session['secret'] is 'A secret phrase!'
session['csrf_token'] is a6acb57eb6e62876a9b1e808aa1302d40b44b945
</code></pre>

<p>and a "Submit This Form" button.  Click the button, and you'll get:</p>

<pre><code>This is the POST result page
session['big'] contains 500 elements
session['secret'] is 'A secret phrase!'
session['csrf_token'] is a6acb57eb6e62876a9b1e808aa1302d40b44b945
</code></pre>

<p>All well and good.  But now change <code>num_elements_to_generate</code> to 3000, clear your cookies, rerun the app and access the page.  You'll get something like:</p>

<pre><code>session['big'] contains 3000 elements
session['secret'] is 'A secret phrase!'
session['csrf_token'] is 709b239857fd68a4649deb864868897f0dc0a8fd
</code></pre>

<p>and a "Submit This Form" button.  Click the button, and this time you'll get:</p>

<pre><code>This is the POST result page
There is no session['big'] set
session['secret'] is 'None'
session['csrf_token'] is 13553dce0fbe938cc958a3653b85f98722525465
</code></pre>

<p>3,000 digits stored in the session variable is too much, so the session variables do not persist between requests.  Interestingly they DO exist in the session on the first page (no matter how many elements you generate), but they will not survive to the next request.  And Flask-WTF, since it does not see a <code>csrf_token</code> in the session when the form is posted, generates a new one.  If this was a form validation step, the CSRF validation would fail.</p>

<p>This seems to be a known Flask (or Werkzeug) bug, <a href="https://github.com/pallets/werkzeug/pull/780" rel="nofollow">with a pull request here</a>.  I'm not sure why Flask isn't generating a warning here - unless it is somehow technically unfeasible, it's an unexpected and unpleasant surprise that it is silently failing to keep the session variables when the cookie is too big.</p>

python session flask wtforms flask-wtforms