Storing pure python datetime.datetime in pandas DataFrame
<p>Since <code>matplotlib</code> doesn't support <a href="https://github.com/pydata/pandas/issues/8113" rel="nofollow">either</a><code>pandas.TimeStamp</code> <a href="http://stackoverflow.com/questions/22048792/how-do-i-display-dates-when-plotting-in-matplotlib-pyplot">or</a><code>numpy.datetime64</code>, and  there are <a href="http://stackoverflow.com/questions/27472548/pandas-scatter-plotting-datetime">no simple workarounds</a>, I decided to convert a native pandas date column into a pure python <code>datetime.datetime</code> so that scatter plots are easier to make.</p>

<p>However:</p>

<pre><code>t = pd.DataFrame({'date': [pd.to_datetime('2012-12-31')]})
t.dtypes # date    datetime64[ns], as expected
pure_python_datetime_array = t.date.dt.to_pydatetime() # works fine
t['date'] = pure_python_datetime_array # doesn't do what I hoped
t.dtypes # date    datetime64[ns] as before, no luck changing it
</code></pre>

<p>I'm guessing pandas auto-converts the pure python <code>datetime</code> produced by <code>to_pydatetime</code> into its native format. I guess it's convenient behavior in general, but is there a way to override it?</p>

<p>The use of <a href="http://pandas.pydata.org/pandas-docs/stable/timeseries.html#converting-to-python-datetimes" rel="nofollow">to_pydatetime()</a> is correct.</p>

<pre><code>In [87]: t = pd.DataFrame({'date': [pd.to_datetime('2012-12-31'), pd.to_datetime('2013-12-31')]})

In [88]: t.date.dt.to_pydatetime()
Out[88]: 
array([datetime.datetime(2012, 12, 31, 0, 0),
       datetime.datetime(2013, 12, 31, 0, 0)], dtype=object)
</code></pre>

<p>When you assign it back to <code>t.date</code>, it automatically converts it back to <code>datetime64</code></p>

<p><a href="http://pandas.pydata.org/pandas-docs/stable/timeseries.html#overview" rel="nofollow">pandas.Timestamp</a> is a datetime subclass anyway :)</p>

<p>One way to do the plot is to convert the datetime to int64:</p>

<pre><code>In [117]: t = pd.DataFrame({'date': [pd.to_datetime('2012-12-31'), pd.to_datetime('2013-12-31')], 'sample_data': [1, 2]})

In [118]: t['date_int'] = t.date.astype(np.int64)

In [119]: t
Out[119]: 
        date  sample_data             date_int
0 2012-12-31            1  1356912000000000000
1 2013-12-31            2  1388448000000000000

In [120]: t.plot(kind='scatter', x='date_int', y='sample_data')
Out[120]: &lt;matplotlib.axes._subplots.AxesSubplot at 0x7f3c852662d0&gt;

In [121]: plt.show()
</code></pre>

<p><a href="http://i.stack.imgur.com/oOWCU.png" rel="nofollow"><img src="http://i.stack.imgur.com/oOWCU.png" alt="enter image description here"></a></p>

<p>Another workaround is (to not use scatter, but ...):</p>

<pre><code>In [126]: t.plot(x='date', y='sample_data', style='.')
Out[126]: &lt;matplotlib.axes._subplots.AxesSubplot at 0x7f3c850f5750&gt;
</code></pre>

<p>And, the last work around:</p>

<pre><code>In [141]: import matplotlib.pyplot as plt

In [142]: t = pd.DataFrame({'date': [pd.to_datetime('2012-12-31'), pd.to_datetime('2013-12-31')], 'sample_data': [100, 20000]})

In [143]: t
Out[143]: 
        date  sample_data
0 2012-12-31          100
1 2013-12-31        20000
In [144]: plt.scatter(t.date.dt.to_pydatetime()  , t.sample_data)
Out[144]: &lt;matplotlib.collections.PathCollection at 0x7f3c84a10510&gt;

In [145]: plt.show()
</code></pre>

<p><a href="http://i.stack.imgur.com/Xh4ZE.png" rel="nofollow"><img src="http://i.stack.imgur.com/Xh4ZE.png" alt="enter image description here"></a></p>

<p>This has an issue at <a href="https://github.com/pydata/pandas/issues/8113" rel="nofollow">github</a>, which is open as of now.</p>

python python-3.x datetime pandas