How to convert/rename categories in dask
<p>I'm trying to rename categories of a dtype 'category' column of a dask dataframe to a series of numbers from 1 to len(categories).</p>

<p>In pandas I was doing it like this:</p>

<pre><code>df['name'] = dd.Categorical(df.name).codes
</code></pre>

<p>but in dask this does not work:</p>

<pre><code>Traceback (most recent call last):
  File "example.py", line 47, in &lt;module&gt;
    sys.exit(main(sys.argv))
  File "example.py", line 25, in main
    df['name'] = dd.Categorical(df.name).codes
AttributeError: module 'dask.dataframe' has no attribute 'Categorical'
</code></pre>

<p>So I tried to get the categories and set them as explained in the <a href="http://pandas.pydata.org/pandas-docs/stable/categorical.html#renaming-categories" rel="nofollow">pandas documentation</a>.</p>

<pre><code>df['name'] = df['name'].astype('category')
cats = df.name.cat.categories
df.name.cat.categories = range(1, len(cats))
</code></pre>

<p>But this yields an exception as well:</p>

<pre><code>Traceback (most recent call last):
  File "example.py", line 50, in &lt;module&gt;
    sys.exit(main(sys.argv))
  File "example.py", line 26, in main
    cats = df.name.cat.categories
  File "[...]/dask/dataframe/core.py", line 3207, in __getattr__
    return self._property_map(key)
  File "[...]/dask/dataframe/core.py", line 3186, in _property_map
    out = self.getattr(self._series._meta_nonempty, key)
  File "[...]/dask/dataframe/core.py", line 258, in _meta_nonempty
    return meta_nonempty(self._meta)
  File "[...]/dask/dataframe/utils.py", line 329, in meta_nonempty
    return _nonempty_series(x, idx)
  File "[...]/dask/dataframe/utils.py", line 308, in _nonempty_series
    entry = s.cat.categories[0]
  File "[...]/pandas-0.19.0-py3.5-linux-x86_64.egg/pandas/indexes/base.py", line 1393, in __getitem__
    return getitem(key)
IndexError: index 0 is out of bounds for axis 0 with size 0
</code></pre>

<p>How can I rename the categories in a dask dataframe column?</p>

<p>You probably want to look at <code>df.column.cat.codes</code>, which has the numbers you're looking for.  Lets work through an example:</p>

<h3>Create Toy dataset in Pandas</h3>

<pre><code>In [1]: import pandas as pd

In [2]: df = pd.DataFrame({'x': ['a', 'b', 'a']})

In [3]: df['x'] = df.x.astype('category')

In [4]: df
Out[4]: 
   x
0  a
1  b
2  a
</code></pre>

<h3>Convert to a Dask.dataframe</h3>

<pre><code>In [5]: import dask.dataframe as dd

In [6]: ddf = dd.from_pandas(df, npartitions=2)
</code></pre>

<h3>Inspect <code>.cat.codes</code> attribute</h3>

<pre><code>In [7]: ddf.x.cat.codes
Out[7]: 
dd.Series&lt;getattr..., npartitions=1, divisions=(0, 2)&gt;

Dask Series Structure:
divisions
0    int8
2     ...
dtype: int8

In [8]: ddf.x.cat.codes.compute()
Out[8]: 
0    0
1    1
2    0
dtype: int8
</code></pre>

<h3>Overwrite category series with codes series</h3>

<pre><code>In [9]: ddf['x'] = ddf.x.cat.codes

In [10]: ddf.compute()
Out[10]: 
   x
0  0
1  1
2  0
</code></pre>

python dask