interactive conditional histogram bucket slicing data visualization
<p>I have a df that looks like: </p>

<pre><code>df.head()
Out[1]:
        A   B   C
city0   40  12  73
city1   65  56  10
city2   77  58  71
city3   89  53  49
city4   33  98  90
</code></pre>

<p>An example df can be created by the following code:</p>

<pre><code>df = pd.DataFrame(np.random.randint(100,size=(1000000,3)), columns=list('ABC'))

indx = ['city'+str(x) for x in range(0,1000000)]
df.index = indx
</code></pre>

<p>What I want to do is:</p>

<p>a) determine appropriate histogram bucket lengths for column A and assign each city to a bucket for column A</p>

<p>b) determine appropriate histogram bucket lengths for column B and assign each city to a bucket for column B</p>

<p>Maybe the resulting df looks like (or is there a better built in way in pandas?)</p>

<pre><code>    df.head()
    Out[1]:
            A   B   C  Abkt Bbkt
    city0   40  12  73  2  1
    city1   65  56  10  4  3
    city2   77  58  71  4  3
    city3   89  53  49  5  3
    city4   33  98  90  2  5
</code></pre>

<p>Where Abkt and Bbkt are histogram bucket identifiers:</p>

<pre><code>1-20 = 1
21-40 = 2
41-60 = 3
61-80 = 4
81-100 = 5
</code></pre>

<p>Ultimately, I want to better understand the behavior of each city with respect to columns A, B and C and be able to answer questions like:</p>

<p>a) What does the distribution of Column A (or B) look like - i.e. what buckets are most/least populated.</p>

<p>b) Conditional on a particular slice/bucket of Column A, what does the distribution of Column B look like - i.e. what buckets are most/least populated.</p>

<p>c) Conditional on a particular slice/bucket of Column A and B, what does the behavior of C look like.</p>

<p>Ideally, I want to be able to visualize the data (heat maps, region identifiers etc). I'm a relative pandas/python newbie and don't know what is possible to develop. </p>

<p>If the SO community can kindly provide code examples of how I can do what I want (or a better approach if there are better pandas/numpy/scipy built in methods) I would be grateful. </p>

<p>As well, any pointers to resources that can help me better summarize/slice/dice my data and be able to visualize at intermediate steps as I proceed with my analysis. </p>

<p><strong>UPDATE:</strong></p>

<p>I am following some of the suggestions in the comments. </p>

<p>I tried:</p>

<p>1) <code>df.hist()</code></p>

<pre><code>ValueError: The first argument of bincount must be non-negative
</code></pre>

<p>2) <code>df[['A']].hist(bins=10,range=(0,10))</code></p>

<pre><code>array([[&lt;matplotlib.axes._subplots.AxesSubplot object at 0x000000A2350615C0&gt;]], dtype=object)
</code></pre>

<p>Isn't <code>#2</code> suppose to show a plot? instead of producing an object that is not rendered? I am using <code>jupyter notebook</code>. </p>

<p>Is there something I need to turn-on / enable in <code>Jupyter Notebook</code> to render the histogram objects? </p>

<p><strong>UPDATE2:</strong></p>

<p>I solved the rendering problem by: <a href="http://stackoverflow.com/questions/10511024/in-ipython-notebook-pandas-is-not-displying-the-graph-i-try-to-plot">in Ipython notebook, Pandas is not displying the graph I try to plot.</a></p>

<p><strong>UPDATE3:</strong></p>

<p>As per suggestions from the comments, I started looking through <a href="http://pandas.pydata.org/pandas-docs/stable/visualization.html" rel="nofollow">pandas visualization</a>, <a href="http://bokeh.pydata.org/en/latest/" rel="nofollow">bokeh</a> and <a href="http://stanford.edu/~mwaskom/software/seaborn/" rel="nofollow">seaborn</a>. However, I'm not sure how I can create linkages between plots. </p>

<p>Lets say I have 10 variables. I want to explore them but since 10 is a large number to explore at once, lets say I want to explore 5 at any given time (r,s,t,u,v). </p>

<p>If I want an interactive hexbin with marginal distributions plot to examine the relationship between r &amp; s, how do I also see the distribution of t, u and v given interactive region selections/slices of r&amp;s (polygons).</p>

<p>I found hexbin with marginal distribution plot here <a href="http://stanford.edu/~mwaskom/software/seaborn/examples/hexbin_marginals.html" rel="nofollow">hexbin plot</a>: </p>

<p><strong>But:</strong></p>

<p>1) How to make this interactive (allow selections of polygons)</p>

<p>2) How to link region selections of r &amp; s to other plots, for example 3 histogram plots of t,u, and v (or any other type of plot).</p>

<p>This way, I can navigate through the data more rigorously and explore the relationships in depth.</p>

<p>As a newbie with insufficient rep, I can't comment, so I'm putting this here as an "answer," though it shouldn't be treated as one; these are just some incomplete suggestions in the same vein as the comments. </p>

<p>Along with the others, I like <code>seaborn</code> though I'm not sure those plots are interactive in the way you are seeking. While I haven't used <code>bokeh</code>, my understanding is that it provides more in the way of interactivity, but regardless of the package, as you move beyond 3 and 4 variables, you can only cram so much into one (family of) charts. </p>

<p>As for in your table directly, the aforementioned <code>df.hist()</code> (by <a href="http://stackoverflow.com/users/5285918/lanery">lanery</a>) is a good start. Once you have those bins, you can then play with the <a href="http://pandas.pydata.org/pandas-docs/stable/groupby.html" rel="nofollow">immensely powerful</a> <code>df.groupby()</code> function. I've been using pandas for two years now, and that function STILL blows my mind. While not interactive, it will definitely help you slice and dice your data as you see fit.</p>

python pandas data-visualization seaborn bokeh