DIsplay pandas dataframe (from source block output) inside an org-mode buffer

Question

I'm trying to setup org-mode as a substitute for jupyter notebook using emacs-jupyter. My problem right now is how to display pandas dataframes. When the dataframe contains cells with a lot of text for example, displaying it as an org table is quite messy. and displaying it with :display plain is ok but not very compact.

What I'd like is to have org-mode display dataframes as an inline image, similar to what happens in a jupyter notebook.

I'm trying to do this using wkhtmltoimage with the :post header argument, but so far was not able to get source block to output a raw html text without the export html wrap (#+begin_export html...#+end_export). I tried with :results value raw but that didn't work.

Any suggestions? Or any other way to accomplish what I want?

I use:

• Emacs 27.2
• Spacemacs v.0.300.0

Answer 1

I guess this is close to what you want. I was a little lazy, here is an html2image package in python. you could also do this with wkhtmltoimage

#+BEGIN_SRC sh
pip install html2image
#+END_SRC

here is a post run block to take an html string and make an image

#+name: html2img
#+BEGIN_SRC python :var html="" :results value
from html2image import Html2Image
import hashlib
import os

m = hashlib.sha256(html.encode('utf-8')).hexdigest()

hf = m + '.html'
pf = m + '.png'

with open(hf, 'w') as f:
    f.write('\n'.join(html.strip().split('\n')[1:-1]))

hti = Html2Image()
hti.screenshot(url=hf, save_as=pf)
os.delete(hf)            
return f'''
,#+attr_org: :width 1200
[[./{pf}]]'''
#+END_SRC

Here is an example:

#+BEGIN_SRC jupyter-python :pandoc :post html2img(*this*) :results raw
import pandas as pd

f = pd.DataFrame([['a', 'b','really really really long text']*13, [1, 2, 3]*13])
f
#+END_SRC

That has this output. It is close to what a notebook does for me.

Answer 2

scimax (based on @aadcg's comment) does have a solution to display pandas dataframes and other figures as images in org-mode.

There are a couple examples they provide, but one uses https://github.com/jkitchin/pycse/blob/master/pycse/orgmode.py, which seems to produce your desired results (may require latex and/or pandoc).

from pycse.orgmode import *

Table([['x', 'y'],
       [1, 2],
       [3, 4]],
      headers='firstrow',
      name='org-data',
      caption='The information about the table',
      attributes=[('latex', ':environment longtable :align |l||l|')])

Answer 3

I have the following in my ~/.ipython/profile_default/startup/50-middle.py startup file.

# this make emacs-jupyter only show Org (not HTML) as results
# https://github.com/gregsexton/ob-ipython
try:
  import IPython
  from tabulate import tabulate
  class OrgFormatter(IPython.core.formatters.BaseFormatter):
      def __call__(self, obj):
        try:
          return tabulate(obj, headers='keys', tablefmt='orgtbl', showindex='always')
        except:
          return None

  ip = get_ipython()
  ip.display_formatter.formatters['text/org'] = OrgFormatter()
except ImportError:
  print("no tabulate")

And when I use emacs-jupyter blocks with Pandas, I see tabular output.

Example:

#+BEGIN_SRC jupyter-python
import numpy as np
import pandas as pd
np.random.seed(42)
pd.DataFrame(np.random.random((3,3)), columns=['a','b','c'])
#+END_SRC

#+RESULTS:
|   |         a |        b |        c |
|---+-----------+----------+----------|
| 0 |   0.37454 | 0.950714 | 0.731994 |
| 1 |  0.598658 | 0.156019 | 0.155995 |
| 2 | 0.0580836 | 0.866176 | 0.601115 |