0

The result of a certain Python (statsmodel module) code in org-mode is abbreviated:

#+BEGIN_SRC python
  import pandas as pd
  import statsmodels.api as sm

  auto_df=pd.read_csv('Auto.csv', usecols= ['mpg', 'horsepower'], na_values= ['?']) 

  auto_df['horsepower'] = pd.to_numeric(auto_df['horsepower'])

  auto_df.dropna(inplace=True)

  X = auto_df['horsepower']
  X = sm.add_constant(X)

  olsmod = sm.OLS(auto_df['mpg'], X)
  olsres = olsmod.fit()
  pred = olsres.get_prediction([1,98])
  return(pred.summary_frame())
#+END_SRC

#+RESULTS:
: mean   mean_se  ...  obs_ci_lower  obs_ci_upper
: 0  24.467077  0.251262  ...     14.809396     34.124758
: 
: [1 rows x 6 columns]

I'm referring to the ellipses in the table below #+Results.

How can I see the full results table?

A comment suggested this could be a a pandas issue, but I don't think so since the output is from a statsmodels function.

And I don't think it's because of statsmodels because here, the full is pred.summary_frame() results table is shown:

#+BEGIN_SRC python
  import pandas as pd
  import statsmodels.api as sm
  import numpy as np

  nsample = 50
  sig = 0.25
  X = np.linspace(0, 20, nsample)
  X = sm.add_constant(X)
  beta = [5., 0.5]
  y_true = np.dot(X, beta)
  y = y_true + sig * np.random.normal(size=nsample)

  olsmod = sm.OLS(y, X)
  olsres = olsmod.fit()

  pred = olsres.get_prediction([1,17])

  return(pred.summary_frame())
#+END_SRC

#+RESULTS:
: mean  mean_se  mean_ci_lower  mean_ci_upper  obs_ci_lower  obs_ci_upper
: 0  13.559089   0.0522      13.454134      13.664043     13.069874     14.048303

One more note, the abbreviate one, from my first example, shows fully in the python shell.

Improve this question
3
  • Please show the complete source block (and show it in text: Org mode prides itself as being your life in plain text - images are a poor substitute). If you are using pandas, then that's where you should concentrate: Org mode probably has nothing to do with it. See e.g.this question
    – NickD
    Commented Mar 26, 2021 at 21:16
  • Thanks @NickD. I updated the question. As I mentioned, since the output in the shell is complete I don't think it's related to a pandas or, more likely, statsmodels option. Agreed?
    – Quinn Culver
    Commented Mar 27, 2021 at 1:20
  • The other advantage of posting text is that one can cut and paste and try exactly what you tried - so thanks for doing that (but note that it was to your advantage to do that: it's unlikely that anybody would type in a program from a screenshot, but if you provide text that can be cut and pasted, it becomes a bit more attractive :-). In any case, I still believe that it's a pandas "feature": see my answer.
    – NickD
    Commented Mar 27, 2021 at 3:19

1 Answer 1

Reset to default
1

This is almost certainly a pandas "feature" and has nothing to do with Org mode, as pointed out in a comment. The following modification to the code produces untruncated output for me:

#+BEGIN_SRC python
  import pandas as pd
  import statsmodels.api as sm
  import numpy as np

  # tell pandas to not worry about truncation
  pd.set_option('display.max_columns', None)
  pd.set_option('display.max_colwidth', None)
  # and to not worry about the width of the "screen"
  pd.set_option('display.width', None)

  nsample = 50
  sig = 0.25
  X = np.linspace(0, 20, nsample)
  X = sm.add_constant(X)
  beta = [5., 0.5]
  y_true = np.dot(X, beta)
  y = y_true + sig * np.random.normal(size=nsample)

  olsmod = sm.OLS(y, X)
  olsres = olsmod.fit()

  pred = olsres.get_prediction([1,17])

  return(pred.summary_frame())
#+END_SRC

#+RESULTS:
: mean   mean_se  mean_ci_lower  mean_ci_upper  obs_ci_lower  obs_ci_upper
: 0  13.406873  0.049386      13.307576       13.50617     12.944031     13.869715

The alignment however is bad: org-babel takes the result, stringifies it and writes it to a temporary file. Since it's just a string, it just gets printed out.

EDIT: I can get good alignment too with a :results drawer header:

#+BEGIN_SRC python :results drawer
 ... no changes to the program ...
#+END_SRC

#+RESULTS:
:results:
        mean   mean_se  mean_ci_lower  mean_ci_upper  obs_ci_lower  obs_ci_upper
0  13.439094  0.049308      13.339955      13.538234     12.976986     13.901202
:end:
Improve this answer
3
  • You are so right! I'd tried the first ` pd.set_option('display.max_columns', None)', (and forgot to mention that in my update) but I guess more was needed. Should I delete this question since it's not about emacs, after all?
    – Quinn Culver
    Commented Mar 27, 2021 at 3:53
  • It's probably useful even if not emacs-related. If people feel strongly about it, they can vote to close the question and/or the moderator can decide to delete it.
    – NickD
    Commented Mar 27, 2021 at 4:12
  • 1
    Thanks so much, @NickD. FYI, I'm actually outputting pandas dataframes as org tables using the pd2org function defined here. It's really nice. I wouldn't have asked this question had I known the pred.summary_frame() is a pandas dataframe. (I thought I'd checked it's type, but I guess not.) Learned something useful anyway!
    – Quinn Culver
    Commented Mar 27, 2021 at 19:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.