eland.eland_to_pandas#
- eland.eland_to_pandas(ed_df: DataFrame, show_progress: bool = False) DataFrame #
Convert an eland.Dataframe to a pandas.DataFrame
Note: this loads the entire Elasticsearch index into in core pandas.DataFrame structures. For large indices this can create significant load on the Elasticsearch cluster and require signficant memory
Parameters#
- ed_df: eland.DataFrame
The source eland.Dataframe referencing the Elasticsearch index
- show_progress: bool
Output progress of option to stdout? By default False.
Returns#
- pandas.Dataframe
pandas.DataFrame contains all rows and columns in eland.DataFrame
Examples#
>>> ed_df = ed.DataFrame('http://localhost:9200', 'flights').head() >>> type(ed_df) <class 'eland.dataframe.DataFrame'> >>> ed_df AvgTicketPrice Cancelled ... dayOfWeek timestamp 0 841.265642 False ... 0 2018-01-01 00:00:00 1 882.982662 False ... 0 2018-01-01 18:27:00 2 190.636904 False ... 0 2018-01-01 17:11:14 3 181.694216 True ... 0 2018-01-01 10:33:28 4 730.041778 False ... 0 2018-01-01 05:13:00 [5 rows x 28 columns]
Convert eland.DataFrame to pandas.DataFrame (Note: this loads entire Elasticsearch index into core memory)
>>> pd_df = ed.eland_to_pandas(ed_df) >>> type(pd_df) <class 'pandas.core.frame.DataFrame'> >>> pd_df AvgTicketPrice Cancelled ... dayOfWeek timestamp 0 841.265642 False ... 0 2018-01-01 00:00:00 1 882.982662 False ... 0 2018-01-01 18:27:00 2 190.636904 False ... 0 2018-01-01 17:11:14 3 181.694216 True ... 0 2018-01-01 10:33:28 4 730.041778 False ... 0 2018-01-01 05:13:00 [5 rows x 28 columns]
Convert eland.DataFrame to pandas.DataFrame and show progress every 10000 rows
>>> pd_df = ed.eland_to_pandas(ed.DataFrame('http://localhost:9200', 'flights'), show_progress=True) 2020-01-29 12:43:36.572395: read 10000 rows 2020-01-29 12:43:37.309031: read 13059 rows
See Also#
eland.pandas_to_eland: Create an eland.Dataframe from pandas.DataFrame