Support Arrow IPC file format with dataframes (#6548)

* Add support for Arrow IPC file format

Add support for Arrow IPC file format to dataframes commands. Support
opening of Arrow IPC-format files with extension '.arrow' or '.ipc' in
the open-df command. Add a 'to arrow' command to write a dataframe to
Arrow IPC format.

* Add unit test for open-df on Arrow

* Add -t flag to open-df command

Add a `--type`/`-t` flag to the `open-df` command, to explicitly specify
the type of file being used. Allowed values are the same at the set of
allowed file extensions.
This commit is contained in:
Aron Nopanen
2022-09-12 16:30:20 -07:00
committed by GitHub
parent 4490e97a13
commit d08212409f
7 changed files with 214 additions and 10 deletions

2
Cargo.lock generated
View File

@ -167,12 +167,14 @@ dependencies = [
"indexmap",
"json-deserializer",
"lexical-core",
"lz4",
"multiversion",
"num-traits",
"parquet2",
"simdutf8",
"streaming-iterator",
"strength_reduce",
"zstd",
]
[[package]]