Jack Wright 23ba613b00
Polars AWS S3 support (#14648)
# Description

Provides Amazon S3 support.

- Utilizes your existing AWS cli configuration. 
- Supports AWS SSO
- Supports
[gimme-aws-creds](https://github.com/Nike-Inc/gimme-aws-creds).
- respects the settings of AWS_PROFILE environment variable for
selecting profile config
- AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION environment
variables for configuring without an AWS config

Usage:
```nushell
polars open s3://bucket/and/path.parquet
```

Supports:
- CSV
- Parquet
- NDJSON / json lines
- Arrow

Doesn't support:
- eager dataframes
-  Avro
- JSON
2024-12-25 06:15:50 -06:00

29 lines
572 B
Rust

use nu_protocol::ShellError;
use polars_io::cloud::CloudOptions;
use url::Url;
use crate::PolarsPlugin;
mod aws;
enum CloudType {
Aws,
}
fn determine_cloud_type(url: &Url) -> Option<CloudType> {
match url.scheme() {
"s3" | "s3a" => Some(CloudType::Aws),
_ => None,
}
}
pub(crate) fn build_cloud_options(
plugin: &PolarsPlugin,
url: &Url,
) -> Result<Option<CloudOptions>, ShellError> {
match determine_cloud_type(url) {
Some(CloudType::Aws) => aws::build_cloud_options(plugin).map(Some),
_ => Ok(None),
}
}