gcp toolkit¶
io module¶
-
class
gcp_toolkit.io.
IO
(bucket_name, staging_dataset, bq_client=None, storage_client=None)¶ Bases:
object
-
bq_to_bucket
(query, path_to_file)¶ Runs query in BigQuery and stores results in Storage
-
bq_to_df
(query, use_builtin=False)¶ Runs a query in BigQuery and loads the results into a pandas Data Frame
-
bucket_to_bq
(path_to_file, table_id, schema=[], csv_delimiter=', ')¶ Loads a csv from Storage into a BigQuery table
-
bucket_to_df
(path_to_file)¶ Reads a file or a group of files matching a pattern into a pandas Data Frame
-
df_to_bq
(df, table_id, schema=[])¶ Loads a pandas Data Frame into a BigQuery table.
-
utils module¶
-
gcp_toolkit.utils.
change_table_schema
(table_id, new_fields, bigquery_client=None)¶ Updates a table schema with new fields
-
gcp_toolkit.utils.
convert_pandas_gbq_schema
(schema)¶ Converts a BigQuery schema in the format of a list of dicts, used by pandas gbq into a list of SchemaFields
-
gcp_toolkit.utils.
create_bucket_folder
(bucket_name, folder_name, storage_client=None)¶ Creates a Folder in Storage