To automatically retrieve information from Google Cloud Storage, create a new fetch task by following these steps. The connector runs each fetch task automatically, based on the schedule that is configured in the configuration file.
To create a new Fetch Task
In the [FetchTasks]
section of the configuration file, specify the number of fetch tasks using the Number
parameter. If you are configuring the first fetch task, type Number=1
. If one or more fetch tasks have already been configured, increase the value of the Number
parameter by one (1). Below the Number
parameter, specify the names of the fetch tasks, starting from zero (0). For example:
[FetchTasks] Number=1 0=MyTask
Below the [FetchTasks]
section, create a new TaskName section. The name of the section must match the name of the new fetch task. For example:
[FetchTasks] Number=1 0=MyTask [MyTask]
In the new section, set the parameters required to authenticate with Google Cloud Storage. The OAuth configuration tool creates a file that contains these parameters, and you can include them in the connector's configuration file using the following syntax:
[MyTask] < "oauth.cfg" [OAUTH]
For more information about including parameters from another file, see Include an External Configuration File
To authenticate with Google Cloud Storage as an ordinary user, set the Username
parameter. You do not need to set this parameter if you are using a service account.
Username
|
The user name to use to authenticate with Google Cloud Storage. |
(Optional) Set one of the following configuration parameters:
ProjectIDs
|
A comma-separated list of project IDs for the projects that you want to process. |
ProjectFilter
|
A filter to choose the projects that you want to process. For information about how to construct the filter, refer to the Google Cloud Storage documentation. |
For example:
[MyTask] < "oauth.cfg" [OAUTH] ProjectIDs=idol-123,idol-987
Or alternatively:
[MyTask] < "oauth.cfg" [OAUTH] ProjectFilter=name:idol*
If you do not set either of these parameters, the connector ingests data from all available projects.
(Optional) You can set additional parameters to choose the buckets and objects that you want to synchronize. For information about these parameters, refer to the Google Cloud Storage Connector Reference.
|