2.5 Data Landing Zone
Foundation - Data Ingestion - Data Ingestion from Offline Sources
Last updated
Foundation - Data Ingestion - Data Ingestion from Offline Sources
Last updated
In this exercise, the goal is to setup your Data Landing Zone Source connector with Azure Blob storage.
Data Landing Zone is an Azure Blob storage interface provisioned by Adobe Experience Platform, granting you to access a secure, cloud-based file storage facility to bring files into Platform. Data Landing Zone supports SAS-based authentication and its data is protected with standard Azure Blob storage security mechanisms at rest and in transit. SAS-based authentication allows you to securely access your Data Landing Zone container through a public internet connection.
Adobe Experience Platform enforces a strict seven-day time-to-live (TTL) on all files uploaded to a Data Landing Zone container. All files are deleted after seven days.
To copy blobs or files to your Adobe Experience Platform Data Landing Zone you'll use AzCopy, a command-line utility. You can download a version for your operating system via https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10.
Unzip the downloade file
Download the sample data file global-context-websiteinteractions.csv, which contains sample website interactions and save it into the folder in which you unzipped azcopy.
Open a terminal window and navigate to the folder on your desktop, you should see the following content (azcopy and global-context-websiteinteractions.csv), for example on OSX:
Log in to Adobe Experience Platform by going to this URL: https://experience.adobe.com/platform.
After logging in, you'll land on the homepage of Adobe Experience Platform.
Before you continue, you need to select a sandbox. The sandbox to select is named --module2sandbox--. You can do this by clicking the text Production Prod] in the blue line on top of your screen. After selecting the appropriate sandbox, you'll see the screen change and now you're in your dedicated sandbox.
In the left menu, go to Sources. In the Sources catalog, search for data landing. On the Data Landing Zone card, click ... and select View Credentials.
Click to copy SASUri.
You'll now ingest data into Adobe Experience Platform using Azure command line tools using AZCopy.
Open a terminal at the location of you azcopy install location and execute the following command to copy a file into AEP's data landing zone:
./azcopy copy <your-local-file> <your SASUri>
Make sure to surround your SASUri with double quotes. Replace <your-local-file> by the path to your local copy of the file global-context-websiteinteractions.csv in the azcopy directory, and replace <your SASUri> by the SASUri value which you copied from the Adobe Experience Platform UI. Your command should look like this:
After executing the above command in your terminal, you'll see this:
Go to your Data Landing Zone in Adobe Experience Platform.
Select Sources, search for data landing and click the Setup button.
This will open the Data Landing Zone. You'll see the file that you just uploaded in the data landing zone's select data panel.
Select your file and select Delimited as data format. You'll then see a preview of your data. Click Next.
You can now start mapping the uploaded data to match the XDM schema of your dataset.
Select Existing dataset and select the dataset Demo System - Event Dataset for Website (Global v1.1). Click Next.
Now you are ready map the incoming source data from your csv file to the target fields from the dataset's XDM schema.
Don't mind the potential errors with the mapping. You'll correct the mapping in the next step.
First of all, click the Clear all mappings button. You can then start with a clean mapping.
Next, click New field type and then select Add new field.
To map the ecid source field, select the field identities.ecid and click Select.
Next, click Map target field.
Select the field --aepTenantId--.identification.core.ecid in the schema structure.
You need to map a couple of other fields, click + New field type followed by Add new field and add fields for this mapping
| source | target |
|---|---|
resource.info.pagename | web.webPageDetails.name |
timestamp | timestamp |
timestamp | _id |
When finished your screen should look like, the screen below. Click Next.
Click Next.
Click Finish.
To monitor you dataflow, go to Sources, Dataflows and click on your dataflow:
Loading the data can take a couple of minutes, when succesful, you'll see a status of Success:
