Configuration

The following steps need to be performed to implement your custom logic when writing to a target system.

Topics

• Define an application for the BAdI implementation
• Create a new configuration
• Associate Cxlink Datalakes to the configuration
• Advanced Configuration
  • Inform of structure changes
  • Kinesis Data Limitation
• Health Checks

---

Updating certificates

When updating certificates, it is necessary to deactivate the mass transfer IDs that are active in the LTRC transaction. If these are not deactivated during the update, the ICM is restarted and the replications are affected.

The process of restarting the ICM is a standard procedure associated with a change in network parameters/configuration. It could also happen with the renewal of the SSL certificates, necessary to establish a secure connection to cloud services (AWS S3).

It is important to note that an ICM restart could potentially result in a SAP SLT job becoming unresponsive if replication is occurring concurrently.

Define an application for the BAdI implementation

You specify the application when you create the configuration; the application links the configuration to a specific BAdI implementation. To do so:

1. In the SAP LT Replication Server system, in transaction SM30, enter the table IUUC_REPL_APPL, and choose the button Maintain.
2. On the next screen choose New Entries and enter:
   • Name: ZLNKAWS_DATALAKES
   • Description: Cxlink Datalakes BAdI or a description of your choice.
3. Save the changes in a new transport request for later transportation.

---

Create a new configuration

The application values that you specify here will be later assigned to a configuration that wants to use the custom logic implemented here

To configure your :

1. Go to transaction LTRC
2. Select Create Configuration to start a configuration wizard.
3. In Specify General Data section add:
   1. Configuration Name: Select a descriptive name that helps you easily identify the configuration purpose.
   2. Description: Specify a more descriptive message for the configuration.
   3. Authorization Group: Leave empty.
4. Select Next.
5. In the Specify Source System section define:
   1. RFC Destination: Select the RFC that connects to the SM59.
   2. Allow Multiple Usage: Keep checked to allow the configuration to be used in multiple environments.
   3. Read from Single Client: Check if you want to restrict the usage of this configuration to the current client. Leave unchecked if no restrictions apply.
6. Select Next.
7. In the Specify Target System section define:
   1. Select Other from the option list.
   2. Select SLT SDK in the Scenario for RFC Communication dropdown.
8. In the Specify Transfer Settings section define:
   1. In Application select the previously created application for BAdI implementation.
   2. In Job Options, set 3 for all values for testing purposes or adapt them to your needs.
   3. Leave the rest of options by default or adapt them to your needs.
9. Select Next.
10. In the Review screen, check the values and select Create to finish the process.
11. Back in the SAP LT Replication Server - Cockpit: Overview screen, you should be able to see the new Mass Transfer.
12. Ensure that the configuration is deactivated before continuing.

---

Associate Cxlink Datalakes to the configuration

Now that you have created the SLT standard configuration, you will need to create the relationship with the storage target for the data. To do so:

1. Go to transaction /n/LNKAWS/SLT_LTR.

2. In the CxLink Data Lakes Configuration - Overview screen, select Import Mass Transfer ID or (F9).

3. Select the previously generated Mass Transfer ID* and select Import Mass Transfer ID.

4. Back in the Cxlink Datalakes Configuration - Overview screen, find the imported MTID and open it.

5. Select New to create a new configuration.

6. In General data section define:

   Amazon Web Services (AWS)

   To add a new Amazon Web Services to the configuration:

   1. Description: A descriptive text for this configuration.
   2. Cloud Service Provider: Select Amazon Web Services.
   3. Service: Indicate your preferred service option: S3 or Kinesis.
   4. Select Next.

   Microsoft Azure

   To add a new Microsoft Azure to the configuration:

   1. Description: A descriptive text for this configuration.
   2. Cloud Service Provider: Select Microsoft Azure.
   3. Azure Tennat: Please select the Azure tenant to be used for this process.
   4. Service: Indicate your preferred service option: Blob or EventHubs.
   5. Select Next.

7. In Access Control section define the Credential to be used:

   Amazon Web Services (AWS)

   Access Mode: Choose between IAM User, Instance Profile or Assume Role:

   • IAM User: In the case of an IAM user, select the user:

   • Instance Profile: No further data required:

   • Assume Role: Select the User/Instance Profile and Assume Role:

   Select Next.

   Microsoft Azure

   Access Mode: Choose between Azure Active Directory (AAD), Shared Access Signature (SAS) and Azure Shared Key (ASK)*:

   • Azure Active Directory (AAD): Select Resource Name:

   • Azure Shared Access Signature (SAS): Select Resource Type and Resource Name:

   • Azure Shared Key (ASK): *For a Blob Service only. Select Resource Type and Resource Name:

   Select Next.

8. In Specify Transfer Method define:

   1. Method: You can define a target for specific operations or the whole process:

      1. Load: Only Initial Load will be sent to the specified target.
      2. Replication: Only Replication data will be sent to the specific target.
      3. Both: Initial Load and Replication data will be sent to the specific target.

   2. Output Format: Choose between JSON or CSV as output format.

      1. JSON: Check Athena compatible if you want to export the data in JSONL format (JSON Lines).
      2. CSV: Specify the ASCII representation of the separator you want to use in the data. (https://www.asciitable.com/)

   3. Target table name conversion. Tables with "/" in their names can provoke path alterations when stored in Amazon S3 or Azure Blob, as they can be assumed as folder separation instead of a single table name. To avoid this behaviour, you can select to replace the '/' symbol for "-" or "__" depending on your needs.

   4. In the Output Settings section:

      Amazon Web Services (AWS)

      Amazon S3 as target

      1. Prefix: You can use the listed dynamic env vars, to generate a custom path based on the values of the exported data.
      2. Custom filename: In the same way, you can use the listed dynamic env vars to build the file names generated to meet your requirements.
      3. Compression: You can enable the compression option to save some space while at rest. You should be able to uncompress the data from your consumers.

      Amazon Kinesis as target

      1. Partition Key: You can use the listed dynamic env vars, to generate a custom partition key for kinesis data stream.
      2. Inform of structure changes: If checked, Cxlink Datalakes will inform of any
         

      Microsoft Azure

      Blobs as target

      1. Compression: You can enable the compression option to save some space while at rest. You should be able to uncompress the data from your consumers.
      2. Inform of structure changes: If checked, Cxlink Datalakes will inform of any

      

      EventHub as target

      1. Compression: You can enable the compression option to save some space while at rest. You should be able to uncompress the data from your consumers.
      2. Inform of structure changes: If checked, Cxlink Datalakes will inform of any

      

   5. Select Next.

   6. Back to the CxLink Data Lakes Configuration - Overview screen, you should see all the configuration values listed.

9. In the Review section, check the values and select Create to finish the process.

10. Repeat the same steps to create as many configurations as needed.

---

Advanced Configuration

In this section you will find all advance configuration needed for Datalakes to work properly.

Inform of structure changes

Cxlink Datalake offers the capability to notify users when modifications are made to the structure of the source system table. This option can be accessed via the transaction /LNKAWS/SLT_LTR in the configuration wizard, under the 'Inform of structure changes' step.

Upon activation of this option and the addition of a new field to a database table affected by this configuration, two distinct files are generated. One file contains the information pertaining solely to the newly added field.

Example

US_OPERATION	US_MTID	US_SID	US_TABNAME	TABNAME	FIELDNAME
I	00X	SLT	ZCSKS		TEST25

The second file contains the final structure of the table, including all the fields.

Example

US_OPERATION	US_MTID	US_SID	US_TABNAME	TABNAME	FIELDNAME
L	00X	SLT	ZCSKS		TEST1
L	00X	SLT	ZCSKS		TEST2
L	00X	SLT	ZCSKS		TEST3
L	00X	SLT	ZCSKS		TEST4
L	00X	SLT	ZCSKS		TEST5

Both files have the same structure:

1. US_OPERATION: As a usual replication, it’s the first field included in the file.
   1. Value ‘L’ → For the file with the final structure.
   2. Value ‘I’ → For the file with only the new field.
2. US_MTID: Mass Transfer ID identificator from SAP LT Replication Server.
3. US_SID: SAP System ID
4. US_TABNAME: Name of the table with the change.
5. Structure DD03P: SAP Standard structure that contains all the technical information about the file.

Further information about the DD03P structure can be obtained by executing the SE11 transaction and specifying the DD_GET_DD03P function module and parameter DD03P_TAB for a specific table.

Ones of the most important fields are:

Fieldname	Type	Description
TABNAME	TABNAME	In this case, empty
FIELDNAME	FIELDNAME	Name of the new field
DDLANGUAGE	DDLANGUAGE	Language key
POSITION	TABFDPOS	Position of the field
KEYFLAG	KEYFLAG	Identify key fields of the table with ‘X’
MANDATORY	MANDATORY	Identify if the field is required
DATATYPE	DATATYPE_D	Datatype in Dictionary
LENG	DDLENG	Length ( Nº of characters )
DECIMALS	DECIMALS	Number of decimals
DOMNAME3L	DOMNAME	Domain name

Kinesis Data Limitation

If you are using Kinesis as a target for data replication, you will need to add a custom configuration to limit the number of records per data chunk to avoid Kinesis limits.

To do so:

1. Go to transaction LTRS
2. Select the desired Mass Transfer ID and double-click to open its properties.
3. Right-click on Performance Options and Add table.
4. Go to Replication Options tab and set the Portion Size (Records) to 500.
5. Save to finish the configuration.

---

Health Checks

You can perform some health checks over the configuration created to ensure that everything is correctly configured before start replication tables.

To perform the checks:

1. Go to transaction /n/LNKAWS/SLT_LTR.

2. In the CxLink Data Lakes Configuration - Overview screen, go to the Health check tab, select Configuration Health Check and Execute.

   Amazon Web Services (AWS)

   3. Double click on the nodes under AWS specific configuration and check the logs.

   4. For S3 Checks output should look similar to:

   5. For Kinesis checks output should look similar to:

   Microsoft Azure

   3. Double click on the nodes under Azure specific configuration and check the logs.

   4. For Blob Checks output should look similar to:

   5. For EventHubs checks output should look similar to:

   ---