Are there any data integration/architecture documentation available?

[Dylan Tuliszewski]

Hello team,

We presented this to our team internally and there are some questions on the technical side of how exactly Data Manager will work. Are you able to provide any technical documentation regarding Data Manager? Below are some questions we have that I cannot find the answers to.

Data integration - how does it integrates with azure and qualys, are these native? Any documentation on the connectors? Can we schedule data imports?

Maintenance - how to set up environment where these connectors/data manager live? Any architectural diagrams showing how data goes from data source into data manager?

Does it have to be a virtual machine?

[Hun Chan · Accepted solution]

Hi Dylan, 

I'll leave the technical documentation and architectural diagrams for my other team members to provide to you. As for the data integration with the Qualys an Azure sources, the Data Manager Adapters has native Azure and Qualys connectors that use API calls to retrieve the relevant information. 

As part of configuring the Data Manager Qualys Connector to connect to the Qualys source, you need to ensure that the Qualys source is API enabled with relevant firewall rules established AND that the credentials to access the source have been provided. The API call that the connector uses is POST ApiUrl/qps/rest/2.0/search/am/hostasset

Likewise for the Azure connector, the Azure AD Device, Azure AD User and the Azure VM are supported. The API calls used for Azure AD Device and Azure VM is Device.Read.All 

The API Call for the Azure AD User is User.Read.All 

You can schedule the data imports by wrapping the configured connector job that you execute in the adapters client using either Task Scheduler (Windows environment) or a Cron job in Unix/Linux environment or Automator and Scheduler in the Mac environment.

Regards,

Hun

[Dylan Tuliszewski]

Thank you Hun! This is very helpful. If we are utilizing the data connectors to collect data from our sources to upload into the Assets tool (CMDB) are we taking cloud data out of the cloud, then uploading back into the cloud (CMDB)? Just trying to wrap my head around how exactly this is working without any diagrams/documents made available.

[Silvia Davis]

Hello @Dylan Tuliszewski,

We will be launching Data Manager soon. In the meantime, you can sign up for early GA here. 

Let me know if you have additional questions.

- Silvia

[Dylan Tuliszewski]

Thank you Silvia, we have already signed up and are awaiting GA. In the meantime, is there any documentation available for our infra and sec teams to review?

[Silvia Davis]

Hi @Dylan Tuliszewski we have documentation available for the customers that signed up for EAP. Let me check on this for you. - Silvia

[Silvia Davis]

Hi, @Dylan Tuliszewski. The documentation will be available only by GA. Stay tuned! I will keep you and the community posted. -Silvia

[Paul Hecker]

I would be very interested in reviewing the documention as well.

Documentation would be greatly appreciated to help us get going as quickly as possible, but also to understand the current limitations.

[Hun Chan]

Hi Dylan,

You are welcome! Yes that is essentially it. You will connect Data Manager to the various data sources in your environment (cloud/on-prem) using the out of the box Adapters. Then Data Manager will correlate, transform and normalise the data from the various data sources into a single pane which you can analyse and query and becomes your single source of truth. The analysed and queried data that you save as a saved search can then be imported back into JSM and populate the schema you create in JSM.

Regards,

Hun