Entity Data Service APIs

General Overview

VMware Secure State provides public Entity Data Service (EDS) APIs documented through a user-friendly Swagger interface for its customers to gain deep insights about their cloud resources. Users can now build automation that queries EDS APIs for cloud resources, resource configurations, connected resource graphs, historic snapshots, and more to combine with their internal or external tools. These APIs can be used to quickly find resources or estimate the blast radius of an incident. Cloud security teams can also use these to discover and aggregate cloud resources across providers, regions, and accounts.

API Overview

To provide in-depth cloud visibility to users, Entity Data Service APIs expose advanced querying capabilities through a set of simple interfaces:

• Search and filter for cloud resources

• Discover other connected resources

• Create aggregations queries to understand distributions of resources

• Get configuration details on a specific cloud resource

• Retrieve complete historic configuration changes

Search and Filter for Resources

Users can search and filter for the resources monitored for their cloud environment through the Query API.

The query below can be used to retrieve all your cloud resources.

curl -X POST \ https://api.securestate.vmware.com/v1/entities/query \ -H 'Authorization: Bearer {access_token}'\ -d '{ "filters": {} }'

Response: { "totalCount": 12000, "results": [ { "partitionKey": "cloudAccountId", "entityId": "AWS.AutoScaling.cloudAccountId.us-west-2.AutoScalingGroup.groupName", "entityType": "AWS.AutoScaling.AutoScalingGroup", "entityName": "GroupId", "cloudAccountId": "cloudAccountId", "region": "us-west-2", "service": "AutoScaling", "provider": "AWS", "properties": [ { "name": "AutoScalingGroupARN", "type": "string", "stringV": "arn:aws:autoscaling:us-west-2:159636093902:autoScalingGroup:a62..." }, ... ], "tags": { "eks:nodegroup-name": "ngId" }, "creationTime": "2020-03-23T23:49:56.000Z", "lastUpdateTime": "2020-05-12T06:00:43.000Z" }, ... ] }

Notice that paginated results of entities with their properties, tags, creation and update times are returned.

Users can also apply filters to retrieve resources with the desired property. Below is an example of retrieving all resources with an environment cloud tag for staging.

curl -X POST \ https://api.securestate.vmware.com/v1/entities/query \ -H 'Authorization: Bearer {access_token}'\ -d '{ "filters": { "query": "tag.Environment = \"staging\"" } }'

Various filters can be applied on cloud resources for properties such as provider, region, resource type, etc. To scope the query to a set of cloud accounts, the cloudAccountIds property can be used. entityNames, entityTypes, providers, regions, and services are also supported as filter options for which multiple values can be passed in to limit the scope for a query. To find resources matching certain configurations, the query filter can be used as shown in the example above. Filters in the query parameter can be passed in for any property or tag by following the “property.{propertyName} = {value}” or “tag.{tagName} = {value}” syntax. As a convention, all such filters must have the property or tag prefix. For example, all resources with an internal IP can be retrieved by passing in this query:

“property.PrivateIPAddress = {Internal IP Value}”

Finally, any such combination of filters can be applied to quickly retrieve your cloud resources at different scopes. The complete schema of the supported filters is described by the queryFilters object in the Swagger docs.

Discover Connected Resources

Users can call into the Graph Query API, built on the Gremlin language, to discover the connected relationship graph of resources. This capability can be used for threat hunting and incident investigation.

Below is an example for retrieving security groups with security group rules for Port 22 connected to instances using the Graph Query API. For instance, consider the below cloud environment.

You can write a query to find all the Security groups with a Security Group Rule with port 22 connected to an EC2 instance, marked in the diagram as the matching query. Resources that don’t follow such a graph topology will be filtered out. Such a query can be created in the Gremlin language by using the ‘has’ operator to sift through resources with matching property values (entityType, property.ToPort, and property.FromPort) and traversing between resources types using the ‘in’ and ‘out’ operators for different edge directions. A partitionKey that maps to a cloud account is required to run a Graph Query. Refer to the request below as an example.

curl -X POST \ https://api.securestate.vmware.com/v1/entities/graph-query \ -H 'Authorization: Bearer {access_token}' \ -d '{ "gremlin": "g.V().has(\"entityType\",\"AWS.EC2.SecurityGroup\").filter(out().has(\"entityType\", \"AWS.EC2.SecurityGroupRule\").has(\"property.FromPort\", lte(22)).has(\"property.ToPort\", gte(22))).in().has(\"entityType\", \"AWS.EC2.Instance\").tree()", "partitionKey": "{cloudAccountId}" }'

Response: { "count": 1, "tree": { "nodes": [ { "id": "AWS.EC2.cloudAccountId.us-east-1.SecurityGroup.sg-instanceId", "type": "entity", "childNodes": [ { "id": "AWS.EC2.cloudAccountId.us-east-1.Instance.i-instanceId", "type": "entity" }, { "id": "AWS.EC2.cloudAccountId.us-east-1.Instance.i-instanceId", "type": "entity" }, ... ] }, ] }, "entities": { "AWS.EC2.cloudAccountId.us-east-1.Instance.i-instanceId": { "partitionKey": "cloudAccountId", "entityId": "AWS.EC2.cloudAccountId.us-east-1.Instance.i-instanceId", "entityType": "AWS.EC2.Instance", "cloudAccountId": "cloudAccountId", "region": "us-east-1", "service": "EC2", "provider": "AWS", "properties": [ ] } } }

The response represents the graph structure in the form of first level nodes and the connected child nodes. The list of entities with all their properties is also returned. Similar advanced traversal queries using Gremlin supported operators are supported.

Create Resource Aggregations

Users can create top-level summaries of the distribution of their cloud resources across providers, regions, resource types, cloud tags, etc. through the Aggregations Query API.

The query below provides an example for creating an aggregation of resources by regions.

curl -X POST \ https://api.securestate.vmware.com/v1/entities/aggregations-query \ -H 'Authorization: Bearer {access_token}' \ -d '{ "aggregations": { "caid": { "fieldName": "region", "aggregationType": "Terms" } } }'

Response: { "totalCount": 10311, "aggregations": { "caid": { "bucketCount": 18, "buckets": { "global": { "count": 3005 }, "us-east-1": { "count": 3442 }, "us-east-2": { "count": 410 }, "us-west-1": { "count": 557 }, "us-west-2": { "count": 458 }, ... } } } }

Total resources and the count of resources in each region are provided in the response. Nested aggregation queries combined with filters allow for the creation of aggregate summaries at the desired scope.

Get Resource Configuration Details

Users can retrieve all the configuration details for a resource by resourceId using the Get Entity API.

curl -X GET \ https://api.securestate.vmware.com/v1/entities/{resourceId} \ -H 'Authorization: {Authorization Token}'

Response: { "partitionKey": "cloudAccount", "entityId": "AWS.AutoScaling.cloudAccount.us-west-2.AutoScalingGroup.groupName", "entityType": "AWS.AutoScaling.AutoScalingGroup", "entityName": "public-servers-group", "cloudAccountId": "cloudAccount", "region": "us-west-2", "service": "AutoScaling", "provider": "AWS", "properties": [ { "name": "AutoScalingGroupName", "type": "string", "stringV": "public-servers-group" }, { "name": "AutoScalingGroupARN", "type": "string", "stringV": "arn:aws:autoscaling:us-west-2:..." }, { "name": "MinSize", "type": "long", "longV": 2 }, { "name": "MaxSize", "type": "long", "longV": 2 }, ... ], "creationTime": "2020-03-17T00:10:22.000Z", "lastUpdateTime": "2020-05-21T07:01:57.000Z" }

The response provides the latest configurations for the resource tracked by VSS.

Retrieve Historic Snapshots

Users can retrieve all the historic configuration changes for a cloud resource using the Changelog API.

The query below can be used to retrieve all historic snapshots for the provided resourceId.

curl -X GET \ https://api.securestate.vmware.com/v1/entities/{resourceId}/changelog \ -H 'Authorization: Bearer {access_token}'

Response { "changeLog": [ { "lastUpdateTime": "2020-04-10T09:00:44.000Z" }, { "lastUpdateTime": "2020-04-07T02:25:49.000Z", "propertyChanges": { "AmiLaunchIndex": { "changeType": "ADD", "newValue": "0" }, "Architecture": { "changeType": "ADD", "newValue": "x86_64" }, "Arn": { "changeType": "ADD", "newValue": "arn:aws:iam::..." }, ... }, "tagChanges": { "Name": { "changeType": "ADD", "newValue": "development" }, "aws:autoscaling:groupName": { "changeType": "ADD", "newValue": "workers-NodeGroup-GroupId" }, ... } }, { "lastUpdateTime": "2020-04-07T02:25:45.000Z" } ] }

Each resource configuration change, whether an add, remove or update, along with its change timestamp is provided in the response.

Swagger Documentation

We have exposed an easy-to-use Swagger interface that documents and provides examples for utilizing all our public APIs. The Swagger documentation can be found here: https://api.securestate.vmware.com/entities