aws-inventory-graph

Description

Explore your AWS platform with, Dgraph, a graph database.

Thanks to Go and its goroutines, we can insert thousand of ressources in few seconds.

Prerequisites

• docker
• docker-compose
• Go (for build only)

Install

Download and move to /usr/local/bin/ a binary from release page

Build

This project uses go.mod, so after cloning this repo, simply run :

go build && chmod +x ./aws-inventory-graph

or

GOBIN=/usr/local/bin/ go install && chmod +x /usr/local/bin/aws-inventory-graph

Usage

Start Dgraph server

make

or

make up

Access to WebUI (dgraph-ratel) : http://localhost:8000

Import ressources

Authentication is based on your .aws/config file.

Usage of aws-inventory-graph:
  -dgraph string
        Dgraph server (ip:port) (default "127.0.0.1:9080")
  -drop
        Drop all nodes and the schema
  -list
        List available ressource types
  -no-schema
        Disable the refresh schema at each run
  -profile string
        Profile from ~/.aws/config (default "default")
  -region string
        AWS Region (default "eu-west-1")
  -type string
        Get the schema for a type (only after importing some data)

Example :

aws-inventory-graph -region us-west-2 -profile xxxx

2019/11/29 17:35:58 Drop all previous data
2019/11/29 17:35:58 Add schema
2019/11/29 17:36:04 List ...
...
2019/11/29 17:36:05 Add ... Nodes
...
2019/11/29 17:36:08 Add ... Edges
...

Get schema for a type

You can get all schemas for types and predicates in dgraph-ratel WebUI:

or with binary, in JSON format :

aws-inventory-graph -type Address | jq

{
  "types": [
    {
      "fields": [
        {
          "name": "name",
          "type": "string"
        },
        {
          "name": "Service",
          "type": "string"
        },
        {
          "name": "Region",
          "type": "string"
        },
        {
          "name": "OwnerId",
          "type": "string"
        },
        {
          "name": "PrivateIpAddress",
          "type": "string"
        },
        {
          "name": "PublicIp",
          "type": "string"
        },
        {
          "name": "Domain",
          "type": "string"
        },
        {
          "name": "AllocationId",
          "type": "string"
        },
        {
          "name": "_Instance",
          "type": "Instance"
        }
      ],
      "name": "Address"
    }
  ]
}

Predicates which are prefixed with a _ are Edges, and they all have a reverse.

Stop and/or Remove Dgraph

Stop :

make stop

Remove :

make rm

Available Ressources

Here the list of currently supported ressources :

• Address
• AutoScalingGroup
• AvailabilityZone
• CacheCluster
• CacheSubnetGroup
• Cidr
• DbCluster
• DbClusterParameterGroup
• DbInstance
• DbParameterGroup
• DbSubnetGroup
• Image
• Instance
• InstanceProfile
• KeyPair
• LaunchConfiguration
• LaunchTemplate
• LoadBalancer
• NatGateway
• OptionGroup
• SecurityGroup
• Snapshot
• Subnet
• TargetGroup
• Volume
• Vpc
• VpcPeeringConnection

All Edges between the Nodes have reversed.

⚠ AWS API is often messy, names are not consistent from one endpoint to another, we try to fix that and keep a global coherance. This is why some Predicates don't match exact names returned by API.

Query examples

See here to get more info about Dgraph’s GraphQL+.

Get Elastic IPs + Instances + NatGateways

{
  Address(func: type(Address)) @filter(has(_Instance) or has(_NatGateway)){
    name dgraph.type PublicIp
    Instance:_Instance {name dgraph.type InstanceId}
    NatGateway: _NatGateway{name dgraph.type NatGatewayID}
  }
}

Get Classic LoadBalancers + AutoScalingGroups + Instances

{
  LoadBalancer(func: type(LoadBalancer))@filter(eq(LoadBalancerType, classic)) @cascade{
    name dgraph.type
    AutoScaling:~_LoadBalancer {
        name dgraph.type
        Instance:_Instance{
            name dgraph.type InstanceId
        }
    }
  }
}

Get Application LoadBalancers + TargetGroups + AutoScalingGroups + Instances

{
  LoadBalancerV2(func: type(LoadBalancer))@filter(eq(LoadBalancerType, application))@cascade{
    name dgraph.type
    TargetGroup:~_LoadBalancer @filter(type(TargetGroup)){
        name dgraph.type
        AutoScalingGroup:~_TargetGroup @filter(type(AutoScalingGroup)){
            name dgraph.type Instance:_Instance{
                name dgraph.type InstanceId
            }
        }
    }
  }
}

Get VpcPeeringConnections + Vpcs

{
  VpcPeeringConnection(func: type(VpcPeeringConnection)){
    name dgraph.type VpcPeeringConnectionId
    AccepterVpc:_AccepterVpc {name dgraph.type VpcId}
    RequesterVpc:_RequesterVpc {name dgraph.type VpcId}
  }
}

Get which Instances have access to which DbInstances

{
  DbInstances(func: type(DbInstance))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup {
      name dgraph.type GroupId
      IngressSecurityGroup:_SecurityGroup @facets {
        name dgraph.type
        Instance:~_SecurityGroup @filter(type(Instance)){
          name dgraph.type InstanceId
        }
      }
    }
  }
}

Get the Cidr which are allowed for access to DbInstances

{
  Rds(func: type(DbInstance))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup {
      name dgraph.type GroupId
      IngressCidr:_Cidr @facets {
        name dgraph.type
      }
    }
  }
}

Get Instance opened worldwide + associated ports

{
  OpenWorldCidr(func: eq(name, "0.0.0.0/0"))@cascade{
    name dgraph.type
    SecurityGroup:~_Cidr @filter(type(SecurityGroup)) @facets {
      name dgraph.type GroupId
      Instance:~_SecurityGroup @filter(type(Instance)){
        name dgraph.type InstanceId
      }
    }
  }
}

Get which KeyPairs give access to which Instances

{
  KeyPair(func: type(KeyPair))@cascade{
    name dgraph.type
    Instance:~_KeyName{
      name dgraph.type InstanceId
    }
  }
}

Get CacheClusters (Elasticache) with Instances which have access to them

{
  Memcached(func: type(CacheCluster))@filter(eq(Engine, memcached))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup @facets {
      name dgraph.type
      IngressSecurityGroup:_SecurityGroup @facets {
        name dgraph.type
        Instance:~_SecurityGroup @filter(type(Instance)){
          name dgraph.type InstanceId
        }
      }
    }
  }
  Redis(func: type(CacheCluster))@filter(eq(Engine, redis))@cascade{
    name dgraph.type
    SecurityGroup:_SecurityGroup @facets {
      name dgraph.type
      IngressSecurityGroup:_SecurityGroup @facets {
        name dgraph.type
        Instance:~_SecurityGroup @filter(type(Instance)){
          name dgraph.type InstanceId
        }
      }
    }
  }
}

Get InstanceProfiles which are not used by Instances

{
  InstanceProfile(func: type(InstanceProfile)) @filter(not has(~_InstanceProfile)) {
    name dgraph.type
  }
}

Get Instances without backup (no linked Snapshot)

{
  Instance(func: type(Instance)) @filter(eq(OwnerName, univadis-prod)) @cascade{
    name dgraph.type
    Volume:~_Instance @filter(not has(~_Volume) and type(Volume)){}
  }
}

Get the sum of Volumes by type

{
  var(func: type(Volume))  @filter(eq(VolumeType, gp2)){
    A as Size
  }
  var(func: type(Volume))  @filter(eq(VolumeType, standard)){
    B as Size
  }
  Volume(){
    gp2:sum(val(A))
    standard:sum(val(B))
  }
}

Author

Thomas Labarussias (@Issif)