mds

package module
v0.0.0-...-125e6b6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 26, 2024 License: Apache-2.0 Imports: 31 Imported by: 0

README

GCE Metadata Server Emulator

This script acts as a GCE's internal metadata server.

It returns a live access_token that can be used directly by Application Default Credentials from any SDK library or return any GCE metadata key-value pairs and attributes.

For example, you can call ADC using default credentials or specifically with ComputeCredentials and also recall any GCE project or instance attribute.

To use, first run the emulator:

./gce_metadata_server -logtostderr --configFile=config.json \
  -alsologtostderr -v 5 \
  -port :8080 \
  --serviceAccountFile certs/metadata-sa.json

Note the credentials for the server can be sourced from a service account key, workload federation, Trusted Platform Module (TPM) or statically provided as environment variable. The example above uses a key.

Then in a new window, export some env vars google SDK's under

export GCE_METADATA_HOST=localhost:8080
export GCE_METADATA_IP=127.0.0.1:8080

and run any application using ADC:

#!/usr/bin/python

from google.cloud import storage
import google.auth

import google.auth.compute_engine
import google.auth.transport.requests

from google.auth.compute_engine import _metadata

## with ADC metadata server
credentials, project = google.auth.default()    
client = storage.Client(credentials=credentials)
buckets = client.list_buckets()
for bkt in buckets:
  print(bkt)

## as compute credential
creds = google.auth.compute_engine.Credentials()
session = google.auth.transport.requests.AuthorizedSession(creds)
r = session.get('https://www.googleapis.com/userinfo/v2/me').json()
print(str(r))

## get arbitrary metadata values directly 
request = google.auth.transport.requests.Request()
print(_metadata.get_project_id(request))
print(_metadata.get(request,"instance/id"))

You can also launch the metadata server directly from your app or use in unit tests:

package main

import (
  	mds "github.com/salrashid123/gce_metadata_server"    
)

func TestSomething(t *testing.T) {

  // use any any credentials (static, real or fake)
  creds, _ := google.FindDefaultCredentials(ctx, "https://www.googleapis.com/auth/cloud-platform")

  serverConfig := &mds.ServerConfig{
		BindInterface: "127.0.0.1",
		Port:          ":8080",
  }

  claims := &mds.Claims{
		ComputeMetadata: mds.ComputeMetadata{
			V1: mds.V1{
				Project: mds.Project{
					ProjectID: "some_project_id",
				},
			},
		},
  }

  f, _ := mds.NewMetadataServer(ctx, serverConfig, creds, claims)

  err = f.Start()
  defer f.Shutdown()

  // optionally set a env var google sdk libraries understand
  // t.Setenv("GCE_METADATA_HOST", "127.0.0.1:8080")
  // do tests here, eg with "cloud.google.com/go/compute/metadata"
  // mid, _ := metadata.ProjectID()

  // or call it directly
  // client := &http.Client{}
  // req, _ := http.NewRequest(http.MethodGet, "http://127.0.0.1:8080/computeMetadata/v1/project/project-id", nil)
  // req.Header.Set("Metadata-Flavor", "Google")
  // res, _ := client.Do(req)  
}

The metadata server supports additional endpoints that simulate other instance attributes normally only visible inside a GCE instance like instance_id, disks, network-interfaces and so on.

For more information on the request-response characteristics:

The script performs the following:

  • returns the access_token and id_token provided by either
    • the serviceAccount JSON file you specify.
    • workload identity federation configuration
    • service account impersonation
    • statically from a provided environment variable
    • service account RSA key on HSM or Trusted Platform Module (TPM)
  • return project attributes (project_id, numeric-project-id)
  • return instance attributes (instance-id, tags, network-interfaces, disks)

You can run the emulator:

  1. directly on your laptop
  2. within a docker container locally.
  3. as a kubernetes service
  4. with some difficulty, bound to the link-local address (169.254.169.254)
  5. within unit tests

The endpoints that are exposed are:

r.Handle("/computeMetadata/v1/project/project-id")
r.Handle("/computeMetadata/v1/project/numeric-project-id")
r.Handle("/computeMetadata/v1/project/attributes/{key}")

r.Handle("/computeMetadata/v1/instance/service-accounts/")
r.Handle("/computeMetadata/v1/instance/service-accounts/{acct}/")
r.Handle("/computeMetadata/v1/instance/service-accounts/{acct}/{key}")
r.Handle("/computeMetadata/v1/instance/network-interfaces/{index}/access-configs/{index2}/{key}")
r.Handle("/computeMetadata/v1/instance/attributes/{key}")
r.Handle("/computeMetadata/v1/instance/{key}")
r.Handle("/")

This is not an officially supported Google product

Note, the real metadata server has some additional query parameters which are either partially or not implemented:

You are free to expand on the endpoints surfaced here..pls feel free to file a PR!

images/metadata_proxy.png

Configuration

The metadata server reads a configuration file for static values and uses a service account for dynamically getting access_token and id_token.

The basic config file format roughly maps the uri path of the actual metadata server and the emulator uses these values to populate responses.

For example, the instance_id, project_id, serviceAccountEmail and other files are read from the values here, for example, see config.json:

{
  "computeMetadata": {
    "v1": {
      "instance": {
        "id": 5775171277418378000,
        "serviceAccounts": {
          "default": {
            "aliases": [
              "default"
            ],
            "email": "metadata-sa@your-project.iam.gserviceaccount.com",
            "scopes": [
              "https://www.googleapis.com/auth/cloud-platform",
              "https://www.googleapis.com/auth/userinfo.email"
            ]
          }
        }
      },
      "oslogin": {},
      "project": {
        "numericProjectId": 708288290784,
        "projectId": "your-project"
      }
    }
  }
}

The field are basically a JSON representation of what the real metadata server returns recursively

$ curl -v -H 'Metadata-Flavor: Google' http://metadata/computeMetadata/v1/?recursive=true | jq '.'

Any requests for an access_token or an id_token are dynamically generated using the credential provided. The scopes for any token uses the values set in the config file

Usage

The following steps details how you can run the emulator on your laptop.

You can either build from source:

go build -o gce_metadata_server cmd/main.go

Or download an appropriate binary from the Releases page

You can set the following options on usage:

Option Description
-configFile configuration File (default: config.json)
-interface interface to bind to (default: 127.0.0.1)
-port port to listen on (default: :8080)
-serviceAccountFile path to serviceAccount json Key file
-impersonate use impersonation
-federate use workload identity federation
-tpm use TPM
-persistentHandle TPM persistentHandle
-pcrs TPM PCR values the key is bound to (comma separated pcrs in ascending order)
-domainsocket listen on unix socket
-allowDynamicScopes Allow access_token scopes to be set dynamically
GOOGLE_PROJECT_ID static environment variable for PROJECT_ID to return
GOOGLE_NUMERIC_PROJECT_ID static environment variable for the numeric project id to return
GOOGLE_ACCESS_TOKEN static environment variable for access_token to return
GOOGLE_ID_TOKEN static environment variable for id_token to return
With JSON ServiceAccount file

Create a GCP Service Account JSON file (you should strongly prefer using impersonation..)

export PROJECT_ID=`gcloud config get-value core/project`
gcloud iam service-accounts create metadata-sa

You can either create a key that represents this service account and download it locally

gcloud iam service-accounts keys create metadata-sa.json \
   --iam-account=metadata-sa@$PROJECT_ID.iam.gserviceaccount.com

or preferably assign your user impersonation capabilities on it (see section below)

You can assign IAM permissions now to the service account for whatever resources it may need to access and then run:

mkdir certs/
mv metadata-sa.json certs

./gce_metadata_server -logtostderr --configFile=config.json \
  -alsologtostderr -v 5 \
  -port :8080 \
  --serviceAccountFile certs/metadata-sa.json
With Impersonation

If you use impersonation, the serviceAccountEmail and scopes are taken from the config file's default service account.

First setup impersonation for your user account:

gcloud iam service-accounts \
  add-iam-policy-binding metadata-sa@$PROJECT_ID.iam.gserviceaccount.com \
  --member=user:`gcloud config get-value core/account` \
  --role=roles/iam.serviceAccountTokenCreator

then,

./gce_metadata_server -logtostderr \
     -alsologtostderr -v 5  -port :8080 \
     --impersonate --configFile=config.json
With Workload Federation

For workload identity federation, you need to reference the credentials.json file as usual:

then just use the default env-var and run:

export GOOGLE_APPLICATION_CREDENTIALS=`pwd`/sts-creds.json
./gce_metadata_server -logtostderr --configFile=config.json \
  -alsologtostderr -v 5 \
  -port :8080 --federate

To use this mode, you must first setup the Federation and then set the environment variable pointing to the ADC file.

for reference, see

where the sts-creds.json file is the generated one you created. For example using the OIDC tutorial above, it may look like

for example, if the workload federation user is mapped to

principal://iam.googleapis.com/projects/1071284184436/locations/global/workloadIdentityPools/oidc-pool-1/subject/alice@domain.com

then that identity should have the binding to use the metadata service account:

# enable federation for principal://
gcloud iam service-accounts add-iam-policy-binding metadata-sa@$PROJECT_ID.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "principal://iam.googleapis.com/projects/$GOOGLE_NUMERIC_PROJECT_ID/locations/global/workloadIdentityPools/oidc-pool-1/subject/alice@domain.com"

ultimately, the sts-creds.json will look like (note:, the service_account_impersonation_url value is not present)

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/1071284184436/locations/global/workloadIdentityPools/oidc-pool-1/providers/oidc-provider-1",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": "/tmp/oidccred.txt"
  }
}

where /tmp/oidcred.txt contains the original oidc token

With Trusted Platform Module (TPM)

If the service account private key is bound inside a Trusted Platform Module (TPM), the metadata server can use that key to issue an access_token or an id_token

Note: not all platforms supports this mode. The underlying go-tpm library is only supported on a few of the targets (linux/darwin + amd64,arm64). If you need support for other platforms, one option is to comment the sections for the TPM, remove the library bindings and compile.

Before using this mode, the key must be sealed into the TPM and surfaced as a persistentHandle. This can be done in a number of ways described here:

Basically, you can either

  • A download a Google ServiceAccount's json file and embed the private part to the TPM. example
  • B Generate a Key on the TPM and then import the public part to GCP. example
  • C remote seal the service accounts RSA Private key, encrypt it with TPM's Endorsement Key and load it securely inside the TPM. example

B is the most secure but C allows for multiple TPMs to use the same key

Anyway, once the RSA key is present as a handle, start the metadata server using the --tpm flag and set the --persistentHandle= value.

TPM based tokens derives the serivceAccount email from the configuration file. You must first edit config.json and set the value of Claims.ComputeMetadata.V1.Instance.ServiceAccounts["default"].Email.

After that, run

./gce_metadata_server -logtostderr --configFile=config.json \
  -alsologtostderr -v 5 \
  -port :8080 \
  --tpm --persistentHandle=0x81008000

we're using a persistentHandle to save/load the key but a TODO is to load from the context tree from files

The TPM based credentials imports a JWT generator library to perform the oauth and id_token exchanges:

If the TPM based key is restricted through a PCR policy, you will need to supply the list of PCRs its bound to using the --pcrs flag: (eg --pcrs=2,3,23)

A TODO enhancement could be to add on support for PKCS-11 systems: eg salrashid123/golang-jwt-pkcs11

also see:

Startup

Use any of the credential initializations described above and on startup, you will see something like:

./gce_metadata_server -logtostderr --configFile=config.json \
  -alsologtostderr -v 5 \
  -port :8080 \
  --serviceAccountFile certs/metadata-sa.json

images/setup_2.png

AccessToken

In a new window, run

curl -s -H 'Metadata-Flavor: Google' --connect-to metadata.google.internal:80:127.0.0.1:8080 \
   http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

{
  "access_token": "ya29.c.EltxByD8vfv2ACageADlorFHWd2ZUIgGdU-redacted",
  "expires_in": 3600,
  "token_type": "Bearer"
}

Please note the scopes used for this token is read in from the declared values in the config file.

Unlike the GCE metadata server, Cloud Run allows you to request a scope dynamically by using the ?scopes= query parameter. If you want this mode enabled, use the --allowDynamicScopes parameter

To mention, if the only use for this is to acquire credentials for use with a GCP SDK, consider any of the "process credential sources":

or if using go, oauth2 directly from a Trusted Platform Module:

*GCP TPM based TokenSource

IDToken

The following endpoints shows how to acquire an IDToken

curl -H "Metadata-Flavor: Google" --connect-to metadata.google.internal:80:127.0.0.1:8080 \
'http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=https://foo.bar'

The id_token will be signed by google but issued by the service account you used

{
  "alg": "RS256",
  "kid": "178ab1dc5913d929d37c23dcaa961872f8d70b68",
  "typ": "JWT"
}.
{
  "aud": "https://foo.bar",
  "azp": "metadata-sa@$PROJECT.iam.gserviceaccount.com",
  "email": "metadata-sa@PROJECT.iam.gserviceaccount.com",
  "email_verified": true,
  "exp": 1603550806,
  "iat": 1603547206,
  "iss": "https://accounts.google.com",
  "sub": "117605711420724299222"
}

Important: To get id_tokens, you must edit config.json and set the value of

  • Claims.ComputeMetadata.V1.Instance.ServiceAccounts["default"].Email

to the value present for the credentials you are using (eg set it to metadata-sa@$PROJECT.iam.gserviceaccount.com (substituting in value for your real $PROJECT))

Unlike the real gce metadataserver, this will NOT return the full identity document or license info :(&format=[FORMAT]&licenses=[LICENSES])

Attributes

To acquire instance or project attributes, simply call the endpoint:

For example, to get the instance id:

curl -s -H 'Metadata-Flavor: Google' --connect-to metadata.google.internal:80:127.0.0.1:8080 \
      http://metadata.google.internal/computeMetadata/v1/instance/id

5775171277418378000

Using Google Auth clients

GCP Auth libraries support overriding the host/port for the metadata server.

Each language library has their own nuances so please read the sections elow

These are not documented but you can generally just set the value of.

If you intend to use the samples in the examples/ folder, add some viewer permission to list gcs buckets (because this is what all the stuff in the examples/ folder shows)

# note roles/storage.admin is over-permissioned...we only need storage.buckets.list on the project...
gcloud projects add-iam-policy-binding $PROJECT_ID  \
     --member="serviceAccount:metadata-sa@$PROJECT_ID.iam.gserviceaccount.com"  \
     --role=roles/storage.admin

then usually just,

export GCE_METADATA_HOST=localhost:8080

and use this emulator. The examples/ folder shows several clients taken from gcpsamples.

Remember to run gcloud auth application-default revoke in any new client library test to make sure your residual creds are not used.

see examples/pyapp

  export GCE_METADATA_HOST=localhost:8080
  export GCE_METADATA_IP=127.0.0.1:8080

  virtualenv env
  source env/bin/activate
  pip3 install -r requirements.txt

  python3 main.py

Unlike the other language SDK's, for python we need to set GCE_METADATA_IP (see google-auth-library-python #1505).

see examples/javaapp

   export GCE_METADATA_HOST=localhost:8080

   mvn clean install exec:java  -q

see examples/goapp

  export GCE_METADATA_HOST=localhost:8080

  go run main.go

see examples/nodeapp

  export GCE_METADATA_HOST=localhost:8080

  npm i
  node app.js

see examples/dotnet

  export GCE_METADATA_HOST=localhost:8080

  dotnet restore
  dotnet run

Note, Google.Api.Gax.Platform.Instance().ProjectId requests the full recursive path

gcloud
export GCE_METADATA_ROOT=localhost:8080

$ gcloud config list
[component_manager]
disable_update_check = True
[core]
account = metadata-sa@mineral-minutia-820.iam.gserviceaccount.com
project = mineral-minutia-820

gcloud uses a different env-var but if you want to use gcloud auth application-default print-access-token, you need to also use GCE_METADATA_HOST and GCE_METADATA_IP

Other Runtimes

Run emulator as container

This emulator is also published as a release-tagged container to dockerhub:

You can verify the image were signed by the repo owner if you really want to (see section below).

Run with containers

To access the local emulator from containers

cd examples/container
docker build -t myapp .
docker run -t --net=host -e GCE_METADATA_HOST=localhost:8080  myapp

then run the emulator standalone or as a container itself:

docker run \
  -v `pwd`/certs/:/certs/ \
  -v `pwd`/config.json:/config.json \
  -p 8080:8080 \
  -t salrashid123/gcemetadataserver  \
      -serviceAccountFile /certs/metadata-sa.json \
      --configFile=/config.json \
      -logtostderr -alsologtostderr -v 5 \
      -interface 0.0.0.0 -port :8080
Running as Kubernetes Service

You can run the emulator as a kubernetes Service and reference it from other pods address by injecting GCE_METADATA_HOST environment variable to the containers:

If you want test this with minikube locally,

## first create the base64encoded form of the service account key
cat certs/metadata-sa.json | base64  --wrap=0 -
cd examples/kubernetes

then edit metadata.yaml and replace the values:

apiVersion: v1
kind: Secret
metadata:
  name: gcp-svc-account
type: Opaque
data:
  metadata-sa.json: "replace with contents of cat certs/metadata-sa.json | base64  --wrap=0 -"
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mds-config
data:
  config.json: |
     "replace with contents of config.json"

Finally test

minikube start
kubectl apply -f .
minikube dashboard --url
minikube service app-service --url

$ curl -s `minikube service app-service --url`

Number of Buckets: 62

needless to say, the metadata Service should be accessed only form authorized pods

Dynamic Configuration File Updates

Changes to the claims configuration file (--configFile=) while the metadata server is running will automatically update values returned by the server.

On startup, the metadata server sets a file listener on that config file and any updates to the values will propagate back to the server without requiring a restart.

ETag

GCE metadata servers return values with ETag headers. The ETag is used to check if a specific attribute or value has changed.

This metadata server will hash the value for the body to return and use that as the ETag. If you update the configuration file with new attributes or values, the ETag for that node will change. The ETag header key is returned in non-canonical format.

Note wait-for-change value is not supported currently so while you can poll for etag changes, you cannot listen and hold.

Finally, since the etag is just a hash of the node, if you change a value then back again, the same etag will get returned for that node.

Static environment variables

If you do not have access to certificate file or would like to specify static token values via env-var, the metadata server supports the following environment variables as substitutions. Once you set these environment variables, the service will not look for anything using the service Account JSON file (even if specified)

export GOOGLE_PROJECT_ID=`gcloud config get-value core/project`
export GOOGLE_NUMERIC_PROJECT_ID=`gcloud projects describe $GOOGLE_PROJECT_ID --format="value(projectNumber)"`
export GOOGLE_ACCESS_TOKEN="some_static_token"
export GOOGLE_ID_TOKEN="some_id_token"
export GOOGLE_ACCOUNT_EMAIL="metadata-sa@PROJECT.iam.gserviceaccount.com"

for example you can use those env vars and specify a fake svc account json key file (fake since its not actually even used)

./gce_metadata_server -logtostderr  \
   -alsologtostderr -v 5 \
   -port :8080 --configFile=`pwd`/config.json  --serviceAccountFile=certs/fake_sa.json

or

docker run \
  -p 8080:8080 \
  -e GOOGLE_ACCESS_TOKEN=$GOOGLE_ACCESS_TOKEN \
  -e GOOGLE_NUMERIC_PROJECT_ID=$GOOGLE_NUMERIC_PROJECT_ID \
  -e GOOGLE_PROJECT_ID=$GOOGLE_PROJECT_ID \
  -e GOOGLE_ACCOUNT_EMAIL=$GOOGLE_ACCOUNT_EMAIL \
  -e GOOGLE_ID_TOKEN=$GOOGLE_ID_TOKEN \  
  -v `pwd`/config.json:/config.json \
  -v `pwd`/certs/fake_sa.json:/certs/fake_sa.json \
  -t salrashid123/gcemetadataserver \
  -port :8080 --configFile=/config.json --serviceAccountFile=/certs/fake_sa.json \
  --interface=0.0.0.0 -logtostderr -alsologtostderr -v 5

curl -v -H "Metadata-Flavor: Google" \
  --connect-to metadata.google.internal:80:127.0.0.1:8080 \
   http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

some_static_token
Extending the sample

You can extend this sample for any arbitrary metadata you are interested in emulating (eg, disks, hostname, etc). Simply add the routes to the webserver and handle the responses accordingly. It is recommended to view the request-response format directly on the metadata server to compare against.

GCE's metadata server's IP address on GCE is a special link-local address: 169.254.169.254. Certain application default credential libraries for google cloud may reference the metadata server by IP address so we're adding this in.

If you use the link-local address, do not set GCE_METADATA_HOST

if you really want to use the link local address, you have two options: use iptables or socat. Both require some setup as root

first create /etc/hosts:

169.254.169.254       metadata metadata.google.internal

for socat

create an IP alias:

sudo ifconfig lo:0 169.254.169.254 up

relay using socat:

sudo apt-get install socat

sudo socat TCP4-LISTEN:80,fork TCP4:127.0.0.1:8080

for iptables

configure iptables:

iptables -t nat -A OUTPUT -p tcp -d 169.254.169.254 --dport 80 -j DNAT --to-destination 127.0.0.1:8080

Finally, access the endpoint via IP or alias over port :80

curl -v -H 'Metadata-Flavor: Google' \
     http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

If you don't mind running the program on port :80 directly, you can skip the socat and iptables and simply start the emulator to on the link address (-port :80 --interface=169.254.169.254) after setting the /etc/hosts variable.

Using Domain Sockets

You can also start the metadata server to listen on a unix domain socket.

To do this, simply specify --domainsocket= flag pointing to some file (eg --domainsocket=/tmp/metadata.sock). Once you do this, all tcp listeners will be disabled.

To access using curl, use its --unix-socket flag

curl -v --unix-socket /tmp/metadata.sock \
 -H 'Metadata-Flavor: Google' \
   http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

While it works fine with things like curl, the main issue with using domain sockets is that the default GCE_METADATA_HOST variable just listens on tcp

And its awkward to do all the overrides for a GCP SDK to "just use" a domain socket...

If you really wanted to use unix sockets, you can find an example of how to do this in the examples/goapp_unix folder

anyway, just for fun, you can pipe a tcp socket to domain using socat (or vice versa) but TBH, you're now back to where you started with a tcp listener..

socat TCP-LISTEN:8080,fork,reuseaddr UNIX-CONNECT:/tmp/metadata.sock
Building with Bazel

If you want to build the server using bazel (eg, deterministic),

## generate dependencies
bazel run :gazelle -- update-repos -from_file=go.mod -prune=true -to_macro=repositories.bzl%go_repositories

## run
bazel run cmd:main -- --configFile=`pwd`/config.json   -alsologtostderr -v 5 -port :8080 --serviceAccountFile=`pwd`/certs/metadata-sa.json 

## to build the image
bazel build cmd:tar-oci-index
  ## oci image at bazel-bin/tar-oci-index/tarball.tar

## to push the image a repo, edit cmd/BUILD.bazel and set the push-image target repository
bazel run cmd:push-image

side note: getting bazel to work with google apis is a bit brittle.

make the following edits to repositories.bzl

### add build_file_proto_mode directive here
    go_repository(
        name = "com_github_googleapis_gax_go_v2",
        importpath = "github.com/googleapis/gax-go/v2",
        build_file_proto_mode = "disable_global",
        sum = "h1:A+gCJKdRfqXkr+BIRGtZLibNXf0m1f9E4HG56etFpas=",
        version = "v2.12.0",
    )

### after upgrading google.golang.org/protobuf-->v1.33.0, i had to comment out 
    #go_repository(
    #    name = "org_golang_google_protobuf",
    #    importpath = "google.golang.org/protobuf",
    #    sum = "h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=",
    #    version = "v1.33.0",
    #)
Building with Kaniko

The container image is built using kaniko with the --reproducible flag enabled:

export TAG=...
docker run    -v `pwd`:/workspace -v $HOME/.docker/config.json:/kaniko/.docker/config.json:ro    -v /var/run/docker.sock:/var/run/docker.sock   \
      gcr.io/kaniko-project/executor@sha256:034f15e6fe235490e64a4173d02d0a41f61382450c314fffed9b8ca96dff66b2  \
      --dockerfile=Dockerfile \
      --reproducible \
      --destination "docker.io/salrashid123/gcemetadataserver:$TAG" \
      --context dir:///workspace/

syft packages docker.io/salrashid123/gcemetadataserver:$TAG
skopeo copy  --preserve-digests  docker://docker.io/salrashid123/gcemetadataserver:$TAG docker://docker.io/salrashid123/gcemetadataserver:latest

This is useful for unit tests and fakes. For additional examples, please see the server_test.go and cmd/main.go

Verify Release Binary

If you download a binary from the "Releases" page, you can verify the signature with GPG:

gpg --keyserver keyserver.ubuntu.com --recv-keys 5D8EA7261718FE5728BA937C97341836616BF511

## to verify the checksum file for a given release:
wget https://github.com/salrashid123/gce_metadata_server/releases/download/v3.4.1/gce_metadata_server_3.4.1_checksums.txt
wget https://github.com/salrashid123/gce_metadata_server/releases/download/v3.4.1/gce_metadata_server_3.4.1_checksums.txt.sig

gpg --verify gce_metadata_server_3.4.1_checksums.txt.sig gce_metadata_server_3.4.1_checksums.txt
Verify Container Image Signature

The images are also signed using my github address (salrashid123@gmail). If you really want to, you can verify each signature usign cosign:

## for tag/version  3.4.0:
IMAGE="index.docker.io/salrashid123/gcemetadataserver@sha256:c3cec9e18adb87a14889f19ab0c3c87d66339284b35ca72135ff9dcd58a59671"

## i signed it directly, keyless:
# $ cosign sign $IMAGE

## which you can verify:
$ cosign verify --certificate-identity=salrashid123@gmail.com  --certificate-oidc-issuer=https://github.com/login/oauth $IMAGE | jq '.'

## search and get 
# $ rekor-cli search --rekor_server https://rekor.sigstore.dev  --email salrashid123@gmail.com
# $ rekor-cli get --rekor_server https://rekor.sigstore.dev  --log-index $LogIndex  --format=json | jq '.'

Testing

a lot todo here, right...thats just life

$ go test -v 

=== RUN   TestBasePathRedirectHandler
--- PASS: TestBasePathRedirectHandler (0.00s)
=== RUN   TestProjectIDHandler
--- PASS: TestProjectIDHandler (0.00s)
=== RUN   TestAccessTokenHandler
--- PASS: TestAccessTokenHandler (0.00s)
=== RUN   TestAccessTokenDefaultCredentialHandler
--- PASS: TestAccessTokenDefaultCredentialHandler (0.00s)
=== RUN   TestAccessTokenComputeCredentialHandler
--- PASS: TestAccessTokenComputeCredentialHandler (0.00s)
=== RUN   TestAccessTokenEnvironmentCredentialHandler
--- PASS: TestAccessTokenEnvironmentCredentialHandler (0.00s)
=== RUN   TestOnGCEHandler
--- PASS: TestOnGCEHandler (0.00s)
=== RUN   TestProjectNumberHandler
--- PASS: TestProjectNumberHandler (0.00s)
=== RUN   TestInstanceIDHandler
--- PASS: TestInstanceIDHandler (0.00s)
PASS
ok  	github.com/salrashid123/gce_metadata_server	0.053s

Documentation

Overview

Creates a Google Cloud Platform local MetadataServer used for test, emulation or to run applications outside of a google cloud environment.

Supports reading credentials from a service account key file, workload federation, statically or from a key saved on a Trusted Platform Module (TPM).

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Claims 

type Claims struct {
	ComputeMetadata ComputeMetadata `json:"computeMetadata"  altjson:"computeMetadata"`
}

Base claims returned by the metadata server Claims are structured in the same format as provided by a 'real' metadata server

eg `curl -v -H 'Metadata-Flavor: Google' http://metadata/computeMetadata/v1/?recursive=true`

type ComputeMetadata 

type ComputeMetadata struct {
	V1 V1 `json:"v1" altjson:"v1"`
}

type Instance 

type Instance struct {
	Attributes  map[string]string `json:"attributes"  altjson:"attributes"`
	CPUPlatform string            `json:"cpuPlatform"  altjson:"cpu-platform"`
	Description string            `json:"description"  altjson:"description"`
	Disks       []struct {
		DeviceName string `json:"deviceName"  altjson:"device-name"`
		Index      int    `json:"index"  altjson:"index"`
		Interface  string `json:"interface"  altjson:"interface"`
		Mode       string `json:"mode"  altjson:"mode"`
		Type       string `json:"type"  altjson:"type"`
	} `json:"disks"  altjson:"disks"`
	GuestAttributes struct {
	} `json:"guestAttributes"  altjson:"guest-attributes"` // Guest attributes endpoint access is disabled.
	Hostname string `json:"hostname"  altjson:"hostname"`
	ID       int64  `json:"id"  altjson:"id"`
	Image    string `json:"image"  altjson:"image"`
	Licenses []struct {
		ID string `json:"id"  altjson:"id"`
	} `json:"licenses" altjson:"licenses"`
	MachineType       string `json:"machineType" altjson:"machine-type"`
	MaintenanceEvent  string `json:"maintenanceEvent" altjson:"maintenence-event"`
	Name              string `json:"name" altjson:"name"`
	NetworkInterfaces []struct {
		AccessConfigs []struct {
			ExternalIP string `json:"externalIp" altjson:"external-ip"`
			Type       string `json:"type" altjson:"type"`
		} `json:"accessConfigs" altjson:"access-configs"`
		DNSServers        []string `json:"dnsServers" altjson:"dns-servers"`
		ForwardedIps      []string `json:"forwardedIps" altjson:"forwarded-ips"`
		Gateway           string   `json:"gateway" altjson:"gateway"`
		IP                string   `json:"ip" altjson:"ip"`
		IPAliases         []string `json:"ipAliases" altjson:"ip-aliases"`
		Mac               string   `json:"mac" altjson:"mac"`
		Mtu               int      `json:"mtu" altjson:"mtu"`
		Network           string   `json:"network" altjson:"network"`
		Subnetmask        string   `json:"subnetmask" altjson:"subnetmask"`
		TargetInstanceIps []string `json:"targetInstanceIps" altjson:"target-instance-ips"`
	} `json:"networkInterfaces" altjson:"network-interfaces"`
	PartnerAttributes struct {
	} `json:"partnerAttributes" altjson:"partner-attributes"`
	Preempted        string `json:"preempted"  altjson:"preempted"`
	RemainingCPUTime int    `json:"remainingCpuTime" altjson:"remaining-cpu-time"`
	Scheduling       struct {
		AutomaticRestart  string `json:"automaticRestart" altjson:"automatic-restart"`
		OnHostMaintenance string `json:"onHostMaintenance" altjson:"on-host-maintenence"`
		Preemptible       string `json:"preemptible" altjson:"preemptible"`
	} `json:"scheduling" altjson:"scheduling"`
	ServiceAccounts map[string]serviceAccountDetails `json:"serviceAccounts" altjson:"service-accounts"`
	Tags            []string                         `json:"tags" altjson:"tags"`
	VirtualClock    struct {
		DriftToken string `json:"driftToken" altjson:"drift-token"`
	} `json:"virtualClock" altjson:"virtual-clock"`
	Zone string `json:"zone" altjson:"zone"`
}

type MetadataServer 

type MetadataServer struct {
	Creds        *google.Credentials // credentials to use
	Claims       Claims              // values for the runtime attributes and values the metadata server returns
	ServerConfig ServerConfig        // base system configuration (listen interface, port, etc)
	// contains filtered or unexported fields
}

Configures and manages the server and is used as a receiver to start and stop the server.

Applications can initialize the metadata server using this struct by providing it with the credentials to use, the claims it provides as well as runtime specifications like the port and interface to use

func NewMetadataServer 

func NewMetadataServer(ctx context.Context, serverConfig *ServerConfig, creds *google.Credentials, claims *Claims) (*MetadataServer, error)

Configure a new MetadataServer instance.

This will not start the instance (to do that, use the .Start() method).

- ServerConfig: This configures the core/baseline runtime. Specify the interface,port and credential scheme to use

- google.Credentials: Credentials to use for the access or id_token

- Claims: The runtime claims returned by the metadata server

func (*MetadataServer) Shutdown 

func (h *MetadataServer) Shutdown() error

Stop a running metadata server

func (*MetadataServer) Start 

func (h *MetadataServer) Start() error

Start running the metadata server using the configuration provided through `NewMetadataServer()`

type OSlogin 

type OSlogin struct {
	Authenticate struct {
		Sessions struct {
		} `json:"sessions" altjson:"sessions"`
	} `json:"authenticate" altjson:"authenticate"`
}

OSLogin configuration to apply

type Project 

type Project struct {
	Attributes       map[string]string `json:"attributes" altjson:"attributes"`
	NumericProjectID int64             `json:"numericProjectId" altjson:"numeric-project-id"`
	ProjectID        string            `json:"projectId" altjson:"project-id"`
}

Project configuration to apply

type ServerConfig 

type ServerConfig struct {
	BindInterface string // interface to bind to (default 127.0.0.1)
	Port          string // port to listen on (default :8080)
	DomainSocket  string // toggle if unix domain sockets should be used.

	Impersonate        bool // toggle if provided default credentials should be impersonated (default: false)
	Federate           bool // toggle if workload federation should be used (default: false)
	AllowDynamicScopes bool // toggle if dynamic scopes are enabled for access_tokens (default: false)

	UseTPM           bool   // toggle if TPM should be used for credentials (default: false)
	TPMPath          string // path to the TPM (default /dev/tpm0)
	PCRs             []int  // list of TPM PCR banks the key is bound to.  If set, the library will attempt to apply PCRSessionPolicy (default: nil)
	PersistentHandle int    // persistent handle for the TPM pointing to the credentials (default: 0)
}

Configures the base runtime for the metadata server. Set the port, bind-address and what mode this server will acquire credentials through

type V1 

type V1 struct {
	Instance Instance `json:"instance" altjson:"instance"`
	Oslogin  OSlogin  `json:"oslogin"  altjson:"oslogin"`
	Project  Project  `json:"project"  altjson:"project"`
}

Configuration of the v1 settings for the metadata server.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.