logging - GCP: How to query logs from labeled resources?

Question

I added labels to my GAE resources and I would like to have different log sinks based on their labels.

I can't find a way to filter them in query logging no bigquery.

question from:https://stackoverflow.com/questions/65833847/gcp-how-to-query-logs-from-labeled-resources

Answer 1

There's a way to add network tags to App Engine Instances too.

I tested this on a simple python "hello world" exaple.

When you have the app ready just add to your app.yaml file two lines. My file looks like this:

runtime: python39
service: my-tagged-app
network:
   instance_tag: gae-tag1

When you deploy this app the instance that will be running it will be "tagged". But you won't see it anywhere alse than in the logs explorer;

Use the query:

resource.type="gae_app"
protoPayload.serviceData.createVersion.request.version.network.instanceTag="gae-tag1"

And you will get this result:

gcloud logging read "gae-tag1"
---
insertId: -oohejhdivvs
logName: projects/myproj/logs/cloudaudit.googleapis.com%2Factivity
operation:
  first: true
  id: 459d7914-20a1-439c-9c4d-08e96964dcb1
  producer: appengine.googleapis.com/admin
protoPayload:
  '@type': type.googleapis.com/google.cloud.audit.AuditLog
  authenticationInfo:
    principalEmail: [email protected]
  authorizationInfo:
  - granted: true
    permission: appengine.versions.create
    resource: apps/myproj/services/czesc-2-tag/versions/2034555122t104822
    resourceAttributes: {}
  methodName: google.appengine.v1.Versions.CreateVersion
  requestMetadata:
    callerIp: 31.91.214.141
    destinationAttributes: {}
requestAttributes:
      auth: {}
      time: '2021-01-22T10:48:27.701290Z'
  resourceLocation:
    currentLocations:
    - us-central1
  resourceName: apps/myproj/services/czesc-2-tag/versions/20210122t104822
  serviceData:
    '@type': type.googleapis.com/google.appengine.v1.AuditData
    createVersion:
      request:
        parent: apps/myproj/services/czesc-2-tag
        version:
          entrypoint:
            shell: ''
          id: 20210122t104822
          network:
            instanceTag: gae-tag1
          runtime: python39
  serviceName: appengine.googleapis.com
  status: {}
receiveTimestamp: '2021-01-22T10:48:28.183925224Z'
resource:
  labels:
    module_id: czesc-2-tag
    project_id: myproj
    version_id: 20210122t104822
    zone: ''
  type: gae_app
severity: NOTICE
timestamp: '2021-01-22T10:48:27.506516Z'

And that's how you can filter out your logs. At least using network tags (which you can also use to define GAE firewall rules).

You can have a look at the similar case discussed here.