Querying Azure APIM Diagnostic Logs

Question

I've enabled diagnostic logs for APIM which are being sent to log-analytics. Scenario All incoming requests to server have RequestTracking_Id header. All backend services use RequestTracking_Id header from the request in logs to track the request. But from in APIM logs, I'm not able to pull this header to query upon.

Is there any way it is possible to have header information of incoming requests in APIM diagnostic logs?

Query:

AzureDiagnostics
| where Type contains "Azure" and Resource contains "APIM-DEV" and backendUrl_s contains "/relativePath" and DurationMs > 2000

The columns which are available on querying are :

TenantId SourceSystem MG ManagementGroupName TimeGenerated Computer activityId_g requestResourceType_s requestResourceId_s collectionRid_s statusCode_s duration_s clientIpAddress_s requestCharge_s requestLength_s responseLength_s resourceTokenUserRid_s region_s partitionId_g error_number_d Severity user_defined_b state_d exec_type_d wait_category_s total_query_wait_time_ms_d max_query_wait_time_ms_d is_parameterizable_s statement_type_s statement_key_hash_s query_param_type_d interval_start_time_d interval_end_time_d logical_io_writes_d max_logical_io_writes_d physical_io_reads_d max_physical_io_reads_d logical_io_reads_d max_logical_io_reads_d execution_type_d count_executions_d cpu_time_d max_cpu_time_d dop_d max_dop_d rowcount_d max_rowcount_d query_max_used_memory_d max_query_max_used_memory_d duration_d max_duration_d num_physical_io_reads_d max_num_physical_io_reads_d log_bytes_used_d max_log_bytes_used_d query_id_d query_hash_s plan_id_d query_plan_hash_s statement_sql_handle_s LogicalServerName_s ElasticPoolName_s DatabaseName_s start_utc_date_t end_utc_date_t wait_type_s delta_max_wait_time_ms_d delta_signal_wait_time_ms_d delta_wait_time_ms_d delta_waiting_tasks_count_d keyProperties_curve_s keyProperties_operations_s keyProperties_attributes_enabled_b algorithm_s identity_claim_xms_mirid_s lastError_transportErrorCode_d subnetId_s backendMethod_s backendUrl_s backendResponseCode_d backendTime_d requestSize_d productId_s userId_s apimSubscriptionId_s backendProtocol_s secretProperties_attributes_enabled_b clientIp_s clientPort_s ruleSetType_s ruleSetVersion_s ruleId_s Message action_s site_s details_message_s details_data_s details_file_s details_line_s hostname_s apiId_s operationId_s apiRevision_s clientIP_s clientPort_d httpMethod_s requestQuery_s userAgent_s httpStatus_d httpVersion_s receivedBytes_d sentBytes_d timeTaken_d sslEnabled_s host_s Level_d isRequestSuccess_b location_s lastError_source_s lastError_reason_s lastError_message_s lastError_section_s method_s url_s responseCode_d responseSize_d cache_s clientProtocol_s lastError_elapsed_d clientTime_d matchedConnections_d systemId_g vnetResourceGuid_g subnetPrefix_s macAddress_s primaryIPv4Address_s ruleName_s direction_s type_s instanceId_s healthyHostCount_d unHealthyHostCount_d requestCount_d latency_d failedRequestCount_d throughput_d priority_d conditions_protocols_s conditions_sourcePortRange_s conditions_destinationPortRange_s conditions_destinationIP_s conditions_sourceIP_s conditions_None_s trustedService_s CorrelationId identity_claim_http_schemas_microsoft_com_identity_claims_scope_s isAccessPolicyMatch_b certificateProperties_attributes_enabled_b certificatePolicyProperties_certificateProperties_subject_s certificatePolicyProperties_certificateProperties_validityInMonths_d certificatePolicyProperties_keyProperties_type_s certificatePolicyProperties_keyProperties_size_d certificatePolicyProperties_keyProperties_reuse_b certificatePolicyProperties_keyProperties_export_b certificatePolicyProperties_certificateIssuerProperties_name_s certificateEnrollmentProperties_id_s certificateEnrollmentProperties_certificateProperties_subject_s certificateEnrollmentProperties_certificateProperties_sha1_s certificateEnrollmentProperties_certificateProperties_sha256_s certificateEnrollmentProperties_certificateProperties_nbf_t certificateEnrollmentProperties_certificateProperties_exp_t certificateEnrollmentProperties_keyProperties_size_d certificateEnrollmentProperties_keyProperties_type_s certificateEnrollmentProperties_secretProperties_type_s certificateEnrollmentProperties_attributes_created_d certificateEnrollmentProperties_attributes_enabled_b certificateEnrollmentProperties_attributes_updated_d ResultDescription keyProperties_type_s keyProperties_size_d secretProperties_type_s certificateProperties_subject_s certificateProperties_sha1_s certificateProperties_sha256_s certificateProperties_nbf_t certificateProperties_exp_t Category OperationName ResultType CallerIPAddress identity_claim_http_schemas_microsoft_com_identity_claims_objectidentifier_g identity_claim_http_schemas_xmlsoap_org_ws_2005_05_identity_claims_upn_s identity_claim_appid_g id_s clientInfo_s requestUri_s httpStatusCode_d vaultProperties_s ResourceId OperationVersion ResultSignature DurationMs SubscriptionId ResourceGroup ResourceProvider Resource ResourceType Type _ResourceId

Answer 1

If you want specific logs according to your application. You can use <log-to-eventhub/> in apim policy in the endpoint level / api level / product level.

Please see below link to know moer about this policy. https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/api-management/api-management-log-to-eventhub-sample.md

In policy, you will get all the request and response objects from context object. Please see the link to know more about Context variables. https://docs.microsoft.com/en-us/azure/api-management/api-management-policy-expressions#ContextVariables

For headers, you can lookup the dictionary object context.Request.Header