amazon web services - AWS Sagemaker processing Job automatically created?

Question

I haven't used sagemaker for a while and today I started a training job (with the same old settings I always used before), but this time I noticed that a processing job has been automatically created and it's running while my training job run (I presume for debugging purpose). I'm sure that this is the first time that it happens.. Is that a new feature introduced by sagemaker? I didn't find any related in documentation, but it's important to know because I don't want extra costs..

This is the image used by the processing job, with a instance type of ml.m5.2xlarge which I didn't set anywhere..

929884845733.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-debugger-rules:latest

question from:https://stackoverflow.com/questions/65882686/aws-sagemaker-processing-job-automatically-created

Answer 1

I can answer my question.. it seems to be a new feature as highlighted here. You can turn it off as suggested in the doc:

To disable both monitoring and profiling, include the disable_profiler parameter to your estimator and set it to True.