Use custom Amazon SageMaker tags to manage project resources and track costs | Amazon Web Services

Amazon SageMaker announced a new feature that you can use to add custom tags to resources created through an Amazon SageMaker Unified Studio project. This will help you enforce labeling standards that align with your organization’s Service Control Principles (SCPs) and help enable resource cost tracking practices established across the organization.

As a SageMaker administrator, you can configure a project profile with tag configurations that will be pushed down to projects that currently use or will use that project profile. The project profile is set to either pass the required tag key-value pairs, or to pass a tag key with a default value that can be modified during project creation. All tags passed to a project will have the effect of tagging resources created by that project. This gives you a control mechanism to enforce that project resources have the expected tags across all projects in the domain.

The first release of custom tags for project resources is supported via an application programming interface (API) through the Amazon DataZone SDKs. In this post, we’ll look at the use cases for custom tags and how to use the AWS Command Line Interface (AWS CLI) to add tags to project resources.

What we hear from customers

As customers continue to build and collaborate using AWS tools for model development, generative artificial intelligence, data processing, and SQL analytics, they see the need to provide control and visibility into the resources they are creating. To support connecting to these AWS tools from SageMaker Unified Studio projects, many different types of resources need to be created across AWS services. These resources are created through AWS CloudFormation stacks (via project environment deployment) by Amazon SageMaker. We hear the following use cases from customers:

• Customers must enforce tagging practices to conform to company policy by using AWS controls such as SCPs to create resources. These controls block resource creation unless specific tags are placed on the resource.
• Customers can also start with policies that enforce the placement of correct tags when creating resources, with the added goal of standardizing resource reporting. By placing identifiable information in sources as they are created, they enforce consistency and completeness in performing cost attribution reporting and traceability.

Customer Swiss Life uses SageMaker as a single solution to catalog, discover, share and manage their corporate data across business domains. They require all resources to have a set of mandatory tags for their financial group in order to bill organizations across the company for created AWS resources.

“Launching project resource tags for Amazon SageMaker gives us visibility into costs incurred across our accounts. This capability allows us to meet our company’s resource tagging guidelines and be confident in assigning costs within our multi-account setup for resources created by Amazon SageMaker projects.”

– Tim Kopacz, software developer at Swiss Life

Prerequisites

To get started with your own tags, you need the following resources:

• SageMaker Unified Studio domain.
• An AWS Identity and Access Management (IAM) entity with permissions to make AWS CLI calls to the domain.
• An IAM entity authorized to make changes to the IAM domain provisioning role. If SageMaker created it for you, it will be called AmazonSageMakerProvisioning-. The provisioning role provisions and manages the resources defined in the selected plans in your account.

How to set project resource tags

The following steps describe how you can configure custom tags for SageMaker Unified Studio project resources:

1. (Optional) Update the SageMaker provisioning role to enable specific tag keys.
2. Create a new project profile with the project resource tags configured.
3. Create a new project with project resource tags.
4. Update an existing project using project source tags.
5. Verify that the sources are marked.

(Optional) Update the SageMaker provisioning role to allow tag key values

Tea AmazonSageMakerProvisioning- the role has an AWS managed policy with a condition aws:TagKeys allows you to create tags with this role only if the tag key starts with AmazonDataZone. In this example, we change the tag key to start with different strings. Go to Create a new project profile with the project resource tags configured unless you need the tag keys to have a different structure (like starts with, contains, etc.)

1. Open the AWS Management Console and go to IAM.
2. In the navigation pane, select Role.
3. Select from the list AmazonSageMakerProvisioning-.
4. Select Authorization table
5. Choose Add permissionand then select Create an inline policy.
6. Below Policy editorselect JSON.
7. Enter the following policies. Add strings under the condition aws:TagKeys. In this example, tag keys starting with TOP or an exact match key tag CostCenter a role will be created.

   {
       "Version": "2012-10-17",
       "Statement": (
           {
               "Sid": "CustomTagsUnTagPermissions",
               "Effect": "Allow",
               "Action": (
                   "codecommit:UntagResource",
                   "iam:UntagRole",
                   "logs:UntagResource",
                   "athena:UntagResource",
                   "redshift-serverless:UntagResource",
                   "scheduler:UntagResource",
                   "bedrock:UntagResource",
                   "neptune-graph:UntagResource",
                   "quicksight:UntagResource",
                   "glue:UntagResource",
                   "airflow:UntagResource",
                   "secretsmanager:UntagResource",
                   "lambda:UntagResource",
                   "emr-serverless:UntagResource",
                   "elasticmapreduce:RemoveTags",
                   "sagemaker:DeleteTags",
                   "ec2:DeleteTags"
               ),
               "Resource": "*",
               "Condition": {
                   "StringEquals": {
                       "aws:ResourceAccount": "${aws:PrincipalAccount}"
                   },
                   "ForAllValues:StringLike": {
                       "aws:TagKeys": (
                           "AmazonDataZone*",
                           "ACME*",
                           "CostCenter"
                       )
                   },
                   "Null": {
                       "aws:ResourceTag/AmazonDataZoneProject": "false"
                   }
               }
           },
           {
               "Sid": "CustomTagsTaggingPermissions",
               "Effect": "Allow",
               "Action": (
                   "cloudformation:TagResource",
                   "codecommit:TagResource",
                   "iam:TagRole",
                   "glue:TagResource",
                   "athena:TagResource",
                   "lambda:TagResource",
                   "redshift-serverless:TagResource",
                   "logs:TagResource",
                   "secretsmanager:TagResource",
                   "sagemaker:AddTags",
                   "emr-serverless:TagResource",
                   "neptune-graph:TagResource",
                   "bedrock:TagResource",
                   "elasticmapreduce:AddTags",
                   "airflow:TagResource",
                   "scheduler:TagResource",
                   "quicksight:TagResource",
                   "emr-containers:TagResource",
                   "logs:CreateLogGroup",
                   "athena:CreateWorkGroup",
                   "scheduler:CreateScheduleGroup",
                   "cloudformation:CreateStack",
                   "ec2:*"
               ),
               "Resource": "*",
               "Condition": {
                   "ForAnyValue:StringLike": {
                       "aws:TagKeys": (
                           "AmazonDataZone*",
                           "ACME*",
                           "CostCenter"
                       )
                   },
                   "StringEquals": {
                       "aws:ResourceAccount": "${aws:PrincipalAccount}"
                   }
               }
           }
       )
   }

It is possible to restrict a specific AWS service tag and de-tag permission based on the plans or features used.

Create a new project profile with the project resource tags configured

Use the following steps to create a new SQL Analytics project profile with custom tags. The example uses AWS CLI commands.

1. Open the AWS CloudShell console.
2. Create a project profile using the following CLI command.
   1. Tea project-resource-tags parameter consists of key (brand key), value (brand value) a isValueEditable (boolean indicating whether the tag value can be modified during project creation or update).
   2. Tea allow-custom-project-resource-tags parameter set to true allows the project creator to create additional key-value pairs. The key must conform to the inline policy AmazonSageMakerProvisioning- role.
   3. Tea project-resource-tags-description parameter is a description field for project resource tags. The maximum number of characters allowed is 2048. The description must be entered every time create-project-profile gold update-project-profile is called

   aws datazone create-project-profile \
     --name "SQL Analytics with Project Resource Tags" \
     --description "Analyze your data in SageMaker Lakehouse using SQL" \
     --domain-identifier "$DOMAIN_ID" \
     --region "$REGION" \
     --status ENABLED \
     --project-resource-tags '(
       {
           "key": "ACME-Application",
           "value": "SageMaker",
           "isValueEditable": false
       },
       {
           "key": "CostCenter",
           "value": "123",
           "isValueEditable": true
       }
     )' \
     --allow-custom-project-resource-tags \
     --environment-configurations '(
       {
           "name": "Tooling",
           "description": "Configuration for the Tooling Environment",
           "environmentBlueprintId": "",
           "deploymentMode": "ON_CREATE",
           "deploymentOrder": 0,
           "awsAccount": {
           "awsAccountId": "$ACCOUNT"
       },
       "awsRegion": {
           "regionName": "$REGION"
       },
           "configurationParameters": {
               "parameterOverrides": (
                   {
                       "name": "enableSpaces",
                       "value": "false",
                       "isEditable": false
                   },
                   {
                       "name": "maxEbsVolumeSize",
                       "isEditable": false
                   },
                   {
                       "name": "idleTimeoutInMinutes",
                       "isEditable": false
                   },
                   {
                       "name": "lifecycleManagement",
                       "isEditable": false
                   },
                   {
                       "name": "enableNetworkIsolation",
                       "isEditable": false
                   }
               )
           }
       },
       {
           "name": "Lakehouse Database",
           "description": "Creates databases in Amazon SageMaker Lakehouse for storing tables in S3 and Amazon Athena resources for your SQL workloads",
           "environmentBlueprintId": "",
           "deploymentMode": "ON_CREATE",
           "deploymentOrder": 1,
           "awsAccount": {
               "awsAccountId": "$ACCOUNT"
           },
           "awsRegion": {
           "regionName": "$REGION"
           },
           "configurationParameters": {
               "parameterOverrides": (
                   {
                       "name": "glueDbName",
                       "value": "glue_db",
                       "isEditable": true
                   }
               )
           }
       },
       {
           "name": "OnDemand RedshiftServerless",
           "description": "Enables you to create an additional Amazon Redshift Serverless workgroup for your SQL workloads",
           "environmentBlueprintId": "",
           "deploymentMode": "ON_DEMAND",
           "awsAccount": {
           "awsAccountId": "$ACCOUNT"
           },
           "awsRegion": {
               "regionName": "$REGION"
           },
           "configurationParameters": {
               "parameterOverrides": (
                   {
                       "name": "redshiftDbName",
                       "value": "dev",
                       "isEditable": true
                       },
                       {
                       "name": "redshiftMaxCapacity",
                       "value": "512",
                       "isEditable": true
                       },
                       {
                       "name": "redshiftWorkgroupName",
                       "value": "redshift-serverless-workgroup",
                       "isEditable": true
                       },
                       {
                       "name": "redshiftBaseCapacity",
                       "value": "128",
                       "isEditable": true
                       },
                       {
                       "name": "connectionName",
                       "value": "redshift.serverless",
                       "isEditable": true
                       },
                       {
                       "name": "connectToRMSCatalog",
                       "value": "false",
                       "isEditable": false
                       }
                   )
               }
           },
           {
               "name": "OnDemand Catalog for Redshift Managed Storage",
               "description": "Enables you to create additional catalogs in Amazon SageMaker Lakehouse for storing data in Redshift Managed Storage",
               "environmentBlueprintId": "",
               "deploymentMode": "ON_DEMAND",
               "awsAccount": {
               "awsAccountId": "$ACCOUNT"
               },
               "awsRegion": {
                   "regionName": "$REGION"
               },
               "configurationParameters": {
                   "parameterOverrides": (
                       {
                           "name": "catalogName",
                           "isEditable": true
                       },
                       {
                           "name": "catalogDescription",
                           "value": "RMS catalog",
                           "isEditable": true
                       }
                   )
               }
           }
     )'

This project profile will have a tag ACME-Application = SageMaker placed on all projects associated with the project profile and cannot be modified by the project creator. Mark CostCenter = 123 may have a value modified by the project creator because isValueEditable property is set to true.

Grant users permission to use the project profile during project creation. IN Authorization either the project profile section Selected users or groups gold Allow all users and groups.

Use allow-custom-project-resource-tags parameter means that the project creator can add his own tags (key-value pair). The key must match the status check in the provisioning role policy (AmazonSageMakerProvisioning-). If allow-custom-project-resource-tagsthe parameter changes to false after project tags are created, the tags created by the project will be removed the next time the project is updated.

Project profile update

Updates to project resource tags are possible via update-project-profile command. The command replaces all values ​​in project-resource-tags section, so be sure to include an exhaustive set of tags. The project profile update will be reflected in the projects after launch update-project or when a new project is created using a project profile. The following example adds a new tag, ACME-BusinessUnit = Retail.

There are three ways to work with project-resource-tags parameter when updating the project profile.

• Passing a non-empty list of project resource tags will replace the tags currently configured in the project profile.
• Passing an empty list of project resource tags clears all previously configured tags:
  • --project-resource-tags '()'
• By not including the project resource tag parameter, the previously configured tags will remain as they are.

aws datazone update-project-profile \
  --domain-identifier "$DOMAIN_ID" \
  --identifier "$PROJECT_PROFILE_ID" \
  --region "$REGION" \
  --project-resource-tags '(
    {
        "key": "ACME-Application",
        "value": "SageMaker",
        "isValueEditable": false
    },
    {
        "key": "CostCenter",
        "value": "123",
        "isValueEditable": true
    },
    {
        "key": "ACME-BusinessUnit",
        "value": "Retail",
        "isValueEditable": false
    }
  )'

Create a new project with project resource tags

The following steps will guide you through creating a new project that will inherit the tags from the project profile and allow the project creator to modify one of the tag values.

1. Create the project using the following CLI command example.
2. Edit CostCenter brand value using --resource-tags parameter.parameter. Tags configured in the project profile where it is isValueEditable attribute is false is automatically moved to the project.

   aws datazone create-project \
     --domain-identifier "$DOMAIN_ID" \
     --region "$REGION" \
     --name "$PROJECT_NAME" \
     --description "New project with tags" \
     --project-profile-id "$PROJECT_PROFILE_ID" \
     --resource-tags '{
           "CostCenter": "456"
       }'

Update an existing project using project source tags

For existing projects associated with a project profile, you must update the project to apply the new tags.

1. Update the project using the following CLI command example.
2. In this scenario, you need to modify the editable value and add a new tag. Label CostCenter will have the default value overwritten as “789” and new ACME-Department = Finance the tag will be added.

   aws datazone update-project \
     --domain-identifier "$DOMAIN_ID" \
     --identifier "$PROJECT_ID" \
     --project-profile-version "latest" \
     --region "$REGION" \
     --resource-tags '{
           "CostCenter": "789",
           "ACME-Department": "Finance"
       }' 

Project level tags (those not configured from the project profile) need to be passed during the project update to be preserved. For brands with isValueEditable = true configured from the project profile, any previously set override must be used, otherwise the value will revert to the default value from the project profile.

Validating sources are marked

Verify that the tags are positioned correctly. An example of a project-created resource is a project’s IAM role. The tags view for this role should show the tags configured from the project profile.

1. Open SageMaker Unified Studio and get the project role from Project details part of the project. The role name starts with datazone_usr_role_.
2. Open the IAM console.
3. In the navigation pane, select Role.
4. Locate the project’s IAM role.
5. Select Tags table
   
   

Conclusion

In this post, we discussed use cases related to tags from customers and walked through how to get started with custom tags in Amazon SageMaker to place tags on project-created resources. By allowing administrators to configure project profiles using standardized tagging configurations, you can now help ensure consistent tagging practices across all SageMaker Unified Studio projects while maintaining SCP compliance. This feature addresses two critical customer needs: enforcing organization labeling standards through automated governance mechanisms and enabling accurate cost attribution reporting across multi-service deployments.

To learn more, visit Amazon SageMaker and get started using project resource tags.

About the authors