Skip to content

Prevent override of region when using endpoint_url - #464

Merged
kyleknap merged 8 commits into
boto:developfrom
kyleknap:endpoint-override-region
Feb 19, 2015
Merged

kyleknap merged 8 commits into
boto:developfrom
kyleknap:endpoint-override-region

Conversation

@kyleknap

Copy link
Copy Markdown
Contributor

This ensures that user's region is not overwritten by the credentialScope if the user supplies an endpoint url. This is needed to be able to make signed calls to STS regionalized endpoints, without getting auth errors due to incorrect region . So for example, in the CLI, to make a GetSessionToken call to us-west-2 you would do:

$ aws sts get-session-token --region us-west-2 --endpoint-url https://sts.us-west-2.amazonaws.com

Before, this would cause a signing error if signed with sigv4 because the credentialScope was set to us-east-1 and that overrode the region that you provided.

cc @jamesls @danielgtaylor

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.0%) to 94.51% when pulling 1a25222 on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

@kyleknap

Copy link
Copy Markdown
Contributor Author

Coveralls is wrong. Coverage should have went up.

@danielgtaylor

Copy link
Copy Markdown
Contributor

LGTM 🚢-it!

Comment thread botocore/endpoint.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Region is only required for sigv4, so you could previously specify an endpoint-url without a region provided you weren't using sigv4. For example, this is a breaking change for commands such as this:

$ aws configure list | grep region
    region                <not set>             None    None
$ aws s3api list-buckets --endpoint-url https://s3.amazonaws.com/

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point so how about instead?
if endpoint_url is None and region_name is None:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that will fix the issue. In the example above endpoint_url is not None so we don't pull in the region override from the heuristics file.

@kyleknap
kyleknap force-pushed the endpoint-override-region branch from d52ac03 to 68a4c9d Compare February 18, 2015 22:38
@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.0%) to 94.51% when pulling 68a4c9d on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

@kyleknap

Copy link
Copy Markdown
Contributor Author

@jamesls
I think I got the s3 case covered now. The credentialScope region is only used now if it is specified in the endpoint JSON, an endpoint url was not specified, and then, if a region was not specified by the user.

@jamesls

jamesls commented Feb 19, 2015

Copy link
Copy Markdown
Member

Is there another change that's needed before the calls to us-west-2 work? I'm seeing a signing error with the latest changes:

$ aws sts get-session-token --region us-west-2 --endpoint-url https://sts.us-west-2.amazonaws.com

A client error (SignatureDoesNotMatch) occurred when calling the GetSessionToken operation: Credential should be scoped to a valid region, not 'us-east-1'.

@kyleknap
kyleknap force-pushed the endpoint-override-region branch from 2028552 to 5755430 Compare February 19, 2015 07:30
Also touched up endpoint code.
@kyleknap
kyleknap force-pushed the endpoint-override-region branch from 5755430 to 57a3240 Compare February 19, 2015 17:11
@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage remained the same at 94.5% when pulling 57a3240 on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

@kyleknap

Copy link
Copy Markdown
Contributor Author

To make it clear what I did in the latest commits, here is a summary:

  1. Separate all of the region determining logic into a helper method _determine_region_name() in the endpoints class. The goal is that this method can be directly ported to clients when we deprecate operation objects and endpoint.region_name. Note that now both the clients and operations rely on this helper method for region name logic. This consolidation will ensure that clients and operation objects will use the same codepath when choosing a region name.

  2. Removed some of the signature version arguments from endpoint methods. We were not using them at all.

  3. Added integration test to test that sts regionalized calls can be made.

Comment thread botocore/endpoint.py

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a backward-incompatible change of a public function. It's possible we might break someone using Botocore.

@danielgtaylor

Copy link
Copy Markdown
Contributor

I think this looks good. We just need to decide if we really want to make that breaking change. 🚢-it.

Comment thread tests/integration/test_sts.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't call sts when you're using session credentials (say on a CI box using an IAM role), so we should put a check where we only run these tests if we have long term credentials.

@coveralls

Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.0%) to 94.51% when pulling 2d215bc on kyleknap:endpoint-override-region into 6a63e18 on boto:develop.

@kyleknap

Copy link
Copy Markdown
Contributor Author

@jamesls
I made the if statement in the setUp that if a session token is being used then skip the test.

@jamesls

jamesls commented Feb 19, 2015

Copy link
Copy Markdown
Member

:shipit:

kyleknap added a commit that referenced this pull request Feb 19, 2015
Prevent override of region when using endpoint_url
@kyleknap
kyleknap merged commit bd1077a into boto:develop Feb 19, 2015
@kyleknap
kyleknap deleted the endpoint-override-region branch February 19, 2015 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants