[air] Fix NoneType error loading TorchCheckpoint through from_uri. - #32386
Merged
Conversation
Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
Yard1
requested changes
Feb 10, 2023
Member
There was a problem hiding this comment.
I don't think we need this, the model should be already encoded when it was put into the URI. I think the only thing that needs to be fixed is the missing check for the case where _data_dict is None:
def __setstate__(self, state: dict):
if state.get("_data_dict", None):
state = state.copy()
state["_data_dict"] = self._decode_data_dict(state["_data_dict"])
super().__setstate__(state)
Contributor
Author
isn't this for decoding ? We encode it when putting into the URI and decode it when retrieving? |
Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
Contributor
Author
|
waiting for https://github.com/ray-project/ray/pull/32479/files to be landed first... |
from_uri.
Yard1
reviewed
Feb 13, 2023
Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
Yard1
approved these changes
Feb 13, 2023
Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
Yard1
reviewed
Feb 17, 2023
Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
edoakes
pushed a commit
to edoakes/ray
that referenced
this pull request
Mar 22, 2023
…ay-project#32386) * Eagerly load TorchCheckpoint upon from_uri. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * typo Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * [no_ci] remove eager load Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * avoid shallow copy Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * Use `mock_s3_bucket_uri` fixture. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * remove conftest import Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * fix Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * revert changes in tune/. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> --------- Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>
peytondmurray
pushed a commit
to peytondmurray/ray
that referenced
this pull request
Mar 22, 2023
…ay-project#32386) * Eagerly load TorchCheckpoint upon from_uri. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * typo Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * [no_ci] remove eager load Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * avoid shallow copy Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * Use `mock_s3_bucket_uri` fixture. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * remove conftest import Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * fix Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * revert changes in tune/. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> --------- Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com>
elliottower
pushed a commit
to elliottower/ray
that referenced
this pull request
Apr 22, 2023
…ay-project#32386) * Eagerly load TorchCheckpoint upon from_uri. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * typo Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * [no_ci] remove eager load Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * avoid shallow copy Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * Use `mock_s3_bucket_uri` fixture. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * remove conftest import Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * fix Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> * revert changes in tune/. Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> --------- Signed-off-by: xwjiang2010 <xwjiang2010@gmail.com> Signed-off-by: elliottower <elliot@elliottower.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
__getstate__and__setstate__to_uri, at which point we should revisit where to put the encode/decode logic.Signed-off-by: xwjiang2010 xwjiang2010@gmail.com
Why are these changes needed?
Related issue number
Closes #32284
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.