Skip to content

Optimize convert_vec_soft_to_tensor_hard by 50x - #160

Merged
bmhowe23 merged 2 commits into
NVIDIA:mainfrom
bmhowe23:pr-opt-conversions
May 13, 2025
Merged

bmhowe23 merged 2 commits into
NVIDIA:mainfrom
bmhowe23:pr-opt-conversions

Conversation

@bmhowe23

@bmhowe23 bmhowe23 commented May 10, 2025 •

Copy link
Copy Markdown
Collaborator

This small PR optimizes one of our utility functions by >50x.

In the long term, we need to consider accessor methods for cudaqx::tensor that don't incur the performance penalty of allocating and deallocating vectors on every .at() access. As best as I can tell, this is not supported very well by the underlying xtensor class.

Signed-off-by: Ben Howe <bhowe@nvidia.com>
@bmhowe23
bmhowe23 requested review from kvmto and melody-ren May 12, 2025 16:59

@melody-ren melody-ren left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks, Ben.

Curious though - (1) why is the change not applied to the 2D version of convert_vec_hard_to_soft? (2) What prompted this change? Based on a bug, profiling results or something else?

@bmhowe23

bmhowe23 commented May 12, 2025 •

Copy link
Copy Markdown
Collaborator Author

Looks good. Thanks, Ben.

Curious though - (1) why is the change not applied to the 2D version of convert_vec_hard_to_soft? (2) What prompted this change? Based on a bug, profiling results or something else?

The key change that had a big improvement was to convert_vec_soft_to_tensor_hard. The change to convert_vec_hard_to_soft was much more minor. I noticed it had a minor improvement in the test case I was dealing (to avoid the repeated push_back's), but I didn't have any data to justify making a change to the 2D version.

I made these discoveries while working on the sliding window decoder prototypes, which was performing very poorly until these changes were made.

@bmhowe23
bmhowe23 enabled auto-merge (squash) May 13, 2025 05:06
@bmhowe23
bmhowe23 merged commit 2c8da06 into NVIDIA:main May 13, 2025
@bmhowe23
bmhowe23 deleted the pr-opt-conversions branch June 4, 2025 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants