Repository navigation
Improve/cleanup annotated_ptr implementation - #4503
Conversation
| ((__hit_prop == cudaAccessPropertyNormal) ? __sm_80::__on::__l2_cop_on_t::_L2_EVICT_NORMAL_DEMOTE | ||
| : static_cast<__sm_80::__on::__l2_cop_on_t>(__hit_prop)), | ||
| __ap_min( | ||
| (static_cast<std::uint32_t>(__hit_ratio) * __sm_80::__l2_eviction_max_way_t::_CUDA_AMPERE_MAX_L2_WAYS), |
There was a problem hiding this comment.
this seems a bug because __hit_ratio is in the range [0-1)
| cudaAccessProperty __miss_prop = cudaAccessPropertyNormal) | ||
| cudaAccessProperty __miss_prop = cudaAccessPropertyNormal) noexcept | ||
| { | ||
| return (__total_bytes <= (size_t{0xFFFFFFFF}) & __total_bytes != 0 & __hit_bytes <= __total_bytes) |
There was a problem hiding this comment.
seems a wrong usage of bitwise and instead of logical and
ðĻ CI finished in 1h 06m: Pass: 63%/174 | Total: 1d 20h | Avg: 15m 21s | Max: 1h 02m | Hits: 76%/156639
|
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| CUB | |
| Thrust | |
| CUDA Experimental | |
| stdpar | |
| python | |
| CCCL C Parallel Library | |
| Catch2Helper |
Modifications in project or dependencies?
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| +/- | CUB |
| +/- | Thrust |
| +/- | CUDA Experimental |
| +/- | stdpar |
| +/- | python |
| +/- | CCCL C Parallel Library |
| +/- | Catch2Helper |
ðâ Runner counts (total jobs: 174)
| # | Runner |
|---|---|
| 123 | linux-amd64-cpu16 |
| 15 | windows-amd64-cpu16 |
| 12 | linux-arm64-cpu16 |
| 10 | linux-amd64-gpu-rtx2080-latest-1 |
| 6 | linux-amd64-gpu-rtxa6000-latest-1 |
| 5 | linux-amd64-gpu-h100-latest-1 |
| 3 | linux-amd64-gpu-rtx4090-latest-1 |
ðĐ CI finished in 55m 09s: Pass: 100%/174 | Total: 1d 10h | Avg: 11m 57s | Max: 52m 41s | Hits: 85%/271675
|
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| CUB | |
| Thrust | |
| CUDA Experimental | |
| stdpar | |
| python | |
| CCCL C Parallel Library | |
| Catch2Helper |
Modifications in project or dependencies?
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| +/- | CUB |
| +/- | Thrust |
| +/- | CUDA Experimental |
| +/- | stdpar |
| +/- | python |
| +/- | CCCL C Parallel Library |
| +/- | Catch2Helper |
ðâ Runner counts (total jobs: 174)
| # | Runner |
|---|---|
| 123 | linux-amd64-cpu16 |
| 15 | windows-amd64-cpu16 |
| 12 | linux-arm64-cpu16 |
| 10 | linux-amd64-gpu-rtx2080-latest-1 |
| 6 | linux-amd64-gpu-rtxa6000-latest-1 |
| 5 | linux-amd64-gpu-h100-latest-1 |
| 3 | linux-amd64-gpu-rtx4090-latest-1 |
ðĐ CI finished in 1h 02m: Pass: 100%/174 | Total: 1d 02h | Avg: 9m 17s | Max: 39m 22s | Hits: 95%/272293
|
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| CUB | |
| Thrust | |
| CUDA Experimental | |
| stdpar | |
| python | |
| CCCL C Parallel Library | |
| Catch2Helper |
Modifications in project or dependencies?
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| +/- | CUB |
| +/- | Thrust |
| +/- | CUDA Experimental |
| +/- | stdpar |
| +/- | python |
| +/- | CCCL C Parallel Library |
| +/- | Catch2Helper |
ðâ Runner counts (total jobs: 174)
| # | Runner |
|---|---|
| 123 | linux-amd64-cpu16 |
| 15 | windows-amd64-cpu16 |
| 12 | linux-arm64-cpu16 |
| 10 | linux-amd64-gpu-rtx2080-latest-1 |
| 6 | linux-amd64-gpu-rtxa6000-latest-1 |
| 5 | linux-amd64-gpu-h100-latest-1 |
| 3 | linux-amd64-gpu-rtx4090-latest-1 |
ðĐ CI finished in 1h 02m: Pass: 100%/174 | Total: 1d 02h | Avg: 9m 13s | Max: 36m 23s | Hits: 95%/272321
|
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| CUB | |
| Thrust | |
| CUDA Experimental | |
| stdpar | |
| python | |
| CCCL C Parallel Library | |
| Catch2Helper |
Modifications in project or dependencies?
| Project | |
|---|---|
| CCCL Infrastructure | |
| +/- | libcu++ |
| +/- | CUB |
| +/- | Thrust |
| +/- | CUDA Experimental |
| +/- | stdpar |
| +/- | python |
| +/- | CCCL C Parallel Library |
| +/- | Catch2Helper |
ðâ Runner counts (total jobs: 174)
| # | Runner |
|---|---|
| 123 | linux-amd64-cpu16 |
| 15 | windows-amd64-cpu16 |
| 12 | linux-arm64-cpu16 |
| 10 | linux-amd64-gpu-rtx2080-latest-1 |
| 6 | linux-amd64-gpu-rtxa6000-latest-1 |
| 5 | linux-amd64-gpu-h100-latest-1 |
| 3 | linux-amd64-gpu-rtx4090-latest-1 |
There was a problem hiding this comment.
Minimal review of 5% of the PR since this is already merged but after a first skim there seem to be issues that need to be addressed.
Haven't reviewed the remaining 95% of the changes, @miscco you have checked that all modifications are correct right?
EDIT: seems that this PR was set to automerge, and when @miscco did a review of the changes, the bot accidentally automerged it. This has happened a few times, it may make sense to update/configure the bot to require a "merge this" sort of comment...
| - secondary property to accesses in address-range: | ||
| ``[ptr + leading_bytes, ptr + total_bytes)`` | ||
| - *primary property* to accesses in address-range: ``[ptr, ptr + leading_bytes)`` | ||
| - *secondary property* to accesses in address-range: ``[ptr + leading_bytes, ptr + total_bytes)`` |
There was a problem hiding this comment.
This seems incorrect since secondary should also be applied to [ptr - halo_bytes, ptr) according to PTX docs.
There was a problem hiding this comment.
I will report the halo range only when I have internal confirmation that PTX createpolicy has this behavior.
|
|
||
| |<-- halo_bytes -->|<-- leading_bytes -->|<-- halo_bytes -->| | ||
| |<-- total_bytes -->| | ||
| 2. Binary range ``[ptr, ptr + leading_bytes)`` and ``[ptr + leading_bytes, ptr + total_bytes)`` with *primary* and |
There was a problem hiding this comment.
This also seems incorrect. The range is ternary, with two secondary and one primary regions, like the diagram that was removed shows.
| @@ -0,0 +1,205 @@ | |||
| //===----------------------------------------------------------------------===// | |||
There was a problem hiding this comment.
I think what we had discussed is that:
- There would be a few
cuda::ptx-likecreatepolicyfunctions (the ones in this file, eventually to be added tocuda::ptx). - There would be a
constexprversions of those functions (seems those are inaccess_property_encodingand not here yet and also don't follow these APIs?).
There would be tests that for all inputs check that these two APIs return the same bit identical results.
- There would be a wrapper over both that picks PTX or emulation depending on whether its called at compile-time.
- Access properties would then be implemented on top of this.
There was a problem hiding this comment.
There would be a few cuda::ptx-like createpolicy functions (the ones in this file, eventually to be added to cuda::ptx).
There would be a constexpr versions of those functions (seems those are in access_property_encoding and not here yet and also don't follow these APIs?).
yes, the plan is to add createpolicy to cuda::ptx. On the other hand, adding their constexpr versions seems outside the cuda::ptx model. I would like to discuss the issue with @ahendriksen.
The functions on access_property_encoding.h have the same parameters of createpolicy.
There would be tests that for all inputs check that these two APIs return the same bit identical results.
There would be a wrapper over both that picks PTX or emulation depending on whether its called at compile-time.
Access properties would then be implemented on top of this.
I added all these tests, see
- https://github.com/NVIDIA/cccl/blob/main/libcudacxx/test/libcudacxx/cuda/annotated_ptr/access_property_constexpr.pass.cpp
- https://github.com/NVIDIA/cccl/blob/main/libcudacxx/test/libcudacxx/cuda/annotated_ptr/annotated_ptr.createpolicy.pass.cpp
- https://github.com/NVIDIA/cccl/blob/main/libcudacxx/test/libcudacxx/cuda/annotated_ptr/annotated_ptr_constexpr.pass.cpp
There was a problem hiding this comment.
Not suggesting adding constexpr versions to the cuda::ptx.
Instead suggestion is to add constexpr-only versions to cuda::_detail::*, and then add a constexpr wrapper that dispatches to either the constexpr only versions or to the ptx versions depending on whether its being evaluated at compile time to cuda::__detail::* as well. We can then tests the constexpr-only vs cuda::ptx, but inside annotated_ptr, cuda::load/store/etc use the "wrapper" that automatically dispatches.
| :ref:`cuda::associate_access_property <libcudacxx-extended-api-memory-access-properties-associate-access-property>` and | ||
| :ref:`cuda::apply_access_property <libcudacxx-extended-api-memory-access-properties-apply-access-property>` | ||
| to *request* the application of properties to memory operations. | ||
| The class ``cuda::access_property`` provides an opaque encoding for *L2 cache memory residence* control and *memory space* properties. It is used in combination with :ref:`cuda::annotated_ptr <libcudacxx-extended-api-memory-access-properties-annotated-ptr>`, :ref:`cuda::associate_access_property <libcudacxx-extended-api-memory-access-properties-associate-access-property>` and :ref:`cuda::apply_access_property <libcudacxx-extended-api-memory-access-properties-apply-access-property>` to *request* the application of properties to memory operations. |
There was a problem hiding this comment.
As a first time reader I would appreciate a link: L2 cache memory residence.
I make this comment with the purpose to link it in #4580
Description
List of modifications:
<cuda/>headersannotated_ptrmatches PTXcreatepolicyconstexpronly when makes senseminors:
__stdwith::cuda::stdis_same_v[[nodiscard]]whenever possible and[[maybe_unused]]noexcept