Commit graph

95 commits

Author SHA1 Message Date
Nekotekina 2212a131ef Fix some -Weffc++ warnings (part 1) 2021-03-31 11:27:09 +03:00
Nekotekina 729757b1dd atomic.hpp: allow raw notify_all by pointer 2021-03-23 20:58:42 +03:00
Nekotekina 393f9d329d atomic.cpp: allow non-standard sizes in atomic wait (>16) 2021-03-23 20:52:56 +03:00
Nekotekina 820390b7ed atomic.cpp: remove raw_notify and simplify internal logic
Also permit zero size in waiters.
2021-03-23 20:19:10 +03:00
Nekotekina 87af905018 Enable -Wunused-parameter 2021-03-06 18:07:08 +03:00
Nekotekina 4f8cac731b Fix SAFE_BUFFERS attribute for GCC 2021-02-09 12:33:50 +03:00
Nekotekina db8e6fe7a7 Enable -Wunused-variable 2021-01-12 14:34:14 +03:00
Nekotekina 6b96807112 Reimplement utils::popcnt64
Implement utils::popcnt128
2020-12-29 15:28:02 +03:00
Nekotekina bd269bccaf types.hpp: remove intrinsic includes
Replace v128 with u128 in some places.
Removed some unused files.
2020-12-21 21:11:25 +03:00
Nekotekina 5f618814f6 atomic.hpp: use u128 as storage for masks/values 2020-12-21 13:51:23 +03:00
Nekotekina fb29933d3d Add usz alias for std::size_t 2020-12-18 12:23:53 +03:00
Nekotekina a6a5292cd7 Use uptr (std::uintptr_t alias) 2020-12-12 16:29:55 +03:00
Nekotekina 36c8654fb8 Remove HERE macro
Some cleanup.
Add location to some functions.
2020-12-10 12:30:22 +03:00
Nekotekina e055d16b2c Replace verify() with ensure() with auto src location.
Expression ensure(x) returns x.
Using comma operator removed.
2020-12-09 15:43:38 +03:00
Nekotekina b16cc618b5 atomic.hpp: add some features and optimizations
Add atomic_t<>::observe() (relaxed load)
Add atomic_fence_XXX() (barrier functions)
Get rid of MFENCE instruction, replace with no-op LOCK OR on stack.
Remove <atomic> dependence from stdafx.h and relevant headers.
2020-12-07 17:13:12 +03:00
RipleyTom af8c661a64 Remove BOM markers 2020-12-06 15:30:12 +03:00
Nekotekina 019f566239 atomic.cpp: more phase splitting in notify_all() 2020-12-04 14:10:53 +03:00
Nekotekina 13a0b88755 atomic.cpp: improve hashtable access
Simplified, tried to use all ptr bits.
Try to prevent self-collisions (bug).
Hashtable size doubled (to 8 MiB).
2020-12-01 10:19:31 +03:00
Nekotekina bd90e3e37f atomic.cpp: shrink and simplify main hashtable
Reduce collision detection to 1 or 0 for now.
I think it should be offloaded to notifiers.
2020-11-26 20:11:26 +03:00
Nekotekina 8b6d615aa6 atomic.hpp: remove/deprecate incomplete notify features
Notification with "phantom value" could be useful in theory.
As an alternative way of sending signal from the notifier.
But current implementation is just useless.
Also fixed slow_mutex (not used anywhere).
2020-11-26 06:31:36 +03:00
Nekotekina 600c0ea5ba atomic.cpp: fix windows/std path
Always check condition after rearming internal semaphore.
2020-11-25 10:41:17 +03:00
Nekotekina 43952e18e2 Implement prefetch_write() and prefetch_exec() wrappers
Do some refactoring to prefetch_read() in util/asm.hpp as well.
Make all these function constexpr because they are no-ops.
2020-11-24 12:31:11 +03:00
Nekotekina 14fa96fdef atomic.cpp: don't use empty notify callback
Instead, allow setting it to null and check for null.
Similar could be done for wait cb, but it's set for all named threads.
2020-11-24 01:09:04 +03:00
Nekotekina d70ab11477 atomic.cpp: remove redundant constexpr
Static initialization is all-zeros anyway.
But constexpr was killing my intellisense.
And probably also affected compile time.

Also make some internal structures hidden ("static").
2020-11-22 12:07:49 +03:00
Nekotekina 60cff6f3d4 Attempt to fix #9271 2020-11-21 07:09:48 +03:00
Nekotekina 5f010c2f4c atomic.cpp: optimize "cond" allocation with TLS
Billions of events can reduce to thousands, saving CPU time a few s.
Only 4 slots are available (arbitrarily), and only 1 is usually used.
Other slots are used only for waiting on multiple atomics.
2020-11-20 19:13:22 +03:00
Nekotekina 6c18e00977 atomic.cpp: use deterministic "cond" allocation
Takes ~80 ns instead of ~40 ns, same about deallocation.
Loops don't exist here, only 4-level semaphore tree.
Worst case only happens with concurrence, not from looping.
This optimization is not really necessary at current state of RPCS3.
This is more like to test C++ compilers and MSVC u128 implementation.
2020-11-20 15:04:17 +03:00
Nekotekina 72ac1aa537 atomic.cpp: use some actual hash for hashtable (std::minstd_rand)
Should only take about 5ns for initialization.
Should also make collisions almost impossible.

I also suspect I fixed some bad bug in process...
2020-11-20 15:04:17 +03:00
Nekotekina 3dd48a24ce atomic.cpp: remove tiny redundancy 2020-11-17 15:22:04 +03:00
Nekotekina 333fe9a561 atomic.cpp: fix notification on windows 2020-11-15 04:40:44 +03:00
Nekotekina badb3dc2dd atomic.cpp/threads: remove old wait callback
Add new wait callback which simply collects statistics.
Shift workarounds towards actual problem detection.
2020-11-14 18:16:27 +03:00
Nekotekina 3ac819ee70 atomic.cpp: fix waiting on multiple variables
Silly bugs are so silly...
2020-11-14 17:25:02 +03:00
Nekotekina 00f877e812 atomic.cpp: fix rare overload scenario
If slot allocation fails, increase reference counter in every slot.
This prevents search routine from "losing" some slots.
2020-11-14 13:56:23 +03:00
Nekotekina e48f160a29 atomic.cpp: continuation of fixing all-ones masks
Also added missing const noexcept.
2020-11-13 12:40:02 +03:00
Nekotekina cfda4d0ade atomic.cpp: optimize raw_notify() for unspecified pointer mode
Remove unnecessary optimization from cond_alloc().
Optimistic case was absolutely dominating anyway.
Although the whole function is a dirty hack.
Now scanning through all threads is faster.
2020-11-13 07:05:33 +03:00
Nekotekina 3888b0429c atomic.cpp: print some stats on stop 2020-11-12 05:52:22 +03:00
Nekotekina 70761a4ef0 atomic.cpp: fix notification bug from few commits before
Make sure forced notification ignores mask comparison.
Also fix masks provided by smaller types to correct ones.
2020-11-12 05:01:44 +03:00
Nekotekina e37df998e9 atomic.cpp: minor optimization
Silly MSVC emits unnecessary initialize-on-first-use code guards.
Didn't check other compiler. But workaround just works anyway.
2020-11-12 03:41:19 +03:00
Nekotekina c30cdfb3a7 atomic.cpp: merge more atomic variables for simplicity
Compress 16-bit ref counter and two 48+64 bit slot allocators.
This allowed to remove some weird unnecessary logic paths.
Adjust hashtable size to keep it the same.
2020-11-12 03:39:05 +03:00
Nekotekina 350b704cd7 atomic.cpp: combine two atomics (ref ctr and wait ptr)
Should fit into cache line perfectly.
Also makes ptr check more robust.
2020-11-12 01:06:12 +03:00
Nekotekina ad013d59f4 atomic.cpp: don't load actual memory on empty mask
It's not an optimization. It just allows some tricks safely.
2020-11-11 23:59:09 +03:00
Nekotekina d391133524 atomic.cpp: minor notification improvement
Check mask before locking the reference.
Can help to not bother non-eligible waiters.
2020-11-11 23:51:41 +03:00
Nekotekina 4b823469f7 atomic.cpp: restore notify_all() behaviour
Restore batch notification with non-blocking attempts.
2020-11-11 22:43:13 +03:00
Nekotekina 7cd1e767be atomic.cpp: implement some atomic wait operations.
Instead of plain waiting while equal to some value,
it can be something like less, or greater, or even bitcount.
But it's a draft and untested. Hopefully doesn't break anything.
2020-11-11 20:46:37 +03:00
Nekotekina c047ab6147 atomic.cpp: waitable atomics got brutally rewritten
Hashtable increased and flatten, tree-alike extensions removed.
Some things simplified, so it can actually decrease perf a bit.
But most platforms shouldn't be affected.
Removed limit of 56 waiters per pointer.
Real limit now is about 65535.
2020-11-11 05:35:27 +03:00
Nekotekina fd7ea82965 atomic.cpp: fix 3-arg notify functions
Third arg forces comparison with itself, and possibly forced wakeup.
2020-11-07 19:18:14 +03:00
Nekotekina 0bd989b118 atomic.cpp: allow waiting on multiple variables simultaneously
Arbitrary maximum set to 8, but really we need 2, maybe 3.
Added atomic_wait::list object for multi-waiting.
Added atomic_wait::get_unique_tsc just in case.
2020-11-07 18:18:02 +03:00
Nekotekina bd0b292b39 atomic.cpp: adjust assumed pointer mask
Increase waiter mask (allow 65535 waiters).
Assume 47-bit pointers instead of 48-bit.
2020-11-07 15:08:46 +03:00
Nekotekina 31f0a0c064 atomic.cpp: add minor safety check 2020-11-07 15:08:46 +03:00
Nekotekina 3e445eff8e atomic.cpp: maintenance of wait() function
Move hashtable allocation into another function.
2020-11-07 15:08:46 +03:00