Nekotekina
2212a131ef
Fix some -Weffc++ warnings (part 1)
2021-03-31 11:27:09 +03:00
Nekotekina
729757b1dd
atomic.hpp: allow raw notify_all by pointer
2021-03-23 20:58:42 +03:00
Nekotekina
393f9d329d
atomic.cpp: allow non-standard sizes in atomic wait (>16)
2021-03-23 20:52:56 +03:00
Nekotekina
820390b7ed
atomic.cpp: remove raw_notify and simplify internal logic
...
Also permit zero size in waiters.
2021-03-23 20:19:10 +03:00
Nekotekina
87af905018
Enable -Wunused-parameter
2021-03-06 18:07:08 +03:00
Nekotekina
4f8cac731b
Fix SAFE_BUFFERS attribute for GCC
2021-02-09 12:33:50 +03:00
Nekotekina
db8e6fe7a7
Enable -Wunused-variable
2021-01-12 14:34:14 +03:00
Nekotekina
6b96807112
Reimplement utils::popcnt64
...
Implement utils::popcnt128
2020-12-29 15:28:02 +03:00
Nekotekina
bd269bccaf
types.hpp: remove intrinsic includes
...
Replace v128 with u128 in some places.
Removed some unused files.
2020-12-21 21:11:25 +03:00
Nekotekina
5f618814f6
atomic.hpp: use u128 as storage for masks/values
2020-12-21 13:51:23 +03:00
Nekotekina
fb29933d3d
Add usz alias for std::size_t
2020-12-18 12:23:53 +03:00
Nekotekina
a6a5292cd7
Use uptr (std::uintptr_t alias)
2020-12-12 16:29:55 +03:00
Nekotekina
36c8654fb8
Remove HERE macro
...
Some cleanup.
Add location to some functions.
2020-12-10 12:30:22 +03:00
Nekotekina
e055d16b2c
Replace verify() with ensure() with auto src location.
...
Expression ensure(x) returns x.
Using comma operator removed.
2020-12-09 15:43:38 +03:00
Nekotekina
b16cc618b5
atomic.hpp: add some features and optimizations
...
Add atomic_t<>::observe() (relaxed load)
Add atomic_fence_XXX() (barrier functions)
Get rid of MFENCE instruction, replace with no-op LOCK OR on stack.
Remove <atomic> dependence from stdafx.h and relevant headers.
2020-12-07 17:13:12 +03:00
RipleyTom
af8c661a64
Remove BOM markers
2020-12-06 15:30:12 +03:00
Nekotekina
019f566239
atomic.cpp: more phase splitting in notify_all()
2020-12-04 14:10:53 +03:00
Nekotekina
13a0b88755
atomic.cpp: improve hashtable access
...
Simplified, tried to use all ptr bits.
Try to prevent self-collisions (bug).
Hashtable size doubled (to 8 MiB).
2020-12-01 10:19:31 +03:00
Nekotekina
bd90e3e37f
atomic.cpp: shrink and simplify main hashtable
...
Reduce collision detection to 1 or 0 for now.
I think it should be offloaded to notifiers.
2020-11-26 20:11:26 +03:00
Nekotekina
8b6d615aa6
atomic.hpp: remove/deprecate incomplete notify features
...
Notification with "phantom value" could be useful in theory.
As an alternative way of sending signal from the notifier.
But current implementation is just useless.
Also fixed slow_mutex (not used anywhere).
2020-11-26 06:31:36 +03:00
Nekotekina
600c0ea5ba
atomic.cpp: fix windows/std path
...
Always check condition after rearming internal semaphore.
2020-11-25 10:41:17 +03:00
Nekotekina
43952e18e2
Implement prefetch_write() and prefetch_exec() wrappers
...
Do some refactoring to prefetch_read() in util/asm.hpp as well.
Make all these function constexpr because they are no-ops.
2020-11-24 12:31:11 +03:00
Nekotekina
14fa96fdef
atomic.cpp: don't use empty notify callback
...
Instead, allow setting it to null and check for null.
Similar could be done for wait cb, but it's set for all named threads.
2020-11-24 01:09:04 +03:00
Nekotekina
d70ab11477
atomic.cpp: remove redundant constexpr
...
Static initialization is all-zeros anyway.
But constexpr was killing my intellisense.
And probably also affected compile time.
Also make some internal structures hidden ("static").
2020-11-22 12:07:49 +03:00
Nekotekina
60cff6f3d4
Attempt to fix #9271
2020-11-21 07:09:48 +03:00
Nekotekina
5f010c2f4c
atomic.cpp: optimize "cond" allocation with TLS
...
Billions of events can reduce to thousands, saving CPU time a few s.
Only 4 slots are available (arbitrarily), and only 1 is usually used.
Other slots are used only for waiting on multiple atomics.
2020-11-20 19:13:22 +03:00
Nekotekina
6c18e00977
atomic.cpp: use deterministic "cond" allocation
...
Takes ~80 ns instead of ~40 ns, same about deallocation.
Loops don't exist here, only 4-level semaphore tree.
Worst case only happens with concurrence, not from looping.
This optimization is not really necessary at current state of RPCS3.
This is more like to test C++ compilers and MSVC u128 implementation.
2020-11-20 15:04:17 +03:00
Nekotekina
72ac1aa537
atomic.cpp: use some actual hash for hashtable (std::minstd_rand)
...
Should only take about 5ns for initialization.
Should also make collisions almost impossible.
I also suspect I fixed some bad bug in process...
2020-11-20 15:04:17 +03:00
Nekotekina
3dd48a24ce
atomic.cpp: remove tiny redundancy
2020-11-17 15:22:04 +03:00
Nekotekina
333fe9a561
atomic.cpp: fix notification on windows
2020-11-15 04:40:44 +03:00
Nekotekina
badb3dc2dd
atomic.cpp/threads: remove old wait callback
...
Add new wait callback which simply collects statistics.
Shift workarounds towards actual problem detection.
2020-11-14 18:16:27 +03:00
Nekotekina
3ac819ee70
atomic.cpp: fix waiting on multiple variables
...
Silly bugs are so silly...
2020-11-14 17:25:02 +03:00
Nekotekina
00f877e812
atomic.cpp: fix rare overload scenario
...
If slot allocation fails, increase reference counter in every slot.
This prevents search routine from "losing" some slots.
2020-11-14 13:56:23 +03:00
Nekotekina
e48f160a29
atomic.cpp: continuation of fixing all-ones masks
...
Also added missing const noexcept.
2020-11-13 12:40:02 +03:00
Nekotekina
cfda4d0ade
atomic.cpp: optimize raw_notify() for unspecified pointer mode
...
Remove unnecessary optimization from cond_alloc().
Optimistic case was absolutely dominating anyway.
Although the whole function is a dirty hack.
Now scanning through all threads is faster.
2020-11-13 07:05:33 +03:00
Nekotekina
3888b0429c
atomic.cpp: print some stats on stop
2020-11-12 05:52:22 +03:00
Nekotekina
70761a4ef0
atomic.cpp: fix notification bug from few commits before
...
Make sure forced notification ignores mask comparison.
Also fix masks provided by smaller types to correct ones.
2020-11-12 05:01:44 +03:00
Nekotekina
e37df998e9
atomic.cpp: minor optimization
...
Silly MSVC emits unnecessary initialize-on-first-use code guards.
Didn't check other compiler. But workaround just works anyway.
2020-11-12 03:41:19 +03:00
Nekotekina
c30cdfb3a7
atomic.cpp: merge more atomic variables for simplicity
...
Compress 16-bit ref counter and two 48+64 bit slot allocators.
This allowed to remove some weird unnecessary logic paths.
Adjust hashtable size to keep it the same.
2020-11-12 03:39:05 +03:00
Nekotekina
350b704cd7
atomic.cpp: combine two atomics (ref ctr and wait ptr)
...
Should fit into cache line perfectly.
Also makes ptr check more robust.
2020-11-12 01:06:12 +03:00
Nekotekina
ad013d59f4
atomic.cpp: don't load actual memory on empty mask
...
It's not an optimization. It just allows some tricks safely.
2020-11-11 23:59:09 +03:00
Nekotekina
d391133524
atomic.cpp: minor notification improvement
...
Check mask before locking the reference.
Can help to not bother non-eligible waiters.
2020-11-11 23:51:41 +03:00
Nekotekina
4b823469f7
atomic.cpp: restore notify_all() behaviour
...
Restore batch notification with non-blocking attempts.
2020-11-11 22:43:13 +03:00
Nekotekina
7cd1e767be
atomic.cpp: implement some atomic wait operations.
...
Instead of plain waiting while equal to some value,
it can be something like less, or greater, or even bitcount.
But it's a draft and untested. Hopefully doesn't break anything.
2020-11-11 20:46:37 +03:00
Nekotekina
c047ab6147
atomic.cpp: waitable atomics got brutally rewritten
...
Hashtable increased and flatten, tree-alike extensions removed.
Some things simplified, so it can actually decrease perf a bit.
But most platforms shouldn't be affected.
Removed limit of 56 waiters per pointer.
Real limit now is about 65535.
2020-11-11 05:35:27 +03:00
Nekotekina
fd7ea82965
atomic.cpp: fix 3-arg notify functions
...
Third arg forces comparison with itself, and possibly forced wakeup.
2020-11-07 19:18:14 +03:00
Nekotekina
0bd989b118
atomic.cpp: allow waiting on multiple variables simultaneously
...
Arbitrary maximum set to 8, but really we need 2, maybe 3.
Added atomic_wait::list object for multi-waiting.
Added atomic_wait::get_unique_tsc just in case.
2020-11-07 18:18:02 +03:00
Nekotekina
bd0b292b39
atomic.cpp: adjust assumed pointer mask
...
Increase waiter mask (allow 65535 waiters).
Assume 47-bit pointers instead of 48-bit.
2020-11-07 15:08:46 +03:00
Nekotekina
31f0a0c064
atomic.cpp: add minor safety check
2020-11-07 15:08:46 +03:00
Nekotekina
3e445eff8e
atomic.cpp: maintenance of wait() function
...
Move hashtable allocation into another function.
2020-11-07 15:08:46 +03:00