Nekotekina
3dd48a24ce
atomic.cpp: remove tiny redundancy
2020-11-17 15:22:04 +03:00
Nekotekina
984e97cbdc
Potential fix for Linux compilation
2020-11-17 15:22:04 +03:00
Nekotekina
333fe9a561
atomic.cpp: fix notification on windows
2020-11-15 04:40:44 +03:00
Nekotekina
badb3dc2dd
atomic.cpp/threads: remove old wait callback
...
Add new wait callback which simply collects statistics.
Shift workarounds towards actual problem detection.
2020-11-14 18:16:27 +03:00
Nekotekina
3ac819ee70
atomic.cpp: fix waiting on multiple variables
...
Silly bugs are so silly...
2020-11-14 17:25:02 +03:00
Nekotekina
00f877e812
atomic.cpp: fix rare overload scenario
...
If slot allocation fails, increase reference counter in every slot.
This prevents search routine from "losing" some slots.
2020-11-14 13:56:23 +03:00
Nekotekina
ab365fe494
Fixed thread pool a bit
...
Use 128-bit allocator instead of queue.
When pool is full (128), threads just terminate as before.
2020-11-13 13:32:44 +03:00
Nekotekina
e48f160a29
atomic.cpp: continuation of fixing all-ones masks
...
Also added missing const noexcept.
2020-11-13 12:40:02 +03:00
Nekotekina
cfda4d0ade
atomic.cpp: optimize raw_notify() for unspecified pointer mode
...
Remove unnecessary optimization from cond_alloc().
Optimistic case was absolutely dominating anyway.
Although the whole function is a dirty hack.
Now scanning through all threads is faster.
2020-11-13 07:05:33 +03:00
Nekotekina
3888b0429c
atomic.cpp: print some stats on stop
2020-11-12 05:52:22 +03:00
Nekotekina
70761a4ef0
atomic.cpp: fix notification bug from few commits before
...
Make sure forced notification ignores mask comparison.
Also fix masks provided by smaller types to correct ones.
2020-11-12 05:01:44 +03:00
Nekotekina
e37df998e9
atomic.cpp: minor optimization
...
Silly MSVC emits unnecessary initialize-on-first-use code guards.
Didn't check other compiler. But workaround just works anyway.
2020-11-12 03:41:19 +03:00
Nekotekina
c30cdfb3a7
atomic.cpp: merge more atomic variables for simplicity
...
Compress 16-bit ref counter and two 48+64 bit slot allocators.
This allowed to remove some weird unnecessary logic paths.
Adjust hashtable size to keep it the same.
2020-11-12 03:39:05 +03:00
Nekotekina
350b704cd7
atomic.cpp: combine two atomics (ref ctr and wait ptr)
...
Should fit into cache line perfectly.
Also makes ptr check more robust.
2020-11-12 01:06:12 +03:00
Nekotekina
ad013d59f4
atomic.cpp: don't load actual memory on empty mask
...
It's not an optimization. It just allows some tricks safely.
2020-11-11 23:59:09 +03:00
Nekotekina
d391133524
atomic.cpp: minor notification improvement
...
Check mask before locking the reference.
Can help to not bother non-eligible waiters.
2020-11-11 23:51:41 +03:00
Nekotekina
4b823469f7
atomic.cpp: restore notify_all() behaviour
...
Restore batch notification with non-blocking attempts.
2020-11-11 22:43:13 +03:00
Nekotekina
7cd1e767be
atomic.cpp: implement some atomic wait operations.
...
Instead of plain waiting while equal to some value,
it can be something like less, or greater, or even bitcount.
But it's a draft and untested. Hopefully doesn't break anything.
2020-11-11 20:46:37 +03:00
Nekotekina
c047ab6147
atomic.cpp: waitable atomics got brutally rewritten
...
Hashtable increased and flatten, tree-alike extensions removed.
Some things simplified, so it can actually decrease perf a bit.
But most platforms shouldn't be affected.
Removed limit of 56 waiters per pointer.
Real limit now is about 65535.
2020-11-11 05:35:27 +03:00
Nekotekina
d1ee7c651f
Make memory locking optional (mlock, VirtualLock).
...
Fix desired locking operation (to fix "sudo" memory).
It was discovered that some systems have outdated configuration.
With too tight limit, it's almost impossible to lock anything in memory.
2020-11-10 06:05:37 +03:00
Nekotekina
dcbe8ef5f4
Import VirtualAlloc2 if available (Win10+)
...
Import MapViewOfFile3 as well.
Unused, this crap is tricky.
2020-11-10 03:04:38 +03:00
Nekotekina
bd5253047b
Improve DYNAMIC_IMPORT
...
Don't call get_proc_address every time if if failed.
Also rename Utilities/dynamic_library.h to util/dyn_lib.hpp
2020-11-10 01:44:31 +03:00
Nekotekina
e98eb4659e
Implement utils::memory_lock
2020-11-09 22:57:36 +03:00
Nekotekina
d1988d384c
vm: remove unnecessary memset() on stop
2020-11-09 22:57:36 +03:00
Nekotekina
b05d12df0e
Hotfix for shm->map_self()
...
Make sure mmap returns 64K-aligned results, as on Windows.
2020-11-08 22:15:59 +03:00
Nekotekina
1c99a2e7fb
vm: add map_self() method to utils::shm
...
Add complementary unmap_self() method.
Move VirtualMemory to util/vm.hpp
Minor associated include cleanup.
Move asm.h to util/asm.hpp
2020-11-08 16:43:15 +03:00
Nekotekina
fd7ea82965
atomic.cpp: fix 3-arg notify functions
...
Third arg forces comparison with itself, and possibly forced wakeup.
2020-11-07 19:18:14 +03:00
Nekotekina
0bd989b118
atomic.cpp: allow waiting on multiple variables simultaneously
...
Arbitrary maximum set to 8, but really we need 2, maybe 3.
Added atomic_wait::list object for multi-waiting.
Added atomic_wait::get_unique_tsc just in case.
2020-11-07 18:18:02 +03:00
Nekotekina
bd0b292b39
atomic.cpp: adjust assumed pointer mask
...
Increase waiter mask (allow 65535 waiters).
Assume 47-bit pointers instead of 48-bit.
2020-11-07 15:08:46 +03:00
Nekotekina
31f0a0c064
atomic.cpp: add minor safety check
2020-11-07 15:08:46 +03:00
Nekotekina
3e445eff8e
atomic.cpp: maintenance of wait() function
...
Move hashtable allocation into another function.
2020-11-07 15:08:46 +03:00
Nekotekina
0a5742587a
atomic.cpp: upgrade raw_notify()
...
Now it searches all semaphores if data arg is nullptr.
Also it tries to wake up all threads if thread_id is 0.
2020-11-06 17:15:40 +03:00
Nekotekina
ad4df2d946
atomic.cpp: refactor native signaling functions
...
Add native_alert() and try_native_alert()
2020-11-06 17:15:40 +03:00
Nekotekina
bfe9580551
atomic.cpp: fix cond_handle data structures
...
Fix a critical bug with possible id out of range.
2020-11-06 16:56:59 +03:00
Nekotekina
1e45437498
Remove unused atomic with lock bit
2020-11-06 16:48:43 +03:00
Nekotekina
557f0c5a8a
atomic.cpp: remove load() from notify functions
...
Only compare masks for overlap for second overload (with mask provided).
Explicit "new value" can be provided in new 3-arg overloads.
Also rename atomic_storage_futex -> atomic_wait_engine.
2020-11-06 04:44:31 +03:00
Nekotekina
9fb8d449fe
atomic.cpp: more improvements
...
Add ref counters for internal semaphores for concurrent notifying.
Lack of them could result in loss of notification.
2020-11-06 00:14:53 +03:00
Nekotekina
1bb7c74c93
atomic.cpp: various cleanups and fixes
...
Add pointer comparison to notifiers (to prevent spurious wakeups).
Fix a bug with a possible double notification in raw_notify().
Fix a bug with incorrect allocatin bit slots for cond_handle.
Add a semaphore counter to track max allowed number of threads.
Use #define for some constants to STRINGIZE them in errors.
Add some error messages when certain limits are reached.
Fix a bug with a wrong check simply throwing std::abort.
Use "special" notify_all patch with batch processing for every arch.
Fix Win7 bug who no one probably noticed.
2020-11-06 00:10:16 +03:00
Nekotekina
5248240e10
atomic.cpp: improvements.
...
Reduced static memory amount for waitable atomics.
Allow notifier to skip notifications if wait/notify masks don't overlap.
Improve raw_notify to wake up the thread by its id, add thread_id arg.
Add optional mask argument to notify_one() and notify_all().
2020-11-05 05:51:43 +03:00
Nekotekina
95dbcf2fd7
Logs: add callback for cpu_thread to set cpu_flag::wait
...
May improve waiting time in critical suspend_all ops.
2020-10-29 18:57:57 +03:00
Eladash
e791678dfc
atomic.cpp: fix memory ordering loads and release stores
2020-10-28 20:45:13 +03:00
Nekotekina
c491b73f3a
SPU: improve accurate DMA
...
Remove vm::reservation_lock from it.
Use lock bits to prevent memory clobbering in GETLLAR.
Improve u128 for MSVC since it's used for bitlocking.
Improve 128 bit atomics for the same reason.
Improve vm::reservation_op and friends.
2020-10-28 03:47:41 +03:00
Nekotekina
c50233cc92
atomics.cpp: add support for waiting on 128-bit atomics
...
Complementarily.
Also refactored to make waiting mask non-template arg.
2020-10-28 03:47:41 +03:00
Nekotekina
6806e3d5c7
atomic.cpp: implement notify callback
...
Notification can be very heavy, especially if we need to wake many threads.
Callback is set for cpu_thread in order to set wait flag accordingly.
2020-10-27 17:56:19 +03:00
Nekotekina
d344701fd5
atomic.cpp: add sparse atomics
...
Set alignment as second template argument (typically cache line).
2020-10-26 01:15:31 +03:00
Nekotekina
af6fcb19a8
atomic.cpp: add std primitive fallback for other platforms
...
Other platforms = not Windows or Linux.
2020-10-25 16:45:22 +03:00
Nekotekina
7db77a5580
atomic.cpp: use new thread alerting API (Win8+)
...
Win7 will remain using old API (keyed events).
2020-10-24 19:06:02 +03:00
Nekotekina
c479d431a4
atomic.cpp: experimental optimization (Win7+)
...
Try non-blocking wake up function first.
2020-10-24 14:16:32 +03:00
Nekotekina
8628fc441d
atomic.cpp: rewrite internals (again)
...
Use individual semaphore for each thread.
Unfortunately, limit max thread waiting for single address (60).
If limit is reached, use busy waiting.
2020-10-24 14:16:32 +03:00
Nekotekina
d48d67767a
atomic.cpp: integrate callback check in ptr_cmp function
2020-10-24 14:16:32 +03:00
