Here mostly are my own views after what I understood about Virtual Threads from different online sources. So please feel free to share your views to let the right information flowing.
Virtual Thread - I think, it is saying that it is not a thread in reality. It is more like a container for your Runnable/Callable tasks with 'Continuation' support. If I understand correct then such 'Continuation' support is already in Python.
Virtual Thread could have been that Runnable task to run on Platform thread but that way
every Runnable task would have been a Virtual task, which may not be useful in every case & may break the backward compatibility.
So I think, this 'Continuation' support was moved to 'Thread' class.
Using Thread class, user can decide whether to use Virtual threads or not.
Why is it important?
As per my understanding-
Virtual Threads using the Platform threads as carrier threads to get the job done by the CPU.
Virtual Threads are used by JVM to manage the tasks in its own context.
Like, JVM application may be used by millions of users requiring millions of threads, one thread for each user.
But, will every user be using the CPU resources at the same time for the task done.
No, may be 10% such users will be there at a particular moment, requiring OS threads.
Here, Virtual Threads come to rescue, as we have limited number of Platform threads.
Virtual Threads will be working on behalf of the users. And when needed, virtual thread of the user will mount itself on the carrier thread i.e. Platform thread to use the platform resources.
Till date, one Platform thread used to be occupied by each Runnable task till its completion & during the sleep() or wait() calls, such platforms threads used to get blocked without doing anything else, wasting their time.
Now, Virtual Thread will take such Runnable task & unmount itself when task is blocked without blocking Platform thread. So Platform thread gets free to take other task.
So initial thought comes - Its good to make every thread, a Vitual Thread. Right?
Platform threads scheduling is done by OS, to avoid starvation for any thread.
But now the tasks are given to Virtual Threads & one Vitual Thread mounts itself to one Platform thread till it is not unmounted. Virtual Thread is unmounted during blocking calls, else JVM doesn't have any reason to unmount a working Vitual Thread in the middle.
So, issue I see -
Number of logical processors : 8
Number of tasks submitted to Virtual Threads : 10
Number of Platform Threads : ForkJoin Pool is used with number of threads as n-1
So pool will be having 7 platform threads & Virtual Threads will ForkJoin pool threads as
career threads i.e. 7
Now 7 virtual threads will be mounted to 7 Platform threads for processing.
No virtual thread will be unmounted till some sleep or wait or blocking call occurs.
Imagine all these 7 threads get rogue to run infinitely or something happens that these threads get into inter-dependence loop such that no Virtual thread is getting unmounted.
So will you get any error?
Probably not, you may not see any spike in CPU threads or any error that no thread can be
created now and this makes the situation very tricky to understand that what is going wrong
due to which application is not responding for new requests & neither is rejecting new
requests. This is what one needs to be aware about.
Then when to prefer to use Virtual Threads?
Use virtual threads when tasks have blocking code like tasks requiring user interactions.
Use Platform threads when tasks are CPU intensive, using Virtual threads for such cases is
not going to help, it may even degrade the performance.
Virtual Thread could have been that Runnable task to run on Platform thread but that way
every Runnable task would have been a Virtual task, which may not be useful in every case & may break the backward compatibility.
So I think, this 'Continuation' support was moved to 'Thread' class.
Using Thread class, user can decide whether to use Virtual threads or not.
Why is it important?
As per my understanding-
Virtual Threads using the Platform threads as carrier threads to get the job done by the CPU.
Virtual Threads are used by JVM to manage the tasks in its own context.
Like, JVM application may be used by millions of users requiring millions of threads, one thread for each user.
But, will every user be using the CPU resources at the same time for the task done.
No, may be 10% such users will be there at a particular moment, requiring OS threads.
Here, Virtual Threads come to rescue, as we have limited number of Platform threads.
Virtual Threads will be working on behalf of the users. And when needed, virtual thread of the user will mount itself on the carrier thread i.e. Platform thread to use the platform resources.
Till date, one Platform thread used to be occupied by each Runnable task till its completion & during the sleep() or wait() calls, such platforms threads used to get blocked without doing anything else, wasting their time.
Now, Virtual Thread will take such Runnable task & unmount itself when task is blocked without blocking Platform thread. So Platform thread gets free to take other task.
So initial thought comes - Its good to make every thread, a Vitual Thread. Right?
Platform threads scheduling is done by OS, to avoid starvation for any thread.
But now the tasks are given to Virtual Threads & one Vitual Thread mounts itself to one Platform thread till it is not unmounted. Virtual Thread is unmounted during blocking calls, else JVM doesn't have any reason to unmount a working Vitual Thread in the middle.
So, issue I see -
Number of logical processors : 8
Number of tasks submitted to Virtual Threads : 10
Number of Platform Threads : ForkJoin Pool is used with number of threads as n-1
So pool will be having 7 platform threads & Virtual Threads will ForkJoin pool threads as
career threads i.e. 7
Now 7 virtual threads will be mounted to 7 Platform threads for processing.
No virtual thread will be unmounted till some sleep or wait or blocking call occurs.
Imagine all these 7 threads get rogue to run infinitely or something happens that these threads get into inter-dependence loop such that no Virtual thread is getting unmounted.
So will you get any error?
Probably not, you may not see any spike in CPU threads or any error that no thread can be
created now and this makes the situation very tricky to understand that what is going wrong
due to which application is not responding for new requests & neither is rejecting new
requests. This is what one needs to be aware about.
Then when to prefer to use Virtual Threads?
Use virtual threads when tasks have blocking code like tasks requiring user interactions.
Use Platform threads when tasks are CPU intensive, using Virtual threads for such cases is
not going to help, it may even degrade the performance.
Virtual Threads provide the task queue where a Vitual Thread keeps itself as unavailable to Platform Threads during blocking call, once blocking call is done then Virtual Thread is available to be picked by Platform threads.
Platform threads are managed by ForkJoin pool which is working in 'Work stealing' style. So which platform thread will pick which Virtual Thread is not decided.
But I think, concern is that this ForkJoin pool will be having as many threads as logical processors in the System. Why?
Again I think, as blocking factor is moved to Virtual threads, so Platform threads can be as many as Logical processors, as none will be blocking for user input.
But discussion on this later, first let's see the simple examples to work with Virtual Threads.
Below code is for simple Producer Consumer problem-
Platform threads are managed by ForkJoin pool which is working in 'Work stealing' style. So which platform thread will pick which Virtual Thread is not decided.
But I think, concern is that this ForkJoin pool will be having as many threads as logical processors in the System. Why?
Again I think, as blocking factor is moved to Virtual threads, so Platform threads can be as many as Logical processors, as none will be blocking for user input.
But discussion on this later, first let's see the simple examples to work with Virtual Threads.
Below code is for simple Producer Consumer problem-
Producer Consumer
Output sample using Platform threads
Few points to note -
1) Threads created in above traditional way are non-daemon threads, so JVM will not shut till
all these threads are executed.
2) Check Thread IDs shown as Thread[#22, if you count then 20 such Platform threads are
created, as we have created them in the loop in the code.
3) Producer thread is also created, so total 21 Platform threads are created.
4) Here, time consumed to create the Platform threads is insignificant, as less threads are
created.
5) Most important point to note - I have given name to each thread, like for consumers one
can see names like 'consumer0', 'consumer1',....'consumer19'
So observe that each thread id# is mapped to the same consumer name till the end.
Means?
It shows, each consumer task is holding one Platform thread till last, even when consumer
is sleeping for 520 ms in between, so during that time Platform thread is idle.
So this one-one mapping hurts when there are millions of Consumers tied to individual
Platform threads, but 90% consumers are sleeping or blocked, making available 90% CPU
power useless.
Bonus : What is 5,main then?
It is showing that 'main' is the parent thread with priority 5
1) Threads created in above traditional way are non-daemon threads, so JVM will not shut till
all these threads are executed.
2) Check Thread IDs shown as Thread[#22, if you count then 20 such Platform threads are
created, as we have created them in the loop in the code.
3) Producer thread is also created, so total 21 Platform threads are created.
4) Here, time consumed to create the Platform threads is insignificant, as less threads are
created.
5) Most important point to note - I have given name to each thread, like for consumers one
can see names like 'consumer0', 'consumer1',....'consumer19'
So observe that each thread id# is mapped to the same consumer name till the end.
Means?
It shows, each consumer task is holding one Platform thread till last, even when consumer
is sleeping for 520 ms in between, so during that time Platform thread is idle.
So this one-one mapping hurts when there are millions of Consumers tied to individual
Platform threads, but 90% consumers are sleeping or blocked, making available 90% CPU
power useless.
Bonus : What is 5,main then?
It is showing that 'main' is the parent thread with priority 5
Virtual Threads Trial
Now uncomment the Virtual Threads block & comment the Platform threads block in above
code and execute that.
Below is the kind of Output one can see-
Now uncomment the Virtual Threads block & comment the Platform threads block in above
code and execute that.
Below is the kind of Output one can see-
Output sample using Virtual Threads
Few points to note -
1) Virtual Threads created above are daemon threads, so JVM will not wait for Virtual threads
to complete. And that's why had to use while(prod.isAlive()); to keep the main thread active.
2) Check Thread IDs shown as VirtualThread[#24, if you count then 20 such Virtual
threads are created, as we have created them in the loop in the code.
3) Producer thread is also created, so total 21 Virtual threads are created.
Same as Platform Threads earlier, right? No, these are Virtual threads like creating Java
Objects with 'Continuation' support, so need very less time as compared to Platform
threads.
4) What is 'Continuation' support? When virtual threads are blocked for User input or some
other reason & once JVM sees this, then Virtual thread is unloaded from the Platform
thread. But some context is needed to continue once unblocked. So that state & context is
saved along with Virtual thread in the Heap. And once virtual thread is ready to be picked
then any available Platform thread will pick it.
5) Observer ForkJoinPool-1-worker-1 ...it is basically a ForkJoin pool of Platform threads
which works on 'Work stealing' concept. So here ForkJoinPool-1, is the pool name &
worker-1 is the Platform thread from that pool.
6) Most important point to note - We have our Virtual threads each having individual
Consumer like 'consumer0', 'consumer1'...'consumer19', but our Consumers are not
blocking any ForkJoin worker i.e. Platform thread.
Observe that same consumer virtual thread is being served by different Worker. Why?
As one consumer is blocked then it is unmounted from that worker & moved to heap along
with its context. Once that Consumer is unblocked then any worker from the pool picks that
Consumer Virtual thread to execute, and who knows either it can be the same worker or
some other worker from the pool. But this way, no worker is blocked & will be busy till there
is some task for it, so its time is not wasted due to blocking calls.
1) Virtual Threads created above are daemon threads, so JVM will not wait for Virtual threads
to complete. And that's why had to use while(prod.isAlive()); to keep the main thread active.
2) Check Thread IDs shown as VirtualThread[#24, if you count then 20 such Virtual
threads are created, as we have created them in the loop in the code.
3) Producer thread is also created, so total 21 Virtual threads are created.
Same as Platform Threads earlier, right? No, these are Virtual threads like creating Java
Objects with 'Continuation' support, so need very less time as compared to Platform
threads.
4) What is 'Continuation' support? When virtual threads are blocked for User input or some
other reason & once JVM sees this, then Virtual thread is unloaded from the Platform
thread. But some context is needed to continue once unblocked. So that state & context is
saved along with Virtual thread in the Heap. And once virtual thread is ready to be picked
then any available Platform thread will pick it.
5) Observer ForkJoinPool-1-worker-1 ...it is basically a ForkJoin pool of Platform threads
which works on 'Work stealing' concept. So here ForkJoinPool-1, is the pool name &
worker-1 is the Platform thread from that pool.
6) Most important point to note - We have our Virtual threads each having individual
Consumer like 'consumer0', 'consumer1'...'consumer19', but our Consumers are not
blocking any ForkJoin worker i.e. Platform thread.
Observe that same consumer virtual thread is being served by different Worker. Why?
As one consumer is blocked then it is unmounted from that worker & moved to heap along
with its context. Once that Consumer is unblocked then any worker from the pool picks that
Consumer Virtual thread to execute, and who knows either it can be the same worker or
some other worker from the pool. But this way, no worker is blocked & will be busy till there
is some task for it, so its time is not wasted due to blocking calls.
So one major question pops-up...Does it mean, Virtual threads are similar to Async
Programming? Now will Asnc programming become less relevant?
Few other similar questions...
Please share your thoughts about the above questions & the content I shared.
I will be adding more examples & practises later.
Programming? Now will Asnc programming become less relevant?
Few other similar questions...
Please share your thoughts about the above questions & the content I shared.
I will be adding more examples & practises later.
