Java network concurrency

Objectives

• Define what concurrency is.
• Define the different ways to handle multiple clients at the same time:
  • Multiprocessing.
  • Multithreading.
  • Asynchronous programming.
• Implement and manage concurrency in Java network applications.

Disclaimer

• This is not a course on concurrency.
• Many many things are not covered.
• Focus on concurrency for network applications.
• Other ways and mechanisms to handle concurrency will be covered in other teaching units.

Explore the code examples

Individually, or in pair/group, take 15 minutes to explore and discuss the code examples.

Answer the questions available in the course material:

• How do the code examples work?
• What are the main takeaways of the code examples?
• What are the main differences between the code examples?

If needed, use the theoretical content to help you.

Concurrency: an introduction

• The ability to manage multiple tasks running at the same time.
• Differs from parallelism, the ability to execute multiple tasks simultaneously.
• Before we dive into concurrency, let's take a step back and remember what a processor is.

What is a processor?

• Piece of hardware that executes millions of instructions every second.
• Also called CPU (Central Processing Unit).
• Processors can have one or multiple cores (called single-core or multi-core processors).

What is a core?

• A physical unit inside a processor that executes instructions.
• Each core can only process one instruction at a time.
• When multiple processors and/or multiple cores are present, they can execute multiple instructions at the same time between them.

What is a process?

• A process is an instance of a program that is being executed.
• Processes are isolated from each other and have their own memory space.
• Expensive to create and manage.
• Processes communicate between each other using Inter-Process Communication (IPC) mechanisms.

What is a thread? (1/3)

• By default, a process is executed by a single processor/core, executing one instruction at a time.
• As processors/cores are super fast, it appears processes/threads are executed simultaneously.
• Under the hood, the processor/core switches between processes/threads very quickly.

What is a thread? (2/3)

• A thread is a sequence of instructions that can be executed by a core.
• A core can manage multiple threads by switching between them.
• The main thread is the thread that starts when the application starts.
• More threads can be created to handle different tasks concurrently.

What is a thread? (3/3)

• You have already used threads with picocli (each command is ran on a separate thread).
• Using threads, a server can manage multiple clients by switching between them: each client is handled by a separate thread.
• Threads are more lightweight than processes.

What problems can concurrency cause?

• What happens when a client connects to the server?
• How to isolate the data sent by one client from the others?
• What happens when multiple clients want to access the same resource (variables) at the same time?

What happens when multiple threads access the same resource?

• The processor can switch between threads at any time.
• The order in which the threads are executed is not guaranteed.
• Threads can access the same resource at the same time.

Handling one client at a time

1. Create a socket to listen to incoming connections.
2. Create a new socket for the client.
3. Handle the connection.

Analogy: a restaurant with one table.

Simple but quite useless...

Handling multiple clients at the same time

Handling multiple clients at the same time is called concurrency.

Concurrency can be achieved with:

• Multiprocessing.
• Multithreading.
• Asynchronous programming.

Multiprocessing

• Create an entirely new process for each client.
• Heavy and slow.
• Not recommended.
• Analogy: a new restaurant for each customer, including the kitchen, waiters, etc.

Multithreading

• Create a new thread for each client.
• More lightweight and faster than multiprocessing.
• Recommended with a thread pool to limit resource usage.
• Not enough or too many threads can slow down the system.
• Analogy: a new or limited number of tables for each customer.

Asynchronous programming

• Handle multiple clients with a single thread.
• Very performant!
• Analogy: a food truck - you get a ticket, wait for the food to be prepared, get your food and leave.
• Out of scope but interesting to know! Node.js is a good example.

Handling multiple threads in Java

• Java provides the ExecutorService interface to manage threads.
• It uses different Executors implementations.
• Each implementation has its own pros and cons.
• You have used threads with picocli (each command is a thread).

@Override
public Integer call() {
  try (ExecutorService executorService = Executors.newFixedThreadPool(2); ) {
    executorService.submit(this::emittersWorker);
    executorService.submit(this::operatorsWorker);
  } catch (Exception e) {
    System.out.println("[Receiver] Exception: " + e);

    return 1;
  }

  return 0;
}

public Integer emittersWorker() {
  // Manage emitters
}


public Integer operatorsWorker() {
  // Manage operators
}

Thread safe variable types and data structures in Java

• A thread-safe variable type and/or data structure is a variable type/data structure that can be accessed by multiple threads at the same time.
• Java provides several thread-safe variable types and data structures.

• Variables types:
  • AtomicBoolean instead of boolean/Boolean.
  • AtomicInteger instead of int/Integer.
  • AtomicLong instead of long/Long.
• Data structures:
  • ConcurrentHashMap instead of HashMap.
  • ConcurrentLinkedQueue instead of LinkedList.
  • CopyOnWriteArrayList instead of ArrayList.
  • CopyOnWriteArraySet instead of HashSet.

• Multiple concurrent variable types and/or data structures can be updated together atomically.
• You must use locks (the equivalent to mutexes and semaphores in C/C++) to avoid inconsistencies.

Locks will not be covered (and not expected to be used) in this teaching unit but you can find additional information in the course material.

What will you do?

• Run full client/server examples and understand how concurrent clients are handled.
• Explore and answer questions about some concurrent strategies.
• Update previous game/application to handle multiple clients (optional).

Now it's your turn!

• Read the course material.
• Do the practical content.
• Ask questions if you have any.

Find the course on GitHub.

Do not hesitate to help each other! There's no need to rush!

Finished? Was it easy? Was it hard?

Can you let us know what was easy and what was difficult for you during this course?

This will help us to improve the course and adapt the content to your needs. If we notice some difficulties, we will come back to you to help you.

GitHub Discussions

You can use reactions to express your opinion on a comment!

Sources

• Main illustration by Brent Olson on Unsplash
• Illustration by Aline de Nadai on Unsplash
• Illustration by Bernard Hermant on Unsplash
• Illustration by alexey turenkov on Unsplash
• Illustration by Andrey Matveev on Unsplash
• Illustration by BoliviaInteligente on Unsplash
• Illustration by BoliviaInteligente on Unsplash
• Illustration by BoliviaInteligente on Unsplash

• Illustration by That's Her Business on Unsplash
• Illustration by Jens Herrndorff on Unsplash
• Illustration by Elliott Stallion on Unsplash
• Illustration by K8 on Unsplash
• Illustration by bckfwd on Unsplash
• Illustration by James Elchico on Unsplash
• Illustration by micheile henderson on Unsplash
• Illustration by Afrah on Unsplash
• Illustration by Pop & Zebra on Unsplash
• Illustration by Kateryna Miller on Unsplash