SIXT Tech

Kotlin Multiplatform (KMP): One Codebase, Many Platforms

Ninad Bhase — Thu, 12 Feb 2026 00:00:00 GMT

Introduction

Kotlin Multiplatform (KMP) is JetBrains’ solution for teams who want to share logic across platforms — built for teams who want to share logic without giving up native UI.
Unlike Flutter or React Native, KMP doesn’t replace your UI layer — it complements it. You still write SwiftUI, Jetpack Compose, or React components natively, ensuring platform-native performance and APIs. It’s about sharing logic, not necessarily UI — although UI sharing is possible with frameworks like Compose Multiplatform.

Note: Compose Multiplatform extends KMP by also sharing UI code (using Jetpack Compose), but that’s optional — KMP works just fine without it.

KMP is Gradle-driven, compiles Kotlin to each platform’s native/bytecode target, and exposes Kotlin artifacts you can consume directly from platform code.

Why Use KMP

Less duplicate logic: single implementation for algorithms, validation, networking.
Consistency: same models, serializers, and business rules across platforms.
Maintainability: fixes and features applied once propagate everywhere.
Flexibility: keep platform-specific UI idiomatic (Compose on Android, SwiftUI on iOS).
Incremental adoption: migrate small pieces at a time — no big-bang rewrite.

️ Core Concepts

Targets

KMP compiles shared code to platform-specific artifacts. Common targets include:

android() or jvm() — compiles to Java bytecode (Android builds into an AAR).
iosX64, iosArm64, iosSimulatorArm64 — iOS frameworks (Objective-C/Swift interop).
js() — compiles to JavaScript (Node/browser).
wasmJs() — compiles to WebAssembly for high-performance web applications.
macosX64, linuxX64, mingwX64 — native/desktop targets.

Source Sets

Source sets organize shared and platform-specific code.

commonMain / commonTest: shared code and tests.
androidMain, iosMain, jvmMain: platform implementations.
expect and actual: expect and actual are Kotlin’s mechanism for defining platform-agnostic APIs — more on this below

Example:

// commonMain
expect fun currentTimestampMillis(): Long

// androidMain
actual fun currentTimestampMillis(): Long = System.currentTimeMillis()

// iosMain
actual fun currentTimestampMillis(): Long = NSDate().timeIntervalSince1970.toLong() * 1000L

Expect / Actual Pattern

This pattern allows you to define APIs in shared code (expect) and implement them in platform-specific code (actual).
It’s the foundation of how Kotlin Multiplatform abstracts platform differences.

For example:

// commonMain
expect class Logger() {
    fun log(message: String)
}

// androidMain
actual class Logger {
    actual fun log(message: String) {
        Log.d("KMP", message)
    }
}

// iosMain
actual class Logger {
    actual fun log(message: String) {
        println("KMP: $message")
    }
}

You’ll often use this for:

Logging
File I/O
Network clients (Ktor engines)
Preferences
System services (camera, GPS, etc.)

Interoperability

Kotlin Multiplatform integrates smoothly with each target’s ecosystem.

Android / JVM:
The shared module compiles to a .aar or .jar file, consumed like any other Gradle dependency.
iOS:
KMP outputs a .framework or .xcframework bundle exposing Objective-C headers.
Swift can then call shared Kotlin code directly.
JavaScript:
Generates JS bundles for Node or browser, allowing Kotlin logic to be reused in web apps.

Example — importing shared code in Swift:

import Shared

let userRepo = SharedUserRepository()
userRepo.getUser(id: "42") { user, error in
    if let user = user {
        print(user.name)
    }
}

Build & Tooling

Gradle Setup

plugins {
    kotlin("multiplatform") version "1.9.x"  // Note: Replace .x with the latest stable version.
    id("com.android.library")
}

kotlin {
    android()
    iosX64()
    iosArm64()
    iosSimulatorArm64()

    sourceSets {
        val commonMain by getting {
            dependencies {
                implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.x")
                implementation("io.ktor:ktor-client-core:2.x")
            }
        }
    }
}

CocoaPods Integration

For iOS projects using CocoaPods, KMP supports automatic generation of .podspec files.

kotlin {
    cocoapods {
        summary = "Shared KMP Module"
        homepage = "https://yourcompany.dev"
        framework {
            baseName = "shared"
        }
    }
}

After running pod install, your iOS app can import Shared like any other pod.

For Swift Package Manager: KMP also supports SPM via XCFrameworks — run ./gradlew :shared:assembleXCFramework and add it to Xcode manually or via Package.swift.

Multiplatform Libraries Ecosystem

Several Kotlin libraries already support KMP out of the box, including:

Ktor — Networking
kotlinx.serialization — JSON serialization
SQLDelight — Type-safe database
Koin or Kotlin Inject — Dependency injection
kotlinx.coroutines — Concurrency

These libraries expose a commonMain implementation and internally manage platform-specific code.

Networking & Serialization

Networking in KMP is commonly implemented using Ktor, and data models are serialized using kotlinx.serialization.

Example setup:

// commonMain
expect fun provideHttpClient(): HttpClient

// androidMain
actual fun provideHttpClient(): HttpClient = HttpClient(OkHttp)

// iosMain
actual fun provideHttpClient(): HttpClient = HttpClient(Ios)

Persistence

You can implement persistence in a multiplatform-friendly way using:

SQLDelight — for type-safe local database access
Settings library or expect/actual for key-value storage

Example using expect/actual for preferences:

// commonMain
expect class Preferences() {
    fun putString(key: String, value: String)
    fun getString(key: String): String?
}

// androidMain
actual class Preferences {
    private val prefs = PreferenceManager.getDefaultSharedPreferences(context)

    actual fun putString(key: String, value: String) {
        prefs.edit().putString(key, value).apply()
    }

    actual fun getString(key: String): String? = prefs.getString(key, null)
}

// iosMain
actual class Preferences {
    private val defaults = NSUserDefaults.standardUserDefaults()

    actual fun putString(key: String, value: String) {
        defaults.setObject(value, forKey = key)
    }

    actual fun getString(key: String): String? = defaults.stringForKey(key)
}

Migration Strategy (From Android-only Kotlin)

One of Kotlin Multiplatform’s biggest advantages is that you don’t need to rewrite your entire app.
You can adopt it gradually — starting with isolated modules and expanding over time.

Here’s a proven step-by-step migration plan:

1. Identify Reusable Logic

Start by locating modules that can easily be shared:

Networking layer (API clients, DTOs)
Models and serialization
Business/domain logic
Validation and utilities

Anything not tied to Android APIs (like Context or View) is a good candidate.

2. Create a Shared KMP Module

Create a new Gradle module (e.g., /shared) using the Kotlin Multiplatform plugin:

plugins {
    kotlin("multiplatform")
    id("com.android.library")
}

Configure targets for Android and iOS:

{
android()
iosX64()
iosArm64()
iosSimulatorArm64()
}

Add commonMain, androidMain, and iosMain source sets to start building shared functionality.

3. Move Common Code Incrementally

Move business logic, API definitions, and models from your Android module into commonMain. Replace Android dependencies (e.g., SharedPreferences) with expect/actual abstractions.

// commonMain
expect fun getAppVersion(): String

// androidMain
actual fun getAppVersion(): String =
BuildConfig.VERSION_NAME

4. Integrate Shared Code Back into Android

Once the shared module builds successfully, integrate it into your Android project.

In androidApp/build.gradle.kts:

implementation(project(":shared"))

Now your Android app uses logic from the shared module transparently.

5. Generate the iOS Framework

Next, build the iOS target to generate an .xcframework:

./gradlew :shared:assembleXCFramework

This produces a Shared.xcframework which can be imported into Xcode.

In Swift:

import Shared

let repo = SharedUserRepository()
repo.fetchUser(id: "42") { user in
print(user.name)
}

6. Gradually Expand

Once both Android and iOS consume shared logic, you can migrate more modules:

Caching and persistence
Analytics
Feature toggles
Utility layers

Avoid moving platform-specific code (UI, sensors, Bluetooth, etc.) until you have solid common infrastructure.

Best Practices During Migration

Keep commonMain platform-agnostic — no Android imports.
Write shared unit tests early in commonTest.
Use dependency injection for platform services.
Test build pipelines on macOS early to avoid CI surprises.
Document all expect/actual pairs to prevent confusion later.

Example of a Small First Step

A realistic starting point is to share your networking layer.
Create models and API clients in commonMain.
Implement platform-specific HTTP engines (OkHttp for Android, Darwin for iOS).
Verify integration on Android first, then export to iOS.

This way, you deliver immediate value while reducing risk.

1.Testing Strategy KMP supports shared unit tests in commonTest:

// Testing Shared Code
// commonTest
class UserRepositoryTest {
    @Test
    fun testUserParsing() {
        val user = User("John", 30)
        assertEquals("John", user.name)
    }
}

Run tests for all targets via: ./gradlew allTests

Performance & Binary Size Address a common concern: Does KMP add bloat? Kotlin/Native binaries are optimized with tree-shaking and minification. Shared code typically adds minimal overhead compared to duplicating logic in Swift and Kotlin separately.
Building KMP in CI/CD macOS runners required for iOS builds (GitHub Actions, Bitrise, etc.) Use ./gradlew build for all targets or separate jobs per platform Cache Gradle and CocoaPods dependencies to speed up builds
Limitations Being honest about trade-offs builds trust: Current Limitations: iOS debugging experience is improving but not as mature as Android Some Kotlin/JVM libraries don’t support Kotlin/Native yet Swift interop for advanced generics can be tricky

Conclusion

Kotlin Multiplatform is not just another cross-platform framework — it’s a multi-target architecture built on top of Kotlin’s language power and Gradle’s flexibility.
Instead of forcing a single UI or runtime across devices, KMP gives you the freedom to share only what makes sense — your business logic, models, and data layers — while keeping the native experience intact for every platform.

For Android developers, KMP feels natural. You can reuse your Kotlin skills, libraries, and build tools while extending your reach to iOS, Desktop, or even Web.
It enables consistency in logic, faster feature parity, and fewer bugs across platforms — without sacrificing performance or native design principles.

The key to success with KMP lies in incremental adoption. Start small — share stable, core modules like networking or validation — then expand gradually as your team and tooling mature.

As the ecosystem evolves with Compose Multiplatform, SQLDelight, Ktor, and Kotlin/Native improvements, Kotlin Multiplatform is fast becoming a production-ready solution for modern, scalable, and maintainable mobile architectures.

Build once. Run everywhere. Stay native.

Spring Reactive Programming: Asynchronous, Non-blocking, and Scalable Solutions

Raksha G Shenoy — Fri, 02 Feb 2024 00:00:00 GMT

Before delving into reactive programming, it’s important to know the concept of a stream. A stream represents a sequence of data transmitted from one system to another, typically following a blocking, sequential, and FIFO (first-in-first-out) pattern. The conventional blocking nature of data streaming often hinders a system’s ability to process real-time data concurrently with streaming. Consequently, the concept of constructing a “reactive” systems architecture emerged to facilitate the seamless processing of data during streaming.

What is Reactive Programming and why is a need for Reactive Programming?

Reactive programming is a programming paradigm that helps to implement non-blocking, asynchronous, and event-driven or message-driven data processing. It models data and events as streams that it can observe and react to by processing or transforming the data. The term, “reactive,” refers to programming models that are built around reacting to changes. It is built around the publisher-subscriber pattern (observer pattern). In the reactive style of programming, we make a request for resources and start performing other things. When the data is available, we get the notification along with data in the callback function. The callback function handles the response as per application/user needs.

Most usual programs work in a way that if they’re busy, new tasks have to wait for their turn. This happens in traditional applications that use blocking or synchronous calls. When many threads are occupied, the program blocks new tasks until the ongoing ones are completed.

However, some modern applications need to handle multiple tasks at the same time. Reactive programming on the server side helps with this. It allows web or server applications to work more efficiently by handling multiple tasks simultaneously. This is done asynchronously, meaning the program doesn’t have to wait for one task to finish before starting another. This improves performance, makes the program more scalable, and better equipped to handle lots of users.

Difference between Blocking and Non-blocking request Processing.

Blocking Request Model:

In a typical MVC application, when a request arrives at the server, a servlet thread is generated and assigned to worker threads for tasks such as I/O and database processing. While these worker threads are occupied with their tasks, the servlet threads go into a waiting state, causing the calls to be blocked. This process is known as blocking or synchronous request processing. Since a server has a finite number of request threads, it constrains the server’s ability to handle a maximum number of requests during peak loads. This limitation may impact performance and hinder the server from fully utilizing its capabilities.

Non-blocking Request Model:

In a non-blocking system, every incoming request is accompanied by both an event handler and a callback. The request thread then delegates the incoming request to a thread pool, which efficiently manages a limited number of threads. Subsequently, the thread pool assigns the request to its handler function and becomes available to handle additional incoming requests from the request thread.

Upon completion of the handler function’s processing, one of the threads from the pool retrieves the response and forwards it to the callback function. Consequently, the threads in a non-blocking system never enter a waiting state, thereby enhancing the productivity and performance of the application.

In this non-blocking paradigm, a single request has the potential to be processed by multiple threads, contributing to the scalability of the application’s performance. The non-blocking nature of threads facilitates improved application performance, as the small number of threads leads to reduced memory utilization and minimizes context switching.

Components of Reactive Programming:

Reactive Streams was conceived with the aim of establishing a standard for asynchronous stream processing of data, incorporating non-blocking backpressure. To fulfill this objective, Java 9 introduced the Reactive Streams API.

Publisher - It is responsible for preparing and transferring data to subscribers as individual messages. A Publisher can serve multiple subscribers but it has only one method, subscribe().
```
    public interface Publisher<T> {
        public void subscribe(Subscriber<? super T> s);
    }
```
Subscriber - A Subscriber is responsible for receiving messages from a Publisher and processing those messages. It acts as a terminal operation in the Streams API. It has four methods to deal with the events received.
1. onSubscribe(Subscription s) - Gets called automatically when a publisher registers itself and allows the subscription to request data.
2. onNext(T t) - Gets called on the subscriber every time it is ready to receive a new message of generic type T.
3. onError(Throwable t) - Is used to handle the next steps whenever an error is monitored.
4. onComplete() - Allows to perform operations in case of successful subscription of data.
```
    public interface Subscriber<T> {
       public void onSubscribe(Subscription s);
       public void onNext(T t);
       public void onError(Throwable t);
       public void onComplete();
    }
```
Subscription: It represents a one to one relationship between the subscriber and publisher. It can be used only once by a single Subscriber. It has methods that allow requesting for data and to cancel the demand:
```
    public interface Subscription {
      public void request(long n);
      public void cancel();
    }
```
Processor: It represents a processing stage that consists of both Publisher and Subscriber.
```
    public interface Processor<T, R> extends Subscriber<T>, Publisher<R> {
     }
```

Spring Webflux

Spring WebFlux is a parallel version of Spring MVC and supports fully non-blocking reactive streams. It supports the back pressure concept and uses Netty as the inbuilt server to run reactive applications. It is built on Project Reactor library that makes it possible to build reactive applications on the HTTP layer.

Two publishers which are used heavily in webflux

Mono: Returns 0 or 1 element.

  Mono<String> mono = Mono.just("Sixt");
  Mono<String> mono = Mono.empty();

Flux: A Publisher that emits 0 to N elements which can keep emitting elements forever. It returns a sequence of elements and sends a notification when it has completed returning all its elements.
```
Flux<Integer> flux = Flux.just(1, 2, 3, 4);
Flux<String> fluxString = Flux.fromArray(new String[]{"A", "B", "C"});
Flux<String> fluxIterable = Flux.fromIterable(Arrays.asList("A", "B", "C"));
Flux<Integer> fluxRange = Flux.range(2, 5);
Flux<Long> fluxLong = Flux.interval(Duration.ofSeconds(10));

flux.subscribe();
```
The data won’t flow or be processed until the subscribe() method is called. log() method is used to trace and observe all the stream signals. The events are logged into the console.

To start with using Web Flux with spring boot below dependency should be added

   <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
    <version>3.0.5</version>
   </dependency>

Concept of Backpressure:

In Reactive Streams, backpressure establishes the mechanism for regulating the transmission of data across streams.

Imagine that server A dispatches 1000 EPS (events per second) to server B. However, server B can only handle 800 EPS, resulting in a shortfall of 200 EPS. As a consequence, server B faces the challenge of falling behind, needing to process the deficit data and transmit it downstream or possibly store it in a database. Consequently, server B grapples with backpressure, and there’s a risk of running out of memory and experiencing failure.

To address this backpressure scenario, several options or strategies can be employed:

Buffer - The deficit data can be buffered for later processing when the server has the capacity. But with a huge load of data coming in, this buffer might increase and the server would soon run out of memory.
Drop - Dropping, i.e. not processing events, should be the last option. Usually, we can use the concept of data sampling combined with buffering to achieve less data loss.
Control - The concept of controlling the producer that sends the data is by far the best option. Reactive Streams provides various options in both push and pull-based streams to control the data that is being produced and sent to the consumer.

Backpressure ensures that both producers and consumers can operate at their optimal processing rates, preventing bottlenecks or system overload.

   @GetMapping(produces = MediaType.TEXT_EVENT_STREAM_VALUE)
   public Flux<UserResponse> retrieveUsers() {
   return userService.retrieveUsers()
           .onBackpressureBuffer(10, BufferOverflowStrategy.DROP_OLDEST)
           .delayElements(Duration.ofMillis(100))
           .log();
}

The onBackpressureBuffer(10, BufferOverflowStrategy.DROP_OLDEST) operator is then applied to the Flux, which limits the buffer size to 10 elements and uses a buffer overflow strategy of dropping the oldest elements when the buffer becomes full. This means that if the downstream subscriber cannot keep up with the rate of emissions, the buffer will store up to 10 elements before it starts dropping older elements to make room for newer ones. The delayElements(Duration.ofMillis(100)) operator is applied to the Flux, which adds a delay of 100 milliseconds before emitting each element. This means that the UserResponse objects emitted by the Flux will be spaced out by at least 100 milliseconds, which can help control the rate of emissions and prevent backpressure issues.

One of the best examples of backpressure in a streaming service context is managing the flow of data between a content provider (such as Netflix) and a client (such as a user’s device) during video streaming. In video streaming, the server needs to send video data to the client continuously to maintain uninterrupted playback. However, the client’s ability to receive and process this data can vary due to factors like network congestion, device capabilities, or the user’s internet connection. Benefits:

Smooth Playback: By employing backpressure mechanisms, Netflix ensures smooth playback for users by dynamically adjusting the delivery of data based on network conditions and device capabilities.

Efficient Resource Utilization: Backpressure helps Netflix optimize the utilization of its streaming infrastructure, preventing overload on servers, CDNs, and network components.

Improved User Experience: By mitigating buffering and playback issues, backpressure contributes to a better overall user experience, leading to higher customer satisfaction and retention.

Use cases of Reactive Programming

Reactive web programming is particularly useful for applications that involve streaming data and real-time interactions. It uses methods that don’t block the program, making it responsive to changes without delay.

High-Concurrency Web Applications: Applications that require handling a large number of concurrent connections, such as real-time messaging systems, chat applications, or multiplayer online games, can benefit from Spring WebFlux. It allows handling many requests with a small number of threads, leading to efficient resource utilization and scalability.
Streaming Data and Media: Applications that deal with streaming data, such as audio/video streaming platforms, real-time analytics, or IoT data processing, can leverage Spring WebFlux. It supports handling continuous streams of data with backpressure mechanisms, ensuring that data processing is efficient and responsive.
Event-Driven Applications: Applications that are event-driven, where actions are triggered based on events or messages, can utilize Spring WebFlux to handle event streams efficiently. This includes applications like real-time notifications, event sourcing systems, or reactive workflows where responsiveness and real-time processing are critical.
Internet of Things (IoT) and Edge Computing: In IoT scenarios where devices generate continuous streams of data, and in edge computing environments where resources are limited, Spring WebFlux can be used to build reactive applications that handle data streams efficiently and can scale based on demand.

Limitations of Reactive Programming

Complexity:
- Learning Curve: Reactive programming introduces new paradigms and abstractions, which can be challenging for developers accustomed to imperative programming.
- Debugging and Tracing: Reactive code can be harder to debug and trace due to its asynchronous nature, making it difficult to pinpoint the source of issues.
Tooling and Ecosystem:
- Limited Tool Support: Compared to traditional Spring MVC, tooling and ecosystem support for WebFlux is less mature, which might limit debugging, profiling, and performance tuning capabilities.
- Integration Challenges: Integrating WebFlux with other libraries or frameworks that are not designed for reactive programming can be challenging.
Limited Use Cases:
- Not Always Beneficial: For applications with low concurrency requirements or that are primarily CPU-bound, the benefits of a reactive model may not justify the added complexity.
- Suitability: Reactive programming is particularly beneficial for I/O-bound applications. For other types of applications, the traditional blocking model might be more straightforward and efficient.
Library and Framework Compatibility:
- Blocking Libraries: Many existing Java libraries are blocking and may not work well with WebFlux, requiring careful selection or modification of libraries.
- Backpressure Handling: Properly handling backpressure (the situation where producers generate data faster than consumers can process) requires careful design and is not always straightforward.
Scalability Concerns:
- Thread Pool Management: Managing thread pools effectively in a reactive system is crucial and can be complex. Poorly managed thread pools can negate the scalability benefits of a non-blocking approach.
- Resource Management: Reactive applications need careful resource management to avoid potential bottlenecks and ensure optimal performance.

While WebFlux offers significant advantages for building scalable, non-blocking applications, it is essential to weigh these disadvantages and consider whether it is the right fit for your specific use case and team expertise.

Conclusion

Reactivity and non-blocking behavior typically don’t enhance the speed of applications directly. Instead, the anticipated advantage lies in the capacity to scale the application efficiently using a limited, constant number of threads and reduced memory demands. This scalability feature contributes to enhanced resilience under heavy loads, ensuring a more predictable scaling pattern for applications.

Dask: A parallel data processing python library for large datasets

Shashank Chandavarkar — Tue, 16 Jan 2024 00:00:00 GMT

While conducting data analytics, we often utilize Pandas to perform specific operations on the data in order to extract valuable insights. Initially, when working on data manipulation, I approached it as a data structure problem and did not make use of any built-in Pandas functions. Later, as I delved deeper into Pandas and explored its functions, I discovered that it was significantly faster than manually iterating over the DataFrame (or using the Pandas apply function, which essentially involves iterating over an axis and applying a function) and performing operations on individual rows. Curious about why these built-in functions were faster, I conducted some research and found that Pandas uses NumPy under the hood, which contributes to its speed. When can convert our dataframe to numpy vectors and perform mathematical operations on these vectors if we want our code to be fast. Writing code to perform these vector calculations is significantly harder when lots of operations are involved and sometimes python functions are easier and faster to implement.

In a specific use case involving a DataFrame was large, I had to iterate over the DataFrame and perform operations. This significantly slowed down my code. Recognizing the need for optimization, I began exploring ways to make the iteration (or the apply function) faster. While numerous alternatives were available, one of them was notably simple and easy to use and understand: a library called Dask. Dask parallelizes the process by breaking down the DataFrame into multiple partitions and performing operations on them concurrently.

Dask offers various methods that serve as alternatives to Pandas functions, performing the same operations but in parallel. In today’s computing environment, where most computers have multiple cores and can take advantage of multiple threads, Dask provides a distinct advantage over traditional Pandas.

Dask can be particularly useful when dealing with large datasets that cannot all fit into your RAM at once. By using Dask, you can split the data into multiple tasks, and these tasks can fit into the RAM more efficiently.

Internals

Dask provides us with multiple collections(could be understood as data structures) in which we can store data. This data is converted into a Task Graph and these tasks in the graph are worked on in parallel by multiple threads present in the system(we can even use Dask in a distributed environment for higher workloads).These task graphs are acyclic in nature and each node can be understood as a python function and each edge as a python object. These nodes can be worked upon by threads, hence achieving parallelism.

Dask basics

Minute level entries had to be generated for multiple vehicles and for each entry some processing had to be done. When multiple vehicles were involved, the number of entries was a lot for my system to process. So I tried it on an AWS EC2 instance and yet it was taking a lot of time to process these entries. When the resource utilisation of the EC2 instance was checked it still had lots more to give as only one core was utilised so parallelisation could work and all of the resources at hand could be used.

After some research, I found that Dask Dataframes could be used and the advantage was that i could easily convert a pandas dataframe to a Dask Dataframe.

import dask.dataframe as dd
dask_df = dd.from_pandas(pandas_df, npartitions=n)

We can create partitions in our dataset by using the arguments npartitions.

To perform any sort of processing on this dataframe we can do:

dask_df = dask_df.map_partitions( func )
df = dask_df.compute()

This function can be a lambda function, or a normal function which takes input as a dask dataframe. The partitions created initially will be mapped to the function specified in the map_partitions, the function will take argument of type dask dataframe. For this processing to happen we have to use the compute function, which starts performing the computations and gives us an output , the output will be a pandas dataframe. For optimal performance Dask’s documentation advices to switch between dask and pandas dataframes , we should do this when the computed data from dask data frame is small enough to fit in the RAM easily.

Dask built-in Function example

Dask has a few functions which resemble the conventional pandas functions. Dask.explode is one such function.

pandas.explode: One of the most useful functions that we can relate is the explode function, when we have to create row wise entry for a every element in a list which is present in a row we can use the explode function.

Example from pandas documentation:

Dask has the same function, which is particularly useful when there are multiple lists in the column that you want to explode and the resultant dataframe would be very large in size due to the nature of the function hence dask is useful here.

import dask.dataframe as dd
dask_df = dd.from_pandas(pandas_df, npartitions=n)
dasf_df = dask_df.explode(column)

Results in my use case

Initially to test the advantages of using dask I took a smaller dataset and check how much advantage dask could give me

Using Pandas:

Using Dask:

We can see a 50%+ step up in the total computation time, for a smaller subset of my usecase. In the 2nd case, it can be noticed that cpu time is not the same as wall time, this is because parallelism was used.

Drawbacks

Dask can be useful only if large datasets are used and if the operations that have to be performed is a dask function(or a combination of them).

Insights

Dask can be a very powerful tool when used along with pandas, but it is mainly useful when the data being worked upon is large, for smaller datasets it is better to use pandas and not rely on dask. When using dask it is better to couple it up with pandas and use dask just as an optimisation tool.

Empowering Scalability: Harnessing the Power of CQRS for High-Performance Systems

Aaliya Khan — Mon, 04 Dec 2023 11:13:00 GMT

In the world of software architecture and design patterns, the Command Query Responsibility Segregation (CQRS) pattern has gained popularity for its ability to improve system scalability, maintainability, and performance.

In this blog, we will explore what CQRS is, why it is essential, how to implement it, and the advantages it offers to developers and organizations.

What is CQRS?

CQRS stands for Command Query Responsibility Segregation. It’s an architectural pattern that suggests separating the read and write operations for a given data model. In traditional systems, a single model is often used for both read and write operations. CQRS advocates splitting this model into two distinct parts: one for handling commands (writes) and another for queries (reads).

Why CQRS

Now that we know what CQRS is, let’s delve into the reasons why this pattern is beneficial:

1.Scalability: CQRS empowers you to scale your read and write components independently, offering a flexible solution to handle varying workloads. To elaborate further, you can deploy multiple instances of your read components or distribute your read databases across servers. This allows you to efficiently manage the increased demand on read operations, which often outnumber write operations. By decoupling the scaling of read and write functionalities, you can optimize resource allocation, ensuring responsive and efficient performance even under heavy loads.

2.Performance Optimization: CQRS enables you to tailor your read models for optimal performance. This includes designing and structuring your read models to cater specifically to the needs of querying operations. By separating read and write concerns, you have the freedom to fine-tune your read models independently, implementing caching mechanisms or denormalization strategies to achieve faster response times. This level of optimization contributes to an overall improvement in system performance and user experience 3.Maintainability: CQRS promotes cleaner code by enforcing a clear separation of concerns. This makes it easier to maintain and evolve your system over time, as you have isolated the logic for handling commands and queries.

4.Flexibility: CQRS allows you to use different data storage technologies for your read and write models. For example, you can use a relational database for writes and a NoSQL database for reads, optimizing each for its respective workload.

CQRS Use Cases: Domains that Benefit from Command Query Responsibility Segregation

E-commerce:

Managing product listings, inventory updates, order placements, and user accounts often involves distinct read and write concerns.

Finance and Banking:

Handling transactions, managing account balances, and generating financial reports may have different requirements for command and query processing. Healthcare:

Logistics and Supply Chain:

Tracking shipments, managing warehouse inventory, and processing orders involve a mix of write and read operations that can benefit from separate optimization.

Social Media:

Social platforms may have intense write operations when users create content or engage in real-time communication. The read side is crucial for displaying user profiles, feeds, and recommendations.

Synchronizing Data in CQRS: Bridging the Divide between Read and Write Databases

Now that we have explored the essence of CQRS and its benefits in separating the concerns of reading and writing data, the next pivotal question arises: How do we ensure that the data remains synchronized between the distinct read and write databases?

Embracing Event Sourcing: CQRS often goes hand in hand with the concept of event sourcing. Events, representing state changes, are stored in an event store on the write side. On the read side, we employ event handlers that subscribe to these events, updating the read database accordingly. This asynchronous process allows for the decoupling of the write and read operations, ensuring a more responsive and scalable system. The write-side components emit events when changes occur. These events are then captured and processed by event handlers on the read side. These handlers play a crucial role in updating the read database based on the evolving state of the system. By executing asynchronously, event handlers prevent bottlenecks in the write-side processing.
CQRS utilises the concept of eventual consistency: While the write side updates its data immediately, the read side might experience a slight delay. This temporary inconsistency between the two sides is a trade-off that allows for improved system responsiveness. Users should be aware of this characteristic and design their applications to gracefully handle it.
Database Replication: For scenarios demanding real-time or near-real-time synchronization, database replication becomes a valuable factor. Replicating data from the write database to the read database ensures that changes are swiftly reflected in the query-optimized read store. This replication can occur synchronously or asynchronously, depending on the performance requirements of the system.
Materialized Views for Performance: Materialized views offer an optimized solution for complex queries on the read side. By precomputing and storing results based on events from the write side, materialized views provide rapid access to information without the need for intensive processing during query execution.
Message Brokers as Intermediaries: The use of message brokers, such as Apache Kafka or RabbitMQ, provides a resilient and scalable communication channel between the write and read sides. Events emitted by the write side are transmitted through the message broker, ensuring loose coupling and facilitating robust data synchronization.
Idempotent Operations: To enhance the reliability of data synchronization, it’s crucial to design systems that handle idempotent operations gracefully. Ensuring that the same event can be safely applied multiple times without adverse effects becomes imperative, particularly when replaying events.
Monitoring and Error Handling: Implementing robust monitoring and error-handling mechanisms is paramount. Logs and alerts can help detect and address issues promptly, guaranteeing the continued integrity of the data synchronization process.Additionally, consider incorporating retry queues for enhanced error handling in the context of CQRS. Retry queues offer a valuable mechanism to automatically reprocess failed or delayed messages, promoting system resilience and minimizing the impact of transient errors. This proactive approach ensures that the data synchronization process can recover gracefully from hiccups and maintain its reliability over time.

In short , while CQRS introduces a clear and powerful separation of concerns in data management, the synchronization of data between the write and read sides requires careful consideration and thoughtful implementation.

Designing a CQRS System

Identify Command and Query Models

Command Model:
- Focuses on handling write operations.
- Contains entities and logic for modifying the system’s state.
Query Model:
- Focuses on handling read operations.
- Contains entities and logic optimized for querying and data retrieval.

Define Command and Query Handlers

Command Handlers:
- Implement the logic to process and execute command requests.
- Update the state of the system based on the received commands.
Query Handlers:
- Implement the logic to process and execute query requests.
- Retrieve data from the system to fulfill read requests.

Introduce Aggregates for Consistency
- Aggregates:
- Define clusters of related entities that form a consistency boundary.
- Ensure atomicity and data consistency for operations within the aggregate.
Implement Domain Models
- Domain Models:
- Represent the core entities and business logic of the system.
- Enforce business rules and encapsulate behavior.
Establish Asynchronous Communication

Event Sourcing:
- Record events as a log of state-changing operations.
- Provides a historical view of changes and supports recovery.
Message Queues:
- Implement asynchronous communication between components.
- Enhances system responsiveness and scalability.

Optimize for Read and Write Operations

Write-Side Optimization:
- Optimize for high-throughput write operations.
- Ensure efficient handling of command requests.
Read-Side Optimization:
- Optimize for complex querying and data retrieval.
- Implement caching and other optimizations to enhance read performance.

Address Consistency Challenges
- Eventual Consistency:
- Accept that consistency may not be instantaneous across all parts of the system.
- Implement strategies to handle eventual consistency, such as background processes for updates.
Monitor and Tune

Monitoring:
- Implement monitoring solutions to track system performance.
- Monitor command and query execution times, event processing, and system health.
Tuning:
- Continuously analyze and tune the system for optimal performance.
- Adjust resource allocation, database indexes, and caching strategies based on monitoring insights.

Implementing CQRS in Spring Boot

Let’s create a simple Spring Boot application to demonstrate the CQRS pattern. In this example, we’ll focus on a hypothetical “Task Management” system.

├── src
│   ├── main
│   │   ├── java
│   │   │   └── com.example.cqrs
│   │   │       ├── command
│   │   │       │   ├── CreateTaskCommand.java
│   │   │       │   └── TaskCommandHandler.java
│   │   │       ├── query
│   │   │       │   ├── TaskDetailsQuery.java
│   │   │       │   └── TaskQueryHandler.java
│   │   │       ├── domain
│   │   │       │   └── Task.java
│   │   │       ├── Application.java
│   │   │       └── TaskController.java
│   │   └── resources
│   └── test

1. Command Model:

The CreateTaskCommand class encapsulates the data needed to create a new task, such as the task title

// CreateTaskCommand.java
public class CreateTaskCommand {
    private String title;
    // other fields, getters, setters
}

2. Command Handler:

The TaskCommandHandler class is responsible for executing the command, creating a new task, and storing it in the database.

// TaskCommandHandler.java
@Service
public class TaskCommandHandler {
    @Autowired
    private TaskRepository taskRepository;

    public void handle(CreateTaskCommand command) {
        Task task = new Task();
        task.setTitle(command.getTitle());
        // set other fields
        taskRepository.save(task);
    }
}

3. Query Model:

he TaskDetailsQuery class represents a query for fetching details of a specific task, identified by its unique identifier (taskId).

// TaskDetailsQuery.java
public class TaskDetailsQuery {
private Long taskId;
// other fields, getters, setters
}

4. Query Handler:

The TaskQueryHandler class handles the execution of the query, retrieving task details from the database.

// TaskQueryHandler.java
@Service
public class TaskQueryHandler {
@Autowired
private TaskRepository taskRepository;

    public TaskDetailsQueryResult handle(TaskDetailsQuery query) {
        Optional<Task> task = taskRepository.findById(query.getTaskId());
        return task.map(TaskDetailsQueryResult::new).orElse(null);
    }
}

5. Domain Model:

The Task class represents the domain model for a task, including fields such as id (unique identifier) and title. It is annotated with @Entity for JPA mapping

// Task.java
@Entity
public class Task {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    private String title;
    // other fields, getters, setters
}

6. Controller:

The TaskController class exposes RESTful endpoints for interacting with tasks. The POST /tasks endpoint allows the creation of a new task by handling CreateTaskCommand requests. The GET /tasks/{taskId} endpoint allows the retrieval of task details by handling TaskDetailsQuery requests.



// TaskController.java
@RestController
@RequestMapping("/tasks")
public class TaskController {
@Autowired
private TaskCommandHandler taskCommandHandler;

    @Autowired
    private TaskQueryHandler taskQueryHandler;

    @PostMapping
    public ResponseEntity<Void> createTask(@RequestBody CreateTaskCommand command) {
        taskCommandHandler.handle(command);
        return new ResponseEntity<>(HttpStatus.CREATED);
    }

    @GetMapping("/{taskId}")
    public ResponseEntity<TaskDetailsQueryResult> getTaskDetails(@PathVariable Long taskId) {
        TaskDetailsQuery query = new TaskDetailsQuery();
        query.setTaskId(taskId);
        TaskDetailsQueryResult result = taskQueryHandler.handle(query);
        return ResponseEntity.ok(result);
    }
}

Query Result:

The TaskDetailsQueryResult class represents the result of a task details query, providing a structured response.

// TaskDetailsQueryResult.java
public class TaskDetailsQueryResult {
    private Long id;
    private String title;
    // other fields, getters, setters

    public TaskDetailsQueryResult(Task task) {
        this.id = task.getId();
        this.title = task.getTitle();
        // set other fields
    }
}

These classes work together to demonstrate a basic CQRS implementation in the context of a simple “Task Management” system using Spring Boot. The primary goal is to showcase how CQRS can be employed to separate the responsibilities of handling commands (write operations) and queries (read operations), leading to a more modular and scalable architecture.

Limitations:

1.Increased Complexity:

Introducing CQRS often results in a more complex system architecture. Developers need to manage separate read and write models, implement event handling, and synchronize data between the two models. This complexity can make the system harder to understand and maintain.

2.Eventual Consistency:

CQRS introduces eventual consistency, meaning that there might be a delay between the time a write operation occurs and when the read model is updated. This inconsistency can be challenging for applications that require immediate and fully consistent query results. Data Synchronization Challenges:

Keeping the read and write models in sync can be complex. Developers need to implement mechanisms to handle data synchronization and address issues such as race conditions and concurrency conflicts.

3.Operational Overhead:

Managing and operating a system based on CQRS may require additional infrastructure and operational considerations. Event sourcing, in particular, introduces challenges related to event storage, replaying events, and managing event versioning.

4.Consistency Challenges in Distributed Systems:

In distributed systems, achieving strong consistency across read and write models can be challenging. Developers may need to choose between eventual consistency and investing in more complex distributed transaction mechanisms.

5. Increased Resource Usage:

Maintaining separate read and write models may lead to increased resource usage, especially in scenarios where frequent updates to the read model are necessary. This can impact scalability and resource efficiency.

Conclusion

In conclusion, Command Query Responsibility Segregation (CQRS) is a nuanced and potent design pattern that has significantly influenced the architecture of modern software systems. While it finds application in diverse domains such as e-commerce, finance, and collaborative tools, the adoption of CQRS comes with its challenges. Navigating the complexities and trade-offs requires a thoughtful approach, and you must weigh the benefits against the learning curve and increased intricacie. As technology continues to evolve, CQRS stands as a valuable tool, offering a strategic approach to building responsive, scalable, and adaptable software solutions.

Thanks For Reading!

Elevating Your Codebase: An Imperative Role of Alerting and Monitoring

Sumukh R R — Sat, 11 Nov 2023 00:00:00 GMT

In the world of software development, where every second counts and user expectations soar, the importance of robust alerting and monitoring within your application or service cannot be overstated. In this blog post, we’ll delve into the critical role these practices play in ensuring the reliability and performance of your applications or services. Furthermore, we’ll explore strategies to improve and standardize alerting and monitoring standards across development teams.

The Crucial Role of Alerting and Monitoring:

Alerting and Monitoring serve as the vigilant guardians of your application or service. These practices provide real-time insights into your application’s health, performance, and security. Without them, you’re navigating in the dark, leaving your systems vulnerable to outages and inefficiencies.

Proactive Issue Mitigation:

Effective alerting and monitoring systems allow you to catch issues before they escalate. Setting up alerts based on predefined thresholds enables proactive intervention. For example, if your server’s CPU usage exceeds 90%, an alert can trigger a notification, prompting immediate action. This level of proactive monitoring can significantly reduce downtime and service disruptions.

Vital Steps to follow when configuring Alerting and Monitoring Systems:

When setting up an alerting and monitoring mechanism in a application or service, it’s essential to consider the below steps to ensure that you choose the right metrics to effectively monitor your system.

Data Collection and Storage:
- Data collection involves gathering data from various sources within your codebase or infrastructure. This data can include system performance metrics, application-specific metrics, logs, and more.
- Use appropriate data collectors and agents to retrieve and send this data to a central repository or monitoring system.
- Once data is collected, it needs to be stored and processed effectively. Data storage solutions like databases or time-series databases are used to store historical data.
- Carefully select the metrics ensuring that they are relevant to your application’s performance and business goals.
Data Visualisation:
- Data visualization is about presenting the collected data in a human-readable format. It helps you understand the system’s behavior, trends, and anomalies.
- Use visualization tools like Grafana, Kibana, Splunk, Dynatrace, New Relic, and more to create charts, graphs, and reports that display metrics over time.
- Visualisation mainly aids in identifying patterns, and bottlenecks, allowing you to make data-driven decisions.
Alerting and Notification:
- Alerting is a critical step that involves setting up rules and thresholds to trigger notifications when specific conditions or anomalies are detected.
- Notification mechanisms such as email, SMS, or integrations with tools like Slack, Opsgenie, PagerDuty, or incident management platforms are used to inform relevant parties when alerts are triggered.
Monitoring:
- Monitoring is the continuous observation of your system’s performance and the responsiveness of your alerting system.
- Regularly review and refine the metrics, thresholds, and alerting rules to ensure they remain relevant and effective.
- Use monitoring to proactively detect and respond to issues, reduce downtime and improve system reliability.

Metrics relevant to application goals, Latency metrics, Error Rates based metrics, Scaling metrics to monitor resource utilization (CPU, memory, network, etc.), and Custom Metrics that address application-specific requirements must be key considerations when selecting appropriate metrics for alerting and monitoring.

Standardizing Alerting Practices:

To improve alerting standards across your application or service, consider the following steps:

Define clear Objectives: Begin by establishing clear objectives for each alert. Define what constitutes a critical issue and what is merely informational. This clarity helps avoid alert fatigue.
Thresholds and Conditions: Always define precise thresholds and conditions for triggering alerts. Make these thresholds data-driven, relying on historical performance data or observed patterns in metrics over time to set realistic and actionable alerts.
Escalation Policies: Implement escalation policies to ensure that alerts are routed to the right teams or individuals based on severity levels and time sensitivity. Escalation policies help prevent alerts from getting lost in the noise.

Importance of Automation and Self-Healing in Streamlining Monitoring practices:

In the realm of monitoring, automation involves the deployment of tools and scripts that can carry out routine monitoring tasks, data collection, analysis, and responses to certain events or alerts. Automated processes can help improve efficiency, reduce manual errors, and ensure consistent and timely monitoring across various components of the IT environment.

Self-Healing, on the other hand, involves creating systems or processes that can automatically detect and respond to certain issues without requiring human intervention. Self-healing mechanisms aim to identify common problems and implement predefined solutions to restore or improve system functionality.

Key Reasons highlighting importance of Automation and Self-healing:

Efficiency and Speed: Automation allows for the rapid execution of monitoring tasks, while self-healing systems can automatically resolve common issues which minimizes downtime and increases overall system efficiency.
Scalability: The large scale of data can be overwhelming for manual monitoring. Automation allows for scalability, ensuring that monitoring practices can adapt to the size and complexity of infrastructure.
Cost Savings: By automating routine monitoring tasks and enabling self-healing mechanisms, organizations can reduce the need for a large, dedicated workforce to manage and respond to alerts which results in improved operational efficiency, reduced downtime, and prevention of financial losses by preventing issues.
Focus on Innovation: By automating repetitive tasks, teams can focus on more strategic and innovative projects.

Security and Compliance:

Integrating security and compliance checks into your alerting and monitoring processes is paramount. Regularly monitor for security breaches, unusual activities, and compliance violations. This safeguards your application’s integrity and user data. The prerequisites for setting up security and compliance are defining clear policies, Continuous Compliance Monitoring, Incident Response Plan Integration, and Regular Training and Drills. For Example, these play a major role in scenarios like Unauthorized access attempts, Data Ex-filtration Attempts, Application Security Breaches, etc.

Continual Improvement:

The landscape of software development is ever-evolving. Continual improvement in these processes is essential for maintaining the health and security of software systems. It involves adapting to changing environments, optimizing resource usage, and aligning with evolving business objectives. By regularly enhancing incident response processes, identifying root causes, and integrating new technologies organizations can ensure early detection of issues and enhance the overall user experience. It also plays a critical role in addressing compliance requirements, fostering efficient collaboration between teams, and enabling proactive risk management. This approach establishes a dynamic and responsive monitoring system that evolves alongside the codebase, promoting resilience, reliability, and long-term success. One major example is to conduct post-incident reviews to learn from past issues and refine your practices.

Conclusion:

In the dynamic world of software development, alerting and monitoring are your silent sentinels, guarding your application or service against unforeseen threats and performance bottlenecks. By standardizing these practices you elevate the reliability and resilience of your applications, ensuring they meet the high expectations of modern users.

Remember, the road to improvement starts with a commitment to vigilance. Invest in robust alerting and monitoring practices, set clear objectives, and adapt as your application evolves. Your users will thank you for the reliability and performance you deliver, and your development teams will operate with greater confidence in the codebase they oversee.

In the end, it’s not just about code – it’s about the experience you create and the trust you build with your users. Alerting and Monitoring are your allies in delivering exceptional software experiences.

Jetpack Compose: A New Era of Android UI Design

Chandan Sharma — Thu, 12 Oct 2023 00:00:00 GMT

Introduction

Creating user-friendly interfaces in Android app development has always been a tough task. But now, Google’s Jetpack Compose is here to change the game. It’s a new toolkit that makes designing Android UIs a whole lot easier. This explanation gives you the inside scoop and practical skills to make stunning user interfaces using code examples.

In the past, we relied on XML-based layouts for Android apps, which could get pretty complicated. But Jetpack Compose is different. It uses a declarative approach, meaning you tell it how you want your UI to look, and it takes care of the rest. This makes UI development simpler, more efficient, and better suited for the dynamic nature of modern Android apps. So, with Jetpack Compose, you can create beautiful and responsive user interfaces that truly connect with your users.

The Evolution of Android UI Design: From XML to Jetpack Compose

Traditional Android UI development involved crafting layouts using XML-based files. While effective, this approach led to boilerplate code, maintenance challenges, and intricate view hierarchies, as seen in the example below

<LinearLayout
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:orientation="vertical">

    <TextView
        android:text="Hello, XML!"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"/>

    <Button
        android:text="Click Me"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"/>
</LinearLayout>

Enter Jetpack Compose, which brings declarative UI to the forefront. This means you describe how the UI should look, and the framework automatically handles rendering and updates.

@Composable
fun ComposeUI() {
    Column(
        horizontalAlignment = Alignment.CenterHorizontally,
        verticalArrangement = Arrangement.Center
    ) {
        Text(text = "Hello, Jetpack Compose!")
        Button(onClick = { /* Handle button click */ }) {
            Text(text = "Click Me")
        }
    }
}

Introducing Jetpack Compose: The Declarative UI Toolkit

In the realm of UI design, clarity, and conciseness are paramount. The declarative UI philosophy championed by Jetpack Compose fundamentally transforms how we approach UI creation. It’s a mindset shift from the traditional imperative approach to one where the focus is on expressing the desired outcome without specifying the step-by-step process to achieve it.

Why Declarative UI?

In traditional imperative UI frameworks, developers specify each element’s properties and layout constraints, often resulting in verbose code and intricate relationships. Jetpack Compose’s declarative approach shifts the focus to describing the desired UI, enhancing code readability, maintenance, and debugging by handling the implementation details automatically.

Example: Declarative UI in Action

Consider a scenario where you want to display a list of items in your app. In the imperative approach, you might iterate through the list, create individual view elements, set their properties, and add them to a container. This can become cumbersome as the list grows. Here’s a simplified example:

@Composable
fun ItemList(items: List<String>) {
    Column {
        items.forEach { item ->
            Text(text = item)
        }
    }
}

In this example, you focus on what the UI should contain (Text components for each item) rather than the mechanics of creating and arranging each element.

Setting Up Jetpack Compose: Getting Started

Embarking on your Jetpack Compose journey involves more than just curiosity; it’s a transformative step toward modern Android UI development. To harness the capabilities of Jetpack Compose, you need to integrate it into your Android project, laying the foundation for a new era of UI design.

Add Dependencies: The first step is to add the necessary Jetpack Compose dependencies to your project. Open your app-level build.gradle file and add the Compose dependencies in the dependencies block:

dependencies {
    // ... other dependencies

    implementation "androidx.compose.ui:ui:$compose_version"
    implementation "androidx.compose.foundation:foundation:$compose_version"
    implementation "androidx.compose.material:material:$compose_version"
    implementation "androidx.compose.runtime:runtime-livedata:$compose_version"
}

Ensure you replace $compose_version with the appropriate version number.

Enable Compose in Activity: To enable Jetpack Compose in an activity, you need to set the setContent of the activity to a Jetpack Compose composable function:

class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            // Your Jetpack Compose UI goes here
        }
    }
}

With these initial steps, Jetpack Compose is integrated into your project, ready to revolutionize your UI development process.

Exploring Composable Functions: The Building Blocks of UI

Composable functions form the foundation of Jetpack Compose, defining modular and reusable UI elements that return descriptions of how they should appear when rendered. This approach streamlines UI component creation, encapsulation, and composition, serving as the cornerstone of Jetpack Compose’s architecture.

1. Composable Functions:

In Jetpack Compose, It’s a special function that you use to create different parts of your app’s user interface. Each composable function defines a specific UI element, and you can use them to assemble complex interfaces.

Code Sample: Let’s create a simple composable function to display a greeting message:

@Composable
fun Greeting(name: String) {
    Text(text = "Hello, $name!")
}

In this example, Greeting is a composable function that takes a name parameter and creates a Text component with a personalized greeting.

2. Modifier:

Modifiers are tools you use to customize how your UI elements look and behave. You can think of them as instructions you give to UI components to style or position themselves in a specific way.

Code Sample: Let’s use a modifier to add padding to a Text component:

Text(
    text = "Hello, Compose!",
    modifier = Modifier.padding(16.dp)
)

Here, the padding modifier adds some space around the Text component, making it look more spaced out.

3. Column and Row Layouts:

Layouts are containers that help you arrange UI elements in a specific order. The Column layout stacks UI elements vertically, while the Row layout arranges them horizontally. These layouts automatically handle the positioning of your components.

Code Sample: Let’s use a Column layout to stack two Text components vertically:

Column(
    verticalArrangement = Arrangement.Center,
    horizontalAlignment = Alignment.CenterHorizontally
) {
    Text("Hello")
    Text("Compose")
}

In this code, the Column arranges the “Hello” and “Compose” Text components vertically in the center of the screen.

Creating Interactive UIs: Adding User Interaction

Jetpack Compose is a powerful tool for crafting interactive and engaging user interfaces in your app. It allows developers to easily add interactions to UI elements like buttons and text fields.

Let’s walk through the process of creating a simple button that changes its text when clicked. This will help you grasp the fundamentals of adding user interaction using Jetpack Compose.

1. Set Up Your Composable Function:

Begin by creating a new composable function that encapsulates your interactive UI element. We’ll call it InteractiveButton.

@Composable
fun InteractiveButton() {
    // Your code will go here
}

2. Manage the Button’s State:

In Jetpack Compose, managing UI state is crucial for creating interactive elements. We’ll use the remember and mutableStateOf functions to handle the state of our button.

@Composable
fun InteractiveButton() {
    var buttonText by remember { mutableStateOf("Click Me") }

    // Continue with your code
}

3. Define the Button Behavior:

Inside the InteractiveButton composable, you can create UI components such as the Button itself. The button’s behavior is defined using the onClick parameter, which updates the buttonText state when the button is clicked.

@Composable
fun InteractiveButton() {
    var buttonText by remember { mutableStateOf("Click Me") }

    Column(
        horizontalAlignment = Alignment.CenterHorizontally,
        verticalArrangement = Arrangement.Center
    ) {
        Text(
            text = buttonText,
            fontSize = 20.sp,
            modifier = Modifier.padding(16.dp)
        )

        Button(
            onClick = { buttonText = "Clicked!" },
            modifier = Modifier.padding(16.dp)
        ) {
            Text(text = "Click Me")
        }
    }
}

In this code:

We’re using the Text composable to display the button’s text, which is controlled by the buttonText state.
The Button composable’s onClick parameter specifies what happens when the button is clicked.

4. Preview and Observe:

Creating a preview function allows you to observe your interactive UI component in action. Add the following preview function to your code:

@Preview
@Composable
fun InteractiveButtonPreview() {
    InteractiveButton()
}

Styling and Theming with Jetpack Compose

Jetpack Compose lets you easily control the look and feel of your app. You can create custom styles, fonts, and colors using the MaterialTheme feature. When building an app, appearance matters as much as functionality. Users appreciate apps that look good and have a cohesive design. Jetpack Compose, along with Material Design principles, simplifies the process of making your app visually appealing and consistent. Let’s dive into how you can give your app a unique and stylish appearance.

1. Using Material Design Components:

Imagine you have a box of Lego pieces - each piece is a UI component. Material Design is like having instructions on how to put those pieces together to build something cool. Jetpack Compose comes with a bunch of pre-designed components that follow these instructions, giving your app a modern and polished look.

To start, wrap your UI components in a MaterialTheme:

@Composable
fun StyledApp() {
    MaterialTheme {
        // Your UI components go here
    }
}

This automatically applies Material Design styles to your app.

2. Customizing the Look:

While Material Design is great, you might want your app to have its own personality. Jetpack Compose lets you customize styles easily.

For example, if you want to change the color of your text, you can use a modifier:

Text(
    text = "Hello, Jetpack Compose!",
    style = MaterialTheme.typography.h4,
    modifier = Modifier.color(Color.Red)
)

3. Making Your Own Themes:

Creating your own theme is like choosing a unique color palette for your app. It’s a great way to match your app’s look to its purpose.

private val MyCustomTheme = darkColors(
    primary = Color(0xFF2D8CFF),
    background = Color.Black
)

@Composable
fun CustomThemedApp() {
    MaterialTheme(colors = MyCustomTheme) {
        // Your UI components go here
    }
}

4. Making Themes for Light and Dark Modes:

Some people like light themes, while others prefer dark themes. Jetpack Compose makes it easy to have both.

@Composable
fun AdaptiveThemedApp() {
    val colors = if (isSystemInDarkTheme()) {
        darkColors()
    } else {
        lightColors()
    }

    MaterialTheme(colors = colors) {
        // Your UI components go here
    }
}

Styling and theming with Jetpack Compose is like giving your app a makeover. With Material Design components, customizable styles, unique themes, and themes that adapt to light and dark modes, you have the tools to create an app that looks great and matches its personality.

State Management in Jetpack Compose

Efficiently managing state is critical in UI development. Jetpack Compose offers state management techniques, such as remember and mutableStateOf, that help you manage and update UI states seamlessly.

Understanding State: Data That Changes

State refers to data that can change during the lifetime of your app. For instance, variables that control what’s displayed on the screen, user input, or any data that updates based on events. In Jetpack Compose, you manage this changing data using a special mechanism.

State Management in Jetpack Compose: The Basics

In Compose, state management revolves around the remember function and the @Composable annotation. This combination allows you to create and hold onto state in a way that’s both efficient and reactive.

1. Using `remember` for State: Remembering Data:

The remember function remembers values across recompositions (UI updates) while ensuring that changes trigger recompositions when necessary.

Code Sample: Counter with State

@Composable
fun Counter() {
    var count by remember { mutableStateOf(0) }

    Button(onClick = { count++ }) {
        Text("Count: $count")
    }
}

We use mutableStateOf to create a piece of state called count.
The remember function ensures that the count value is retained across recompositions.

2. Separation of UI Logic: The @Composable Annotation:

The @Composable annotation indicates that a function defines a UI component. When the state changes, only the relevant @Composable functions are recomposed, not the entire UI. This ensures efficient updates.

Sometimes, you need to manage state that multiple composables share. In such cases, you can “hoist” the state to a common ancestor and pass it down to the relevant composables.

@Composable
fun App() {
    var sharedCounter by remember { mutableStateOf(0) }

    Counter(counter = sharedCounter)
    OtherComponent(counter = sharedCounter)
}

@Composable
fun Counter(counter: Int) {
    Button(onClick = { /* Update sharedCounter */ }) {
        Text("Count: $counter")
    }
}

@Composable
fun OtherComponent(counter: Int) {
    // Use counter
}

The App composable holds the shared state sharedCounter.
Both Counter and OtherComponent receive and use the same counter value.

The Future of Android UI Design: Jetpack Compose and Beyond

Jetpack Compose is set to reshape Android UI design. Its growing community, focus on animations, interactivity, and future potential for cross-platform compatibility promise a brighter future for UI development.

The Composable Revolution: Jetpack Compose revolutionizes Android UI design, making it easier to create powerful and visually appealing user interfaces.
Transformative Declarative Syntax: Jetpack Compose’s declarative syntax, state management, theming, and more empower developers to build modern Android apps.
A Path Toward Efficiency: As Jetpack Compose evolves, it promises a growing community, cross-platform compatibility, innovative design tools, and immersive interactions.

Conclusion: Embrace the Composable Revolution

Jetpack Compose has ushered in a new era of Android UI design, enabling developers to create powerful and visually appealing user interfaces with ease. Its declarative syntax, state management capabilities, theming, and more make it a versatile toolkit for crafting modern Android apps. As you explore and incorporate Jetpack Compose into your projects, you’ll discover its potential to transform your development workflow and enhance user experiences.

Mobile App Security Testing: Planning and Initiating Testing

Rashmi Kumari — Tue, 10 Oct 2023 00:00:00 GMT

In today’s digital age, for various purposes like travel, banking, shopping, socializing, entertainment, learning and many more, we rely on mobile apps. However, with the increased dependency on mobile apps, there is an equally growing concern about the security of these applications. A single breach to the application can lead to compromise of user data, financial loss, and this can damage the brand’s reputation. To mitigate these risks, it is essential to plan and execute thorough security testing for mobile applications. Based on my experience, outlined below are the best practices to be followed while planning and strategizing security testing of mobile applications.

Step 1: Understanding the Mobile App and Its Environment

Before directly jumping into security testing, let’s first have a clear understanding of the mobile application and its environment. Which includes:

Understanding the App’s Functionality: Gain insights regarding the purpose of mobile application, the sensitive data it handles, user authentication mechanisms, and third-party integrations.
Identifying Platforms and Devices: Determine the platforms (iOS, Android, etc.) and devices (smartphones, tablets) the app will run on. Security considerations may vary based on the platform and device.
Legal and Compliance Requirements: Understand the legal and compliance requirements specific to the industry or region, such as GDPR(General Data Protection Regulation), HIPAA(privacy of protected health information ), or PCI DSS (Payment Card Industry Data Security Standard).

Step 2: Define Goal of Mobile App’s security

Before testing, it’s important to understand what is the goal of security testing. The goals can be –

Data Protection: Ensure that user data is encrypted and stored securely.
Authentication and Authorization: Verify that only authorized users can access certain features and data.
Secure Communication: Ensure that data transmitted between the mobile app and server is encrypted. Protocol like HTTPS encrypts data transmitted over the network and ensures data protection.
Code Vulnerabilities: Identify and mitigate vulnerabilities like SQL injection and other code-related issues.
Secure APIs: Check the security of APIs used by mobile app.
Penetration Testing: Test the application for potential vulnerabilities that attackers might exploit.

Step 3: Who can perform Security Testing

Understanding of who can perform security testing in an agile team or building a capable security testing team is essential. This team may consist of:

Security Testers: Experts in security testing methodologies and tools.
Developers: Developers familiar with the app’s codebase can identify vulnerabilities more effectively.
Quality Assurance (QA) Testers: QA testers can provide valuable insights from a user’s perspective.

Step 4: Define a Security Testing Strategy

It’s always good to have a security testing strategy that outlines the objectives, scope and approach for testing. Some known security testing strategies are :

Static Analysis: Examining the application’s source code for vulnerabilities without executing it.
Dynamic Analysis: Assessing the app’s behaviour during runtime to identify vulnerabilities related to memory usage, network interactions, data storage etc.
Penetration Testing: Simulating real-world attacks to find weaknesses in the app’s defences. Astra and Cipher are well known service provider for Penetration testing. Few mobile app penetration testing tools are-
- Core Impact Pro (Android, iOS, and Windows)
- zANTI (Android)
- Ianalyzer (iOS)
Authentication and Authorization Testing: Evaluating how the app handles user access and permissions.
Secure Code Review: Manual review of the application’s source code to identify security vulnerabilities.

Step 5: Choose Appropriate Security Testing Tools

Select the testing tools that align with objectives and scope of mobile app. Few popular tools for mobile app security testing include:

Mobile Device Emulators: These tools simulate various mobile devices and operating systems for testing app compatibility and vulnerabilities.
Static Application Security Testing (SAST) Tools: SAST tools analyse the app’s source code for vulnerabilities. Ex: Sonarqube
Dynamic Application Security Testing (DAST) Tools: DAST tools test the running app for vulnerabilities. Ex: OWASP ZAP, Burp Suite
API Testing Tools: API tools evaluate the security of APIs used by app. Ex: APIsec

Other popular tool is Mobile Security Framework (MobSF). It is all-in-one mobile application (Android/iOS/Windows) testing tool capable of performing SAST and DAST analysis.

Always,check for current state of these tools and consider using up-to-date tools.

Step 6: Create Test Plan and Data

Always a well-defined test plan ensures the success of testing. Test plan may include:

Scope: Define the scope of the testing, that includes which parts of the app will be tested, the devices, operating systems that will be tested on.
Testing Methodologies: Methodologies such as penetration testing, code review, or automated scanning should be used
Test Cases: Test case development to address each security goal
Timeline: Timeline for testing and prioritize critical issues to be addressed first.
Data: Prepare test data ex. sample user profiles, various inputs etc

Step 7: Execute Security Testing

Now that testing scope, tools and techniques are already identified, based on our requirement and mobile application, it’s time for the execution of security testing. Pay attention to:

Data Security: Ensure that sensitive user data like passwords or personal information are encrypted and securely stored.
Authentication: Verify that the authentication process is robust and protects against unauthorized access.
Authorization: Check that users can only access the features and data they are authorized to access.
Input Validation: Test the application for vulnerabilities related to input validation, such as SQL injection or cross-site scripting (XSS).
API Security: Assess the security of any APIs used by the app.
Session Management: Ensure that session tokens are handled securely.

Step 8: Reporting and Fixing the issues

Analyse the results obtained from the security testing, classify vulnerabilities based on severity, and prioritize them for correction. Also it is good practice to connect with security experts to identify false positives. Share the report with the development team and work closely with them to prioritize and address the issues. Always, create a detailed report containing Summary of Findings & Detailed Vulnerability Analysis.

Step 9: Re-test and Validate Fixes

After the development team addresses the vulnerabilities, always test again to verify that the issues are resolved and that the fixes haven’t introduced new vulnerabilities.

Step 10: Regularly Update and Monitor

Security testing is not a one-time activity; it should be an ongoing process. Regularly re-evaluate your mobile application’s security, especially when new features are added or changes are made to the codebase. Penetration testing can be done when any major feature release happens or once in 6 months or a year, depending on project requirement. Also, it is recommended to do security testing for any latest security threats and vulnerabilities updates and always keep a tab on legal compliances, for any updates or changes.

Conclusion

In an era of increasing cyber threats, securing your mobile application is not optional—it’s a necessity. Security testing for mobile applications is a crucial step in ensuring the safety of both users and the reputation of the app. By following these steps and maintaining a proactive approach to security, one can significantly reduce the risk of security breaches and protect the integrity of your mobile application.

Remember, security is an ongoing process and regular security assessments are crucial to maintain a secure mobile app environment.

Java Virtual Threads vs Platform Threads Performance Comparison under high load.

Utkarsh Verma — Sun, 20 Aug 2023 00:00:00 GMT

Platform Thread vs Java Thread.

Before we begin comparing the performance of virtual threads to platform threads we need to understand the key differences between them. Java’s platform thread is just a wrapper over the OS thread. Since OS threads are managed via the underlying operating system, their scheduling and optimisations are practically inaccessible to JVM. This is where virtual threads comes into action, part of project loom JEP425 virtual threads provide a one-to-one mapping over the underlying OS threads. In this case JVM is responsible to mapping virtual threads to OS threads, and in scenarios where Java App is busy in a non CPU bound task such as Network call or a DB call. JVM can just un-map that thread corresponding to non CPU bound task and free up that OS thread as well, there by allowing full control over scheduling optimisations.

Setting up Spring Application.

For our case we will consider a Spring Boot application (**version 3) which will be running on a TomCat Server, and performing an intensive non CPU bound process. Java Virtual Threads is available on version 19 and 20 in preview mode currently to enable them we need to set preview flags in our build.gradle file. We will be using Java 19 here.

[** Prior versions to 3 may use TomCat 9 which is not optimised to virtual threads.]

compileJava {
 options.compilerArgs += ['--enable-preview']
}

tasks.withType(JavaExec) {
 jvmArgs += '--enable-preview'
}

tasks.withType(Test).all {
 jvmArgs += '--enable-preview'
}

We will have a standard GET Endpoint running on our application that will invoke a non CPU intensive task.

package com.application.virtualthread.controller;

import com.application.virtualthread.service.NonCPUBoundService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/thread-application")
public class AppController {

    @Autowired
    private NonCPUBoundService nonCPUBoundService;

    @GetMapping("/run")
    public ResponseEntity<String> invokeVirtualThreadApplication() {
        nonCPUBoundService.executeNonCPUBoundTask();
        return new ResponseEntity<>(HttpStatus.OK);
    }
}

To mimic a non CPU intensive task in which the application might be busy waiting for some other resource over network, we will use Thread.sleep here for 5 secs.


package com.application.virtualthread.service;

import org.springframework.stereotype.Service;

@Service
public class NonCPUBoundService {
    public void executeNonCPUBoundTask() {
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

}

For this 5 sec non CPU bound task, If we consider platform threads; JVM is quite aware that CPU is not being utilised but the OS is clueless over it, so scheduling optimisations can’t be done. Its here where virtual threads beat it, since JVM has a one-to-one mapping it can simply un-map the virtual thread freeing the OS thread for other requests. Spring’s dispatcher servlet maps request onto Tomcat’s thread pool. To use virtual threads as tomcat’s thread pool, we must make some configuration edits:

package com.application.virtualthread.configuration;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.web.embedded.tomcat.TomcatProtocolHandlerCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.concurrent.Executors;

@Slf4j
@Configuration
public class AppConfig {

    // Enables tomcat virtual pool thread if flag is set true in application.properties.
    @Bean
    @ConditionalOnProperty(name = "virtual-thread.flag", havingValue = "true")
    public TomcatProtocolHandlerCustomizer<?> protocolHandlerVirtualThreadCustomizer() {
        log.info("Using Virtual Threads!!!");
        return protocolHandler -> protocolHandler.setExecutor(Executors.newVirtualThreadPerTaskExecutor());
    }
}

Creating a high load traffic.

Tomcat has default max-thread configured to 200. This means it can serve upto 200 parallel request without any performance degradation. However, if we consider a high-load traffic environment of 1000 parallel requests, at an instant 200 requests will be served instantly and the other 800 requests will have to wait till the threads become free(in our case 5 sec per thread). With virtual threads this issue is resolved as for those 5 sec of non cpu bound task, the thread is freed. For our load-testing we will be using the Apache Benchmark tool to simulate a concurrent traffic of 1000 requests.

// paste this in terminal to hit the endpoint
ab -n 1000 -c 1000 http://localhost:8080/thread-application/run

Performance Reports

Platform Thread

Virtual Thread

The above Benchmark reports suggest Platform thread bound requests took a median latency of 15 sec while those of virtual threads took around 5 secs only. You can visit this repository for implementation details: GitHub

Thanks for reading!

Elevating Code Quality and Speed: Exploring BDD with TDD for Effective software development

Raghav Saboo — Fri, 14 Jul 2023 11:13:00 GMT

In the fast-paced world of software development we live in today, it’s always a challenge to deliver top-notch applications quickly and efficiently. Thorough testing is an essential part of the development process to achieve that. By combining Behaviour-Driven Development (BDD) with Test-Driven Development (TDD), we can enhance the effectiveness of the testing process and ultimately improve the overall quality of our software.

In this blog , we will dive into the world of BDD and TDD, exploring their fundamental principles, benefits, and practical implementation techniques

Behaviour Driven Development

BDD is an agile software development methodology that focuses on aligning business stakeholders, developers, and testers around a shared understanding of the system’s behaviour and requirements.

In BDD, the emphasis is on defining and describing the expected behaviour of the system in a human-readable format. This is generally done through the creation of structured scenarios written in a Given-When-Then format. These scenarios describe the preconditions (Given), the actions or events (When), and the expected outcomes (Then) of a particular feature or behaviour.

Test Driven Development

TDD is a software development practice that prioritises writing tests before the actual code. It involves initially writing tests that are expected to fail, followed by writing the minimum amount of code required to pass those tests. Once the code passes the tests, the focus shifts to refactoring it for improved efficiency, design and cleanliness.

Phases of TDD

RED: In this phase, you write the failing test cases that represents the desired behaviour or functionality you want to implement.
Green: Once you have the failing tests, quickly write the minimum amount of code required to make the tests pass.
Refactor: After the tests pass, our aim is to improve the code without changing its behaviour. This phase focuses on enhancing the code’s design, readability, maintainability and performance. You can restructure the code, apply the appropriate design pattern, extract methods, eliminate duplication, and make it more efficient or easier to understand.

Advantages of TDD

● Fast Feedback Loops: TDD enables rapid feedback loop between development and testing. Developers can run tests frequently, ideally after each code change, to ensure that everything is functioning as expected. This immediate feedback enables quick identification and resolution of issues, resulting in faster development cycles.

● Improved Code Quality: TDD promotes writing high-quality code. By writing tests first, developers are forced to think about the requirements first. Thus gaining a clearer understanding of the desired functionality and can design code that meets the requirements. This leads to cleaner, more maintainable, and less error-prone code.

● Gamification of Development: After writing extensive test cases in the red phase, the given problem can now be treated as a competitive programming problem where you need to pass the given test cases in the most efficient way. Thus coding now becomes more fun!

● Increased Test Coverage: Since test cases for each feature are written before the code, this ensures a higher rate of test coverage. It encourages developer to write all possible test cases, leading to more comprehensive test suites.

● Enhanced Design and Architecture: Kent Beck author of TDD by Example points out we can’t do 2 things at once easily, understand the solution to the problem and engineer the code right. We could end up over engineer the solution beyond what the test requires or get into analysis paralysis. In TDD, developers focus on the expected behaviour and design their code accordingly. This leads to more modular, loosely coupled, and easily extensible code, promoting good software architecture.

● Refactoring Safety and Greater Code Confidence: TDD provides developers with a safety net when making changes to existing code. Since the codebase is well-covered with tests, developers can refactor or modify code with confidence, knowing that the tests will quickly catch unintended consequences.

● Better documentation and collaboration: TDD promotes collaboration between developers and testers. Tests can serve as documentation that define the expected behaviour, developers and testers can have a common understanding of the requirements.

Limitations of TDD:-

● Time-consuming and Initial Learning Curve: TDD can be time-consuming, especially in the beginning, as it requires writing tests before writing the actual code. This extra effort may extend the development time initially. However, proponents argue that the time saved from debugging and maintenance in the long run can offset this initial investment.

● False sense of security: While TDD helps catch many bugs early, it does not guarantee a bug-free application. There may still be edge cases that are not adequately covered by the tests. Developers must be cautious not to rely solely on the existing test suite and should perform additional quality assurance measures to ensure overall product quality.

BDD with TDD: The behavioural specifications defined in BDD scenarios guide the creation of test cases in TDD. The development follows a Red-Green-Refactor cycle, where failing test cases drive the implementation of code and continuous refactoring increases the code quality.

The objective of BDD/TDD is to test the behaviour of the system, not just the methods of individual classes. Rather than solely aiming to increase test coverage, we should shift our focus to the specific behaviours these tests are intended to validate.

Example for BDD/TDD in Go:

Consider a simple example where we need to calculate the employee payout given the parameters hourly pay rate, no of hours worked, commissions, no. of projects and employer cost.

BDD Phase

We create a list of all possible expected behaviours of the software in Given-When-Then format. Using a table where we represent each variable that can change with a column. The last row represents the final expected output. We then confirm the behaviour from business stakeholders.

Given, Hourly Rate to be 10$ and Employer Cost to be 100$ for each employee. Commission is given for each project. The output (Payout) is expected in decimal format.

RED Phase

Here we create the Employee struct with a function to compute the payout. We then create a test file which contains a test for each behaviour in the above table. Now we run the tests to see them fail, thus completing our red phase.

type Employee struct {
	Name          string
	Id            int
	HourlyPayRate float32 `default:"10"`
	HoursWorked   int
	EmployerCost  float32 `default:"100"`
	HasCommision  bool
	Commision     float32
	NoOfProjects  int
}

func (Employee) ComputePayout() (payout float32) {
	panic("Compute payout func not implemented")
}

func (suite *EmployeeComputePayrollTestSuite) Test_EmployeePayout_Returns_Float() {
	assert.Equal(suite.T(), reflect.TypeOf(suite.Employee.ComputePayout()).String(), "float32")
}

func (suite *EmployeeComputePayrollTestSuite) Test_EmployeePayout_NoCommission_NoHours() {
	assert.Equal(suite.T(), suite.Employee.ComputePayout(), float32(100))
}

func (suite *EmployeeComputePayrollTestSuite) Test_EmployeePayout_NoCommission() {
	suite.Employee.HoursWorked = 10
	assert.Equal(suite.T(), suite.Employee.ComputePayout(), float32(200))
}

func (suite *EmployeeComputePayrollTestSuite) Test_EmployeePayout_WithCommission() {
	suite.Employee.HoursWorked = 10
	suite.Employee.HasCommision = true
	suite.Employee.Commision = 10
	suite.Employee.NoOfProjects = 10
	assert.Equal(suite.T(), suite.Employee.ComputePayout(), float32(300))
}

func (suite *EmployeeComputePayrollTestSuite) Test_EmployeePayout_Commission_Disabled() {
	suite.Employee.HoursWorked = 10
	suite.Employee.HasCommision = false
	suite.Employee.Commision = 10
	assert.Equal(suite.T(), suite.Employee.ComputePayout(), float32(200))
}

GREEN Phase

In this stage, our primary objective is to expedite the successful execution of all the tests by quickly implementing the necessary logic.

func (emp Employee) ComputePayout() float32 {
	payout := emp.HourlyPayRate * float32(emp.HoursWorked)
	payout = payout + emp.EmployerCost
	if emp.HasCommision {
		payout = payout + (emp.Commision * float32(emp.NoOfProjects))
	}
	return payout
}

REFACTOR Phase

This phase focuses on the generation of clean code by removing duplication, applying the appropriate design patterns, formatting long methods, removing code smells, etc.

In this phase, you can ensure that your code conforms to the specification by running tests. This allows you to modify the code and verify that it still functions correctly. For example, you might decide that the term ‘employer cost’ is too broad, and instead, you want to separate it into ‘employer office cost’ and ‘employer support cost’ with default values of $60 and $40, respectively. As a result, the final refactored method would be:

func (emp Employee) ComputePayout() (payout float32) {
	payout = (emp.HourlyPayRate * float32(emp.HoursWorked)) + emp.EmployerOfficeCost + emp.EmployerSupportCost
	if emp.HasCommision {
		payout += emp.Commision * float32(emp.NoOfProjects)
	}
	return
}

Conclusion

Adopting Behaviour-Driven Development (BDD) with Test-Driven Development (TDD) can significantly elevate both code quality and development speed. By focusing on the desired behaviours of the system rather than just individual methods, BDD/TDD encourages a more comprehensive and efficient testing approach. Embracing BDD/TDD enables teams to deliver functionality to customers faster while maintaining a high level of code quality, avoiding potential long-term setbacks caused by neglecting tests.

Bug Bashing: A Fun and Effective Way to Improve Software Quality

Hanamant V K — Wed, 07 Jun 2023 00:00:00 GMT

What is Bug Bash?:

A bug bash is a collaborative effort aimed at uncovering a large number of bugs within a short time interval. During a bug bash, various participants beyond testers, such as developers, product managers, engineering managers, designers, marketers, solution engineers, writers, customer support representatives, and even executives like the CTO and CEO, can also join in.

Why Bug Bash?

Discovering a substantial number of bugs in a very short time frame is a key goal. However, this approach may also lead to the identification of some invalid defects. For instance, if a colleague is not well-acquainted with the product, they might assume it’s supposed to function in a specific manner and subsequently report invalid defects. Nevertheless, we can embrace this side effect because it contributes to comprehensive test coverage.
Involving individuals with diverse backgrounds and experiences in a bug bash unleashes a wealth of perspectives from various participants, resulting in the identification of bugs, product shortcomings, opportunities for improvement, and usability issues, among other insights.
Bug bashes foster enhanced collaboration among team members and cultivate a shared sense of responsibility for quality, both within the team and across the organization.
Participation in bug bashes provides an opportunity for members of other teams to gain a deeper understanding of different aspects of the product.

When to conduct a bug bash?

An effective practice is to organize two bug bashes: one with a larger number of participants and another with a select group.

First Bug Bash: This should occur when the feature reaches stability in the staging environment. At this point, most major issues in the product should be resolved, and any non-blocker issues are acceptable. This phase typically falls around the halfway mark to the release date or possibly even earlier.
Second Bug Bash: This is scheduled once we deem the product ready for release. Its primary purpose is to prevent any unexpected issues after the go-live phase, which may include regressions or the emergence of new issues.

How to conduct a bug bash?

Arrange the bug bash at a time that suits all participants, send out event invitations well in advance, and keep the event duration within a range of 1 to 1.5 hours.
In advance, share any pertinent documents or links related to the feature or necessary tools.
Request that all participants install the required tools and ensure they have the ‘application under test’ installed prior to the bug bash.
Verify that all pertinent systems are deployed and accessible in the testing environment.
Provide an overview of the bug bash’s testing scope and what will be examined, maintaining focus among participants.
Define the roles of the moderator, participants, and the stager, who will facilitate the bug bash and assist participants in accessing essential tools and devices.
Consider organizing teams to test specific feature areas and emphasize critical aspects.
Guarantee that all participants have access to relevant systems, environments, and necessary resources ahead of the bug bash.
Compile a list of known issues and problems to minimize duplicate reporting.
Distribute test data and request participants to confirm its previous use.
Establish a bug filing process, such as sharing an Excel sheet or creating a common ticket in JIRA.
Set up a communication channel, such as a Slack channel or Teams meeting, for participants to use during the bug bash.
Ponder the idea of acknowledging participants with items such as pizza or snacks after the bug bash.
Show respect for participants: As bug bashes involve individuals with varying technical expertise, expect a mix of both bugs and feedback. If an issue seems invalid, attempt to understand it from the participant’s perspective. If the decision is to close the issue, provide a detailed explanation and acknowledge the participant’s effort with a comment.
Express appreciation to the entire team: Following the conclusion of the bug bash, extend gratitude to all participants for their valuable time and extend invitations for their participation in future bug bashes.

Post bug bash activities

Issue Triage : After the bug bash, review all the issues and feedback collected during the event. Prioritize them based on severity and impact on the application.
Bug Tracking: Ensure that all the bugs and feedback reported during the bug bash are documented in your bug tracking system, such as JIRA or a similar tool. Assign them to relevant team members for further investigation and resolution.
Bug Reproduction: If the reported bugs are not immediately reproducible, work with the participants or testers to gather additional information or steps to reproduce the issues. This will help the development team in resolving the bugs more efficiently.
Bug Resolution: Developers should start working on fixing the identified bugs. Depending on the complexity of the issues, this might take some time. Keep the communication lines open with participants to provide updates on bug resolutions.
Regression Testing: After the bugs are fixed, perform regression testing to ensure that the changes did not introduce new issues or break existing functionality.
Documentation: Update your documentation, including user manuals and release notes, to reflect any changes made as a result of the bug bash.
Feedback Analysis: Review the feedback provided by participants. Look for common themes and trends in the feedback to identify areas that might need improvement or further development.
Reporting: Prepare a post-bug bash report summarizing the key findings, including the number of bugs identified, their severity, and any critical issues that were resolved. Share this report with the relevant stakeholders and the bug bash participants.
Acknowledgment and Rewards: Recognize and reward participants for their contributions. You can announce the winners of categories like “best bug hunter” and “most critical bug reporter” and distribute rewards accordingly.
Lessons Learned: Conduct a retrospective meeting with your team to discuss what went well during the bug bash and what could be improved for future events. Use this feedback to enhance your bug bash process.
Planning for Future Bug Bashes: Start planning for the next bug bash, taking into account the lessons learned from the current one. Consider incorporating any process improvements or changes suggested during the retrospective.
Continuous Improvement: Continuously iterate on your bug bash process to make it more effective and efficient. Implement changes based on feedback and lessons learned to ensure that future bug bashes yield better results.

Remember that post-bug bash activities are essential for ensuring that the issues identified during the event are addressed promptly and that the development team can make necessary improvements to the application. Effective communication and collaboration with participants and team members are key to the success of these activities.

How do we measure the success of the bug bash?

Number of Bugs Found: One of the primary metrics for measuring success is the number of bugs discovered during the bug bash. A successful bug bash should uncover a significant number of bugs, including critical and non-critical issues.
Bug Severity: Categorize the bugs based on their severity (e.g., critical, major, minor). A successful bug bash should identify critical issues that could have a severe impact on the application.
Bug Validity Rate: Assess the validity of reported bugs. Measure the percentage of bugs reported during the bug bash that are valid and require attention. This metric helps you filter out invalid defects and focus on real issues.
Collaboration and Participation: Assess the level of collaboration among participants from various roles and departments. A successful bug bash should promote cross-functional collaboration and engagement.
Stakeholder Satisfaction: Gather feedback from relevant stakeholders, such as product managers and executives, to assess their satisfaction with the bug bash outcomes and its impact on the product’s quality.
Acknowledgment and Rewards: Evaluate the effectiveness of acknowledging and rewarding participants for their contributions. Recognize individuals or teams who made significant contributions to the bug bash.
Continuous Improvement: Measure the extent to which the bug bash process is continuously improving over time. Are changes being implemented based on feedback and lessons learned?

Conclusion

A bug bash is a delightful and collaborative gathering centered around testing, where we unite in pursuit of a shared goal: ensuring top-notch quality. These bug bashes serve as a highly effective means of pinpointing and resolving issues before significant releases. Are you looking forward to the upcoming bug bash? Join us in conquering those bugs together!

Appreciate your time and attention!

Key Metrics to Assess the Effectiveness of Automation Testing

Francis Susairaj — Sat, 27 May 2023 00:00:00 GMT

Introduction:

Automation testing plays a crucial role in ensuring the delivery of a high-quality product. While the significance of automation tests is widely acknowledged, it is important to determine how to quantify their impact, assess their value in terms of effort and resources, and measure the success of test automation. Outlined below are metrics that can be employed to evaluate the effect of automation on the overall application quality, along with some best practices to generate data for these metrics.

1. Requirements Coverage

The quality of software heavily relies on the accuracy of its defined requirements by the business and project teams. Therefore, requirement coverage becomes a vital measure to determine the extent of application testing. By mapping test cases to requirements, teams can verify that all desired functionalities are adequately tested. This approach mitigates risks, facilitates traceability, and helps achieve the desired quality standards for the software. There are several approaches to calculate these figures, all aiming to ascertain whether all product features have been automated.

It is recommended to categorize automation tests based on the criticality of business flows, such as critical/high/medium/low.
Mapping automation tests to documented test cases enhances visibility and traceability of test execution with respect to product features.

2. Test Execution Duration

One key metric for measuring the success of automation is the runtime of tests. Total test duration quantifies the time required to execute automated tests. Although this metric does not directly measure quality, extensive and time-consuming automation suites can deter developers due to long waiting periods or time-consuming reporting. By understanding the factors that influence execution duration and implementing optimization strategies, teams can significantly improve the efficiency and speed of their automated testing processes. Prioritizing test cases, leveraging parallel execution, optimizing individual test cases, managing the test environment and data, and adopting continuous integration and continuous testing practices are all key steps towards achieving optimized automation test execution duration.

Consider breaking down larger test suites to achieve faster reporting while ensuring critical tests are not missed.
Parallel test execution should be contemplated during the framework design phase, taking into account appropriate design patterns, thread-safe data structures, and test data management.
Integrate automation tests into a continuous integration and continuous testing pipeline. By automating the execution of tests triggered by code changes, teams can identify defects earlier and reduce the overall execution duration.

3. Test Pass or Fail Percentage

This metric indicates the percentage of tests that have recently passed or failed in relation to the total planned tests. Analyzing the results of test suites over time provides an overview of testing progress and instills confidence in the automated tests. This metrics provides insights into the success rate of executed test cases and highlights areas that require attention or further testing. By considering factors such as test case quality, test data, test environment, and test execution strategy, organizations can optimize their testing processes and improve the pass or fail percentage. Regular monitoring and analysis of this metric contributes to overall software quality improvement, defect identification, and efficient resolution.

The objective should not be limited to achieving a 100% pass rate but also encompassing adequate coverage and quick turnaround time for analyzing failures.
Flaky tests are inevitable, often caused by factors like poor test data, environments, or coding practices. It is advisable to have separate configurations for rerunning or retrying such tests, ensuring test execution duration remains manageable.
A stable and consistent test environment is essential for reliable test results. Environmental issues, such as hardware or software inconsistencies, can lead to false failures and impact the overall pass or fail percentage.

4. Equivalent Manual Testing Efforts

EMTE measures the effort required to execute tests manually compared to automating them. This metric enables more accurate planning of release timelines and resource allocation. It serves as a metric and allows QA professionals to focus on other aspects of quality and testing. Understanding the factors influencing equivalent manual testing efforts helps organizations strike a balance between automation and manual testing. By identifying suitable test candidates, assessing risks, adopting an iterative approach, and continuously evaluating the testing strategy, organizations can optimize their testing efforts and ensure comprehensive software quality assurance.

It is essential to recognize that automation does not aim to replace manual testing entirely but rather saves time by avoiding repetitive tests for each release.
A significant portion of exploratory manual testing should still be conducted to add value to the build’s quality.

5. Script Maintenance Time

Every piece of automation test code requires maintenance, which may involve updating UI locators, adapting to changes in API contracts, addressing intermittent failures, or resolving discrepancies between local and Jenkins environments. This metric quantifies the time invested in the maintenance process, providing insights into the overall value of test automation for the project. Recognizing the factors that impact maintenance efforts and adopting effective strategies can help manage and optimize maintenance time. By emphasizing modular test design, maintaining comprehensive documentation, conducting regular code reviews, leveraging version control, integrating scripts into a CI pipeline, and investing in training, organizations can ensure sustainable and efficient automation script maintenance.

Adopt a modular and reusable test design approach. By structuring automation scripts in a modular manner, changes in individual components can be isolated and addressed more efficiently, minimizing the overall maintenance time.
If more time is spent on maintenance rather than writing new tests, it may be necessary to assess the framework design, coding practices, and testing environment.
It is advisable to allocate time to resolve issues and develop new tests to maintain confidence in the automated tests within the team.

Conclusion

In conclusion, the above-mentioned five points are common and self-explanatory, yet they are often overlooked. Additional metrics can be incorporated based on individual requirements, such as return on investment (ROI), defect distribution, build stability, and execution trends. These metrics should be applied to all layers of automation testing in the test pyramid, whether it involves unit tests, component tests, or end-to-end tests. While these metrics do not serve as quality gates for
the application under test, they reflect the value of automated tests, which ultimately impacts the overall product quality. Poor performance in these metrics may indicate gaps in automation practices, potentially impeding testing progress or creating a false impression that all cases are automated, leading to a decline in product quality.

Thanks for reading…!!!

Implementing dynamic filtering on joined queries using JPA specification and criteria API

Vishal Attarde — Mon, 27 Mar 2023 00:00:00 GMT

Introduction:

In most web applications, we would have come across a requirement to filter, sort, and paginate the data by joining multiple tables. If you are using Spring JPA then there are multiple ways to retrieve and paginate the data on joined table.

Writing native queries.
Using JPQL query.
Using inbuilt repository methods.

These options are good for the use case where we will have to construct the query based on a fixed set of conditions/where clauses. We can’t add where clause at runtime using these options.

Spring JPA also provides the capability to generate dynamic SQL queries with help of Criteria API and Specification. In this article, we will see how we can leverage JPA Criteria query support to build generic specifications which can retrieve rows from joins on multiple tables with sorting and pagination.

Context

Let us consider the following simple database schema:

Employee entity with basic details and Many-to-One association with Department entity.
Department entity with basic details and One-to-Many associations with Employee entity

Filtering on joined tables using criteria API

To illustrate how to use search and filter data based on conditions let’s consider the following implementation of the problem where we wanted to get employees from the specific department with a salary greater than 90k. Please refer to the comments in the code snippet for a better explanation.

/* Create tuple query */

CriteriaQuery<Tuple> tupleQuery = entityManager.getCriteriaBuilder().createTupleQuery();
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
Root<Employee> employeeRoot = tupleQuery.from(Employee.class);

/* Define specification for getting employees for which salary gte 90k and from "Engineering" dept */

Specification<Employee> specification = (root, query, criteriaBuilder) -> {

   /* Join tables Employee and Department. We will have to pass JoinType that can be INNER for inner joins,
LEFT for left outer join and RIGHT for right outer join */
   
   Join<Object, Object> joinDepartment = root.join("department", JoinType.INNER);
   Predicate employeeSalaryPredicate = criteriaBuilder.greaterThanOrEqualTo(root.get("salary"), 90);
   Predicate departmentNamePredicate = criteriaBuilder.equal(joinDepartment.get("name"), "Engineering");
   
   /*     We can select all columns from the join for that we will have to pass Join objects to multiselect().
      Alternatively, if we have to select individual columns then we can pass the expression with an alias as given below.
      e.g  query.multiselect(root.get("id").alias("employeeId"),root.get("salary").alias("employeeSalary"),joinDepartment.get("name").alias("deptName"));
   */
    
   query.multiselect(root, joinDepartment);
   return criteriaBuilder.and(employeeSalaryPredicate, departmentNamePredicate);
};

/* Apply specification. Get the predicate from the specification and use it in tupleQuery  */
Predicate predicate = specification.toPredicate(employeeRoot tupleQuery, builder);
tupleQuery.where(predicate);

/* Get TypedQuery from tupleQuery */
TypedQuery<Tuple> typedQuery = entityManager.createQuery(tupleQuery);

/* Get result */
List<Tuple> result = typedQuery.getResultList();

/* Retrieving column data or entity
   If all columns are selected in multiselect() function then we can get Entity object as shown in the code.
   If few columns expressions are provided to multiselect function then we can retrieve them as shown below:
   1: Using alias as provided in multiselect() function
      Integer employeeId = tuple.get("employeeId",Integer.class)

   2:Using index of the position in the result tuple
      Integer employeeId = tuple.get(0,Integer.class)
      Double employeeSalary = tuple.get(1,Double.class)
 */
 
Tuple tuple = result.get(0);
Employee emp = tuple.get(0, Employee.class);
Department dept = tuple.get(1, Department.class);

Dynamic queries with specification:

In this section, we will build a filtering model which can allow us to create any level of nested AND/OR queries by leveraging JPA criteria query and specification support.

Define the filter object to capture details of the filter such as the field on which the filter is to be applied, operator, value, and join identifier to get the correct path of the attribute.
Define the AbstractFilter class with a method to convert the filter to predicate with the help of criteria query support. AbstractFilter class is extended by classes that are
- SimpleFilter - This will be used for creating a single filter.
- AndFilter - This will be used to define the conjunction of multiple filters of any type.
- OrFilter - This will be used to define the disjunction of multiple filters of any type.
We will define a function to convert the Filter to a predicate. We will need information about all the joins to get the correct attribute path. This function will create the predicate as per the filter operator. Please refer to the method getPredicateFromFilter from the AbstractFilter class.
Let’s define GenericSpecification class which converts complex filters to predicates. This class can be used for creating a specification for any root entity.
Implementation:

public class Filter {

   /* Name of the variable from Entity class on which filter has to be applied */
   String field;
   
   /* Filter operator */
    FilterOperator operator;
    
   /* Filter value */
    Object value;
    
   /* Join identifier */
    String entityName;
}

public abstract class AbstractFilter {

    public abstract Predicate toPredicate(Root root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder, Map<String, Join<Object, Object>> attributeToJoin);

    public Predicate getPredicate(Filter filter, CriteriaBuilder criteriaBuilder, Path expression) {

        Predicate predicate = null;
        switch (filter.getOperator()) {
            case EQUAL_TO:
                predicate = criteriaBuilder.equal(expression, filter.getValue());
                break;
            case LIKE:
                predicate = criteriaBuilder.like(expression, "%" + filter.getValue() + "%");
                break;
            case IN:
                predicate = criteriaBuilder.in(expression).value(filter.getValue());
                break;
            case GT:
                predicate = criteriaBuilder.greaterThan(expression, (Comparable) filter.getValue());
                break;
            case LT:
                predicate = criteriaBuilder.lessThan(expression, (Comparable) filter.getValue());
                break;
            case GTE:
                predicate = criteriaBuilder.greaterThanOrEqualTo(expression, (Comparable) filter.getValue());
                break;
            case LTE:
                predicate = criteriaBuilder.lessThanOrEqualTo(expression, (Comparable) filter.getValue());
                break;
            case NOT_EQUAL:
                predicate = criteriaBuilder.notEqual(expression, filter.getValue());
                break;
            case IS_NULL:
                predicate = criteriaBuilder.isNull(expression);
                break;
            case NOT_NULL:
                predicate = criteriaBuilder.isNotNull(expression);
                break;
            default:
                log.error("Invalid Operator");
                throw new IllegalArgumentException(filter.getOperator() + " is not valid operator");
        }
        return predicate;
    }

    public Predicate getPredicateFromFilter(Filter filter, Root root, CriteriaBuilder criteriaBuilder, Map<String, Join<Object, Object>> attributeToJoin) {
        Assert.notNull(filter,"Filter must not be null");
        if (attributeToJoin != null && attributeToJoin.get(filter.getEntityName()) != null) {
            return  getPredicate(filter, criteriaBuilder, attributeToJoin.get(filter.getEntityName()).get(filter.getField()));
        } else {
            return getPredicate(filter, criteriaBuilder, root.get(filter.getField()));
        }
}

/* SimpleFilter */
public class SimpleFilter extends AbstractFilter {

    private Filter filter;

    @Override
    public Predicate toPredicate(Root root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder, Map<String, Join<Object, Object>> attributeToJoin) {
        return getPredicateFromFilter(filter,root,criteriaBuilder,attributeToJoin);
    }
}
-----------------------------------------------------

/* AndFilter */
public class AndFilter extends AbstractFilter{

    private List<AbstractFilter> filters;

    @Override
    public Predicate toPredicate(Root root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder, Map<String, Join<Object, Object>> attributeToJoin ) {
        return criteriaBuilder.and(filters.stream().map(filter -> filter.toPredicate(root,query,criteriaBuilder,attributeToJoin)).collect(Collectors.toList()).toArray(Predicate[]::new));
    }
}

-------------------------------------------------------

/* OrFilter */
public class OrFilter extends AbstractFilter {

    private List<AbstractFilter> filters;

    @Override
    public Predicate toPredicate(Root root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder, Map<String, Join<Object, Object>> attributeToJoi) {
       return criteriaBuilder.or(filters.stream().map(filter -> filter.toPredicate(root,query,criteriaBuilder,attributeToJoi)).collect(Collectors.toList()).toArray(Predicate[]::new));
    }
}

public class GenericSpecification<T> implements Specification<T> {

   private AbstractFilter filter;
   private JoinDataSupplier<T> joinDataSupplier;

   @Override
   public Predicate toPredicate(Root<T> root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder) {

      if (joinDataSupplier != null && filter !=null) {
         return filter.toPredicate(root, query, criteriaBuilder, joinDataSupplier.getJoinData(root,query));
      }
      return criteriaBuilder.conjunction();
   }
}

Example

Let’s consider a hypothetical example where we need to retrieve employee data whose salary is greater than 90k from cost center “CostCenter-1” or employee data whose salary is greater than 80k from cost center “CostCenter-2”. This query requires a join between Employee and Department tables. Let’s create a specification from the filter and join data. We can use the same tuple-typed query as explained in Section 2 to get the results for the given specification. Please refer to the following code snippet for the same.

public Specification<Employee> getEmployeeDepartmentJoinSpecification(AbstractFilter filter) {

   GenericSpecification<Employee> serviceOrderSpecification = new GenericSpecification<Employee>();
   serviceOrderSpecification.setFilter(filter);
   serviceOrderSpecification.setJoinDataSupplier(new JoinDataSupplier<Employee>() {
      @Override
      public Map<String, Join<Object, Object>> getJoinData(Root<Employee> root, CriteriaQuery<?> query) {

         Map<String, Join<Object, Object>> attributeToJoinMap = new LinkedHashMap<>();
         Join<Object, Object> joinDepartment = root.join("department", JoinType.INNER);
         attributeToJoinMap.put("Department", joinDepartment);
         query.multiselect(root.get("id"), root.get("firstName"), joinDepartment.get("name"));
         return attributeToJoinMap;
      }
   });
   return serviceOrderSpecification;
 }


//Creating filter

SimpleFilter salaryFilter = new SimpleFilter(new Filter("salary",FilterOperator.GTE,90,"Employee"));
SimpleFilter departmentNameFilter = new SimpleFilter(new Filter("costCenter",FilterOperator.EQUAL_TO,"CostCenter-1","Department"));
AndFilter firstAndFilter = new AndFilter(Arrays.asList(salaryFilter,departmentNameFilter));

SimpleFilter salaryFilter1 = new SimpleFilter(new Filter("salary",FilterOperator.GTE,80,"Employee"));
SimpleFilter departmentNameFilter1 = new SimpleFilter(new Filter("costCenter",FilterOperator.EQUAL_TO,"CostCenter-2","Department"));
AndFilter secondAndFilter = new AndFilter(Arrays.asList(salaryFilter1,departmentNameFilter1));
OrFilter orFilter = new OrFilter(Arrays.asList(firstAndFilter,secondAndFilter));

/* we can use this specifcation for tuple query */
Specification<Employee> specification = getEmployeeDepartmentJoinSpecification(orFilter);

The query generated from the above filters:

select e1_0.id, e1_0.department_id, e1_0.email_id, e1_0.first_name, e1_0.last_name, e1_0.salary, d1_0.id, d1_0.cost_centre, d1_0.name from employee e1_0 join department d1_0 on d1_0.id=e1_0.department_id where e1_0.salary>=? and d1_0.cost_centre=? or e1_0.salary>=? and d1_0.cost_centre=?

We can generate any combination of AND/OR queries with this filtering model. e.g: [ (A or B) and (U or V)]

Pagination and Sorting

Adding sorting to the query

We can make use of QueryUtils.java class from the spring-data-jpa library. This gives util method to get an Order object which can be applied to a tuple query.

tupleQuery.orderBy(org.springframework.data.jpa.repository.query.QueryUtils.toOrders(sort, root, builder));

Adding pagination to the query

There is no direct support for pagination when we are dealing with criteria queries, but we can still achieve this with the help of spring data Pageable and Page interface. PageRequest is an implementation of Pageable which we will use here to provide pagination metadata like page number, page size, and sorting metadata. We can make use of the PageableExecutionUtils.java class from the spring-data-jpa library to get the paged data from a query. We will also need to know the total number of rows that can be retrieved with given filters, so we will have to create a count criteria query. Please refer to the following code snippet for the implementation details, this implementation is inspired by SimpleJpaRepository.java from spring-data-jpa library.

/* Function to get paged data */
public <S> Page<Tuple> getPage(TypedQuery<Tuple> query, final Class<S> rootClass, Pageable pageable, Specification<S> spec) {
   if (pageable.isPaged()) {
      query.setFirstResult((int) pageable.getOffset());
      query.setMaxResults(pageable.getPageSize());
   }
   return PageableExecutionUtils.getPage(query.getResultList(), pageable,() -> executeCountQuery(getCountQuery(spec, rootClass)));
}

/* Function to execute count query */
public long executeCountQuery(TypedQuery<Long> countQuery) {
   List<Long> totals = countQuery.getResultList();
   long total = 0L;
   for (Long element : totals) {
      total += element == null ? 0 : element;
   }
   return total;
}

/* Function to create count query */
protected <S> TypedQuery<Long> getCountQuery(Specification<S> spec, Class<S> rootClass) {
   CriteriaBuilder builder = entityManager.getCriteriaBuilder();
   CriteriaQuery<Long> query = builder.createQuery(Long.class);
   Root<S> root = query.from(rootClass);
   Predicate predicate = spec.toPredicate(root, query, builder);
   if (predicate != null) {
      query.where(predicate);
   }
   query.select(builder.count(root));
   query.orderBy(Collections.<Order> emptyList());
   return entityManager.createQuery(query);
}

Conclusion

We can construct any level of nested queries with the help of the above filtering model and specification. This implementation will not be helpful if the query has to be generated on properties from JSON type columns.You can find the working code here

Thanks for reading…!!!

Performance testing with Gatling

Sandal Iqbal — Tue, 21 Mar 2023 00:00:00 GMT

Measuring the performance of an application is critical to business expansion and growth. One of the ways to achieve this is through load and performance tests. Load Testing ensures that your application can perform as expected in production. Just because your application will pass a functional test, this does not mean that it can perform the same under a load. Load testing identifies where and when your application breaks, so you can fix the issue before shipping to production.

What is load testing?

Load testing is a mechanism which helps us to identify the performance bottleneck in a website and take corrective measures which brings in a positive user experience. Be it a micro-service, a REST service or a website, load testing can help to identify the culprit bringing down the performance of the application and give an indication of optimal resources required to run smoothly.

What factors influence the performance of a website?

When we have a visitor on a website his expectation is that the site loads quick enough without having to wait for a long time. This basically depends on the response time to load different contents of the web page. This response time depends on two factors:

The time taken by the frontend browser to load and display elements on the web page, and
The time taken by the backend server to give response to a request These affect the performance of the website.

When should you perform load testing?

Ideally we should periodically keep monitoring the performance of our website to ensure the performance does not degrade over time. Most suitably whenever a new service or an application is being developed its performance should be measured and it should be released to production only when it meets the performance numbers expected from the application and all the performance parameters are within predefined threshold values.

How load testing works?

Load testing is focused on heavy load simulation. A load generator creates virtual users by sending requests to a server. These requests are increased in number over a period of time and response time of the server is measured. From the metrics collected we get an idea about the performance of the system and the max load that the system can handle.

What is Gatling?

Gatling is a powerful open-source load testing tool. It has excellent support for http protocol which makes it very efficient to test the performance of any http server. It has expressive DSL that makes it possible to write scenarios that are easy to understand. Its underlying architecture is asynchronous which makes it possible to implement virtual users as messages rather than as dedicated threads hence making it possible to use system resources very efficiently and running thousands of concurrent users without any issue.

How to perform load testing with Gatling?

Writing a performance test is very similar to writing any other automation test. The test class in which we are going to define our performance test must extend Simulation class of Gatling. Simulation is the parent class your tests must extend so Gatling can launch them. We define our test class like so:

public class GatlingTest extends Simulation {


}

Now within this class we define the rest of the elements of the performance test. To test the performance of an application we need to build a scenario which represents the action that a user would perform on the website. So we create a scenario like below which is sending an http get request to the server:

scenario("Scenario")
  .exec(http("Home").get("https://gatling.io"));

The exec method is used to execute an action. In this case it is sending an http get request.

Scenarios are usually built with a ScenarioBuilder.

ScenarioBuilder scn = scenario("Scenario").exec(http("Home").get("https://gatling.io"));

The next important class is HttpProtocolBuilder using which we define various elements of the http request that we are going to send for eg. the base url, user agent, accept headers etc. We define it like this:

HttpProtocolBuilder httpProtocol = http.baseUrl("https://gatling.io”)
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
.acceptLanguageHeader("en-US,en;q=0.5")
.acceptEncodingHeader("gzip, deflate”);

ScenarioBuilder and HttpProtocolBuilder are the main and necessary elements to get started with performance test. Once these are defined we use the setUp method. This is a mandatory piece in your Simulations. It is essential to call the setUp method exactly once in the constructor to register the test components. This has to be done like this:

setUp(
  scn.injectOpen(atOnceUsers(1))
).protocols(httpProtocol);

Now the complete simulation looks like this:

public class GatlingTest extends Simulation {

	ScenarioBuilder scn = scenario("Scenario").exec(http("Home").get("https://gatling.io"));
	HttpProtocolBuilder httpProtocol = http.baseUrl("https://gatling.io”)
		.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
		.acceptLanguageHeader("en-US,en;q=0.5")
		.acceptEncodingHeader("gzip, deflate”);

	setUp(
		scn.injectOpen(atOnceUsers(1))
	).protocols(httpProtocol);
}

This will inject 1 user into the scenario. They’re different injection profiles for different use cases:

injectOpen: to specify the arrival rate of users with no control on number of concurrent users
injectClosed: to specify and control concurrent number of users

Some examples of injectOpen:

scn.injectOpen(
    nothingFor(4), 
    atOnceUsers(10), 
    rampUsers(10).during(5), 
    constantUsersPerSec(20).during(15)
)

Some examples of injectClosed:

scn.injectClosed(
    constantConcurrentUsers(10).during(10), 
    rampConcurrentUsers(10).to(20).during(10)
  )

We need to choose the right injection profile based on the website usage. Once the simulation is defined we can execute it in many ways. One of the ways is to use io.gatling.gradle plugin and then run the simulations from commandline using gradle:

gradle gatlingRun

The simulation will run and at the end a beautiful performance test report is generated.

Hope this helps in getting started with performance testing.

Unraveling Patterns: Exploring the Fascinating World of Clustering Algorithms

Sourabh Kashyap — Thu, 09 Mar 2023 00:00:00 GMT

Clustering is a popular technique in machine learning used for grouping data points based on their similarities. It is a type of unsupervised learning method where there is no predefined output variable or label. Instead, the algorithm attempts to discover patterns and structure within the data by grouping similar data points.

What is Clustering? Clustering is a process of grouping a set of data points in such a way that data points in the same group (called a cluster) are more similar to each other than those in other groups. The goal of clustering is to identify patterns or structures within the data that are not immediately apparent.

Classification of Clustering Algorithms

Clustering algorithms can be broadly classified into two categories based on the approach the algorithm takes to group the data.

Hierarchical Clustering
Partitioning Clustering

Both of the classifications are explained below.

Hierarchical Clustering

In hierarchical clustering, clusters are formed by recursively dividing the data into smaller clusters until each cluster contains only one data point. There are two types of hierarchical clustering: agglomerative and divisive.

Agglomerative Clustering begins with each data point as a separate cluster and at each iteration, we merge the closest pair of clusters and repeat this step until only a single cluster is left. Basically, it uses a bottom-up approach. In hierarchical clustering, we have a concept called as proximity matrix which stores the distance between each point. Multiple distance metrics can be used for deciding the closeness of two clusters.
- Euclidean distance: distance((x, y), (a, b)) = √((x — a)² + (y — b)²)
- Manhattan Distance: distance (x1, y1) and (x2, y2) = |x1 — x2| + |y1 — y2|
Divisive Clustering or the top-bottom approach works in the opposite way. Instead of starting with n clusters, it starts with a single cluster and assigns all the points to that cluster. Therefore, it doesn’t matter if we have 10 0r 1000 data points, all these data points will belong to the same cluster at the beginning. Now at each iteration, we split the farthest point in the cluster and repeat the process until each cluster contains only a single data point.

Partitioning Clustering

In partitioning clustering, the data is divided into non-overlapping clusters by minimizing the distance between the data points within each cluster and maximizing the distance between different clusters. Popular clustering algorithms include several partitioning methods such as k-means, k-medoids, fuzzy c-means etc, working of few of them are explained below:

K-Means Clustering: K-Means is a widely used clustering algorithm that groups data points into K clusters. The algorithm randomly selects K data points as centroids and then assigns each data point to the nearest centroid. The centroids are then updated by taking the mean of all the data points assigned to them. The K number of means obtained will become the new centroids for each cluster. The process is repeated until convergence. You have to specify the number of clusters — k as an input to the algorithm.

_src:https://upload.wikimedia.org/wikipedia/commons/e/ea/K-means_convergence.gif_

Hierarchical Clustering: Hierarchical clustering is a bottom-up approach that creates a hierarchical structure of clusters. The algorithm starts by assigning each data point to its own cluster and then iteratively merges the closest pairs of clusters until all data points are in one cluster.
Density-Based Clustering: This is a non-parametric clustering method that groups the points which are closely located (high density of points). Density-based clustering is based on the idea that clusters are dense regions of data points separated by areas of lower density. The algorithm identifies areas of high density and expands them until the density falls below a threshold. The resulting regions are clusters.

The algorithm takes the following inputs: * Epsilon Parameter — ε: Degree of the closeness of points in the same cluster. It is also referred to as the radius of the circle created around each point to check the density * MinPts: Minimum number of points required inside the circle to be classified as a core point

_src:https://miro.medium.com/proxy/1*tc8UF-h0nQqUfLC8-0uInQ.gif_

Expectation-Maximization Clustering (Gaussian Mixture Models): Expectation-Maximization (EM) is a probabilistic clustering algorithm that models each cluster as a probability distribution. The algorithm iteratively assigns data points to clusters based on their probabilities and updates the parameters of the probability distributions.

_src:https://upload.wikimedia.org/wikipedia/commons/6/69/EM_Clustering_of_Old_Faithful_data.gif_

From Manual Planning to Smart Clustering: How We Transformed Our Delivery and Collection Workflow

Here, we will give a bit of insight into how we leveraged the clustering to provide better suggestions to our back-office agents to schedule Delivery and Collection orders of vehicles.

Each day we have several delivery and collection orders coming into our pipeline and the branch manager has to manually go and plan a tour and assign an agent to these orders, hence this planning part is an additional burden on them. We have taken a step in the direction of automatic tour planning by leveraging the power of clustering, we try to generate intelligent tour suggestions that the branch manager can review and import.

Our tour generation takes distance and time as the clustering parameter along with our custom set of rules to further refine the suggestions.

We will further keep on refining our recommendations and also try to generate tours based on the available manpower. Also will plan to incorporate an optimized route for a particular tour.

We’re just getting started! stay tuned for updates about the new cool features around the recommendations……..

adios…

Introduction To Object Relational Mapping in Java

Abhay Shalghar — Mon, 20 Feb 2023 00:00:00 GMT

Programmers using object-oriented programming languages often struggle with integrating the database structure with their code because relational databases use tables to represent data, while object-oriented languages like Java use connected objects.

OOP developers also face the challenge of connecting their application to a relational database using structured query language (SQL), which can be time-consuming and require understanding of raw SQL coding. SQL query builders provide a layer of abstraction that helps to simplify the process and provide more information about the data.

Object-Relational Mapping (ORM) is a programming technique that enables software developers to work with object-oriented programming languages, such as Java, Python, or Ruby, to interact with relational databases, such as MySQL, Oracle, or Microsoft SQL Server.

ORM is an abstraction layer that helps to bridge the gap between the object-oriented programming paradigm and the relational database model. It allows developers to use objects to represent database entities, such as tables, rows, and columns, and to manipulate them more naturally and intuitively.

By using ORM, developers can reduce the amount of boilerplate code required to interact with databases, avoid writing complex SQL queries manually, and improve the maintainability and scalability of their software applications. Additionally, ORM provides a higher level of security and protection against SQL injection attacks, as it automatically escapes user input and parameterizes queries.

ORM has become a popular technique in modern software development, as it helps to simplify database access, enhance code readability and maintainability, and increase developer productivity. In this context, many ORM frameworks have been developed, such as Hibernate, JPA, Django ORM, or SQLAlchemy, which provide a set of tools and utilities to facilitate the integration of object-oriented programming languages with relational databases.

Object Relational Mapping with SpringBoot

Spring Boot is a popular Java framework for building web applications. It comes with many powerful features, including support for Object-Relational Mapping (ORM) through various ORM libraries such as Hibernate and Spring Data JPA. In this context, let’s discuss Spring Boot’s Object-Relational Mapping support using Spring Data JPA.

Spring Data JPA is a subproject of the Spring Data project that simplifies database access using the Java Persistence API (JPA). Spring Data JPA provides a powerful way to map Java objects to relational databases, and it does so using a repository abstraction that greatly simplifies database access. Here are the steps to use Spring Data JPA for object-relational mapping in a Spring Boot application:

1. Configure the database

The first step in using Spring Data JPA is to configure the database connection. This can be done in the application.properties file or application.yml file. Here’s an example:

spring.datasource.url=jdbc:mysql://localhost:3306/mydb
spring.datasource.username=root
spring.datasource.password=1234
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.jpa.hibernate.ddl-auto=update

This configuration file sets the MySQL database connection details such as URL, username, password, and driver class name. It also sets the Hibernate DDL auto property to update the database schema automatically.

2. Create an entity class

The next step is to create a Java class that represents an entity in the database. An entity class typically has properties that map to columns in a database table. Here’s an example:

@Entity
@Table(name = "employees")
public class Employee {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "first_name")
    private String firstName;

    @Column(name = "last_name")
    private String lastName;

    @Column(name = "email")
    private String email;

    // getters and setters
}

In this example, the Employee class is annotated with @Entity, which tells Spring Data JPA that this class is an entity. The @Table annotation is used to specify the name of the database table that this entity maps to. The @Id annotation specifies the primary key of the entity, and the @GeneratedValue annotation is used to generate the primary key automatically. The @Column annotation is used to map properties to columns in the database table.

3. Create a repository interface

The next step is to create a repository interface that extends the JpaRepository interface. This interface provides methods for common CRUD operations such as save(), findById(), findAll(), and deleteById(). Here’s an example:

public interface EmployeeRepository extends JpaRepository<Employee, Long> {
}

In this example, the EmployeeRepository interface extends the JpaRepository interface and specifies the Employee entity class and the primary key type as Long.

4. Use the repository

Now that the entity and repository are defined, you can use them in your Spring Boot application. Here’s an example of using the repository to save and retrieve an employee:

public class EmployeeService {
    @Autowired
    private EmployeeRepository employeeRepository;
    
    public Employee save(@RequestBody Employee employee) {
        return employeeRepository.save(employee);
    }
    
    public Employee findById(@PathVariable Long id) {
        return employeeRepository.findById(id).orElse(null);
    }
}

In this example, the EmployeeService is a service class that uses the EmployeeRepository to perform CRUD operations on the Employee entity. The @Autowired annotation is used to inject the EmployeeRepository instance into the service. The save(), findById(), findAll(), and deleteById() methods are implemented using methods provided by the JpaRepository interface.

Overall, Spring Boot provides a convenient way to perform ORM using Spring Data JPA. By following the above steps, you can easily map Java objects to tables in a relational database.

Advantages of Using ORMs

Reduced development time: ORM frameworks provide a way to easily map objects to database tables, eliminating the need for developers to write complex SQL queries manually. This can significantly reduce the time required for database integration and data persistence implementation.
Increased productivity: With the help of ORM frameworks, developers can focus more on the business logic of the application and less on the database operations, which can increase their productivity.
Improved maintainability: ORM frameworks provide a clean separation between the application’s business logic and the persistence layer, making it easier to maintain and modify the application code.
Improved performance: ORM frameworks provide caching and optimization mechanisms that can improve application performance by reducing the number of database queries.
Database independence: ORM frameworks provide a way to write database-independent code, which means that the same code can be used with different types of databases without any modifications.
Type safety: With the help of ORM frameworks, developers can write type-safe queries, reducing the possibility of runtime errors and improving code quality.
Simplified database schema evolution: ORM frameworks provide support for database schema evolution, allowing developers to modify the database schema without affecting the application’s code.

Disadvantages of Using ORMs

While there are many benefits to using Object-Relational Mapping (ORM) frameworks like Hibernate, JPA, and Entity Framework, there are also some potential drawbacks to consider. Here are some of the main disadvantages of using ORM:

Performance overhead: ORM frameworks can add performance overhead to the application, as they often generate complex SQL queries that can be slower than manually optimized queries.
Limited control over SQL queries: ORM frameworks generate SQL queries automatically, which means that developers have limited control over the queries that are executed. This can lead to suboptimal queries that do not take advantage of database-specific features.
Complexity: ORM frameworks can be complex to configure and use, especially for developers who are not familiar with the framework. This can result in longer development times and more difficult maintenance.
Learning curve: Developers may need to learn a new set of APIs and conventions to work with the ORM framework, which can result in a steep learning curve.
Increased memory usage: ORM frameworks often use caching mechanisms to improve performance, which can result in increased memory usage.
Dependency on the ORM framework: Applications that use an ORM framework are often tightly coupled to the framework, which can make it difficult to switch to a different framework or use native SQL queries.
Limited support for complex queries: ORM frameworks are not always well-suited for complex queries that require custom SQL, which can be a limitation for applications that require advanced querying capabilities.

Conclusion

In conclusion, Object-Relational Mapping (ORM) frameworks have become a popular solution for software development projects that require data persistence. By providing a way to easily map objects to database tables, ORM frameworks like Hibernate, JPA, and Entity Framework help developers to write cleaner, more maintainable code and reduce the time required for database integration.

While there are some potential drawbacks to using ORM, such as performance overhead and limited control over SQL queries, the benefits of using ORM can outweigh the disadvantages, especially for applications with a high degree of data persistence requirements. Overall, Java developers can leverage the power of ORM frameworks in conjunction with Spring Boot to build efficient and scalable applications that meet the complex demands of modern software development.

Efficient Data Management in React with React Query

Vanmeeganathan P K — Mon, 30 Jan 2023 00:00:00 GMT

React Query is a lightweight library for fetching, caching, and updating asynchronous data in React applications. It is designed to work with a variety of data sources, including REST APIs and GraphQL endpoints.

It is used for managing, caching, and synchronizing asynchronous data in React applications. It makes it easy to work with asynchronous data and API requests in React.

React Query provides a set of hooks that can be used to fetch, cache, and update data in your application. You can use the useQuery hook to execute a query and subscribe to updates, and the useMutation hook to execute mutations (i.e., updates) to the data. It also provides several configuration options to customize the behavior of the library to fit your specific needs.

The latest version of React Query is renamed as TanStack Query and it has support for Solid, Svelte, and Vue as well. However, we will stick to React for this blog.

React Query - How it works?

We begin, by defining a query by calling a hook (e.g. useQuery) and passing it a unique key representing the data to be fetched.
The hook will return a tuple containing three elements: a status (such as “loading”, “error”, or “success”), the data that was fetched, and a function to refetch the data.
If the data is already stored in the cache, it will be returned immediately. Otherwise, React Query will make a request for the data and store it in the cache for future use.
The status and data returned by the hook can be used in your React component to render the UI. For instance, you could show a loading spinner while the data is being fetched or display an error message if there was a problem.
If the data changes, React Query will automatically update the cache and re-render the component with the new data.
In addition, React Query offers several other features, including cache invalidation, deduplication of queries, and the ability to cancel ongoing requests.

Setting up React Query in a React application

To incorporate React Query into a React app, you’ll need to install the library. This can be done by running the following command in the terminal:

npm i @tanstack/react-query

After installation, follow the below steps:

Create a client object for making HTTP requests to your API using a library like axios or fetch.
Wrap your top-level component with the QueryClientProvider component and pass in the client object from step 1. This will provide the client object to all components in your app.

import { QueryClientProvider, QueryClient } from '@tanstack/react-query';

const client = QueryClient();
const App = () => (
  <QueryClientProvider client={client}>
    {/* Your app goes here */}
  </QueryClientProvider>
);

Now that you’ve set up React Query, you can use the useQuery and useMutation hooks to fetch and update data in your React app.

useQuery Hook

Queries are used to request data from an asynchronous source, such as a server, by providing a unique key and a function that returns a promise.
The useQuery hook allows a query to be subscribed to in a component or custom hook.
The query result returned by useQuery contains information about the query’s status, data, and any errors that occurred.
Queries can be in a loading, success, or error state and also have a fetchStatus of fetching, paused, or idle.
Queries can be refetched manually or automatically and can also be canceled to stop unnecessary data fetching.

Consider the below example :

import { useQuery } from '@tanstack/react-query';

const fetchTodoList = async () => {
    const response = await fetch('/api/todos');
    return response.json();
};
function MyComponent() {
    const { isLoading, isError, data, error } = useQuery({
        queryKey: ['uniqueIdentifier'],
        queryFn: fetchTodoList,
    })
    if (isLoading) {
        return <span>Loading...</span>
    }
    if (isError) {
        return <span>Error: {error.message}</span>
    }
    return (
        <ul>
            {data.map((todo) => (
                <li key={todo.id}>{todo.title}</li>
            ))}
        </ul>
    )
}

The useQuery hook accepts two arguments:-

A unique identifier for the query (in this case, a string ‘uniqueIdentifier’)
A function that performs the actual query.

When the component containing the useQuery hook is rendered, the hook will execute the query function and return the data, error, and loading state.

The component can then use these values to render the appropriate UI based on the current state of the query.

useMutation Hook

Mutations are used to create/update/delete data or perform server side-effects. For this purpose, React Query exports a useMutation hook.

import { useMutation } from '@tanstack/react-query';

function AddTodo() {
    const mutation = useMutation({
        mutationFn: (newTodo) => {
            return axios.post('/todos', newTodo)
        },
    })
    return (
        <div>
            {mutation.isLoading ? (
                'Adding todo...'
            ) : (
                <>
                    {mutation.isError ? (
                        <div>An error occurred: {mutation.error.message}</div>
                    ) : null}
                    {mutation.isSuccess ? <div>Todo added!</div> : null}
                    <button
                        onClick={() => {
                            mutation.mutate({ id: new Date(), title: 'Do Laundry' })
                        }}
                    >
                        Create Todo
                    </button>
                </>
            )}
        </div>
    )
}

In the above code,

The AddTodo function, which is defined in this code, has a button that, when clicked, sends a POST request to an API endpoint to add a to-do item.
The useMutation hook takes a function that returns a promise as an argument. useMutation hook returns a function that you can call to execute the mutation.
We then use this function as the onClick handler for the button element.
When the button is clicked, the addTodo function will be called and the mutation will be executed.
The useMutation hook will handle the loading and error state for the mutation automatically.

Query Invalidation

Often while working with complex front-end applications, we would encounter stale data, and the need to refresh them.
This can be done by invalidating the query.
The QueryClient object has an invalidateQueries object which can be used to mark queries as stale and also refetch the same.

The below code will invalidate all the queries in the application.

queryClient.invalidateQueries();

The below code will invalidate every query with a key that starts with todos.

queryClient.invalidateQueries();

Benefits of Using React Query for Data Management

Some of the benefits of using React Query include:

Declarative API:

React Query provides a declarative API for managing data and removes the need for managing data fetching and caching manually.

Automatic cache invalidation:

React Query handles cache invalidation and removes the need for manually invalidating the cache when data changes.

Refetching on the interval:

React Query allows to specify an interval for refetching data to keep the data up-to-date in the application.

Concurrent mode support:

React Query works well with React’s concurrent mode, allowing one to fetch data in the background while rendering the UI.

Suspense support:

React Query can be used with React’s Suspense feature to manage loading states in your application.

Conclusion

React Query helps simplify the process of managing data in a React application. It makes it easier to build performant and scalable applications.

Happy Coding !

What is Swagger, and how does it work?

Ajay Kumar Singh — Wed, 11 Jan 2023 13:00:00 GMT

In today’s software development world, the software development lifecycle depends on APIs. If you are a software developer, you must be integrating APIs into your application.

APIs (Application Programming Interfaces) allow us to expose the data and functionality to the public for use, and then we can add that data to the applications we build.

Swagger is one of the most popular tools for web developers to document REST APIs.

How does Swagger work?

Before we get into Swagger, let’s take a quick look at API documentation and the OpenAPI specification as they relate to Swagger.

API Documentation Overview

An API acts as a bridge between an application and a web server, orchestrating communication between the two. Behind every API is a complex layer of information, like API calls, data formats, requests, and syntax. Therefore, well documenting your API is essential to ensuring a positive developer experience, such as with code examples, screenshots, syntax references, and other pertinent information.

API documentation is like a primary resource containing instructions explaining what is possible with your API and how to effectively use and integrate with it. It also serves as a place for developers to return with questions about syntax or functionality. The best API documentation has all these answers; that’s why documentation is valuable. Let’s understand what you should include in the API documentation.

Your documentation must completely describe the API’s functionality, be accurate and educational, and inspire usage. API documentation should include the following:

• An overview of the API

• Guides and tutorials (step-by-step examples)

• Examples and use cases

• Operations and endpoint support

• The response an API returns from a request

OpenAPI Specification Overview

The OpenAPI specification (previously known as the Swagger specification) defines a standard, programming language-agnostic interface for REST APIs. The standard is important so that the developer who writes REST APIs is compliant with its best practices, such as versioning, safety, and error handling. So we can say that OpenAPI is similar to a template with a set of rules and constraints that explain how you could describe an API. It is usually written in YAML or JSON file format and is readable by humans and machines.

What is Swagger?

Swagger is a set of open-source tools for writing REST-based APIs. If you have to write an API from scratch, you probably have to question if you’ve included all the necessary information that a user will need to be successful. Swagger simplifies API documentation and saves time by running it through the OpenAPI specification to ensure compliance.

The tool is built around the OpenAPI specification and contains three components:

• Swagger Editor,

• Swagger UI,

• Swagger Codegen.

Now the OpenAPI specification is known as the Swagger specification. The difference now is that OpenAPI is the instruction, and Swagger is the implementation of those instructions. So, Swagger provides the tools for implementing the OpenAPI specification. And both the Swagger and OpenAPI specifications describe the structure of the REST API, so machines can mock them. The API documentation process is a lot easier for the developer to generate and maintain, all because of the automations made by OpenAPI and Swagger.

Swagger API Examples and Features

In today’s software realm, there are no systems running online without exposing an API. We have moved from monolithic systems to microservices. And the whole design of microservices is based on REST APIs.

Let’s understand the features of each Swagger component: Swagger Editor, Swagger UI, and Swagger Codegen.

Swagger Editor

The Swagger Editor is an open-source browser-based editor where you can design and edit the API documentation and OpenAPI specifications. Swagger Editor can be downloaded, installed, and run locally; accessed through the browser; or used with a host version such as SwaggerHub. Below are some key features of the Swagger editor.

The Swagger editor is a convenient tool that addresses this issue. We can write up our documentation in YAML or JSON, and it can automatically get compared against the Swagger spec. All the mistakes get flagged, and we can see all the suggested alternatives. This way, when we publish our documentation, we can be sure that it’s error-free.

Swagger Editor has an editor on the left panel where we can plug in all of our requests and response data. The editor supports the YAML or JSON format. We can view the documentation on the right panel, just as the end-user will see it.

live bug report

If we are getting any errors, the live error report will share the row that we need to adjust in order to be compliant with the specifications.

Autocomplete Option

If we don’t want to rewrite every time we have a new GET method, using this autocomplete option, we can build our documentation faster.

Ease of use

As an open-source tool, Swagger Editor allows others to fully customise documentation to meet their own needs and requirements, making it easy to use.

Below is an example view of the Swagger Editor. Take a look around to familiarise yourself with its features.

image source

What we like:

The Swagger Editor makes it convenient for us to design our specifications before we dive into the coding. The editor tells us exactly what the API will ask for what the request will look like, and what response we can expect. This way, we already have a template to follow, which can help reduce future errors and overall coding time.

Swagger UI

After creating the documentation with the help of an editor, we need a way to share it with the users. Swagger UI displays OpenAPI specifications as interactive API documentation. It takes the YAML file and converts it into user-facing documentation that lets our users try out the API calls directly in the browser. Some key features include:

Easily Integrated

Swagger UI easily integrates with existing and new applications.

Flexible Setup

Swagger UI can be run in several ways, such as in the cloud, locally, as node packages, etc.

Interactive

Swagger UI has a “try it out” button that turns the query parameters into fields. This allows us to make a call against an actual API.

Review the following example of the Swagger UI display:

image source

Swagger Codegen

So far, we’ve created interactive API documentation with Swagger Editor and Swagger UI; now it is time to implement the server logic so that our API can go live. Swagger Codegen generates server stubs, client SDKs, and client libraries from an OpenAPI specification. Swagger Codegen has the following features:

Generate server stubs.

Swagger Codegen gives you many choices of servers and frameworks, such as Go server, Java server, Scala server, and Node server. You can choose the server that supports your backend implementation.

Client-side SDKs

You can quickly and easily build client SDKs for APIs in languages such as JavaScript, Java, C#, Swift, etc. A client SDK contains wrapper classes that you can use to call the API from your application without having to deal with HTTP requests and responses.

Below is a screenshot of the Swagger Codegen GitHub Readme. You can start contributing to the open-source project at any time.

image source

What we like:

Swagger Codegen simplifies our build process so the team can focus better on the API’s implementation and adoption.

Must Know Concepts of System Design

Priyanka M — Fri, 06 Jan 2023 00:00:00 GMT

System design is the process of defining the architecture, interfaces, and data for a system that satisfies specific requirements. It requires a systematic approach and requires you to think about everything in infrastructure, from the hardware and software, all the way down to the data. These decisions are required to be taken carefully keeping in mind not only the Functional Requirements but also the NFRs Scalability, Reliability, Availability, and Maintainability.

Let’s discuss some very important concepts about designing a system every developer should be aware of.

1. Distributed Cache

A distributed cache is a system that pools together the random-access memory (RAM) of multiple networked computers into a single in-memory data store used as a data cache to provide fast access to data. Distributed caches are especially useful in environments with high data volume and load.

Popular use cases for Distributed cache:

· Storing web session data

· Application acceleration

· Decreasing network usage/costs

· Extreme scaling

Some of the widely used distributed caches are Redis, Hazelcast and Memcached.

2. Distributed Messaging Queues

A distributed messaging queue is used to send messages between distributed components of a system. Most of the messaging patterns follow the publish-subscribe model where the senders of the messages are called publishers and those who want to receive the messages are called subscribers.

Some benefits of using a distributed messaging queue are:

· We can quickly scale and improve performance depending on live traffics.

· Many message servers have an in-built setup for replications to provide fault-tolerant and durable data.

It provides the benefits of reliability, scalability, and persistence.

Some of the popular distributed messaging queues are Kafka, ActiveMQ, and Kestrel.

3. Distributed File System

A distributed file system, or DFS, is a file system that spans across multiple file servers or multiple locations. The data is accessed and processed as if it was stored on the local client machine. The DFS makes it convenient to share information and files among users on a network in a controlled and authorized way.

Examples include Hadoop Distributed File System(HADOOP), and Ceph.

4. Distributed Databases

With distributed databases, data is physically stored across multiple sites(either on the same network or on entirely different networks) and independently managed.

· Distributed databases have the advantage of modular development, which means we can add a computer and local data to the distributed system anytime without interruption.

· They are also fault-tolerant and continue to function at reduced performance until the error is fixed.

Distributed databases can be homogenous(all the physical locations have the same underlying hardware and run the same operating systems and database applications) or heterogenous(the hardware, operating systems, or database applications may be different at each location).

Though there are many distributed databases to choose from, some examples of distributed databases include Apache Ignite, Apache Cassandra, Apache HBase, and Google Cloud Spanner.

5. Load Balancer

A load balancer is a system that acts as a reverse proxy and distributes network or application traffic across a number of servers. It helps in increasing the capacity (concurrent users),reliability and availability of applications and websites for users.

It provides benefits like predictive analytics that determine traffic bottlenecks before they happen.

Examples of some load balancers are Nginx, HAProxy, Kemp Loadmaster.

6. Distributed Coordination System

The distributed coordination system handles the communication and cooperation between processes. It is a centralized service for providing configuration information, naming, synchronization and group services over large clusters in distributed systems.

Most commonly used distributed coordination system is Apache Zookeeper.

7. Metrics, Monitoring and Alerting

These are used to monitor the health and performance of a distributed system and detect and troubleshoot problems. They help you understand trends in usage or behavior, and to understand the impact of changes you make. If the metrics fall outside of your expected ranges, these systems can send notifications to the concerned person to take a look and identify the possible causes.

The types of values you monitor and the information you track will probably change as your infrastructure evolves.

Some example are Prometheus and Datadog.

8. Distributed locks

Distributed locks provide mutually exclusive access to shared resources in a distributed environment.

They are used for two main reasons:

· Efficiency of a system. Taking a lock saves you from unnecessarily doing the same work twice.

· Correctness of data. It prevents from data corruption, data loss, and data inconsistency.

Distributed locks need to decide whether to use an optimistic or a pessimistic locking policy:

· Optimistic locking policies assume that the resource being read will not change while reading it, and therefore do not lock the resource during read operations.

·Pessimistic locking policies assume that the resource being read will change while reading it, and therefore lock the resource during read operations.

An example of distributed lock is Redlock.

9. Distributed Storage

A distributed storage is a system that can split data across multiple physical servers across data centres. Distributed storage can spread files, block storage or object storage across multiple physical servers.

Advantages of a distributed storage are:

· Scalability: the primary motivation for distributing storage is to scale horizontally.

· Redundancy: distributed storage systems can store more than one copy of the same data, for high availability, backup, and disaster recovery purposes.

· Performance: distributed storage can offer better performance than a single server in some scenarios.

Distributed storage is the basis for massively scalable cloud storage systems like Amazon S3 and Microsoft Azure Blob Storage.

10. Distributed Logging

In distributed systems, each of the applications emits its own logs and often stores them in different locations. With distributed systems, logging is the easy part. What’s much harder is to make sense of this ocean of logs from a logical point of view. This is where centralized logging comes into picture. Centralized logging collects and aggregates logs from multiple services into a central location where they are indexed in a database. The log data can be searched, filtered, and grouped in the log management software by fields like status, host, severity, origin, and timestamp.

11. Stream Processing

Stream processing is a data management technique that involves ingesting a continuous data stream to quickly analyze, filter, transform or enhance the data in real time. Once processed, the data is passed off to an application, data store or another stream processing engine.

Stream processing is key if you want analytics results in real time. Architecturally, stream processing involves a framework of data processing software that sits in between the data source (the producers) and the users of that data (the consumers).

Some Popular stream processing frameworks include Apache Flink and Apache Spark.

12. Full-Text Search

A full-text search is a comprehensive search method that compares every word of the search request against every word within the document or database. It enables users to search for any word in an app or website. The underlying datastructure used here is inverted index, which maps the words or number to its locations in a document or a set of documents.

Lets understand with an example. Imagine you have the following data in a document data store:

[
{
"id": 1,  
"name": "Jon Smith",
"description": "Architect, Family"
},
{
"id": 2,
"name": "Frank Jones",
"description": "Family, Basketball"
},
{
"id": 3,
"name": "Jon Frank Whitaker",
"description": "Basketball, Architect"
},
]

If you want to find the information in record with id 2, then any database engine can efficiently retrieve that record for you because they will have an index that looks something like this:

1 => [Location On Disk: 987654]
2 => [Location On Disk: 549877]
3 => [Location On Disk: 654722]

If you need to find all the records that contain the word “Jon” however, the database engine will not be efficient at that query because it will have to scan through each record looking through the entire name field and the entire description field searching for the word “Jon”. A full text search index, on the other hand would look something like this:

"Architect"
1 => [Location On Disk: 987654]
3 => [Location On Disk: 654722]
"Basketball"
2 => [Location On Disk: 549877]
3 => [Location On Disk: 654722]
"Family"
1 => [Location On Disk: 987654]
2 => [Location On Disk: 549877]
"Frank"
2 => [Location On Disk: 549877]
3 => [Location On Disk: 654722]
"Jon"
1 => [Location On Disk: 987654]
3 => [Location On Disk: 654722]
"Jones"
2 => [Location On Disk: 549877]
"Smith"
1 => [Location On Disk: 987654]
"Whitaker"
3 => [Location On Disk: 654722]

Here, you can quickly see that records 1 and 3 contain the text “Jon” and you can efficiently load those records from disk.

Conclusion:

These are the high level concepts which can be used for designing any scalable, highy available and real-time systems.

Thanks for reading!

Mocking with Mockery in Golang

Arpan Basak — Tue, 03 Jan 2023 00:00:00 GMT

Adding unit tests is paramount when it comes to ensuring the reliability and maintainability of the code written. It not only serves as a safety net to catch bugs early, but also safeguards future integrations. Pertaining to writing unit tests, mocking becomes an even more indispensable tool. Mocking facilitates isolation of specific components and emulating their behavior, creating controlled environments for testing without relying on external dependencies. We will be exploring how to create mocks using Mockery, a popular mocking library for Go.

What is Mockery?

Mockery is a tool that generates mock implementations of interfaces. Please do keep in mind that the interfaces need to be present, to be able to be mocked. Mockery heavily relies on interfaces to substitute real implementations with mocks. It enhances modularity, enables test isolation and facilitates dynamic behavior replacement during testing. We will understand more on this with the example mentioned later.

Github link can be found in Mockery.

Salient Features of Mockery

Automatic mock generation: Mockery automates the process of creating mock implementations for interfaces. You simply provide the interface you want to mock and Mockery generates the corresponding mock implementation. This can save a lot of time and effort, especially when dealing with interfaces that have numerous methods.
Simple command-line interface: Being a command-line tool, its usage is pretty straightforward. Just run a command with desired options, you will have your mocks generated. Not only that, we can customize various aspects too. We will understand further when we deep dive on the options/flags.
Support for testify/mock assertions: The generated mocks can easily be integrated with the assertions provided by the ‘testify/mock’ package, allowing us to seamlessly use ‘On’, ‘Return’, ‘AssertExpectations’ etc.
Mocking unexported interfaces: Mockery supports generation of mocks for unexported interfaces. This is quite useful when you have internal interfaces that you want to mock for testing purposes, but you don’t want to expose them to the external API of your package.

Getting started with Mockery

Before starting, we need to make sure to have Mockery installed. We can do it using below command :

go get github.com/vektra/mockery/v2/../

Once installed, we can use Mockery to generate mocks for the interfaces.

Mockery Commands

1. Generate Mocks for All interfaces

To generate mocks for all interfaces in the current directory and its subdirectories

mockery --all

2. Generate Mocks for a Specific interface

To generate mocks for a specific interface, we can use name flag

mockery --name InterfaceName

3. Specify output directory

By default, Mockery generates mocks in the ./mocks directory. We can use output flag to specify different output directory.

mockery --all --output path/to/output

4. Include subdirectories

Include subdirectories when generating mocks, we can use recursive flag.

mockery --all --recursive

5. Specify package name

Use the keeptree flag to preserve the directory structure when generating mocks. This can be useful to maintain the same package structure in mocks directory.

mockery --all --keeptree

6. Generate Mocks for a specific package

Create mocks for interfaces defined in a specific package. Handy in combination with output flag.

mockery --all --dir path/to/package

7. Define Output Package Name

To set custom package name for the generated mocks, we can use outputpkg flag.

mockery --all --outputpkg customMocks

Example: Using Mockery to test an interface

Let’s consider we have an interface DataProvider which represents an external service to fetch data. In our case, we will just return a random number between 0 and the number passed.

// data_provider.go
package service

// DataProvider is an interface for fetching and retrieving data.
type DataProvider interface {
	GetRandomNumber(id int) (int, error)
}

Let’s create the implementation of the interface DataProvider to return a random number.

// data_provider_impl.go
package service

import (
	"errors"
	"math/rand"
)

// DataProviderImpl is the concrete implementation of the DataProvider interface.
type DataProviderImpl struct{}

// GetRandomNumber simulates fetching a random number between 0 and id.
func (d *DataProviderImpl) GetRandomNumber(id int) (int, error) {
	if id < 0 {
		return 0, errors.New("Invalid id")
	}
	// Simulate fetching a random number between 0 and id
	return rand.Intn(id + 1), nil
}

Now, we need to consume the DataProvider to get the random data. Let’s create that. Additionally, we will check if the random number we fetched, is even or odd.

// data_consumer.go
package service

// ConsumeData is a function that uses a DataProvider to fetch and process data.
func ConsumeData(provider DataProvider, id int) (string, error) {
	// Use GetRandomNumber to get a random number between 0 and id
	randomNumber, err := provider.GetRandomNumber(id)
	if err != nil {
		return "", err
	}

	// Check whether the value is even or odd
	result := checkEvenOrOdd(randomNumber)

	// Return the result
	return result, nil
}

// checkEvenOrOdd checks whether the given value is even or odd.
func checkEvenOrOdd(value int) string {
	if value % 2 == 0 {
		return "Even"
	}
	return "Odd"
}

Now, let’s create the mocks for DataProvider interface using Mockery. We will use below command to do so.

mockery  --output ./mocks --dir ./service --all

On running the command, you will see below execution. What this will do, is create a DataProvider.go inside mocks package since we have specified –output ./mocks in the mockery command.

22 Jan 24 08:33 IST INF Starting mockery dry-run=false version=v2.36.1
22 Jan 24 08:33 IST INF Using config:  dry-run=false version=v2.36.1
22 Jan 24 08:33 IST INF Walking dry-run=false version=v2.36.1
22 Jan 24 08:33 IST INF Generating mock dry-run=false interface=DataProvider qualified-name=RandomizedEvenOdd/service version=v2.36.1
22 Jan 24 08:33 IST INF writing mock to file dry-run=false interface=DataProvider qualified-name=RandomizedEvenOdd/service version=v2.36.1

Auto generated mocked DataProvider.go will be like below. Please make sure to re-generate mocks whenever there is an update in the interfaces. For example, we add a new function in the interface.

// Code generated by mockery v2.36.1. DO NOT EDIT.

package mocks

import mock "github.com/stretchr/testify/mock"

// DataProvider is an autogenerated mock type for the DataProvider type
type DataProvider struct {
	mock.Mock
}

// GetRandomNumber provides a mock function with given fields: id
func (_m *DataProvider) GetRandomNumber(id int) (int, error) {
	ret := _m.Called(id)

	var r0 int
	var r1 error
	if rf, ok := ret.Get(0).(func(int) (int, error)); ok {
		return rf(id)
	}
	if rf, ok := ret.Get(0).(func(int) int); ok {
		r0 = rf(id)
	} else {
		r0 = ret.Get(0).(int)
	}

	if rf, ok := ret.Get(1).(func(int) error); ok {
		r1 = rf(id)
	} else {
		r1 = ret.Error(1)
	}

	return r0, r1
}

// NewDataProvider creates a new instance of DataProvider. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewDataProvider(t interface {
	mock.TestingT
	Cleanup(func())
}) *DataProvider {
	mock := &DataProvider{}
	mock.Mock.Test(t)

	t.Cleanup(func() { mock.AssertExpectations(t) })

	return mock
}

With the mock generated, let’s use it to write test cases for ConsumeData which has an external dependency on DataProvider.

// data_consumer_test.go
package test

import (
	"EvenOdd/mocks"
	"EvenOdd/service"
	"errors"
	"testing"

	"github.com/stretchr/testify/assert"
)

// TestConsumeDataForSingleInput tests the ConsumeData function with a single input.
func TestConsumeDataForSingleInput(t *testing.T) {
	// Create an instance of the mock
	mock := &mocks.DataProvider{}

	// Set the expected return value for the GetRandomNumber method
	mock.On("GetRandomNumber", 5).Return(3, nil).Once()

	// Call ConsumeData that using the mocked DataProvider
	result, err := service.ConsumeData(mock, 5)

	// Assert that the result and error are as expected
	assert.Equal(t, "Odd", result)
	assert.NoError(t, err)

	// Assert that the GetRandomNumber method was called with the expected input
	mock.AssertExpectations(t)
}

// TestConsumeDataForMultipleInputs tests the ConsumeData function with multiple values.
func TestConsumeDataForMultipleInputs(t *testing.T) {
	// Create an instance of the mock
	mock := &mocks.DataProvider{}

	// Set the expected return values for the GetRandomNumber method
	mock.On("GetRandomNumber", 20).Return(10, nil).Once()
	mock.On("GetRandomNumber", 30).Return(15, nil).Once()
	mock.On("GetRandomNumber", 10).Return(5, nil).Once()

	// Set the expected return value for an error scenario
	mock.On("GetRandomNumber", -1).Return(0, errors.New("Invalid id")).Once()

	// Added multiple inputs for testing
	testCases := []struct {
		input int
		want  string
		err   error
	}{
		{input: 20, want: "Even", err: nil},
		{input: 30, want: "Odd", err: nil},
		{input: 10, want: "Odd", err: nil},
		{input: -1, want: "", err: errors.New("Invalid id")},
	}

	for _, tc := range testCases {
		result, err := service.ConsumeData(mock, tc.input)

		// Assert that the result and error are as expected
		assert.Equal(t, tc.want, result)
		assert.Equal(t, tc.err, err)
	}

	// Assert that the GetRandomNumber methods were called with the expected inputs
	mock.AssertExpectations(t)
}

The first test case is with a single input. Whereas, if we want to write a test using multiple inputs, we can do it in a similar way as the second test case. Also, in second test case, I have added error scenario as well.

To run the test, go inside test package and run command (go test) :

go test

PASS
ok      RandomizedEvenOdd/test    0.400s

Voila! There you go!

In this example, the mock is used to simulate the behavior of the DataProvider interface, allowing us to control the output and assert that the interaction with the external dependency is as expected.

By incorporating Mockery, we can easily create maintainable and effective mocks for the interfaces, allowing us to write thorough tests for our Go code.

Code github link can be found in RandomizedEvenOdd.

Hope this helps in getting started with Mockery!

DnD with React DnD for better UX

Vanmeeganathan P K — Mon, 19 Dec 2022 00:00:00 GMT

Drag and Drop (DnD) is a user interface concept using which one can select an object/element in the viewport and can move it (ie drag) to a desired location on the view port (ie drop).

Drag and Drop (DnD) makes it easy to copy, move, reorder and delete items with the help of mouse clicks. It is much more intuitive from the point of user experience than other user actions which may require clicking, typing etc. Often, the UI gets simpler and cleaner on account of using drag & drop.

Some of the different use-cases include uploading files, rearranging data in a table/section like in a Trello board, matching values in one section to an other value in another section.

React DnD

HTML5’s Drag and Drop API supports drag and drop operations in a HTML application. DnD operations involve moving around objects in the screen. They are inherently complex and there are few libraries which help us make our work much simpler. React DnD is a popular library with MIT license and around 18.1k stars in GitHub at the time of writing this article.

The Quest for a better UX using DnD

At Sixt R&D, we had business requirements which demanded a lot of complex operations. They could be made much simpler using DnD operations which are more intuitive to the users.

Required Drag and Drop Actions

Our UX design demanded the below drag and drop operations between different sections like Dragging a row from a table into the body of an accordion in a section which has a list of accordions. Rows to be draggable based on validation with some existing data. Dragging a marker located on top of Google Map into the body of an accordion in a section which has a list of accordions. Markers to be draggable based on validation with some existing data. This was coupled with validations on the draggable objects as well as the droppable zones.

We solved these requirements elegantly using React DnD.

Working with React DnD

React DnD is a simple to use library which can be installed as below.

npm install react-dnd react-dnd-html5-backend

React-dnd requires a ‘backend’ to work. We will go with HTML5Backend, the most popular one. There are other third party backends like touch backend which are also available. Here, we will stick to HTML5Backend.

Setting up the Context for Drag and Drop

In order to use the Drag and Drop functionalities, we need to enclose the components with the DndProvider component. It is usually done near the top of the application tree. This will inject the DndProvider component with the property ‘backend’ having value as HTML5Backend.

import { HTML5Backend } from 'react-dnd-html5-backend'
import { DndProvider } from 'react-dnd'

export default class MyApplication{
  render() {
    return (
      <DndProvider backend={HTML5Backend}>
        /* Our Drag-and-Drop Application */
      </DndProvider>
    )
  }
}

useDrag Hook

In order to make a component draggable, you can use the useDrag hook provided by the React Dnd library. It can be used as below :

const [, drag] = useDrag(() => ({
    type: type,
    item: {},
    canDrag: true,
  }
), [dependency]);

useDrag returns a drag object which needs to set to the object that is to be dragged like below.

<div ref={drag}>
  ....
</div>

useDrag hook accepts a range of properties. Some of the important ones are listed below :-

type

Type of the dragged item. This is helpful to distinguish between draggable elements, when there are different types of draggable objects that behave differently when dragged.

item

When dropped, this object will appear as input to the callback function drop supplied to the useDrop hook.

canDrag

When set to true, the object is draggable. When false, it is not draggable. It can be used when we need to control draggability of an object.

When the dependency value changes, the values are recomputed within the useDrag hook. It is useful in scenarios when the draggability of an object changes. Failure to update this, will lead to canDrag and item values not getting updated when required.

useDrop Hook

In order to make an element a container into which dragged objects can be dropped, we can use the useDrop hook.

const [, drop] = useDrop(
    () => ({
      accept: ['type 1','type 2',],
      canDrop: () => true,
      drop: (item) => {
          ......
      }
    }
),[dependency]);

// here item is the value passed to drag object property 'item'

useDrop returns a drop object which needs to be set to the DOM object that will behave as the container into which objects will be dragged like below.

<div ref={drop}>
  ....
</div>

useDrop hook accepts a range of properties. Some of the important ones are listed below :-

It accepts an array of item types that can be dragged into the drop element.

drop

It accepts a callback function that gets triggered when a drop action occurs. It takes an input object, which will be populated with the item object passed as a property to the dragged object.

canDrop

When set to true, dragged object can be dropped into the DOM element. It can be used when we need to control if a dragged object can be dropped into a particular drop container.

When the dependency value changes, the values are recomputed within the useDrop hook. Failure to update this, will lead to canDrop and other properties inside useDrop hook not getting updated when required.

*React Dnd supports more such features which make it a truly versatile library that can used to create complex apps that require a lot of conditional drag and drop options. Do give it a try if you need to implement such functionalities. *

Happy coding !

SIXT Tech

Kotlin Multiplatform (KMP): One Codebase, Many Platforms

Introduction

Why Use KMP

️ Core Concepts

Targets

Source Sets

Expect / Actual Pattern

Interoperability

Build & Tooling

Gradle Setup

CocoaPods Integration

Multiplatform Libraries Ecosystem

Networking & Serialization

Persistence

Migration Strategy (From Android-only Kotlin)

1. Identify Reusable Logic

2. Create a Shared KMP Module

3. Move Common Code Incrementally

4. Integrate Shared Code Back into Android

5. Generate the iOS Framework

6. Gradually Expand

Best Practices During Migration

Example of a Small First Step

Conclusion

Spring Reactive Programming: Asynchronous, Non-blocking, and Scalable Solutions

What is Reactive Programming and why is a need for Reactive Programming?

Difference between Blocking and Non-blocking request Processing.

Blocking Request Model:

Non-blocking Request Model:

Components of Reactive Programming:

Spring Webflux

Concept of Backpressure:

Use cases of Reactive Programming

Limitations of Reactive Programming

Conclusion

Dask: A parallel data processing python library for large datasets

Internals

Dask basics

Dask built-in Function example

Results in my use case

Drawbacks

Insights

Empowering Scalability: Harnessing the Power of CQRS for High-Performance Systems

What is CQRS?

Why CQRS

CQRS Use Cases: Domains that Benefit from Command Query Responsibility Segregation

E-commerce:

Finance and Banking:

Logistics and Supply Chain:

Social Media:

Synchronizing Data in CQRS: Bridging the Divide between Read and Write Databases

Designing a CQRS System

Identify Command and Query Models

Command Model:

Query Model:

Define Command and Query Handlers

Command Handlers:

Query Handlers:

Introduce Aggregates for Consistency

Implement Domain Models

Establish Asynchronous Communication

Event Sourcing:

Message Queues:

Optimize for Read and Write Operations

Write-Side Optimization:

Read-Side Optimization:

Address Consistency Challenges

Monitor and Tune

Monitoring:

Tuning:

Implementing CQRS in Spring Boot

1. Command Model:

2. Command Handler:

3. Query Model:

4. Query Handler:

5. Domain Model:

6. Controller:

Query Result:

Limitations:

1. Using `remember` for State: Remembering Data: