Automating Database Migrations in Java with Flyway

Database migrations are a crucial aspect of software development, particularly in environments where continuous integration and delivery (CI/CD) are standard practice. As your application grows and evolves, so too must the database schema it depends on. Manually managing these schema changes can lead to errors and consume significant time.

Enter Flyway, an invaluable open-source tool tailored for simplifying database migrations. Flyway introduces version control to your database, allowing you to migrate your schema safely and with reliability. In this article, we'll explore how to automate database migrations in multi-module gragle java project using Flyway, ensuring that managing database changes becomes a streamlined, error-resistant process.

More details on flyway

Understanding Multi-Project Builds in Gradle

While some smaller projects or monolithic applications might manage with just one build file and a unified source structure, larger projects are frequently organized into multiple, interdependent modules. The term "interdependent" is key here, highlighting the need to connect these modules via a singular build process.

Gradle caters to this setup with its multi-project build capability, often termed as a multi-module project. In Gradle's terminology, these modules are called subprojects.

A multi-project build is structured around one root project and can include several subprojects beneath it.

The directory structure should look as follows:

├── .gradle
│   └── ?
├── gradle
│   ├── libs.versions.toml
│   └── wrapper
├── gradlew
├── gradlew.bat
├── settings.gradle.kts (1)
├── sub-project-1
│   └── build.gradle.kts (2) 
├── sub-project-2
│   └── build.gradle.kts (2) 
└── sub-project-3
    └── build.gradle.kts (2)

(1) The settings.gradle.kts file should include all subprojects.
(2) Each subproject should have its own build.gradle.kts file.

Leveraging Gradle Sub-Modules for Clean Architecture

Clean Architecture is a design pattern that emphasizes separation of concerns, making software easier to maintain and test. One of the practical ways to implement this architecture in a project involves using Gradle's sub-module structure to organize your codebase. Here's how you can align Clean Architecture with Gradle sub-modules:

Clean Architecture Layers:
Core:

• Contains business logic, domain models, and application rules. Has no dependency on External or Web.
• Should be independent of framework-specific implementations where possible.

External:

• Handles external actions or integrations, such as database migrations or third-party service interactions.
• May depend on Core for business logic but should not depend on Web.

Web:

• The entry point, exposing a REST API and handling HTTP requests.
• Depends on Core for business logic and may depend on External for integrations.

├── .gradle
│   └── ?
├── gradle
│   ├── libs.versions.toml
│   └── wrapper
├── gradlew
├── gradlew.bat
├── settings.gradle.kts (1)
├── sub-project-1
│   └── build.gradle.kts (2) 
├── sub-project-2
│   └── build.gradle.kts (2) 
└── sub-project-3
    └── build.gradle.kts (2)

Step 1: Create a Java-based Gradle project and name it "SchoolStaff".

Step 2: Go to Spring Initializr and generate a REST API project named Web.

Step 3: Create a Java-based Gradle project and name it External.

Step 4: Create a Java-based Gradle project and name it Core.

Root build.gradle.kts

SchoolStaff/
├── Core/
│   ├── src/
│   │   └── main/
│   │       ├── java/         # Business logic and domain objects
│   │       └── resources/    # Core-specific resources (if any)
│   └── build.gradle.kts
├── External/
│   ├── src/
│   │   └── main/
│   │       ├── java/         # External integration code
│   │       └── resources/    # db/migration and other external resources
│   └── build.gradle.kts
├── Web/
│   ├── src/
│   │   └── main/
│   │       ├── java/         # REST controllers and entry-point logic
│   │       └── resources/    # Application-specific configuration
│   └── build.gradle.kts
├── build.gradle.kts          # Root Gradle build
└── settings.gradle.kts       # Project module settings

settings.gradle.kts

plugins {
    id("java")
}

allprojects {
    group = "school.staff"
    version = "1.0.0"

    repositories {
        mavenLocal()
        mavenCentral()
    }
}

subprojects {
    apply(plugin = "java")

    dependencies {
        testImplementation(platform("org.junit:junit-bom:5.10.0"))
        testImplementation("org.junit.jupiter:junit-jupiter")
    }

    tasks.test {
        useJUnitPlatform()
    }
}

Required dependencies for the "Web" project.

rootProject.name = "SchoolStaff"

include("Core", "External", "Web")

Required dependencies for the "Core" project.

dependencies {
    implementation(project(":Core"))
    implementation(project(":External"))
}

Required dependencies for the "External" project.

dependencies {
    runtimeOnly(project(":External"))
}

We use the following plugin for Flyway migration:

import java.sql.DriverManager
import java.util.Properties
// Function to load properties based on the environment
fun loadProperties(env: String): Properties {
    val properties = Properties()
    val propsFile = file("../web/src/main/resources/application-$env.properties")

    if (propsFile.exists()) {
        propsFile.inputStream().use { properties.load(it) }
    } else {
        throw GradleException("Properties file for environment '$env' not found: ${propsFile.absolutePath}")
    }

    return properties
}
// Set the environment (default is 'dev' if no argument is passed)
val env = project.findProperty("env")?.toString() ?: "dev"

// Load properties for the chosen environment
val dbProps = loadProperties(env)
buildscript {
    dependencies {
        classpath("org.flywaydb:flyway-database-postgresql:11.1.0") // This is required for the flyway plugin to work on the migration, otherwise it will throw an error as No Database found
        classpath("org.postgresql:postgresql:42.7.4")
    }
}
plugins {
    id("java-library")
    id("org.flywaydb.flyway") version "11.0.1"
}

group = "school.staff"
version = "unspecified"

repositories {
    mavenLocal()
    mavenCentral()
}

dependencies {
    implementation("org.springframework.boot:spring-boot-starter-data-jpa:3.4.0")
    implementation("org.postgresql:postgresql:42.7.4")
    implementation("org.flywaydb:flyway-core:11.0.1")
    implementation("org.flywaydb:flyway-database-postgresql:11.0.1")
    implementation("org.flywaydb:flyway-gradle-plugin:11.0.1")

    implementation (project(":Core"))
    testImplementation(platform("org.junit:junit-bom:5.10.0"))
    testImplementation("org.junit.jupiter:junit-jupiter")
}

tasks.test {
    useJUnitPlatform()
}

// Task to create the database if it doesn't exist
tasks.register("createDatabase") {
    doLast {
        val dbUrl = dbProps["spring.datasource.url"] as String
        val dbUsername = dbProps["spring.datasource.username"] as String
        val dbPassword = dbProps["spring.datasource.password"] as String

        // Extract the base URL and database name
        val baseDbUrl = dbUrl.substringBeforeLast("/")+ "/"
        val dbName = dbUrl.substringAfterLast("/")

        // Connect to the PostgreSQL server (without the specific database)
        DriverManager.getConnection(baseDbUrl, dbUsername, dbPassword).use { connection ->
            val stmt = connection.createStatement()
            val resultSet = stmt.executeQuery("SELECT 1 FROM pg_database WHERE datname = '$dbName'")
            if (!resultSet.next()) {
                println("Database '$dbName' does not exist. Creating it...")
                stmt.executeUpdate("CREATE DATABASE \"$dbName\"")
                println("Database '$dbName' created successfully.")
            } else {
                println("Database '$dbName' already exists.")
            }
        }
    }
}

flyway {
    url = dbProps["spring.datasource.url"] as String
    user = dbProps["spring.datasource.username"] as String
    password = dbProps["spring.datasource.password"] as String
    locations = arrayOf("classpath:db/migration")
    baselineOnMigrate = true
}

 //Ensure classes are built before migration
tasks.named("flywayMigrate").configure {
    dependsOn(tasks.named("createDatabase"))
    dependsOn(tasks.named("classes"))
}

This approach is well-suited for production environments, as it ensures controlled and reliable migrations. Instead of running migrations automatically on each application startup, we execute them only when necessary, providing greater flexibility and control.

We are also utilizing the application.properties file in the Spring application to manage database connections and credentials. The baselineOnMigrate = true setting ensures that the initial migration is used as the baseline for future migrations.

 plugins {
    id("org.flywaydb.flyway") version "11.0.1"
}

We can use JPA Buddy to generate all the migration files within the External project's resources/db/migration directory.

V1__Initial_Migration

flyway {
    url = dbProps["spring.datasource.url"] as String
    user = dbProps["spring.datasource.username"] as String
    password = dbProps["spring.datasource.password"] as String
    locations = arrayOf("classpath:db/migration")
    baselineOnMigrate = true
}

From the root project, we can execute the Flyway migration using the following command:

CREATE TABLE _user
(
    id                 UUID NOT NULL,
    created_by         UUID,
    created_date       TIMESTAMP WITH TIME ZONE,
    last_modified_by   UUID,
    last_modified_date TIMESTAMP WITH TIME ZONE,
    first_name         VARCHAR(255),
    last_name          VARCHAR(255),
    email              VARCHAR(255),
    password           VARCHAR(255),
    tenant_id          UUID,
    CONSTRAINT pk__user PRIMARY KEY (id)
);

This will apply all the migration files to the database.

Conclusion

We've explored how to automate database migrations using Flyway within a Gradle multi-module project, which is crucial for maintaining schema consistency in CI/CD environments.

We also covered how Gradle supports multi-project builds, organizing complex projects into manageable subprojects, each with its own build configuration, unified under a root build script.

Lastly, we aligned Clean Architecture with Gradle modules, structuring the project into Core, External, and Web layers, promoting a clean separation of concerns and dependency management.

These practices enhance modularity, automation, and maintainability, setting the stage for scalable, error-free software development.

The above is the detailed content of Flyway Migrations in Multi-Module Gradle Projects (Clean Architecture). For more information, please follow other related articles on the PHP Chinese website!