Enhanced Logging in ktor server with CallLogging Feature

When building web applications, especially APIs, it’s crucial to have robust logging to monitor requests, performance, and errors. Ktor provides a convenient CallLogging feature that can be easily integrated into your server setup to log important information about each HTTP request and response. This logging can be fine-tuned to include custom formatting, highlighting, and filtering.

In this article, we will explore how to use Ktor’s CallLogging to log detailed information for each incoming request, including HTTP method, request path, query parameters, response status, and more. We will also demonstrate how to color-code the logs for better readability in the console.

Call logger

Setting Up Call Logging in Ktor

To get started, you need to install the CallLogging feature in your Ktor server. Here is a code snippet that demonstrates a highly customized logging configuration:

install(CallLogging){
    level = Level.INFO
    filter { call -> call.request.path().startsWith("/") }
    format { call ->
        val status = call.response.status()
        val httpMethod = call.request.httpMethod.value
        val userAgent = call.request.headers["User-Agent"]
        val path = call.request.path()
        val queryParams =
            call.request.queryParameters
                .entries()
                .joinToString(", ") { "${it.key}=${it.value}" }
        val duration = call.processingTimeMillis()
        val remoteHost = call.request.origin.remoteHost
        val coloredStatus =
            when {
                status == null -> "\u001B[33mUNKNOWN\u001B[0m"
                status.value < 300 -> "\u001B[32m$status\u001B[0m"
                status.value < 400 -> "\u001B[33m$status\u001B[0m"
                else -> "\u001B[31m$status\u001B[0m"
            }
        val coloredMethod = "\u001B[36m$httpMethod\u001B[0m"
        """
        |
        |------------------------ Request Details ------------------------
        |Status: $coloredStatus
        |Method: $coloredMethod
        |Path: $path
        |Query Params: $queryParams
        |Remote Host: $remoteHost
        |User Agent: $userAgent
        |Duration: ${duration}ms
        |------------------------------------------------------------------
        |
  """.trimMargin()
    }
}

Key Features of This Configuration

1. Log Level:

The level = Level.INFO specifies that the logging will capture INFO level messages. This can be adjusted based on your needs (e.g., DEBUG, WARN, ERROR).

2. Filtering:

The filter function ensures that only requests starting from the root path (”/”) are logged. You can customize this to log requests to specific paths or APIs.

3. Custom Format:

The format block allows you to create a personalized log message that includes key details about each request:

• Status: The HTTP response status code (e.g., 200, 404), is color-coded based on the status range.

• Method: The HTTP method (GET, POST, etc.), is displayed in cyan.

• Path: The request path.

• Query Parameters: The query parameters in a key-value format.

• Remote Host: The IP address of the client making the request.

• User Agent: The User-Agent header, which tells you what browser or client made the request.

• Duration: The time it took for the server to process the request.

This gives you a full picture of each request’s lifecycle in a clean and organized format.

4. Color Coding:

The log output uses ANSI escape codes to add colors:

• Green for success (2xx),

• Yellow for redirection (3xx) or unknown status,

• Red for error statuses (4xx, 5xx),

• Cyan for HTTP methods.

These colors help differentiate important information quickly when scanning through logs.

Example Output

When a request comes into the server, the log output would look something like this:

------------------------ Request Details ------------------------
Status: 200
Method: GET
Path: /api/products
Query Params: id=123, sort=asc
Remote Host: 192.168.1.1
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
Duration: 45ms
------------------------------------------------------------------

This format makes it easy to read and troubleshoot requests at a glance, with different colors highlighting important parts of the log.

Benefits of Using Call Logging in Ktor

1. Improved Debugging: Logs provide essential information for troubleshooting failed requests or performance bottlenecks, allowing developers to understand the request flow.

2. Performance Monitoring: Tracking the processing time of each request helps in identifying slow endpoints.

3. Security Insights: Logging details such as the client’s IP and User-Agent can help detect suspicious activity or malformed requests.

4. Customizable Output: As shown in the code, the format can be adjusted to fit your exact logging needs. You can add or remove details based on what’s important for your application.

GitHub Example: https://github.com/piashcse/ktor-E-Commerce
Medium: https://piashcse.medium.com/enhanced-logging-in-ktor-server-with-calllogging-feature

Read More

Ktor Client in Kotlin Multiplatform

 

Kotlin Multiplatform (KMP) allows you to write code that can be shared across multiple platforms, including Android, iOS, desktop and the web. One of the libraries that make KMP even more powerful is Ktor, a framework for building asynchronous servers and clients in connected systems. In this blog post, we’ll walk through setting up a Ktor client in a KMP project and making a sample HTTP request.

ktor client

Setting Up the Ktor Client

First, let’s look at the code snippet that configures the Ktor client:

val client = HttpClient {
    defaultRequest {
        url {
            takeFrom("https://api.themoviedb.org/") // Base url
            parameters.append("api_key", "59cd6896d8432")
        }
    }
    expectSuccess = true
    install(HttpTimeout) {
        val timeout = 30000L
        connectTimeoutMillis = timeout
        requestTimeoutMillis = timeout
        socketTimeoutMillis = timeout
    }
    install(Logging) {
        logger = Logger.DEFAULT
        level = LogLevel.HEADERS
        logger = object : Logger {
            override fun log(message: String) {
                println(message)
            }
        }
    }
    install(ContentNegotiation) {
        json(Json {
            ignoreUnknownKeys = true
        })
    }
}

Let’s break down what each part of this configuration does.

Default Request Configuration

The defaultRequest block sets up default parameters for all requests made by this client:

defaultRequest {
    url {
      takeFrom("https://api.themoviedb.org/") // Base url
        parameters.append("api_key", "59cd6896d8432")
    }
}

Here, we’re specifying a base URL and appending an API key to every request. This is useful for APIs that require authentication or other constant query parameters.

Timeout Configuration

Next, we configure the timeouts for the client:

install(HttpTimeout) {
    val timeout = 30000L
    connectTimeoutMillis = timeout
    requestTimeoutMillis = timeout
    socketTimeoutMillis = timeout
}

We set a 30-second timeout for connecting, requesting, and socket operations. This ensures that our client doesn’t hang indefinitely.

Logging Configuration

The Logging feature helps us log HTTP requests and responses:

install(Logging) {
    logger = Logger.DEFAULT
    level = LogLevel.HEADERS
    logger = object : Logger {
        override fun log(message: String) {
            println(message)
        }
    }
}

Here, we log the headers of the requests and responses. The custom logger prints these logs to the console.

Content Negotiation

Finally, we configure content negotiation to handle JSON data:

install(ContentNegotiation) {
    json(Json {
        ignoreUnknownKeys = true
    })
}

This tells the client to use JSON for content negotiation and to ignore unknown keys in the JSON response, making the client more resilient to changes in the API.

Using the Ktor Client

Once the client is configured, you can use it to make HTTP requests. Here’s an example of making a GET request to fetch data for a specific person:

suspend fun fetchPersonData(personId: String): Person {
    return client.get {
        url {
            encodedPath = "3/person/$personId"
        }
    }.body()
}

In this function, we make a GET request to the /3/person/{personId} endpoint. The encodedPath specifies the endpoint, and the personId is dynamically included in the URL. The body() function is used to parse the response into a Person object.

Example Usage

You can call the fetchPersonData function in a coroutine to get the data for a specific person:

fun main() = runBlocking {
    val personId = "12345"
    val personData = fetchPersonData(personId)
    println(personData)
}

In this example, we use runBlocking to start a coroutine, fetch the data for a person with ID 12345, and print the result.

Conclusion

Configuring a Ktor client in a Kotlin Multiplatform project is straightforward and highly customizable. By setting up default requests, timeouts, logging, and content negotiation, you can ensure your HTTP client is robust and efficient. Making requests and handling responses is seamless, allowing you to build powerful and responsive applications. Happy coding!

Ref: https://piashcse.medium.com/ktor-client-in-kotlin-multiplatform-27e6ac59ae01

Read More

Exposed BaseTable for generating createdAt and updatedAt in Ktor part-4

 ktor BaseTable in terms of IntIdTable

import org.jetbrains.exposed.dao.*
import org.jetbrains.exposed.sql.ReferenceOption
import org.joda.time.DateTime
import org.joda.time.DateTimeZone

fun currentUtc(): DateTime = DateTime.now(DateTimeZone.UTC)

abstract class BaseIntIdTable(name: String) : IntIdTable(name) {
    val createdAt = datetime("createdAt").clientDefault { currentUtc() }
    val updatedAt = datetime("updatedAt").nullable()
}

abstract class BaseIntEntity(id: EntityID<Int>, table: BaseIntIdTable) : IntEntity(id) {
    val createdAt by table.createdAt
    var updatedAt by table.updatedAt
}

abstract class BaseIntEntityClass<E : BaseIntEntity>(table: BaseIntIdTable) : IntEntityClass<E>(table) {

    init {
        EntityHook.subscribe { action ->
            if (action.changeType == EntityChangeType.Updated) {
                try {
                    action.toEntity(this)?.updatedAt = currentUtc()
                } catch (e: Exception) {
                    //nothing much to do here
                }
            }
        }
    }
}

val BaseIntEntity.idValue: Int
    get() = this.id.value

object Users : BaseIntIdTable("users") {
    val name = varchar("name", length = 60)
    val role = reference("roleId", Roles, onDelete = ReferenceOption.NO_ACTION).nullable()
}

class User(id: EntityID<Int>) : BaseIntEntity(id, Users) {
    companion object : BaseIntEntityClass<User>(Users)

    var name by Users.name
    var role by Role optionalReferencedOn Users.role
}

ktor BaseTable in terms of IdTable<String> . Primary key will be in string

import org.jetbrains.exposed.dao.*
import org.jetbrains.exposed.dao.id.EntityID
import org.jetbrains.exposed.dao.id.IdTable
import org.jetbrains.exposed.sql.Column
import org.jetbrains.exposed.sql.javatime.datetime
import java.time.LocalDateTime
import java.time.ZoneOffset
import java.util.UUID// generating utc time
fun currentUtc(): LocalDateTime =  LocalDateTime.now(ZoneOffset.UTC)abstract class BaseIntIdTable(name: String) : IdTable<String>(name) {
    override val id: Column<EntityID<String>> =varchar("id", 50).clientDefault { UUID.randomUUID().toString() }.entityId()
    val createdAt = datetime("created_at").clientDefault { currentUtc() }
    val updatedAt = datetime("updated_at").nullable()
}

abstract class BaseIntEntity(id: EntityID<String>, table: BaseIntIdTable) : Entity<String>(id) {
    val createdAt by table.createdAt
    var updatedAt by table.updatedAt
}
abstract class BaseIntEntityClass<E : BaseIntEntity>(table: BaseIntIdTable) : EntityClass<String, E>(table){
    init {
        EntityHook.subscribe { action ->
            if (action.changeType == EntityChangeType.Updated) {
                try {
                    action.toEntity(this)?.updatedAt = currentUtc()
                } catch (e: Exception) {
                    //nothing much to do here
                }
            }
        }
    }
}

object Users : BaseIntIdTable("users") {
    val name = varchar("name", length = 60)
    val role = reference("roleId", Roles, onDelete = ReferenceOption.NO_ACTION).nullable()
}

class User(id: EntityID<Int>) : BaseIntEntity(id, Users) {
    companion object : BaseIntEntityClass<User>(Users)

    var name by Users.name
    var role by Role optionalReferencedOn Users.role
}

Ref:
https://piashcse.medium.com/exposed-basetable-for-generating-createdat-and-updatedat-in-ktor-part-3-581511fdfa38

Read More

Get child data in parent table by backReferencedOn in kotlin Exposed Ktor part-3

UserId is a foreign key in UserHasType table . Now if we want to get UserHasTypeTable data as child data in UsersTable we need to point it as val userType by UserHasTypeEntity backReferencedOn UserHasTypeTable.user_id

object UsersTable : IdTable<String>("users") {
    override val id: Column<EntityID<String>> = text("id").uniqueIndex().entityId()
    val user_name = text("user_name")
    val email = text("email")
    val password = text("password")
    val mobile_number = text("mobile_number").nullable()
    val email_verified_at = text("email_verified_at").nullable() // so far unkmown
    val remember_token = text("remember_token").nullable()
    val verification_code = text("verification_code").nullable() // verification_code
    val created_at = datetime("created_at").defaultExpression(CurrentDateTime()) // UTC time
    val updated_at = datetime("updated_at").nullable()
    val is_verified = text("is_verified").nullable() // email verified by validation code
    override val primaryKey = PrimaryKey(id)
}

class UsersEntity(id: EntityID<String>) : Entity<String>(id) {
    companion object : EntityClass<String, UsersEntity>(UsersTable)
    var user_name by UsersTable.user_name
    var email by UsersTable.email
    var password by UsersTable.password
    var mobile_number by UsersTable.mobile_number
    var email_verified_at by UsersTable.email_verified_at
    var remember_token by UsersTable.remember_token
    var verification_code by UsersTable.verification_code
    var created_at by UsersTable.created_at
    var updated_at by UsersTable.updated_at
    var is_verified by UsersTable.is_verified
    val userType by UserHasTypeEntity backReferencedOn UserHasTypeTable.user_id
    fun userResponse() = UsersResponse(
        id.value,
        user_name,
        email,
        mobile_number,
        email_verified_at,
        remember_token,
        is_verified,
        userType.userHasTypeResponse()
    )
}

data class UsersResponse(
    val id: String,
    val userName: String,
    val email: String,
    val mobileNumber: String?,
    val emailVerifiedAt: String?,
    val rememberToken: String?,
    val isVerified: String?,
    var userType: UserHasType
)

UserHasTypeTable and UserHasTypeEntity

object UserHasTypeTable : IdTable<String>("user_has_type") {
    override val id: Column<EntityID<String>> = text("id").uniqueIndex().entityId()
    val user_id = reference("user_id", UsersTable.id)
    val user_type_id = text("user_type_id")
    val created_at = text("created_at")
    val updated_at = text("updated_at")
    override val primaryKey = PrimaryKey(id)
}

class UserHasTypeEntity(id: EntityID<String>) : Entity<String>(id) {
    companion object : EntityClass<String, UserHasTypeEntity>(UserHasTypeTable)
    var user_id by UserHasTypeTable.user_id
    var user_type_id by UserHasTypeTable.user_type_id
    var created_at by UserHasTypeTable.created_at
    var updated_at by UserHasTypeTable.updated_at
    //var users by UsersEntity referencedOn  UserHasTypeTable.user_id
    fun userHasTypeResponse() = UserHasType(id.toString(), user_type_id)
}

data class UserHasType(
    val id: String, val user_type_id: String
)

User controller

class UserController {
fun login(loginBody: LoginBody) = transaction {
    val query = UsersTable.leftJoin(UserHasTypeTable).select { UsersTable.email eq loginBody.email }
    val result = UsersEntity.wrapRows(query).first()   if(loginBody.password ==  result.password)
      return@transaction result.userResponse()
   else  
      null
   }
}

UserRoute

fun Route.userRoute(userController: UserController) {
post("login") {
    val loginBody = call.receive<LoginBody>()
   
    val db = userController.login(loginBody)
    db.let {
call.respond(JsonResponse.success(loginResponse,HttpStatusCode.OK))
    }
  }
}

Ref:  https://piashcse.medium.com/get-child-data-in-parent-table-by-backreferencedon-in-kotlin-exposed-ktor-part-3-80bb14675871

Read More

Kotlin Exposed create an entity with reference for Ktor part-2

 

PostgreSql UserTable

UserTable Entity for Ktor

import org.jetbrains.exposed.dao.Entity
import org.jetbrains.exposed.dao.EntityClass
import org.jetbrains.exposed.dao.id.EntityID
import org.jetbrains.exposed.dao.id.IdTable
import org.jetbrains.exposed.sql.Column

object UsersTable : IdTable<String>("users") {
    override val id: Column<EntityID<String>> = text("user_id").uniqueIndex().entityId()
    val user_name = text("user_name")
    val email = text("email")
    val password = text("password")
    val mobile_number = text("mobile_number").nullable()
    val email_verified_at = text("email_verified_at").nullable() // so far unkmown
    val remember_token = text("remember_token").nullable()
    val verification_code = text("verification_code").nullable() // verification_code
    val created_at = text("created_at").nullable()
    val updated_at = text("updated_at").nullable()
    val is_verified = text("is_verified").nullable() // email verified by validation code
    override val primaryKey = PrimaryKey(id)
}

class UsersEntity(id: EntityID<String>) : Entity<String>(id)  {
    companion object : EntityClass<String, UsersEntity>(UsersTable)
    var userId by UsersTable.id
    var user_name by UsersTable.user_name
    var email by UsersTable.email
    var password by UsersTable.password
    var mobile_number by UsersTable.mobile_number
    var email_verified_at by UsersTable.email_verified_at
    var remember_token by UsersTable.remember_token
    var verification_code by UsersTable.verification_code
    var created_at by UsersTable.created_at
    var updated_at by UsersTable.updated_at
    var is_verified by UsersTable.is_verified
    var userType by UserTypeEntity via UserTypeTable
    fun userResponse() = UsersResponse(userId.value, user_name,email, mobile_number, email_verified_at, remember_token, created_at, updated_at, is_verified)
}
data class UsersResponse(  val userId :String,
                           val userName :String,
                           val email :String,
                           val mobileNumber:String?,
                           val emailVerifiedAt :String?,
                           val rememberToken:String?,
                           val createdAt:String?,
                           val updatedAt :String?,
                           val isVerified :String?)

UserHasTypeTable Entity

import org.jetbrains.exposed.dao.Entity
import org.jetbrains.exposed.dao.EntityClass
import org.jetbrains.exposed.dao.id.EntityID
import org.jetbrains.exposed.dao.id.IdTable
import org.jetbrains.exposed.sql.Column

object UserHasTypeTable : IdTable<String>("user_has_type") {
    override val id: Column<EntityID<String>> = text("user_has_type_id").uniqueIndex().entityId()
    val user_id = text("user_id").references(UsersTable.id)
    val user_type_id = text("user_type_id")
    val created_at = text("created_at")
    val updated_at = text("updated_at")
    override val primaryKey = PrimaryKey(id)
}

class UserHasTypeEntity(id: EntityID<String>) : Entity<String>(id)  {
    companion object : EntityClass<String, UserHasTypeEntity>(UserHasTypeTable)
    var id_ by UserHasTypeTable.id
    var user_id by UserHasTypeTable.user_id
    var user_type_id by UserHasTypeTable.user_type_id
    var created_at by UserHasTypeTable.created_at
    var updated_at by UserHasTypeTable.updated_at
    var users by UsersEntity referencedOn  UserHasTypeTable.user_id
}

Ref: https://piashcse.medium.com/kotlin-exposed-create-entity-with-reference-for-ktor-part-2-1905836c400c

Read More

PostgreSQL database connection in Ktor part-1

PostgreSQL database connection in Ktor. 

 build.gradle dependency

// Exposed ORM library
implementation "org.jetbrains.exposed:exposed-core:0.35.1"
implementation "org.jetbrains.exposed:exposed-dao:0.35.1"
implementation "org.jetbrains.exposed:exposed-jdbc:0.35.1"implementation "org.postgresql:postgresql:42.2.2"
implementation 'com.zaxxer:HikariCP:3.4.2'

resource/hikari.properties

dataSourceClassName=org.postgresql.ds.PGSimpleDataSource
dataSource.user=postgres
dataSource.password=p123
dataSource.databaseName=db_ktor
dataSource.portNumber=5432
dataSource.serverName=localh

Database helper class

object DatabaseFactory {
    fun init() {
        // Database.connect(hikari())
        initDB()
        transaction {
            //create(Fruits, UserTable, ProductTable)
        }
    }

    private fun initDB() {
        // database connection is handled from hikari properties
        val config = HikariConfig("/hikari.properties")
        val ds = HikariDataSource(config)
        Database.connect(ds)
    }

    // database connection for h2 d
    private fun hikari(): HikariDataSource {
        val config = HikariConfig()
        config.driverClassName = "org.h2.Driver"
        config.jdbcUrl = "jdbc:h2:file:~/documents/db/h2db"
        config.maximumPoolSize = 3
        config.isAutoCommit = false
        config.transactionIsolation = "TRANSACTION_REPEATABLE_READ"
        config.validate()
        return HikariDataSource(config)
    }
}

Application class

fun main() {
    //val environment = System.getenv("KTOR_ENVIRONMENT") ?: "development"
    val configName = "application.conf"
    val appEngineEnv = applicationEngineEnvironment {
        config = HoconApplicationConfig(ConfigFactory.load(configName))
        log = LoggerFactory.getLogger("ktor.application")
        module {
            DatabaseFactory.init()
        }
        connector {
            host = config.property("ktor.deployment.host").getString()
            port = config.property("ktor.deployment.port").getString().toInt()
        }
    }

    embeddedServer(Netty, appEngineEnv).start(wait = true)
}

Read More