This year has been an exciting time as I 1transitioned to a new team and learned many new things.
We held an annual retrospective within the team, and while preparing for it, I reflected on my experiences and thoughts about this year at work.

Design

One of the biggest lessons I learned this year after joining a new team was about design.
In my previous team, most tasks involved maintenance, with little opportunity for new development. As a result, I rarely participated in large-scale design meetings.
Joining the new team gave me the chance to repeatedly participate in designing new modules and subsequent batch development, which proved to be a valuable learning experience.

It was very helpful to participate in the design meeting to think, draw, and discuss the structure.
I could see how to discuss it and what parts to point out.
At first, I had little experience in design, so there were many designs that I thought “Oh, I see.”
Later, there were many cases where I thought, “I thought this way, but the team members’ ideas are better.”

!!! Initially, with limited experience, I often just acknowledged the designs without much input. Later, I found myself thinking, “I would approach this differently, but my teammates have great ideas,” which made the process more engaging.

While I am still learning, I hope to have a greater influence on design next year.
There are some design-related insights I want to document here.

Synchronization

We developed a synchronization server closely tied to the client app.
It provides values to the client to differentiate between an initial synchronization and the current synchronization state.

The design follows these principles:

  1. The value must be unintelligible to the client. Therefore, we encrypt data that only we can understand.
    • If the client could interpret the value, it might modify the data directly, potentially leading to uncontrollable synchronization scenarios or other issues.
  2. If the client modifies the value arbitrarily, a synchronization error should occur.
    • This is for the same reason as above. In case of an error, the client is guided to perform an initial synchronization to restart the process.

This approach reflects the know-how we’ve gained through our synchronization server development.

Batch

One of the services we manage involves four batch servers.
These servers were added one by one as needed, which has made the setup cumbersome and confusing.

When designing batches for new modules, we aimed not to follow the existing structure. Instead, we created a batch module that allows for the addition of batch jobs.
We developed batch servers connected via Kafka, designed to perform multiple job tasks, and implemented additional tasks when necessary.

This experience taught me the importance of initial design, not just for services but also for batch servers.
Additionally, I learned that delete operations must be performed safely and precisely.

Language

Studying and applying Golang to batch development also stands out in my memory.
I had previously studied Golang but had set it aside. Revisiting it to optimize batch performance led to designing and developing a batch in Golang.

The importance of language became clear through the following steps.

  1. The performance of the existing Python batch was poor.
  2. There was obvious room for improvement, so we upgraded the batch by optimizing parts of the Python code.
  3. Golang’s goroutines (lightweight threads) and high performance seemed well-suited for our needs.
  4. We redesigned and developed the batch using Golang.
  5. This resulted in nearly 100x performance improvement on a single EC2 instance.

Through this process, I realized.

  1. Studying without actual development does not last long.
  2. Code decays and becomes forgotten.
  3. Considering the characteristics of the language is essential (ex, Golang’s efficient goroutines).

Automation

We achieved significant automation across various tasks last year.
I believe automation has these advantages.

  1. Tedious tasks no longer need to be done manually.
  2. Developing automation tools for repetitive tasks is a fun challenge.

Our team automated several tasks this year.

Slack Notifications for Errors or Issues in WAS and Batch

We implemented Slack notifications for 500 errors in a new module we developed. This module is maintained so that no unknown 500 errors occur. If any arise, we are notified immediately and address them. (Though I suspect there are some 500 errors in other modules)
Even though we set up Grafana and metrics, they often go unused. Implementing such initial server-side features greatly aids server management.

Versioning, Release Notes, and Documentation

Our deployment process is quite tedious.
We need to document changes for the QA team, submit the changes for approval, and then write release notes, which can be a hassle. To address this, we automated the process.

We created a documentation program using Python. By adhering to a well-defined Git commit and PR convention, the program calculates major/minor/patch versions automatically and compiles the changes.
The compiled information and version updates are sent to Slack as QA documentation and change logs, and are also updated on GitHub releases.

We dockerized this Python program and added it as a step in CircleCI for each module to automate the process. This organized the tedious and laborious task.

Automation improved work efficiency and service stability. Additionally, tackling automation projects exposed me to new areas of interest and kept the work engaging.

I also realized that automation is easier than expected.
Documentation automation, in particular, always seemed inconvenient. However, it turned out to be simpler and faster to implement than I anticipated.

Testing

While developing new modules from scratch, we conducted various tests to improve performance and meet requirements.
The types of tests we performed are as follows:

   
Unit Test Unit-level tests necessary for TDD
API Test Tests conducted outside the server through actual API calls
Scenario Test Tests verifying user use-case flows (scenarios)
Performance Test Load and performance tests using tools like Pinpoint or Locust

Kent Beck’s statement that “tests provide peace of mind and trust” resonated deeply with me.
Through extensive testing, we gained confidence in our service both before and after its launch.

We also added scenario tests, which were not used in previous modules, and learned the importance of having diverse types of tests beyond just unit and API tests.
Here’s what it’s worth looking back on.

Duplicated Tests

With multiple test types, overlaps occur. Having too many tests can slow down the process, so defining clear test layers is crucial.

Performance Testing

I couldn’t participate in setting up or running performance tests due to other development priorities, which I regret.

Test Readability

There’s a tendency to justify sloppy test code with “it’s just a test.” However, this made reviews cumbersome and understanding the tests during code changes time-consuming.
I strongly dislike tests with conditional branches. If needed, I believe the tests should be split. Some team members disagree, so it may just be a matter of preference.

This year has been enjoyable. It highlighted the importance of team dynamics.
I learned a lot about development culture, such as pair programming, code reviews, and retrospectives. Our team fosters an open environment where people can share their thoughts and even uncomfortable topics freely, which I find wonderful.