Blogs on Bossagyu Blog

Three Models of Technical Proficiency: Dreyfus, Shu-Ha-Ri, and the Four Stages of Competence

Sun, 19 Apr 2026 00:00:00 +0900

Overview

The process of acquiring a skill is rarely uniform. We start by relying on manuals, then learn to judge based on context, and eventually reach a stage where the right action comes without conscious thought. To capture these gradual changes, many models have been proposed over the decades.

This article introduces three representative frameworks for technical proficiency—the Dreyfus model, Shu-Ha-Ri, and the Four Stages of Competence—and compares their origins, stages, and characteristics. The goal is to offer a toolkit for understanding where we are, designing our next step, and supporting the learning of others.

Why Understand the Stages of Proficiency

Thinking of proficiency as a set of stages provides three main benefits.

First, self-awareness. Knowing which stage you are in clarifies what to work on next. The skills that matter for a beginner (executing procedures correctly) differ greatly from those that matter for a mid-level practitioner (judgment in context). Awareness of stages prevents misdirected effort.

Second, learning design. Rather than chasing a vague final goal, stage-based milestones make progress easier to sustain and measure. Many models also suggest that the optimal way to learn changes from stage to stage.

Third, understanding others. In mentoring or team leadership, estimating someone’s current stage helps you choose the right support and level of challenge.

That said, models are not reality. Real learning is continuous and often uneven across domains. The frameworks below are best used as guideposts for thinking, not as rigid categorizations.

The Dreyfus Model

Origin and Background

The Dreyfus model was proposed in 1980 by brothers Stuart E. Dreyfus and Hubert L. Dreyfus at UC Berkeley, in a report titled “A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition.” Commissioned by the U.S. Air Force Office of Scientific Research, the report drew on pilot training, chess players, drivers, and other skilled practitioners.

Stuart specialized in operations research while Hubert specialized in philosophy (phenomenology). Together they formalized a key insight: the judgments of experts often cannot be reduced to explicit rules.

The model later became popular in software engineering thanks to books such as Andy Hunt’s Pragmatic Thinking and Learning, and has become a shared vocabulary for discussing engineer growth.

Five Stages

The Dreyfus model describes the journey from novice to expert in five stages.

flowchart LR
A[Novice] --> B[Advanced Beginner]
B --> C[Competent]
C --> D[Proficient]
D --> E[Expert]

クリックで拡大

1. Novice Detaches from context and follows given rules. Following the rules is itself challenging, leaving little room to see exceptions or the bigger picture. Think of measuring ingredients from a recipe, or operating the pedals and wheel by the manual when learning to drive.

2. Advanced Beginner Begins to notice contextual features through experience, but cannot yet distinguish important features from minor ones, and treats them with equal weight. This is the stage where learners ask many questions and encounter situations their rules don’t cover.

3. Competent Selects important information from a pool of signals and can set goals and plans on their own. Rules have become internalized. At the same time, learners begin to sense that “there is no single right answer,” and may feel the weight of decision-making.

4. Proficient Grasps situations holistically. Pattern recognition from past similar cases comes first; detailed rule-based reasoning fills in afterward. This is a hybrid stage where intuition and analysis work together.

5. Expert Arrives at answers intuitively, embedded in the situation itself, without explicit rule-following. A seasoned doctor estimates a diagnosis at a glance; a master player narrows candidate moves before deliberate analysis. However, when the environment is unusual, even experts fall back on deliberate analysis.

Cognitive Shifts Between Stages

The key insight of this model is that as stages progress, the center of cognition shifts from analysis to pattern recognition to intuition. This is not merely a speed-up but a qualitative change in what the practitioner perceives.

Moving up requires more than rule-following—it requires experience in interpreting rules in context. The Dreyfus brothers repeatedly note that mastery is tacit knowledge that textbooks alone cannot teach.

Shu-Ha-Ri

Origin and Background

Shu-Ha-Ri (守破離) is a Japanese model of training stages that emerged from traditional arts such as martial arts, tea ceremony, and flower arrangement. It appears explicitly in Fuhaku Hikki, a text by the Edo-period tea master Kawakami Fuhaku (1719-1807). Its roots trace back further, to Zeami’s (1363-1443) Fūshikaden and its idea of Jo-Ha-Kyū, and to the teachings of Sen no Rikyū.

A poem often attributed to Rikyū reads, “Keep the forms of rules and manners until you break them, and even when you leave them, do not forget the origin.” This line neatly captures the spirit of Shu-Ha-Ri: preserve the form, then break it, then transcend it—without losing its root.

In recent years, Shu-Ha-Ri has been applied to Western practices such as agile development, Scrum adoption, and design pattern learning, becoming a shared vocabulary among Japanese-speaking software professionals.

Three Stages

Shu-Ha-Ri is a three-stage model built on a master-apprentice relationship.

flowchart LR
A[Shu
Follow the form] --> B[Ha
Break the form]
B --> C[Ri
Leave the form]
C -.new form.-> A

クリックで拡大

Shu (守) Practice the forms taught by the master faithfully, without adding personal interpretation. The goal is to imprint the form onto body and mind. The “why” often comes later; what matters first is precise repetition.

Ha (破) Building on the forms absorbed in Shu, the learner incorporates forms from other schools and personal adaptations, breaking the form. This is not mere rebellion but thoughtful comparison and testing. Here, the learner finally feels the reason behind the form.

Ri (離) Leave the form and reach an independent mode of being. This is not discarding the form—by now, the form has become part of the self. Externally, the practitioner appears free and unconstrained. The phrase “do not forget the origin” warns that even at Ri, the root remains in Shu.

Differences from Western Models

Where the Dreyfus model describes changes in an individual’s cognition, Shu-Ha-Ri presumes that forms are passed within a community. The form encodes the history of the school and the wisdom of predecessors; inheriting it accurately is the purpose of Shu.

Shu-Ha-Ri also describes how the learner’s agency changes. Reception dominates in Shu, dialogue and testing in Ha, creation in Ri. The model implies a maturation of the whole person, not only a skill upgrade.

Modern Applications

Learning design patterns: Apply patterns literally (Shu), adapt them to project context (Ha), and eventually design from first principles without needing patterns (Ri).
Scrum adoption: Start by following the Scrum Guide strictly, then adapt events and roles once the team has matured.
Onboarding in general: As a guide for how a mentor’s stance should shift from prescribing procedures to reviewing decisions to co-creating systems.

The Four Stages of Competence

Origin and Background

The “Four Stages of Competence” (also known as the Conscious Competence Ladder) is commonly attributed to Noel Burch of Gordon Training International in the 1970s. However, similar ideas are sometimes attributed to Abraham Maslow or Thomas Gordon, so the origin is not entirely settled.

The distinctive feature of this model is that it combines two axes: whether a learner has a capability, and whether they are aware of having or lacking it. By focusing on awareness as much as ability, the model is particularly good at describing the learner’s psychological state.

Four Stages

The four stages can be organized as a 2x2 matrix.

quadrantChart
title Four Stages of Competence
x-axis Low capability --> High capability
y-axis Unaware --> Aware
quadrant-1 Conscious Competence
quadrant-2 Conscious Incompetence
quadrant-3 Unconscious Incompetence
quadrant-4 Unconscious Competence

クリックで拡大

1. Unconscious Incompetence The learner is not aware of what they cannot do, and thus sees no need to learn. Escape from this stage usually requires external feedback or a vivid encounter with one’s own limits.

2. Conscious Incompetence The learner recognizes their inability. This is psychologically the hardest stage, where self-doubt can derail learning. It helps to have an environment that offers small wins and a relationship that makes failure safe.

3. Conscious Competence The learner can perform, but only with deliberate effort and attention. Execution is slow and tiring, yet consistently produces results. Repetition here builds the foundation for the next stage.

4. Unconscious Competence The learner performs naturally, without conscious thought. Attention is freed for other things. A new challenge emerges: because the skill is tacit, it becomes hard to explain “why” to others.

Psychological Significance and a Fifth Stage

Unlike other models, this one explicitly names the valley of Conscious Incompetence. Learning starts with confronting one’s incompetence; how well we support learners at this stage largely determines whether they continue.

Some variants add a fifth stage: Conscious Competence of Unconscious Competence, sometimes called “reflective competence.” This is the state of being able to re-examine one’s tacit skill and articulate it for others—a state mentors and teachers aspire to.

Comparing the Three Frameworks

At first glance the three models look similar, but they focus on different aspects.

Dimension	Dreyfus Model	Shu-Ha-Ri	Four Stages of Competence
Number of stages	5	3	4 (5 in variants)
Focus	Quality of judgment and action	Relationship to form	Combination of ability and awareness
Cultural background	Western / phenomenology	Japanese / traditional arts	Western / education psychology
Period of formulation	1980	Edo period (with older roots)	1970s
Implied learner	An individual cognition changing	A member inheriting form within a community	A self moving between conscious and unconscious
Top-stage ideal	Intuitive judgment	Free action beyond form	Automatic, effortless execution

Commonalities

Beneath the surface, the models share structural features.

A direction from external dependence to autonomy. Rules, forms, or conscious effort are the starting point; they become internalized over time.
The top stage is described as mastery that resists easy verbalization. Dreyfus’s Expert, Shu-Ha-Ri’s Ri, and the Four Stages’ Unconscious Competence all point to territory beyond explicit instruction.
Progression is not linear. Earlier skills remain as foundations rather than being discarded.

Differences

The points of focus clearly diverge.

The Dreyfus model focuses on the quality of judgment, tracing the cognitive shift from rule-application to intuitive pattern recognition.
Shu-Ha-Ri focuses on relationship with form and the community that transmits it.
The Four Stages of Competence focuses on states of awareness, especially the pivot of realizing one’s own incompetence.

Which to Use When

To analyze patterns of behavior and judgment → the Dreyfus model
To describe a learning environment with a master, mentor, or shared form → Shu-Ha-Ri
To support learners psychologically, especially near the valley of doubt → the Four Stages of Competence

They complement each other, and it is often useful to view a single learner through multiple lenses.

Applications: Learning and Product Design

These frameworks are useful not only as knowledge but as tools. Three common applications:

1. Designing Your Own Learning

Assess your current stage through multiple models. If you look solid as a Competent on Dreyfus but are still at Conscious Competence on the awareness axis, you have discovered a concrete next step. Noticing that you lack “experience breaking the form” can motivate you to seek exposure to other schools (other teams, tools, or languages).

2. Mentoring and Supporting Others

Different stages call for different involvement.

For novices and the unconsciously incompetent: provide explicit rules and a safe environment for repetition.
For the consciously incompetent: stack small wins and affirm that failure is part of the process.
For Competent learners / those at Ha: hand over decision-making responsibility and support through decision reviews.
For Proficient, Expert, or Ri learners: create opportunities to articulate their tacit knowledge for others.

Misjudging the stage turns help into imposition or detachment. Being stage-aware improves the precision of involvement.

3. Product and App Design

These models also inform how we design for users with varying levels of expertise.

Onboarding: Provide explicit guides, step-by-step flows, and constraints for beginners (the Shu stage).
Mode switching: Offer shortcuts and advanced modes for experienced users, and reduce boilerplate hints (Ha and Ri).
Supporting Conscious Incompetence: Users who “don’t know what they don’t know” benefit from diagnostics and tutorials that point to what needs learning.
Feedback that promotes stage transitions: Rather than just returning errors, show hints for the learner’s next step.

Viewing a product through these lenses tends to clarify feature priorities and information architecture.

Summary

Among the many frameworks describing technical proficiency, the Dreyfus model, Shu-Ha-Ri, and the Four Stages of Competence offer complementary lenses.

The Dreyfus model captures changes in the quality of judgment.
Shu-Ha-Ri captures changes in the relationship with form.
The Four Stages of Competence captures the interplay of awareness and ability.

Models are maps, not the territory. Real learners change continuously, and their stages can vary across domains. Still, a map helps us estimate where we are and choose the next step. Use these frameworks as situation-appropriate tools—for your own learning, your relationships with others, and the products you build.

References

Stuart E. Dreyfus, Hubert L. Dreyfus, “A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition” (1980, U.S. Air Force Office of Scientific Research)
Andy Hunt, Pragmatic Thinking and Learning: Refactor Your Wetware (2008, Pragmatic Bookshelf)
Kawakami Fuhaku, Fuhaku Hikki
Zeami, Fūshikaden
Noel Burch, “The Four Stages for Learning Any New Skill” (1970s, Gordon Training International)

Getting Started with FastAPI: Building a CRUD API Tutorial

Sat, 28 Feb 2026 00:00:00 +0900

Overview

FastAPI is a high-performance web framework for building APIs with Python. Its key features include:

Type hint-based: Leverages Python type hints for automatic request/response validation
Auto-generated documentation: Swagger UI (OpenAPI) is automatically generated, allowing you to test APIs from the browser
High performance: Built on Starlette + Pydantic, delivering performance comparable to Node.js and Go

In this article, we’ll walk through building a TODO app CRUD API using FastAPI.

Prerequisites

Python 3.11 or higher
uv installed (see Setting Up a Python Development Environment on Mac with UV)

Environment Setup

First, create a project and install the required packages.

1
2
3


uv init fastapi-todo
cd fastapi-todo
uv add fastapi uvicorn

fastapi is the web framework itself, and uvicorn is the ASGI server.

Hello World

Let’s start by verifying FastAPI works with a minimal setup. Create a main.py file.

1
2
3
4
5
6
7
8


from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
 return {"message": "Hello, FastAPI!"}

Start the server:

1

uv run uvicorn main:app --reload

The --reload option automatically reloads the server when code changes are detected.

Open http://localhost:8000 in your browser and you’ll see:

1

{"message": "Hello, FastAPI!"}

Swagger UI

One of FastAPI’s greatest features is its auto-generated API documentation. Visit http://localhost:8000/docs to see the Swagger UI.

You can test APIs directly from here, eliminating the need for curl or Postman during development.

Implementing the CRUD API

Let’s implement the CRUD API for a TODO app.

Defining Models

First, define the TODO data structure using Pydantic models. Replace main.py with the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from fastapi import FastAPI, HTTPException
from pydantic import BaseModel


app = FastAPI()


class TodoCreate(BaseModel):
 title: str
 completed: bool = False


class TodoResponse(BaseModel):
 id: int
 title: str
 completed: bool


todos: dict[int, dict] = {}
next_id: int = 1

TodoCreate: Request body schema for creating and updating TODOs
TodoResponse: Response type definition
todos: In-memory data store (dict)
next_id: Auto-incrementing ID counter

Create (POST)

1
2
3
4
5
6
7


@app.post("/todos", response_model=TodoResponse)
def create_todo(todo: TodoCreate):
 global next_id
 todo_data = {"id": next_id, "title": todo.title, "completed": todo.completed}
 todos[next_id] = todo_data
 next_id += 1
 return todo_data

By specifying response_model=TodoResponse, the response type is reflected in the Swagger UI.

List (GET)

1
2
3


@app.get("/todos", response_model=list[TodoResponse])
def list_todos():
 return list(todos.values())

Get by ID (GET)

1
2
3
4
5


@app.get("/todos/{todo_id}", response_model=TodoResponse)
def get_todo(todo_id: int):
 if todo_id not in todos:
 raise HTTPException(status_code=404, detail="Todo not found")
 return todos[todo_id]

Returns a 404 error via HTTPException when the ID doesn’t exist.

Update (PUT)

1
2
3
4
5
6
7


@app.put("/todos/{todo_id}", response_model=TodoResponse)
def update_todo(todo_id: int, todo: TodoCreate):
 if todo_id not in todos:
 raise HTTPException(status_code=404, detail="Todo not found")
 todo_data = {"id": todo_id, "title": todo.title, "completed": todo.completed}
 todos[todo_id] = todo_data
 return todo_data

Delete (DELETE)

1
2
3
4
5
6


@app.delete("/todos/{todo_id}")
def delete_todo(todo_id: int):
 if todo_id not in todos:
 raise HTTPException(status_code=404, detail="Todo not found")
 del todos[todo_id]
 return {"detail": "Todo deleted"}

Complete Code

Here’s the full main.py with all endpoints combined:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58


from fastapi import FastAPI, HTTPException
from pydantic import BaseModel


app = FastAPI()


class TodoCreate(BaseModel):
 title: str
 completed: bool = False


class TodoResponse(BaseModel):
 id: int
 title: str
 completed: bool


todos: dict[int, dict] = {}
next_id: int = 1


@app.post("/todos", response_model=TodoResponse)
def create_todo(todo: TodoCreate):
 global next_id
 todo_data = {"id": next_id, "title": todo.title, "completed": todo.completed}
 todos[next_id] = todo_data
 next_id += 1
 return todo_data


@app.get("/todos", response_model=list[TodoResponse])
def list_todos():
 return list(todos.values())


@app.get("/todos/{todo_id}", response_model=TodoResponse)
def get_todo(todo_id: int):
 if todo_id not in todos:
 raise HTTPException(status_code=404, detail="Todo not found")
 return todos[todo_id]


@app.put("/todos/{todo_id}", response_model=TodoResponse)
def update_todo(todo_id: int, todo: TodoCreate):
 if todo_id not in todos:
 raise HTTPException(status_code=404, detail="Todo not found")
 todo_data = {"id": todo_id, "title": todo.title, "completed": todo.completed}
 todos[todo_id] = todo_data
 return todo_data


@app.delete("/todos/{todo_id}")
def delete_todo(todo_id: int):
 if todo_id not in todos:
 raise HTTPException(status_code=404, detail="Todo not found")
 del todos[todo_id]
 return {"detail": "Todo deleted"}

Testing

Start the server and test through the Swagger UI.

1

uv run uvicorn main:app --reload

Visit http://localhost:8000/docs and test in the following order:

1. Create a TODO

Open POST /todos, click “Try it out”, and enter:

1
2
3
4


{
 "title": "Go shopping",
 "completed": false
}

Click “Execute” to get a response like:

1
2
3
4
5


{
 "id": 1,
 "title": "Go shopping",
 "completed": false
}

2. List TODOs

Execute GET /todos to see the list of created TODOs.

3. Update

Use PUT /todos/1 to change completed to true:

1
2
3
4


{
 "title": "Go shopping",
 "completed": true
}

4. Delete

Execute DELETE /todos/1 to delete the TODO. Verify with GET /todos that an empty array is returned.

Summary

In this article, we built a TODO CRUD API using FastAPI. Here are the key advantages:

Automatic request/response validation with Pydantic models
Interactive API documentation via Swagger UI
Simple API specification definition using just Python type hints

We used an in-memory data store in this tutorial, but in real applications, you would use an ORM like SQLAlchemy to connect to a database.

Setting Up a Python Development Environment on Mac with UV - Python environment setup
Event-Driven Backend for Syncing GCS with Gemini File Search API - Practical FastAPI example

Designing an Event-Driven Backend to Sync GCS with Gemini File Search API

Sun, 15 Feb 2026 00:00:00 +0900

Overview

I participated in the “Law x Digital” Hackathon (3rd edition) hosted by Japan’s Digital Agency and developed a cross-source legal document search product called “Lawve.” Lawve enables natural language search across e-Gov legal data and user-uploaded documents.

This article focuses on the backend architecture design that I was responsible for. The backend’s primary responsibility is to “automatically sync the state of Gemini File Search API whenever files are added to or removed from Google Cloud Storage (GCS).”

System Architecture

Here is the overall system architecture of Lawve.

flowchart TB
subgraph User
U[Browser]
end
subgraph GCP
subgraph Frontend
FE[Cloud Run
Next.js]
end
subgraph Backend
CR[Cloud Run
FastAPI]
end
subgraph Storage & Data
GCS[(Cloud Storage)]
FS[(Firestore)]
end
subgraph Event Infrastructure
EA[Eventarc]
end
subgraph AI
GEMINI[Gemini File
Search API]
end
end
subgraph External API
EGOV[e-Gov
Law API]
end
U --> FE
FE --> GEMINI
FE --> FS
FE --> EGOV
FE -->|File Upload| GCS
GCS -->|Event Notification| EA
EA -->|CloudEvents| CR
CR -->|Register/Delete| GEMINI
CR -->|Download| GCS

クリックで拡大

Component Roles

Component	Role
Cloud Run (Next.js)	Frontend: search UI, file upload, search result display
Cloud Run (FastAPI)	Backend: syncs Gemini File Search API in response to GCS events
Cloud Storage	Document storage, serves as the Single Source of Truth
Eventarc	Routes GCS file change events to Cloud Run
Gemini File Search API	Provides full-text and semantic search for documents
Firestore	Manages document metadata and comments
e-Gov Law API	Retrieves legal data

This article focuses on the backend Cloud Run (FastAPI) design.

Event-Driven Architecture Design

Why Event-Driven?

In a legal document search product, documents need to be automatically registered with Gemini File Search API when placed in GCS. While a synchronous API approach from the frontend was an option, I chose an event-driven architecture for the following reasons:

Loose coupling: The frontend only needs to place files in GCS without knowing about the backend
Reliability: Eventarc reliably detects GCS events and delivers them to Cloud Run
Tool compatibility: Files placed via CLI or scripts are synced in the same way

Event Routing with Eventarc

Eventarc monitors two types of GCS events and routes them to the Cloud Run endpoint.

Event	Trigger Condition
`google.cloud.storage.object.v1.finalized`	When a file is uploaded to GCS
`google.cloud.storage.object.v1.deleted`	When a file is deleted from GCS

The Cloud Run endpoint receives events in CloudEvents format.

1
2
3
4
5
6
7
8


@app.post("/")
async def handle_storage_event(request: Request):
 headers = dict(request.headers)
 body = await request.body()
 event = from_http(headers, body)

 result = event_handler.process(event)
 return result

File Upload Processing Flow

When a file is placed in GCS, it is processed through the following flow:

Event reception and path filtering: Only files with the file-search/archive/ prefix are processed
Metadata extraction: Automatically extract law_id and source_type from the GCS path
Delete existing documents: Remove documents with the same source_path to prevent duplicates
File download: Download from GCS to a local temporary file
Register with File Search Store: Upload to Gemini File Search API with metadata

1
2
3
4
5
6


class EventHandler:
 def _handle_event_by_type(self, event_type, bucket_name, file_name):
 if event_type == "google.cloud.storage.object.v1.finalized":
 self._handle_file_upload(bucket_name, file_name)
 elif event_type == "google.cloud.storage.object.v1.deleted":
 self._handle_file_delete(bucket_name, file_name)

File Deletion Processing Flow

When a file is deleted from GCS, the corresponding document is also deleted from Gemini File Search API.

Event reception and path filtering
Search File Search Store by metadata: Identify the matching document using the source_path metadata
Delete document: Remove the document from File Search Store

Integration with Gemini File Search API

Unified GCP Ecosystem

Lawve adopts a policy of unifying the entire infrastructure on GCP. While other RAG services such as OpenAI and Pinecone were options, I chose Gemini File Search API for its seamless integration with Cloud Run, GCS, and Eventarc.

File Search Store Overview

Gemini File Search Store is a managed service that vectorizes and indexes registered documents to provide semantic search. The backend performs the following operations:

Document registration: Upload files with metadata
Document search: Search existing documents by metadata
Document deletion: Remove documents that are no longer needed

Metadata-Based Management

Each document is tagged with three metadata fields for management purposes.

Metadata	Purpose	Example
`law_id`	Legal document ID for unique identification	`323AC0000000205`
`source_type`	Document classification	`user`, `doc`, `admin`
`source_path`	Full GCS path for unique document identification	`gs://bucket/file-search/archive/user/323AC0000000205/medical-law.txt`

Since source_path is the GCS path itself, it uniquely maps GCS files to File Search Store documents.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


metadata = {
 "law_id": law_id,
 "source_path": source_path,
 "source_type": source_type
}

custom_metadata = [
 {"key": key, "string_value": value}
 for key, value in metadata.items()
]

Design Patterns for Extensibility

Path-Based Metadata Extraction

The backend is designed to process any file placed in a specific GCS path, not limited to legal documents. The path convention is as follows:

1

gs://<bucket>/file-search/archive/<source_type>/<law_id>/<file_name>

The source_type and law_id are automatically extracted from the path using a regular expression.

1
2
3
4
5
6
7
8


# Path pattern
GCS_PATH_PATTERN = r'file-search/archive/([^/]+)/([^/]+)/.+'

def parse_gcs_path(gcs_path: str) -> Tuple[Optional[str], Optional[str]]:
 match = re.search(GCS_PATH_PATTERN, gcs_path)
 if match:
 return match.group(1), match.group(2)
 return None, None

This design allows any type of document, such as internal documents or technical documentation, to be managed through the same mechanism. You can add new classifications simply by changing the source_type, and no backend code changes are required as long as the path convention is followed.

Guaranteeing State Synchronization Between GCS and Gemini File Search API

GCS serves as the Single Source of Truth, and the design principle is to always keep the File Search Store state in sync with GCS.

Idempotent uploads: When the same file is re-uploaded, existing documents are deleted before re-registration. This prevents duplicates while supporting content updates.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


def process_file_upload(self, bucket_name, file_path):
 source_path = f"gs://{bucket_name}/{file_path}"

 # Check and delete existing documents
 self._delete_existing_documents(source_path)

 # New upload
 law_id = extract_law_id(file_path)
 local_file_path = self.gcs_client.download_to_temp_file(bucket_name, file_path)
 # ...
 success = self.store_client.upload_file(local_file_path, file_path, metadata)
 return success

Cascading deletes: When a file is deleted from GCS, the corresponding document is automatically removed from the File Search Store. The system searches by source_path metadata and deletes the matching document.

1
2
3
4
5
6
7
8


def process_file_delete(self, bucket_name, file_path):
 source_path = f"gs://{bucket_name}/{file_path}"

 documents = self.store_client.find_documents_by_source_path(source_path)
 if not documents:
 return False

 return self.store_client.delete_document(documents[0])

Local Download Design

Registration from GCS to Gemini File Search API is done by first downloading to a local temporary file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


def download_to_temp_file(self, bucket_name, file_path):
 _, ext = os.path.splitext(file_path)
 if not ext:
 ext = ".txt"

 temp_fd, temp_path = tempfile.mkstemp(suffix=ext)
 os.close(temp_fd)

 blob.download_to_filename(temp_path)
 return temp_path

While direct upload would be simpler, downloading locally first provides the following benefits:

Extensibility: Enables future file transformation steps such as converting Excel to CSV before upload
MIME detection: Preserving the file extension enables accurate MIME type detection
Debugging: Allows inspection of local file contents when issues occur

Error Handling and Operational Design

Always Return 200 OK

The Cloud Run endpoint always returns 200 OK to Eventarc requests, even when errors occur.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


@app.post("/")
async def handle_storage_event(request: Request):
 try:
 headers = dict(request.headers)
 body = await request.body()
 event = from_http(headers, body)
 result = event_handler.process(event)
 return result
 except Exception as e:
 return {
 "status": "error",
 "message": f"Event processing failed: {str(e)}",
 "note": "Error logged and notified via Slack"
 }

This design prevents Eventarc retries. If the endpoint returns 4xx/5xx, Eventarc resends the event, causing the same error to repeat. This results in duplicate Slack notifications and polluted logs. By returning 200 OK and handling errors through Slack notifications and logging, operational issues are avoided.

Error Monitoring via Slack Notifications

When a File Search Store upload fails, an error notification is sent via Slack Webhook. The notification includes the GCS path and error details, enabling operators to quickly identify the problem.

While this Slack-based monitoring is sufficient for hackathon-scale development, a production environment would require more robust monitoring and recovery mechanisms, such as Cloud Monitoring alerts, Cloud Logging integration, and Dead Letter Queues for reprocessing failed events.

Lazy Initialization for Client Management

Gemini API clients and GCS clients use lazy initialization.

1
2
3
4
5


@property
def client(self) -> Optional[genai.Client]:
 if self._client is None and self.api_key:
 self._client = genai.Client(api_key=self.api_key)
 return self._client

This allows the application to start even without API keys configured, making it easier to work in test environments or local development without providing all environment variables.

Summary

The Lawve backend was built on the following design principles:

Loosely coupled automatic sync via event-driven architecture: Using Eventarc to keep the frontend and backend decoupled while automatically reflecting GCS changes in Gemini File Search API
Metadata-driven extensible design: Automatically extracting metadata from path conventions to support documents beyond legal data
GCS as Single Source of Truth: Idempotent uploads and cascading deletes ensure the GCS and File Search Store states always match

By combining an event-driven architecture with GCP managed services, I was able to build a reliable document synchronization platform with minimal code. The simplicity of making documents searchable just by placing files in GCS was a significant advantage during the time-limited hackathon development.

The source code is available on GitHub.

Setting Up a Python Development Environment on Mac with UV (Python environment setup)
How to Use the OpenAI Response API (using OpenAI API)

Building a Cleaning Reminder Bot with AWS Lambda + LINE

Sat, 07 Feb 2026 19:00:00 +0900

Overview

I built a cleaning reminder LINE Bot for family use. This article explains the technical implementation details.

See the completed Bot here: Cleaning Reminder Bot

Architecture

flowchart LR
subgraph User
A[LINE User]
end
subgraph AWS
B[API Gateway]
C[Lambda
process_user_message]
D[(S3
JSON)]
E[Lambda
push_message_periodically]
F[EventBridge
hourly]
end
A -->|Send message| B
B --> C
C <--> D
E <--> D
F -->|Trigger| E
E -->|Notify| A

クリックで拡大

Services Used

Service	Purpose
API Gateway (HTTP API)	Receive LINE Webhooks
Lambda	Message processing, scheduled notifications
S3	Store per-user task data
EventBridge	Scheduled execution

Why This Architecture

Serverless: Pay only for what you use, zero operational overhead
S3: Cheaper than RDS, simple JSON management
SAM: Infrastructure as Code for deployment management

Project Structure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


clean-bot/
├── lib/ # Core modules
│ ├── clean_task.py # Task state management
│ ├── message.py # Command parser
│ ├── line.py # LINE API
│ └── s3_client.py # S3 operations
├── test/ # Tests
├── line_clean_bot.py # Lambda entry point
├── template.yaml # SAM template
└── Makefile # Development commands

Implementation Details

1. Lambda Entry Points

Two Lambda functions are defined.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


# line_clean_bot.py

def process_user_message(event, context):
 """LINE message webhook handler"""
 body = json.loads(event.get('body'))

 # Get user ID or group ID
 if body['events'][0]['source']['type'] == 'user':
 line_id = body['events'][0]['source']['userId']
 else:
 line_id = body['events'][0]['source']['groupId']

 message = body['events'][0]['message']['text']

 # Fetch user data from S3
 s3client = S3client(BUCKET_NAME)
 obj_key = line_id + '.json'
 if not s3client.check_exist_object(obj_key):
 s3client.update_object(obj_key, '{"tasks": []}')

 # Process message and reply
 # ...

def push_message_periodically(event, context):
 """Send reminder notifications on schedule"""
 current_time = datetime.now() + timedelta(hours=9) # JST

 s3client = S3client(BUCKET_NAME)
 for obj_list in s3client.list_objects():
 clean_task = CleanTask(s3client.get_object_body(obj_list.key))

 # Check if notification conditions are met
 if clean_task.should_notify(current_time):
 # Send message
 # ...

2. Task Deadline Management

Tasks are evaluated based on “days since last completed.”

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


# lib/clean_task.py

def __evaluate_cleanup_timing(self, task):
 """Determine if task is overdue"""
 task_time = datetime.strptime(task['updated_at'], self.date_format)
 return (self.now - task_time).days >= int(task['duration'])

def get_todo_tasks(self):
 """Get list of overdue tasks"""
 return [task for task in self.tasks
 if not task.get('paused', False)
 and self.__evaluate_cleanup_timing(task)]

3. Notification Logic

Notifications are sent only on user-specified days and times.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


def should_notify(self, current_time):
 """Determine if notification should be sent"""
 if not self.notification['enabled']:
 return False

 # Check day of week
 weekday_map = {0: 'Mon', 1: 'Tue', 2: 'Wed', 3: 'Thu', 4: 'Fri', 5: 'Sat', 6: 'Sun'}
 current_weekday = weekday_map[current_time.weekday()]
 if current_weekday not in self.notification['days']:
 return False

 # Check time
 if current_time.hour < self.notification['hour']:
 return False

 # Check if already notified today
 # ...

 return True

4. Command Parser

Commands are parsed using simple string splitting.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


# lib/message.py

def get_return_message(self, message, s3client):
 task_operation = self.__get_task_operation_name(message) # First word
 task_name = self.__get_task_name(message) # Second word

 if task_operation == 'done':
 for task_name in self.__get_all_task_name(message):
 self.clean_task.update_task_updated_at(task_name)
 s3client.update_object(self.object_keyname, self.clean_task.get_json())
 return "Completed"

 if task_operation == 'add':
 duration = self.__get_duration(message) # Third word
 self.clean_task.add_task(task_name, duration)
 # ...

5. Data Structure (JSON)

One JSON file per user is stored in S3.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


{
 "tasks": [
 {
 "task_name": "vacuum",
 "updated_at": "2024-01-01 12:00:00",
 "duration": 7,
 "paused": false
 }
 ],
 "notification": {
 "enabled": true,
 "days": ["Mon", "Wed", "Fri"],
 "hour": 7,
 "last_notified_at": "2024-01-01 07:00:00"
 }
}

SAM Template

Infrastructure is defined using AWS SAM.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52


# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Parameters:
 ChannelAccessToken:
 Type: String
 NoEcho: true
 BucketName:
 Type: String
 Default: your-bucket-name

Globals:
 Function:
 Runtime: python3.11
 Timeout: 300
 Environment:
 Variables:
 CHANNEL_ACCESS_TOKEN: !Ref ChannelAccessToken
 BUCKET_NAME: !Ref BucketName

Resources:
 # Message processing Lambda
 ProcessUserMessageFunction:
 Type: AWS::Serverless::Function
 Properties:
 Handler: line_clean_bot.process_user_message
 CodeUri: .
 Policies:
 - S3CrudPolicy:
 BucketName: !Ref BucketName
 Events:
 ApiEvent:
 Type: HttpApi
 Properties:
 Path: /process_user_message
 Method: ANY

 # Scheduled notification Lambda
 PushMessagePeriodicallyFunction:
 Type: AWS::Serverless::Function
 Properties:
 Handler: line_clean_bot.push_message_periodically
 CodeUri: .
 Policies:
 - S3CrudPolicy:
 BucketName: !Ref BucketName
 Events:
 ScheduleEvent:
 Type: Schedule
 Properties:
 Schedule: cron(0 * * * ? *)  # Every hour at minute 0

Deployment

1. Prerequisites

AWS CLI configured
SAM CLI installed
LINE Messaging API channel access token obtained

2. Build & Deploy

1
2
3
4
5


# Build
sam build

# Deploy
sam deploy --parameter-overrides ChannelAccessToken=your_token

3. Configure LINE Webhook

Set the API endpoint URL output after deployment as the Webhook URL in the LINE Developers console.

Local Development

1
2
3
4
5
6
7


# Start local API server
sam local start-api --env-vars env.json

# Run tests
curl -X POST http://localhost:3000/process_user_message \
 -H "Content-Type: application/json" \
 -d @events/line_message.json

Cost

Estimated monthly cost (assuming 100 users):

Service	Cost
Lambda	Nearly free (within free tier)
API Gateway	Nearly free (within free tier)
S3	A few cents/month
Total	A few cents to a few dollars/month

Summary

Serverless architecture minimizes operational costs
S3 + JSON for simple data management
SAM for infrastructure as code
“Days elapsed” deadline management suited for cleaning tasks

Source code is available on GitHub.

How to Use the OpenAI Response API

Wed, 24 Dec 2025 10:00:00 +0900

Overview

This article summarizes how to use the OpenAI Response API for text generation. It covers the official documentation, how to obtain an API key, and a minimal Python example.

What is the Response API?

The Response API is OpenAI’s unified API for text generation, tool calls, and multi-turn interactions. It works for both one-off requests and conversational workflows.

Official documentation

The most reliable source for the latest specs is the official documentation.

Response API reference: https://platform.openai.com/docs/api-reference/responses

How to get an API key

You need an OpenAI API key to use the Response API. Follow these steps:

Sign in to the OpenAI dashboard
Open the API Keys page: https://platform.openai.com/api-keys
Create a new key with “Create new secret key”

Store the key in an environment variable for safety.

1

export OPENAI_API_KEY="<YOUR_API_KEY>"

Python sample code

Install the Python SDK with uv:

1

uv pip install openai

Below is a minimal sample that calls the Response API.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


from openai import OpenAI

client = OpenAI()

response = client.responses.create(
 model="gpt-4.1-mini",
 input="Summarize the key features of the Response API in 3 lines.",
)

print(response.output_text)

Example output

1
2
3


The Response API is a unified endpoint for one-off and conversational generation.
It lets you retrieve model output with a simple request structure.
Start with the docs and expand as your use case grows.

Summary

The Response API is a straightforward way to generate text with OpenAI. Check the docs, prepare an API key, and try the Python SDK for a smooth start.

Introduction to ChatGPT 4o (overview of ChatGPT’s latest features)
Designing an Event-Driven Backend to Sync GCS with Gemini File Search API (a practical example of API integration patterns)

How to Use Codex CLI

Fri, 12 Dec 2025 10:00:00 +0900

Overview

This article summarizes how to access Codex from your local environment via the command line with Codex CLI. It covers installation, sign-in methods, command execution examples, and common commands.

How to Install Codex CLI

If you already have Node.js, you can install it via npm:

1

npm install -g codex-cli

If you use Homebrew, you can also install it as follows:

1
2


brew tap codexcli/tap
brew install codex

After installation, run the following and confirm the version is displayed:

1

codex --version

Codex CLI requires authentication on the first run. You can sign in with either an API key or your ChatGPT account, depending on your use case.

1

codex login --token <YOUR_API_KEY>

You can also store it in an environment variable to avoid typing it every time:

1

export CODEX_API_KEY=<YOUR_API_KEY>

If you prefer the browser-based flow, omit the token and run codex login:

1

codex login

This opens a browser and shows the ChatGPT sign-in screen. Sign in with your email, Google, or Apple account, grant access, and the token will be saved when you return to the CLI.

After signing in, run the following command in either case. If your account information appears, authentication succeeded.

1

codex whoami

How to Run Codex (with examples)

Generate code with a one-off prompt

1

codex run "Write FizzBuzz in Python"

Summarize while attaching a file as context

1

codex run "Summarize this file" --file README.md

Try interactive chat mode

1

codex chat

In chat mode, you can enter prompts to receive responses in sequence. Press Ctrl + D to exit.

Slash commands in chat

In chat mode, type a / prefix to perform helper actions.

Command	Purpose
`/help`	Display available commands
`/new`	Reset the conversation history and start a new thread
`/file <path>`	Attach the specified file as context
`/exit`	Exit chat

How to Use the Commands

Here are the main commands:

Command	Description
`codex login --token <API_KEY>`	Sign in with an API key
`codex login`	Start browser authentication with your ChatGPT account
`codex whoami`	Check the currently signed-in account
`codex run "<prompt>"`	Send a one-off prompt and get the result
`codex run "<prompt>" --file <path>`	Run with a file included as context
`codex chat`	Use interactive chat mode
`codex history`	Review recent execution history

Summary

Once installed and signed in, you can immediately start using Codex from the command line. Knowing the frequently used commands helps you quickly generate scripts or summarize files.

How to Use GitHub Copilot More Effectively with Chat Tools

Tue, 07 Oct 2025 08:39:41 +0900

Overview

This article explains how to use Chat Tools to make GitHub Copilot Chat even more powerful. By using Chat Tools, you can perform various tasks directly within your conversation with Copilot.

What Are Chat Tools?

Chat Tools are features within GitHub Copilot Chat that allow you to execute special tasks. For example, you can fetch terminal outputs or retrieve information from URLs—all from within the chat interface.

How to Use Chat Tools

Here’s how to use Chat Tools in VSCode.

Basically, you can execute them by typing a command in the following format:

1

#<command> <args>

That’s all there is to it.

Common Chat Tool Commands

Below are some of the most useful Chat Tool commands:

Command	Description
`#codebase`	Searches the entire current workspace.
`#selection`	Adds the currently selected code in the editor as context to the prompt.
`#terminal_selection`	Adds the selected output in the terminal as context to the prompt. Useful when referencing terminal output after an error occurs.
`#fetch_webpage`	Retrieves content from a specified URL and adds it as context to the prompt.

There are many more useful commands available—refer to the official documentation for details:

GitHub Copilot Chat Tools

Summary

In this article, we covered how to use Chat Tools within GitHub Copilot Chat. By taking advantage of these tools, you can make Copilot even more convenient and powerful—so give them a try!

How to Use Redis with Go

Sun, 05 Oct 2025 20:25:02 +0900

Overview

This article explains how to use Redis with Go.

Starting Redis

For an explanation of Redis and how to start it using Docker, refer to this article.

Start Redis:

1
2
3


docker run -d --name my-redis \
 -p 6379:6379 \
 redis:7.4.5-alpine

Check the connection using redis-cli:

1

docker exec -it my-redis redis-cli ping

If it returns PONG, the connection is successful.

Create a Go Project

Create a new Go project:

1
2
3


mkdir redis-go
cd redis-go
go mod init redis-go

Using Redis in Go

Here is a sample code that connects to Redis, writes, and reads data:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50


package main

import (
 "context"
 "fmt"
 "log"
 "time"

 "github.com/redis/go-redis/v9"
)

func main() {
 // Connect to local Redis on port 6379 without a password
 rdb := redis.NewClient(&redis.Options{
 Addr: "127.0.0.1:6379",
 Password: "", // no password
 DB: 0,
 DialTimeout: 3 * time.Second,
 ReadTimeout: 1 * time.Second,
 WriteTimeout: 1 * time.Second,
 })
 defer rdb.Close()

 ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
 defer cancel()

 // Ping to test connection
 if err := rdb.Ping(ctx).Err(); err != nil {
 log.Fatalf("redis ping failed: %v", err)
 }
 log.Println("connected to redis ✅")

 // Basic operations (SET/GET)
 if err := rdb.Set(ctx, "hello", "world", 10*time.Minute).Err(); err != nil {
 log.Fatalf("SET error: %v", err)
 }
 val, err := rdb.Get(ctx, "hello").Result()
 if err != nil {
 log.Fatalf("GET error: %v", err)
 }
 fmt.Println("hello =", val)

 // Example using Set type
 key := "team:dev"
 if err := rdb.SAdd(ctx, key, "alice", "bob", "alice").Err(); err != nil {
 log.Fatalf("SADD error: %v", err)
 }
 members, _ := rdb.SMembers(ctx, key).Result()
 fmt.Println("members =", members)
}

Install the dependencies:

1

go mod tidy

Run the program:

1

go run main.go

If you see the following output, everything is working correctly:

1
2
3


2025/10/05 20:30:00 connected to redis ✅
hello = world
members = [alice bob]

Summary

This article explained how to use Redis with Go. When working with Redis in Go, use the go-redis package.

How to Run Redis with Docker (how to start Redis)
Setting Up a Go Development Environment on Mac and Running Hello World (Go environment setup)

How to Run Redis with Docker

Fri, 03 Oct 2025 07:29:31 +0900

Overview

This article explains how to run Redis with Docker.

What is Redis?

Redis is an open-source in-memory data structure store that can be used as a database, cache, and message broker. It provides high-speed read/write performance and supports various data structures such as strings, lists, sets, and hashes.

Running Redis with Docker

1. Install Docker

First, make sure Docker is installed. If it is not installed, please download and install it from the official Docker website.

2. Start a Redis Container

Run the following command to start a Redis container:

1

docker run --name redis -d -p 6379:6379 redis:7.4.5-alpine

1

docker exec -it redis sh

3. Verify Redis Operation

Inside the Redis container, run the following command to start the Redis client:

1

redis-cli

Try setting and getting data:

1
2
3


set name "hoge"
get name
"hoge"

At this point, you can confirm that Redis is working correctly.

4. Stop and Remove the Redis Container

To stop the Redis container, run:

1

docker stop redis

To remove the Redis container, run:

1

docker rm redis

Summary

In this article, we explained how to run Redis using Docker. Redis is widely used as a high-performance database, and with Docker, it can be easily set up. Give Redis a try and see how it can help you.

How to Use Redis with Go (practical example of using Redis in Go)

How to Use AWS Toolkit in Visual Studio Code

Mon, 11 Aug 2025 20:33:30 +0900

Overview

This article explains how to use AWS Toolkit in Visual Studio Code (VSCode).

Installing AWS Toolkit

Launch VSCode.
Click the Extensions icon in the sidebar on the left.
Type “AWS Toolkit” in the search bar and select “AWS Toolkit for Visual Studio Code” from the list.
Click the “Install” button to start the installation.

How to Set the Asia Pacific Tokyo Region

By default, the region is set to us-east-1, so you need to change it to Asia Pacific Tokyo.
The procedure is a bit tricky, so follow the steps below:

In the left sidebar, open “AWS Explorer” and click the hamburger menu.

Click Show or Hide Regions.
From the displayed list, select “Asia Pacific (Tokyo) ap-northeast-1”.

Now, you can use AWS Toolkit in the Asia Pacific Tokyo region.

How to Update Visual Studio Code

Sun, 27 Jul 2025 18:12:16 +0900

Overview

This article explains how to update Visual Studio Code (VSCode) on Mac. In addition to manual GUI updates, it also covers command-line updates and auto-update settings.

Updating via GUI

Launch VSCode.
Go to Code → “Check for Updates”.

When the update starts, you will see a screen like this:

Once the download is complete, “Restart to Update” will be displayed.

After restarting, click “About Visual Studio Code” to confirm that the version has been updated.

Updating via Command Line

If you installed VSCode via Homebrew, you can update it from the terminal.

Check the current version

1

code --version

Update with Homebrew

1

brew upgrade --cask visual-studio-code

If VSCode was not installed via Homebrew, first remove the existing VSCode from your Applications folder, then install it via Homebrew. Your settings and extensions can be restored via account sync, so removing it is safe.

1

brew install --cask visual-studio-code

After this, you can use brew upgrade --cask visual-studio-code for future updates.

Auto-Update Settings

VSCode has an update.mode setting that controls update behavior. Open Settings (Cmd + ,) and search for update mode to change it.

Value	Behavior
`default`	Automatically downloads and installs updates in the background
`manual`	Checks for updates but requires manual installation
`start`	Only checks for updates when VSCode starts
`none`	Completely disables automatic updates

To configure directly in settings.json:

1
2
3


{
 "update.mode": "default"
}

Unless you have a specific reason, default is the recommended setting.

Troubleshooting

“Check for Updates” shows no updates available

Auto-update may be disabled. Check that update.mode is not set to none in your settings.

Update download fails

This may be caused by proxy or firewall settings. Check the following configuration:

1
2
3
4


{
 "http.proxy": "http://proxy.example.com:8080",
 "http.proxyStrictSSL": false
}

Extensions stop working after update

This may be due to extension compatibility issues. Try the following steps:

Open the Command Palette (Cmd + Shift + P)
Run “Extensions: Disable All Installed Extensions”
Restart VSCode
Re-enable extensions one by one to identify the problematic one

If the issue persists

Reinstalling VSCode may resolve the problem.

1
2


# If installed via Homebrew
brew reinstall --cask visual-studio-code

If you sync your settings via a GitHub or Microsoft account, your configuration and installed extensions will be automatically restored after reinstallation.

Summary

This article covered how to update VSCode.

GUI: Code → “Check for Updates” — a few clicks to complete
Command line: brew upgrade --cask visual-studio-code for quick updates
Auto-update: Control behavior with the update.mode setting

By updating regularly, you can take advantage of new features and security patches. After updating, always check the version information to confirm the update was successful.

How to Set Up Gemini CLI on Mac

Tue, 22 Jul 2025 23:21:19 +0900

Overview

This article explains how to set up Gemini CLI.

How to Set Up Gemini CLI

1. Install Gemini CLI

Use Homebrew to install Gemini CLI.

1

brew install gemini-cli

2. Configure Gemini CLI

Run the gemini command.

1

gemini

You can choose to log in or enter an API key to use it.
This time, we will select “Login with Google” to log in.

The screen transitions to the login page.

Once logged in, Gemini CLI becomes available.

3. How to Use Gemini CLI

You can ask questions or request tasks using Gemini CLI.

This time, we will ask it to solve the FizzBuzz problem.

1

gemini

Type “Please write the code for FizzBuzz” into the Gemini CLI prompt.

After inputting, Gemini will generate the code for you.

Summary

By setting up Gemini CLI, you can use Gemini AI directly from the terminal. This enables efficient development and task management using AI. Gemini CLI is very user-friendly and makes terminal work smoother. Give it a try!

Introduction to ChatGPT 4o (comparison with ChatGPT)
How to Use the OpenAI Response API (API integration in practice)

How to Customize Indent Guides in VSCode

Tue, 22 Jul 2025 21:01:13 +0900

Overview

In Visual Studio Code (VSCode), you can customize the colors of indent guides.
This article explains how to change the colors of indent guides.
While extensions like indent-rainbow often have issues such as guides being cut off, you can achieve this simply by editing the VSCode settings file.

How to Change the Colors of Indent Guides

Open the Settings File

Open the VSCode settings file.
Open the Command Palette (Ctrl + Shift + P) and select Preferences: Open Settings (JSON).

Add Settings

Add the following settings as a reference:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


{
 "workbench.colorCustomizations": {
 "editorIndentGuide.background1": "#006400",
 "editorIndentGuide.background2": "#008000",
 "editorIndentGuide.background3": "#00a000",
 "editorIndentGuide.background4": "#006400",
 "editorIndentGuide.background5": "#008000",
 "editorIndentGuide.background6": "#00a000",
 "editorIndentGuide.activeBackground1": "#00ff00",
 "editorIndentGuide.activeBackground2": "#00ff00",
 "editorIndentGuide.activeBackground3": "#00ff00",
 "editorIndentGuide.activeBackground4": "#00ff00",
 "editorIndentGuide.activeBackground5": "#00ff00",
 "editorIndentGuide.activeBackground6": "#00ff00",
 "editorWhitespace.foreground": "#393A3D"
 },
}

Setting	Description
`editorIndentGuide.background<number>`	Specifies the background color of the indent guide. `<number>` corresponds to the depth of the indent.
`editorIndentGuide.activeBackground<number>`	Specifies the background color of the active indent guide. `<number>` corresponds to the depth of the indent.
`editorWhitespace.foreground`	Specifies the color of whitespace characters.

After adding the settings, the colors of the indent guides will be changed as shown in the image below.

Conclusion

In VSCode, you can easily customize the colors of indent guides.
By simply adding the necessary items to the settings file, you can change them to your preferred colors.
This improves code readability and makes it easier to visually grasp the depth of indents.
Give it a try!

How to Enable GitHub Copilot for Markdown

Mon, 21 Jul 2025 10:00:00 +0900

Overview

By default, GitHub Copilot is disabled for Markdown files.
This article explains how to enable GitHub Copilot for Markdown.

Configuring GitHub Copilot

Open the GitHub Copilot plugin page in VSCode.

Click the gear icon to open the settings.

In the settings screen, change markdown from false to true.

Summary

Enabling GitHub Copilot for Markdown makes editing Markdown files more efficient. You can enable it easily by changing the settings in VSCode, so give it a try!

How to Upgrade JetBrains Products to a Major Version

Thu, 12 Jun 2025 20:46:53 +0900

Overview

When upgrading major versions of JetBrains products such as IntelliJ or PyCharm, you need to manually download the new version.
This article explains how to update easily while preserving your settings.

Installing the JetBrains Toolbox App

When upgrading to a major version of a JetBrains product, it is recommended to use the JetBrains Toolbox App.

Download and install the JetBrains Toolbox App from the link above.

Update

Launch the JetBrains Toolbox App and select the product you want to update.

In this example, we will update PyCharm. Click on it to start the update.

Once the update is complete, the new version will be installed as shown below.

Summary

When upgrading to a major version of a JetBrains product, using the JetBrains Toolbox App makes the update process easy.

Running the Feast Tutorial on macOS

Mon, 13 Jan 2025 11:49:53 +0900

Overview

In this article, we’ll follow the Feast tutorial and run it on a Mac.

Prerequisites

Refer to Setting up a Python development environment on Mac with UV to prepare your environment. Once you’ve set up UV, install Feast in that environment.

Installing Feast and Launching the UI

1

uv pip install feast==0.40.1

Note
As of 2025/01/13, there is a known bug in Feast causing the UI to fail on the latest version. (Issue)

Initialize a Feature Repository:

1
2
3


feast init my_feature_repo

Creating a new Feast repository in /Users/kouhei/Program/ML/feast/my_feature_repo.

You’ll see a new repository like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


tree my_feature_repo
.
└── my_feature_repo
 ├── README.md
 ├── __init__.py
 └── feature_repo
 ├── __init__.py
 ├── data
 │ └── driver_stats.parquet
 ├── example_repo.py
 ├── feature_store.yaml
 └── test_workflow.py

Apply the repository configuration:

1
2


cd my_feature_repo/feature_repo
feast apply

Then launch the Feast UI:

1

feast ui

Access http://0.0.0.0:8888/p/my_feature_repo to see the interface.

Manipulating Data in Feast

1. Creating a Training Dataset

Starting from step 5 in the tutorial, you’ll use Jupyter Notebook. Install it first:

1

uv pip install jupyter

Launch the notebook:

1

jupyter notebook

In a Jupyter Notebook, run the following code to create a dataset for training:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


from feast import FeatureStore
import pandas as pd
from datetime import datetime

entity_df = pd.DataFrame.from_dict({
 "driver_id": [1001, 1002, 1003, 1004],
 "event_timestamp": [
 datetime(2021, 4, 12, 10, 59, 42),
 datetime(2021, 4, 12, 8, 12, 10),
 datetime(2021, 4, 12, 16, 40, 26),
 datetime(2021, 4, 12, 15, 1, 12)
 ]
})

store = FeatureStore(repo_path=".")

training_df = store.get_historical_features(
 entity_df=entity_df,
 features=[
 'driver_hourly_stats:conv_rate',
 'driver_hourly_stats:acc_rate',
 'driver_hourly_stats:avg_daily_trips'
 ],
).to_df()

print(training_df.head())

# Train model
# model = ml.fit(training_df)

Below is an example of the notebook output:

2. Materializing the Online Store

To populate the Online Store, run:

1

feast materialize 1970-01-01T00:00:00Z 2025-01-04T01:24:24Z

Note
In the provided sample, the command feast materialize-incremental $CURRENT_TIME may not work, so we specify a broader time range here.

1
2
3
4
5
6
7
8


01/04/2025 10:28:40 AM root WARNING: _list_feature_views will make breaking changes. ...
Materializing 2 feature views from 1970-01-01 09:00:00+09:00 to 2025-01-04 10:24:24+09:00 into the sqlite online store.

driver_hourly_stats_fresh:
 0%| | 0/5 ...
100%|███████████████████████████████████████████████████████████████| 5/5 ...
driver_hourly_stats:
100%|███████████████████████████████████████████████████████████████| 5/5 ...

3. Retrieving Data from the Online Store

Use Jupyter Notebook to fetch data from the Online Store:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from pprint import pprint
from feast import FeatureStore

store = FeatureStore(repo_path=".")

feature_vector = store.get_online_features(
 features=[
 'driver_hourly_stats:conv_rate',
 'driver_hourly_stats:acc_rate',
 'driver_hourly_stats:avg_daily_trips'
 ],
 entity_rows=[{"driver_id": 1001}]
).to_dict()

pprint(feature_vector)

# Make prediction
# model.predict(feature_vector)

Example output:

1
2
3
4


{'acc_rate': [0.5004482269287109],
 'avg_daily_trips': [691],
 'conv_rate': [0.3067885637283325],
 'driver_id': [1001]}

We’ve successfully materialized data in the Online Store and fetched it using Feast.

Summary

We followed the Feast tutorial to prepare training data, materialize that data into the Online Store, and retrieve it. By managing both training and inference datasets in Feast, you can avoid training-serving skew, enhancing the consistency of your ML workflows.

Setting Up a Python Development Environment on Mac with UV (Python environment setup)
Setting Up a Local Environment Using Pyenv and venv (traditional Python environment setup)
How to Read Parquet Files on macOS (Feast sample data format)

Setting Up a Python Development Environment on Mac with UV

Wed, 01 Jan 2025 14:39:53 +0900

Overview

This article introduces how to set up a Python development environment on Mac using UV.

What is UV?

UV is a package management tool announced in mid-2024.
It is written in Rust and is characterized by faster performance compared to other package managers.

For official explanations, please refer to this site.

Usage

Installing UV

1

curl -LsSf https://astral.sh/uv/install.sh | sh

Add UV to your PATH:

1
2


source $HOME/.local/bin/env (sh, bash, zsh)
source $HOME/.local/bin/env.fish (fish)

Verify installation:

1
2


uv --version
uv 0.5.13 (c456bae5e 2024-12-27)

How to Use

Create a virtual environment:

1

uv venv

Activate the virtual environment:

1

source .venv/bin/activate

Install packages:

1

uv pip install <package name>

For more details on UV, refer to the official documentation.

Summary

This article explained how to set up a Python development environment on Mac. As of January 2025, UV is both fast and easy to use, making it a valuable tool for setting up a Python development environment.

Setting Up a Local Environment Using Pyenv and venv (traditional Python environment setup)

Improving Team Performance by Revisiting Scrum Ceremonies

Mon, 18 Nov 2024 09:36:05 +0900

Overview

In Scrum teams, ceremonies are essential for aligning team members, sharing progress, addressing challenges, and planning for the next sprint. However, when these ceremonies are conducted out of habit without revisiting their purpose, they can become time-consuming and ineffective.

This article explores ways to improve team performance by revisiting and optimizing Scrum ceremonies.

Clarify the Purpose of Each Ceremony

Each Scrum ceremony serves a distinct purpose, and ensuring that the agenda aligns with this purpose is crucial. Below are the purposes of the main Scrum ceremonies, based on the Scrum Guide with some interpretive notes.

Ceremony	Purpose
Daily Scrum	Plan the day to achieve the sprint goal.
Sprint Review	Inspect the sprint’s outcome and adapt for future work.
Product Backlog Refinement	Organize the backlog to support achieving the product goal.
Sprint Retrospective	Reflect on the sprint to identify improvements for quality and effectiveness.
Sprint Planning	Define the work to be completed during the next sprint.

Enforce Timeboxing

Adhering to timeboxes is vital for maintaining efficient ceremonies. Timeboxing limits the duration of meetings to a set timeframe, ensuring focus and productivity. If a ceremony exceeds its timebox, reflect on why and plan adjustments to prevent recurrence.

Tips for Efficient Discussions in Specific Ceremonies

Certain ceremonies, like the Daily Scrum, Sprint Review, and Product Backlog Refinement, often consume more time than necessary. Here are some strategies to enhance their effectiveness.

Daily Scrum

The Daily Scrum should focus on clarifying what the team needs to accomplish that day. Any other discussions should be deferred to smaller, focused meetings involving only relevant members.

Tips:

Keep it under 15 minutes.
Limit reporting to essential information.
Defer in-depth discussions to separate, focused sessions.

Sprint Review

The Sprint Review aims to gather feedback from stakeholders to guide future planning. Prepare for the session by clarifying what feedback is needed and from whom.

Tips:

Prepare in advance to showcase progress effectively.
Clearly define what feedback you want and why.

Product Backlog Refinement should ideally take up no more than 10% of the sprint duration. Avoid delving too deeply into items that are not imminent, and focus instead on high-priority tasks.

Tips:

Avoid excessive detail on future backlog items; focus on what’s next.
Use relative estimation techniques and avoid overly precise estimates, acknowledging inherent uncertainty.

Summary

This article discussed methods to conduct Scrum ceremonies more efficiently, ultimately improving team performance. By enforcing timeboxes, aligning discussions with ceremony objectives, and focusing on value-driven conversations, teams can optimize their workflows and deliver better results.

How to Read Parquet Files on macOS

Sun, 15 Sep 2024 16:52:04 +0900

Overview

This article introduces a simple way to read Parquet files on macOS via the command line.

Reading with parquet-cli

We will use the Parquet file provided as a sample by Feast.

Install parquet-cli on macOS using Homebrew.

1

brew install parquet-cli

Check the meta information.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


$ parquet meta driver_stats.parquet

File path: driver_stats.parquet
Created by: parquet-cpp-arrow version 18.1.0
Properties:
pandas: {"index_columns": [{"kind": "range", "name": null, "start": 0, "stop": 1807, "step": 1}], "column_indexes": [{"name": null, "field_name": null, "pandas_type": "unicode", "numpy_type": "object", "metadata": {"encoding": "UTF-8"}}], "columns": [{"name": "event_timestamp", "field_name": "event_timestamp", "pandas_type": "datetimetz", "numpy_type": "datetime64[ns]", "metadata": {"timezone": "UTC"}}, {"name": "driver_id", "field_name": "driver_id", "pandas_type": "int64", "numpy_type": "int64", "metadata": null}, {"name": "conv_rate", "field_name": "conv_rate", "pandas_type": "float32", "numpy_type": "float32", "metadata": null}, {"name": "acc_rate", "field_name": "acc_rate", "pandas_type": "float32", "numpy_type": "float32", "metadata": null}, {"name": "avg_daily_trips", "field_name": "avg_daily_trips", "pandas_type": "int32", "numpy_type": "int32", "metadata": null}, {"name": "created", "field_name": "created", "pandas_type": "datetime", "numpy_type": "datetime64[us]", "metadata": null}], "creator": {"library": "pyarrow", "version": "18.1.0"}, "pandas_version": "2.2.3"}
ARROW:schema: /////xgGAAAQAAAAAAAKAA4ABgAFAAgACgAAAAABBAAQAAAAAAAKAAwAAAAEAAgACgAAAHAEAAAEAAAAAQAAAAwAAAAIAAwABAAIAAgAAABIBAAABAAAADsEAAB7ImluZGV4X2NvbHVtbnMiOiBbeyJraW5kIjogInJhbmdlIiwgIm5hbWUiOiBudWxsLCAic3RhcnQiOiAwLCAic3RvcCI6IDE4MDcsICJzdGVwIjogMX1dLCAiY29sdW1uX2luZGV4ZXMiOiBbeyJuYW1lIjogbnVsbCwgImZpZWxkX25hbWUiOiBudWxsLCAicGFuZGFzX3R5cGUiOiAidW5pY29kZSIsICJudW1weV90eXBlIjogIm9iamVjdCIsICJtZXRhZGF0YSI6IHsiZW5jb2RpbmciOiAiVVRGLTgifX1dLCAiY29sdW1ucyI6IFt7Im5hbWUiOiAiZXZlbnRfdGltZXN0YW1wIiwgImZpZWxkX25hbWUiOiAiZXZlbnRfdGltZXN0YW1wIiwgInBhbmRhc190eXBlIjogImRhdGV0aW1ldHoiLCAibnVtcHlfdHlwZSI6ICJkYXRldGltZTY0W25zXSIsICJtZXRhZGF0YSI6IHsidGltZXpvbmUiOiAiVVRDIn19LCB7Im5hbWUiOiAiZHJpdmVyX2lkIiwgImZpZWxkX25hbWUiOiAiZHJpdmVyX2lkIiwgInBhbmRhc190eXBlIjogImludDY0IiwgIm51bXB5X3R5cGUiOiAiaW50NjQiLCAibWV0YWRhdGEiOiBudWxsfSwgeyJuYW1lIjogImNvbnZfcmF0ZSIsICJmaWVsZF9uYW1lIjogImNvbnZfcmF0ZSIsICJwYW5kYXNfdHlwZSI6ICJmbG9hdDMyIiwgIm51bXB5X3R5cGUiOiAiZmxvYXQzMiIsICJtZXRhZGF0YSI6IG51bGx9LCB7Im5hbWUiOiAiYWNjX3JhdGUiLCAiZmllbGRfbmFtZSI6ICJhY2NfcmF0ZSIsICJwYW5kYXNfdHlwZSI6ICJmbG9hdDMyIiwgIm51bXB5X3R5cGUiOiAiZmxvYXQzMiIsICJtZXRhZGF0YSI6IG51bGx9LCB7Im5hbWUiOiAiYXZnX2RhaWx5X3RyaXBzIiwgImZpZWxkX25hbWUiOiAiYXZnX2RhaWx5X3RyaXBzIiwgInBhbmRhc190eXBlIjogImludDMyIiwgIm51bXB5X3R5cGUiOiAiaW50MzIiLCAibWV0YWRhdGEiOiBudWxsfSwgeyJuYW1lIjogImNyZWF0ZWQiLCAiZmllbGRfbmFtZSI6ICJjcmVhdGVkIiwgInBhbmRhc190eXBlIjogImRhdGV0aW1lIiwgIm51bXB5X3R5cGUiOiAiZGF0ZXRpbWU2NFt1c10iLCAibWV0YWRhdGEiOiBudWxsfV0sICJjcmVhdG9yIjogeyJsaWJyYXJ5IjogInB5YXJyb3ciLCAidmVyc2lvbiI6ICIxOC4xLjAifSwgInBhbmRhc192ZXJzaW9uIjogIjIuMi4zIn0ABgAAAHBhbmRhcwAABgAAACwBAADcAAAApAAAAHAAAAA0AAAABAAAAPz+//8AAAEKEAAAABgAAAAEAAAAAAAAAAcAAABjcmVhdGVkAGr///8AAAIAKP///wAAAQIQAAAAIAAAAAQAAAAAAAAADwAAAGF2Z19kYWlseV90cmlwcwBo////AAAAASAAAABg////AAABAxAAAAAcAAAABAAAAAAAAAAIAAAAYWNjX3JhdGUAAAAA0v///wAAAQCQ////AAABAxAAAAAgAAAABAAAAAAAAAAJAAAAY29udl9yYXRlAAYACAAGAAYAAAAAAAEAxP///wAAAQIQAAAAJAAAAAQAAAAAAAAACQAAAGRyaXZlcl9pZAAAAAgADAAIAAcACAAAAAAAAAFAAAAAEAAUAAgABgAHAAwAAAAQABAAAAAAAAEKEAAAACgAAAAEAAAAAAAAAA8AAABldmVudF90aW1lc3RhbXAACAAMAAYACAAIAAAAAAADAAQAAAADAAAAVVRDAAAAAAA=
Schema:
message schema {
optional int64 event_timestamp (TIMESTAMP(NANOS,true));
optional int64 driver_id;
optional float conv_rate;
optional float acc_rate;
optional int32 avg_daily_trips;
optional int64 created (TIMESTAMP(MICROS,false));
}


Row group 0: count: 1807 16.88 B records start: 4 total(compressed): 29.796 kB total(uncompressed):29.760 kB
--------------------------------------------------------------------------------
 type encodings count avg size nulls min / max
event_timestamp INT64 S _ R 1807 2.78 B 0 "2021-04-12T07:00:00.00000..." / "2024-12-28T14:00:00.00000..."
driver_id INT64 S _ R 1807 0.07 B 0 "1001" / "1005"
conv_rate FLOAT S _ R 1807 5.42 B 0 "1.9221554E-4" / "0.9998668"
acc_rate FLOAT S _ R 1807 5.42 B 0 "2.1329636E-4" / "0.99993944"
avg_daily_trips INT32 S _ R 1807 3.14 B 0 "0" / "999"
created INT64 S _ R 1807 0.05 B 0 "2024-12-28T15:20:28.266000" / "2024-12-28T15:20:28.266000"

View the data using head.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


$ parquet head driver_stats.parquet

{"event_timestamp": 1734102000000000000, "driver_id": 1005, "conv_rate": 0.27734742, "acc_rate": 0.7152132, "avg_daily_trips": 823, "created": 1735399228266000}
{"event_timestamp": 1734105600000000000, "driver_id": 1005, "conv_rate": 0.57354224, "acc_rate": 0.9831811, "avg_daily_trips": 851, "created": 1735399228266000}
{"event_timestamp": 1734109200000000000, "driver_id": 1005, "conv_rate": 0.3287562, "acc_rate": 0.6172164, "avg_daily_trips": 116, "created": 1735399228266000}
{"event_timestamp": 1734112800000000000, "driver_id": 1005, "conv_rate": 0.045716193, "acc_rate": 0.032996926, "avg_daily_trips": 741, "created": 1735399228266000}
{"event_timestamp": 1734116400000000000, "driver_id": 1005, "conv_rate": 0.12863782, "acc_rate": 0.8951942, "avg_daily_trips": 534, "created": 1735399228266000}
{"event_timestamp": 1734120000000000000, "driver_id": 1005, "conv_rate": 0.9555806, "acc_rate": 0.62216556, "avg_daily_trips": 216, "created": 1735399228266000}
{"event_timestamp": 1734123600000000000, "driver_id": 1005, "conv_rate": 0.75297666, "acc_rate": 0.37602386, "avg_daily_trips": 954, "created": 1735399228266000}
{"event_timestamp": 1734127200000000000, "driver_id": 1005, "conv_rate": 0.46957988, "acc_rate": 0.6454945, "avg_daily_trips": 360, "created": 1735399228266000}
{"event_timestamp": 1734130800000000000, "driver_id": 1005, "conv_rate": 0.6702387, "acc_rate": 0.36532214, "avg_daily_trips": 396, "created": 1735399228266000}
{"event_timestamp": 1734134400000000000, "driver_id": 1005, "conv_rate": 0.019627139, "acc_rate": 0.528229, "avg_daily_trips": 833, "created": 1735399228266000}

Check the schema.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33


$ parquet schema driver_stats.parquet
{
"type" : "record",
"name" : "schema",
"fields" : [ {
"name" : "event_timestamp",
"type" : [ "null", "long" ],
"default" : null
}, {
"name" : "driver_id",
"type" : [ "null", "long" ],
"default" : null
}, {
"name" : "conv_rate",
"type" : [ "null", "float" ],
"default" : null
}, {
"name" : "acc_rate",
"type" : [ "null", "float" ],
"default" : null
}, {
"name" : "avg_daily_trips",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "created",
"type" : [ "null", {
"type" : "long",
"logicalType" : "local-timestamp-micros"
} ],
"default" : null
} ]
}

Conclusion

This article explained how to easily read Parquet files on macOS.

Running the Feast Tutorial on macOS (using Feast with Parquet files)

Setting Up a Go Development Environment on Mac and Running Hello World

Sun, 15 Sep 2024 16:52:04 +0900

Overview

This guide introduces how to quickly set up a Go development environment on Mac and run a Hello World program.

Installing Go

Install Go using Homebrew.

1

> brew install go

Check the installed version.

1
2


> go version
go version go1.21.3 darwin/arm64

Running Hello World

Save the following code as main.go.

1
2
3
4
5
6
7


package main

import "fmt"

func main() {
 fmt.Printf("Hello World\n")
}

Run the code.

1
2


> go run hello.go
Hello World

Build the binary and execute it.

1
2
3
4
5
6
7


> go build hello.go

> ls
hello* hello.go

> ./hello
Hello World

Conclusion

That’s how you can set up a Go development environment on Mac and quickly run a Hello World program.

Research on gRPC (practical gRPC implementation)
How to Use Redis with Go (Redis + Go in practice)

Research on gRPC

Sun, 01 Sep 2024 17:53:57 +0900

What is gRPC?

gRPC is a protocol developed by Google to implement RPC.
It uses Protocol Buffers to serialize data, enabling high-speed communication.
API specifications are pre-defined in a .proto file using IDL, from which source code for both the server and client sides is generated.

Differences between REST and gRPC

REST is resource-oriented, whereas RPC focuses on method invocation, with data being a byproduct.

Advantages and Disadvantages

Advantages

High performance with HTTP/2
Data transfer via Protocol Buffers
- Development follows a schema-first approach due to the use of IDL
Flexible streaming modes

Disadvantages

Non-compliance with HTTP/2
Limited browser support
Variability in feature implementation depending on the programming language
Serialized binary data is not human-readable
REST is also sufficiently fast

.proto Files

gRPC uses Protocol Buffers as its serialization format.
Schema definitions are made in files with the .proto extension, and code for various languages is generated using the protoc command.
In Protocol Buffers, all values have types, which can be divided into scalar types and message types.

Scalar Types

Numeric, string, boolean, byte array

Message Types

Message types with multiple fields
Multiple message types can be defined in a single .proto file

1
2
3
4
5


message Person {
 int32 id = 1;
 string name = 2;
 string email = 3;
}

Conducting the gRPC Quick Start

This time, we will perform the gRPC Quick Start using a Python environment.
https://grpc.io/docs/languages/python/quickstart/

Set up the necessary Python environment.

1
2


python -m pip install grpcio
python -m pip install grpcio-tools

Download the sample code.

1
2


git clone -b v1.64.0 --depth 1 --shallow-submodules https://github.com/grpc/grpc
cd grpc/examples/python/helloworld

Start the server.

1
2
3
4


python greeter_server.py

# Output
Server started, listening on 50051

Open another terminal and start the client.

1
2
3
4
5


python greeter_client.py

## Response
Will try to greet world ...
Greeter client received: Hello, you!

Communication between the gRPC client and server was successfully established.

Modifying the .proto File

This time, we will modify the helloworld.proto file to add a new method.

Move to the directory containing the helloworld.proto file.

1

cd grpc/examples/protos

Modify the file as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


syntax = "proto3";

option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";
option objc_class_prefix = "HLW";

package helloworld;

// The greeting service definition.
service Greeter {
 // Sends a greeting
 rpc SayHello (HelloRequest) returns (HelloReply) {}

 // Add the following line
 rpc SayHelloAgain (HelloRequest) returns (HelloReply) {}

 rpc SayHelloStreamReply (HelloRequest) returns (stream HelloReply) {}

 rpc SayHelloBidiStream (stream HelloRequest) returns (stream HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
 string name = 1;
}

// The response message containing the greetings
message HelloReply {
 string message = 1;
}

Generate the gRPC code.

1
2
3


cd examples/python/helloworld

python -m grpc_tools.protoc -I../../protos --python_out=. --pyi_out=. --grpc_python_out=. ../../protos/helloworld.proto

The following files are regenerated.

1
2
3
4
5


ls -l

-rw-r--r--@ 1 xx xx 1823 9 1 18:12 helloworld_pb2.py
-rw-r--r--@ 1 xx xx 578 9 1 18:12 helloworld_pb2.pyi
-rw-r--r--@ 1 xx xx 7018 9 1 18:12 helloworld_pb2_grpc.py

The _pd file that is updated contains the auto-generated Protocol Buffers definition classes and is generally not manually modified.

Update the greeter_server.py file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


from concurrent import futures
import logging

import grpc
import helloworld_pb2
import helloworld_pb2_grpc


class Greeter(helloworld_pb2_grpc.GreeterServicer):
 def SayHello(self, request, context):
 return helloworld_pb2.HelloReply(message="Hello, %s!" % request.name)
 # Add the following function
 def SayHelloAgain(self, request, context):
 return helloworld_pb2.HelloReply(message="Hello Again, %s!" % request.name)

def serve():
 port = "50051"
 server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
 helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
 server.add_insecure_port("[::]:" + port)
 server.start()
 print("Server started, listening on " + port)
 server.wait_for_termination()


if __name__ == "__main__":
 logging.basicConfig()
 serve()

Update the greeter_client.py file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


def run():
 # NOTE(gRPC Python Team): .close() is possible on a channel and should be
 # used in circumstances in which the with statement does not fit the needs
 # of the code.
 print("Will try to greet world ...")
 with grpc.insecure_channel("localhost:50051") as channel:
 stub = helloworld_pb2_grpc.GreeterStub(channel)
 response = stub.SayHello(helloworld_pb2.HelloRequest(name="you"))
 print("Greeter client received: " + response.message)
 response = stub.SayHelloAgain(helloworld_pb2.HelloRequest(name="you"))
 print("Greeter client received: " + response.message)

Restart the server and run the client.

1
2
3
4
5
6


python greeter_server.py
python greeter_client.py

# Output
Greeter client received: Hello, you!
Greeter client received: Hello Again, you!

The newly added method is confirmed to be functioning correctly.

Summary

This time, we researched gRPC and followed the official documentation tutorial. gRPC allows for schema-first development and enables high-performance communication via HTTP/2. Recently, it has been gaining attention as an alternative to REST APIs, making it a technology worth familiarizing yourself with.

Setting Up a Go Development Environment on Mac and Running Hello World (Go environment setup)

Introduction to ChatGPT 4o

Tue, 14 May 2024 23:22:39 +0900

Introduction of GPT-4o

On May 13, 2024, OpenAI announced a new GPT model, ChatGPT-4o.

GPT-4o has the following improvements compared to previous models:

Realization of natural conversation
Faster response time
Enhanced multilingual support
Higher reliability

Notably, the improvements in inference speed and quality allow it to perform exceptionally well in real-time dialogue systems, enabling smoother and more natural communication.

Real-Time Dialogue Applications

The following video demonstrates real-time interaction with ChatGPT-4o using a smartphone.

As you can see in the video, ChatGPT-4o can respond at a level close to human conversation.
This allows users to enjoy very natural conversations.

Previously, the ChatGPT series converted voice to text and then inputted it into GPT-4 for dialogue.
This meant that emotional information such as tone was lost in the process.

However, ChatGPT-4o is trained directly from voice, taking into account information such as tone, leading to more natural conversations.
Additionally, it returns responses directly from voice, eliminating the text interpretation step and allowing for faster responses.

Enhanced Multilingual Support

GPT-4o also enhances multilingual support.
Previously, questions asked in English received high-accuracy responses, but questions in Japanese resulted in lower accuracy.

With the enhanced multilingual support, high-accuracy responses can now be obtained even in Japanese.

Summary

This time, we introduced ChatGPT-4o, announced by OpenAI.
The accuracy of the ChatGPT model has remarkably improved, feeling more and more like human conversation.
I believe that how well engineers can utilize generative AI will greatly affect productivity, so I will continue to monitor trends closely.

Hello ChatGPT-4o

Generating Images with ChatGPT (image generation with ChatGPT)
How to Use the OpenAI Response API (OpenAI API integration)

Migrating from Evernote to Notion

Mon, 29 Apr 2024 19:32:38 +0900

Overview

As of April 26, 2024, Evernote’s Japanese corporation was dissolved, which does not mean Evernote services are ending, but it certainly raises concerns about its future. Given the many restrictions on Evernote’s free plan and the cost associated with its premium plan, I considered switching to another note-taking app.

After some deliberation, I chose Notion for the following reasons:

It possesses all essential features required for a note-taking app and is a viable alternative to Evernote.
Notion’s free plan is less restrictive and adequate for my needs, allowing me to cut costs on premium subscriptions.
Notion offers an import tool specifically for Evernote, facilitating a low-cost migration.

This article details the process of migrating from Evernote to Notion.

Migration Process

Notion provides an Evernote import feature, making the migration straightforward:

Accessing Import Settings

Navigate to Settings in the Notion app.

Click Settings to access the Import option.

After clicking on Import, select Evernote as the import source.

Importing Notebooks

Once linked, you can choose the notebooks you wish to import. Here’s a critical tip: While it seems possible to select multiple notebooks for import, doing so may lead to excessively long processing times or even errors. Therefore, it is advisable to import one notebook at a time.

Even when importing one notebook at a time, be prepared for the process to take several hours, depending on the size of the notebook.

Post-Import Review

After the import, the formatting, images, links, and labels were preserved well, allowing for a seamless transition to Notion.

Conclusion

This guide outlines the steps to migrate from Evernote to Notion. With Notion’s robust free features and straightforward migration process, it’s worth considering if you’re looking to move away from Evernote.

Setting Up a Global gitignore for All Projects

Tue, 16 Apr 2024 23:16:25 +0900

Overview

Adding a .gitignore file to a project allows you to exclude certain files and directories from being tracked by git. However, it can be tedious to repeatedly add default directories like .idea generated by IDEs to .gitignore for each new project. This article explains how to apply .gitignore settings globally across all projects.

Applying gitignore Globally

Git by default checks for ignore settings in ~/.config/git/ignore. Therefore, by placing your ignore configurations in this file, you can have them applied to all your projects.

While it’s common to create a .gitignore_global and register it under core.excludesfile, this method requires unnecessary configuration in .gitconfig, which is why the approach described here is recommended.

Steps to Apply gitignore Globally

Create a Directory for Ignore File

Create a directory to store your ignore file, if it doesn’t already exist:

1

mkdir -p ~/.config/git/

Create and Configure the Ignore File

Create the ignore file and write the patterns for files and directories you want to ignore globally:

1

vim ~/.config/git/ignore

Example of what to include:

1
2
3


.idea/
*.log
node_modules/

Applying the Ignore Settings

These settings will now automatically apply to all projects managed under your user account. If there are already tracked files you wish to ignore, use git rm --cached to stop tracking them without deleting the files.

Reference

Git - gitignore

By following these steps, you can efficiently manage your ignore settings across all your git projects without having to replicate configurations in each project repository.

Using Bluesky API for Automated Posting with Python

Sun, 07 Apr 2024 23:52:09 +0900

Overview

Bluesky is a decentralized social network initiated by Jack Dorsey, former CEO of Twitter.
Built on the ATProtocol, it can be seen as a decentralized version of Twitter, where there is no central authority. The move towards decentralization in social networking might be following the trend seen in finance, from centralized to decentralized cryptocurrencies.

This article compiles methods for executing Bluesky’s API using Python.

Steps to Use Bluesky API

Generating an API Password
Setting Up Python Environment
Script Creation and Execution

Generating an API Password

To execute the API, you need to generate an API password using your account name.

First, verify your account name displayed on your Bluesky profile, minus the initial @. For instance, if my account is bossagyu.bsky.social, that’s the account name I’ll use.

Next, generate the API execution password.

Navigate to Settings → App Passwords to create the password.

Click Add App Password.

After clicking the add button, name your password for management purposes. The name itself is not the password but helps with organization.

Once the password is generated, ensure to copy it as it won’t be displayed again. If you forget to copy, simply generate a new one.

Setting Up the Python Environment

Prepare your Python environment. For setup using venv, refer to this guide.

According to the official documentation, Python version 3.7.1 or higher is required.

After setting up your environment, install the ATProtocol library to interact with it.

1

$ pip install atproto

Verify installation:

1
2
3


$ pip list | grep atproto

atproto 0.0.46

With this, preparation is complete.

Script Creation and Execution

Create a script to post on Bluesky.

1
2
3
4
5
6
7
8
9


from atproto import Client

client = Client()

user_name = "bossagyu.bsky.social"
password = "*******" # Enter the generated API password

client.login(user_name, password)
client.send_post(text='Posting via API.')

That’s all for the script. You can now post to Bluesky using the API.

Let’s run it:

1

$ python post_bluesky.py

Upon execution, the post successfully appears on Bluesky.

Conclusion

This article covered how to execute Bluesky’s API using Python. While Bluesky is still under development, unlike Twitter, which has limitations and charges for API use, Bluesky offers a free API, making it an excellent option for those looking to experiment with social networking APIs at no cost.

Setting Up a Local Environment Using Pyenv and venv (Python environment setup)
Setting Up a Python Development Environment on Mac with UV (modern Python environment setup)

Generating Images with ChatGPT

Sun, 31 Mar 2024 17:35:07 +0900

Overview

Unlike image generation models like Stable Diffusion, ChatGPT can also generate images. This guide explains how to do so.
For those using the ChatGPT paid plan, images can be generated without any additional fees, allowing for the creation of commercially usable images with minimal effort.

This tutorial utilizes DALL-E, a feature of ChatGPT Plus, to generate images. For more information on DALL-E3, refer to OpenAI’s official page.

How to Generate Images

Select “Explore GPTs” from the sidebar.

Type DALL-E in the search bar and perform a search.

Click on “Start Chat” to begin generating images.

Then, simply enter a description of the image you wish to generate, and the image will be created.

Generating an Example Image

Let’s try generating an image of a dog wearing pink sunglasses, as used in this blog.

For a start, we’ll input the prompt “a dog wearing pink sunglasses”.

Additionally, like using regular ChatGPT, you can refine the generated image by entering additional prompts.
This time, let’s input “Make it anime-style” as an additional prompt.

You can see it has an anime-style appearance. By issuing further orders, you can gradually get closer to your desired image.

Conclusion

This guide has explained how to generate images using ChatGPT.

It’s incredibly convenient to be able to generate images easily, but just like with Stable Diffusion, it’s quite challenging to get the exact image you want without adjusting the prompts. The need for prompt refinement seems less critical than with Stable Diffusion, suggesting a possible difference in model performance.

Note that if you overuse this feature, you may receive a message asking you to wait, indicating there may be a limit on the number of generations. For those with access to a machine source, running Stable Diffusion locally on a Mac might still be the best option.

Introduction to ChatGPT 4o (overview of ChatGPT 4o features)

Using Enums in TypeScript

Sat, 23 Mar 2024 13:11:13 +0900

Overview

This article explains how to use Enums in TypeScript.

What are Enums?

Enums (enumerated types) represent a collection of related values. While many languages implement Enums, JavaScript does not. However, TypeScript supports Enums, enriching the JavaScript experience.

How to Use Enums

Enums can be defined as follows:

1
2
3
4
5
6
7


enum Status {
 zero,
 one,
 two
}

console.log(Status.zero); // 0

By default, Enums are assigned numerical values, starting from 0. The compiled JavaScript code would look like this:

1
2
3
4
5
6
7


var Status;
(function (Status) {
 Status[Status["zero"] = 0] = "zero";
 Status[Status["one"] = 1] = "one";
 Status[Status["two"] = 2] = "two";
})(Status || (Status = {}));
console.log(Status.zero); // 0

You can also assign string values to Enums:

1
2
3
4
5
6
7


enum Status {
 zero = 'zero',
 one = 'one',
 two = 'two'
}

console.log(Status.zero); // 'zero'

When comparing string values, you can write as follows:

1
2
3
4
5
6
7
8


const stringZero: String = 'zero';
const value = stringZero as StringStatus;

if (value === StringStatus.zero) {
 console.log('value is zero');
} else {
 console.log('value is not zero');
}

Summary

This article explained how to use Enums in TypeScript. By utilizing Enums, you can improve the readability and maintainability of your code.

Setting Up TypeScript Development Environment Easily with Volta (TypeScript environment setup)

Setting Up TypeScript Development Environment Easily with Volta

Sun, 10 Mar 2024 13:11:13 +0900

Overview

This article explains how to easily set up a TypeScript development environment using Volta, with MacOS as the target platform.

What is Volta?

Volta is a version management tool for Node.js, featuring the following characteristics as presented on Volta’s official site:

Fast: Built in Rust, enabling swift Node.js version switching.
Reliable: Ensures everyone on a project uses the same tools.
Universal: Can be used across package managers, node runtimes, and OSes.

While nodebrew has been commonly used, the use of Volta seems to be increasing lately.

Installing Volta and Node.js

Installing Volta is as simple as running the following command:

1

curl https://get.volta.sh | bash

If you’re using zsh and the path hasn’t been automatically set, use the following commands:

1
2
3


echo 'VOLTA_HOME=$HOME/.volta' >> ~/.zshrc
echo 'export PATH=$VOLTA_HOME/bin:$PATH' >> ~/.zshrc
source ~/.zshrc

Verify the installation. If a version is displayed, Volta has been successfully installed:

1

volta -v

Next, use Volta to install Node.js. If no version is specified, the latest LTS version will be installed:

1

volta install node

Installing yarn and Creating a TypeScript Project

Differences between npm and yarn

Both npm and yarn are package managers for Node.js. Their features include:

npm:

Released a year after Node.js (2010).
Stands for Node Package Manager.
Automatically generates a package-lock.json file.
Installed automatically with Node.js.

yarn:

Released in 2016.
Developed by Facebook, Google, Exponent, and Tilde.
Compatible with npm (can use the same package.json).
More strict in locking module versions than npm.
Faster installation than npm.

While yarn appears superior, recent updates to npm have minimized the differences. Here, we will use yarn to create a TypeScript project.

Installing yarn

Install yarn using Volta:

1

volta install yarn

Confirm the installation by checking if yarn is listed:

1

volta list

Creating a TypeScript Project

Initialize yarn:

1

yarn init -y

Install Node.js:

1

volta pin node@20.0.0

Install TypeScript:

1

yarn add typescript

Install ts-node:

1

yarn add --dev ts-node

Create a tsconfig.json file. This is a TypeScript configuration file that details compilation settings. In this case, you can leave it at the default settings generated:

1

yarn tsc --init

Test a sample program:

1
2
3
4


echo "console.log('Hello, TypeScript!');" > hello.ts
yarn ts-node hello.ts

# If "Hello, TypeScript!" is displayed, it's a success.

The test script ran successfully. This completes the setup of the TypeScript development environment.

Summary

This article described how to set up a TypeScript development environment easily using Volta. By using Volta, managing Node.js versions becomes straightforward, making the development environment setup smoother. Additionally, specifying the Node.js version with Volta in your project’s package.json can help eliminate version discrepancies among developers, adding another layer of convenience to your workflow.

Using Enums in TypeScript (how to use TypeScript Enums)

Explaining Capacity and Performance Management in ITIL v4

Tue, 27 Feb 2024 08:53:36 +0900

Overview

This article explains capacity and performance management as outlined in ITIL v4. I will also apply what I have understood to my own experiences, explaining the process of capacity and performance management.

What are Capacity and Performance Management?

Capacity and Performance Management involves managing the performance of services and the resources supporting them. The goal is to optimize the performance of services and ensure that the capacity of services is appropriately maintained.

Processes in Capacity and Performance Management

The processes in capacity and performance management include:

Establishing Capacity and Performance Control
Analyzing and Improving Service Capacity and Performance

Establishing Capacity and Performance Control

Establishing capacity and performance control involves agreeing with stakeholders on the usage and performance standards for the IT resources used by the service, and deciding on the timing, baseline, and format for evaluation.

It is realized through the following steps:

Identifying service capacity and performance requirements
Agreeing on service capacity and performance requirements
Deciding on capacity and performance requirements
Designing capacity and performance evaluation metrics and reports

Applying my experiences to these processes, I understood them as follows:

Identifying service capacity and performance requirements
- As an internal platform provider, I identified latency performance requirements (99%ile in Nms) demanded by internal users.
- Based on these thresholds, we conducted performance verification and measured throughput per instance.
- We calculated the cost based on the throughput measurements.
Agreeing on service capacity and performance requirements
- We agreed on latency performance and throughput with stakeholders.
Deciding on capacity and performance requirements
- This remained unchanged from what was agreed upon.
Designing capacity and performance evaluation metrics and reports
- We used a tracing tool called Dynatrace to measure and report performance.

Analyzing and Improving Service Capacity and Performance

Analyze issues in usage and performance conditions from service output logs and incident information.

It is realized through the following steps:

Analyzing capacity and performance
Reporting capacity and performance
Planning and designing capacity and performance

Applying my experiences to these activities, I understood them as follows:

Analyzing capacity and performance
- We used Dynatrace to analyze latency performance and throughput.
- We identified performance issues from incident information.
Reporting capacity and performance
- We visualized latency performance and throughput using Dynatrace’s dashboard.
Planning and designing capacity and performance
- Anticipating growth in users, we forecasted that the current capacity would be insufficient.
- We made plans to increase capacity based on demand forecasts and executed them.

Summary

In this article, I explained capacity and performance management based on what I have learned and applied it to my own experiences. I understood that capacity and performance management involves managing aspects similar to availability management, but from the perspective of capacity and performance. In my experience, discussions about performance and availability often go hand in hand, and I rarely deal with them independently.

Understanding Availability Management in ITIL v4 (related ITIL v4 article)
Explaining Business Analysis in ITIL v4 (related ITIL v4 article)

How to Use Stable Diffusion Web UI on Mac

Mon, 12 Feb 2024 11:24:59 +0900

Overview

This article will guide you through installing Stable Diffusion Web UI on Mac and using it locally.

What is Stable Diffusion?

Stable Diffusion is a type of image processing technology using AI. By inputting text, it can generate images corresponding to that text.

Ways to Use Stable Diffusion

There are two main ways to use Stable Diffusion:

Using web applications like Hugging Face or Dream Studio
Using Stable Diffusion Web UI locally

This article focuses on using Stable Diffusion Web UI locally. While web applications are easy for trial purposes, they might have limitations or costs for generating a significant number of images. Thus, local usage is recommended for more extensive needs.

How to Use Stable Diffusion Web UI Locally

We will use stable-diffusion-web-ui published by AUTOMATIC1111 for this purpose.

Prepare the environment
Install stable-diffusion-web-ui
Place the model files
Start stable-diffusion-web-ui and generate images

1. Preparing the Environment

First, we need to install Python and other necessary libraries using homebrew.

Install homebrew:

1

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Set the homebrew path:

1

export PATH="$PATH:/opt/homebrew/bin/"

Install related libraries:

1

brew install cmake protobuf rust pyenv git wget

Set up the Python environment using pyenv. This allows you to use multiple versions of Python. For building the Python environment, refer to this article.

1
2


pyenv install 3.10.6
pyenv local 3.10.6

2. Installing stable-diffusion-web-ui

Clone the repository using git clone:

1
2


git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui

Set up a virtual environment using venv to keep the environment clean:

1
2


python -m venv venv
source venv/bin/activate

Now the environment is ready.

3. Placing the Model Files

Next, download the model files and place them in the stable-diffusion-webui/models/Stable-diffusion/ directory. Model files can be downloaded from sites like:

For this example, we’ll download the blue_pencil model from Civitai.

Search for blue_pencil in Civitai’s search bar, select blue_pencil from the search results, and click the Download button.

Move the downloaded model to the directory:

1

mv ~/Downloads/bluePencilXL_v401.safetensors models/Stable-diffusion/

4. Starting stable-diffusion-web-ui and Generating Images

Finally, start stable-diffusion-web-ui and generate images:

1

./webui.sh

After launching, input the text in the prompt and generate images:

Select the downloaded blue_pencil model in Stable Diffusion checkpoint.
Enter elements you want in the image in the prompt.
Enter elements you don’t want in the image in the Negative prompt.
Click Generate.

By inputting “a dog wearing pink sunglasses,” the output matched the input. You can generate various images by inputting different texts.

Summary

This article introduced how to install and use Stable Diffusion Web UI on Mac locally. Using it locally allows you to generate images freely without the limitations or costs associated with web applications.

Explaining Business Analysis in ITIL v4

Fri, 09 Feb 2024 09:00:56 +0900

Overview

This article consolidates my learning and understanding of business analysis as per ITIL v4. I will also apply my own experiences to explain the business analysis process.

What is Business Analysis?

Business analysis involves analyzing a business or any other component to propose solutions for meeting those needs or solving business challenges. It is important to note that the term “business” analysis does not solely pertain to business entities.

Examples of Business Analysis Techniques

Examples of business analysis techniques include:

SWOT Analysis
User Stories

For specific methods, I defer to other sites as they are not the main focus here. Personally, I often use Customer Journey Maps, although it is not listed as an example here.

The Business Analysis Process

The business analysis process includes two main processes:

Designing and Maintaining Business Analysis Approaches
Identifying Business Analysis and Solutions

Designing and Maintaining Business Analysis Approaches

This process focuses on establishing a consistent and effective approach to business analysis by addressing the current and anticipated needs of the organization. It is executed as follows:

Analyzing the organization and requirements
Reviewing business analysis approach methodologies
Implementing the business analysis approach

In my organization, we rarely conduct analysis at the business layer level. Generally, this process involves analyzing specific requirements using certain methodologies.

Identifying Business Analysis and Solutions

This process emphasizes analyzing stakeholders’ needs and requirements. It includes identifying and proposing solutions to address the stakeholders’ needs and requirements. It is executed as follows:

Collecting and analyzing information from stakeholders
Defining solution options and identifying recommended solutions
Providing support to the solution delivery team
Evaluating and assessing the performance of the solution

Applying my own experiences to these activities, I understood them as follows:

For 1,2

Collecting information from stakeholders and conducting analysis.
Identifying the issues to be solved (why) from the analysis results and determining what solutions are available (what).

For 3

Considering how to solve the identified why and what with the product team.
Deciding on the resolution method and evaluation criteria to determine how success will be measured.

For 4

Regularly evaluating how well the solution meets the established criteria.
I recommend automating data collection and visualizing it with tools like Grafana for visibility.

Summary

This article explained business analysis based on my learning and experiences. Personally, I’ve understood that business analysis does not only target businesses and includes processes beyond analysis.

Reference

Business analysis management: ITIL 4 Practice Guide

Understanding Availability Management in ITIL v4 (related ITIL v4 article)
Explaining Capacity and Performance Management in ITIL v4 (related ITIL v4 article)

Complete Guide to Using GitHub Copilot in VSCode

Sun, 04 Feb 2024 22:34:51 +0900

Overview

This article comprehensively covers how to set up GitHub Copilot in VSCode, enable it for Markdown files, and use Chat tools. A GitHub Copilot account is required as a prerequisite.

Getting Started with GitHub Copilot in VSCode

Installing the Extension

First, you need to install the extension in VSCode. Open VSCode, click on the icon with four squares in the left menu, enter “copilot” in the search text input, and click “install” to start the installation.

Linking with GitHub

After clicking install and completing the installation, the following screen will appear. Click on “Sign in to GitHub.”

You will be asked to allow access to your GitHub account. Click “Allow” to grant permission.

Click “Authorize Visual Studio Code” to give permission.

This completes the integration of GitHub Copilot with VSCode, and you’re now ready to use it.

How to Use

Basically, as you write code, suggestions will automatically appear. Use the following commands to write code efficiently with the suggested completions.

Cheat Sheet

Function	Key
Accept suggestion	Tab
Reject suggestion	Esc
Open Copilot	Ctrl + Enter
Next suggestion	Alt/Option + ]
Previous suggestion	Alt/Option + [
Trigger inline Copilot	Alt/Option + \

Enabling Completions for Markdown Files

By default, GitHub Copilot has completions disabled for Markdown files. If you write blogs or documentation in Markdown, enabling this feature is quite useful.

How to Enable

Open the GitHub Copilot plugin page in VSCode
Click the gear icon to open settings
Change markdown from false to true

This enables Copilot completions in Markdown files. This blog was also written with GitHub Copilot assistance, significantly improving efficiency.

Using Chat Tools for Even More Power

GitHub Copilot Chat has a feature called “Chat tools” that lets you execute various tasks within the chat.

How to Use Chat Tools

Simply enter commands in the format #<command> <args>.

Common Chat Tool Commands

Command	Function
`#codebase`	Search the entire current workspace
`#selection`	Add currently selected code in the editor as context
`#terminal_selection`	Add currently selected terminal output as context. Useful for referencing error output.
`#fetch_webpage`	Fetch content from a URL and add it as context

For more useful commands, check the official documentation:

Github Copilot Chat tools

Summary

This article covered how to set up and use GitHub Copilot in VSCode.

Basic Setup: Extension installation and GitHub integration
Shortcuts: Tab/Esc to accept/reject suggestions
Markdown Support: Change markdown setting to true
Chat Tools: Add context with #codebase and #selection

GitHub Copilot not only provides code completions but also assists with writing text. Give it a try and see how it can help you.

Understanding Availability Management in ITIL v4

Tue, 30 Jan 2024 20:34:58 +0900

Overview

This article explains availability management as defined in ITIL v4. I will also relate my own experiences to explain the processes involved in availability management.

What is Availability Management?

Availability management involves activities to ensure the availability of a service. The objective of availability management is to ensure that the service delivers an agreed level of availability to meet the needs of customers and users.

Processes in Availability Management

There are two main processes in availability management:

Establishing Service Availability Control
Analyzing and Improving Service Availability

Establishing Service Availability Control

Establishing service availability control ensures the availability of a service. It is realized through the following steps:

Identifying service availability requirements
Agreeing on service availability requirements
Deciding on availability measurement requirements
Designing availability metrics and reporting

Applying my own experiences to these processes, I understood them as follows:

Identifying service availability requirements
- Identifying the types of users and the business risks of service downtime.
- For my service, which is an internal platform, I identified the impacts on various services using the platform.
Agreeing on service availability requirements
- Agreeing on the service’s availability (e.g., 99% uptime) in the form of an SLA.
- We also clarified downtime criteria and exceptions for uptime calculations.
Deciding on availability measurement requirements
- As the availability requirements were agreed upon earlier, there were no specific measurement requirements.
Designing availability metrics and reporting
- Primarily designed around the ‘downtime/uptime’ formula.
- For reporting, we created a dashboard to visualize the availability metrics.

Analyzing and Improving Service Availability

This process, as the name suggests, involves analyzing and improving service availability. It is realized through the following steps:

Analyzing service availability
Reporting on service availability
Planning and designing for service availability

Applying my own experiences to these processes, I understood them as follows:

Analyzing service availability
- Confirming that service availability is being achieved and compiling the data.
Reporting on service availability
- Reflecting availability on a dashboard that is accessible to everyone.
Planning and designing for service availability
- In case of incidents that affect availability, we formulated plans for prevention of recurrence.

Reference

Availability management: ITIL 4 Practice Guide

Explaining Business Analysis in ITIL v4 (related ITIL v4 article)
Explaining Capacity and Performance Management in ITIL v4 (related ITIL v4 article)

How to Check if an S3 Object Exists in Python boto3

Sat, 27 Jan 2024 21:41:37 +0900

Overview

This article explains how to check whether an S3 object exists using Python. boto3 provides two API levels: resource and client. I’ll cover both methods.

I used this approach when developing a LINE Bot where I needed to check if a user’s configuration file existed in S3.

resource vs client: Which Should You Use?

boto3 has two API levels:

API	Characteristics	Best For
`resource`	High-level API, object-oriented	Simple operations, readability
`client`	Low-level API, closer to AWS API	Fine-grained control, performance

Conclusion: For simple existence checks, I recommend using client’s head_object. Here’s why:

head_object only retrieves object metadata, making it lightweight
resource’s load() internally calls head_object anyway
client is closer to the AWS API with more predictable behavior

Using boto3.client (Recommended)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


import boto3
from botocore.exceptions import ClientError

def check_s3_object_exists(bucket_name: str, object_key: str) -> bool:
 """Check if an S3 object exists"""
 s3 = boto3.client('s3')
 try:
 s3.head_object(Bucket=bucket_name, Key=object_key)
 return True
 except ClientError as e:
 error_code = e.response['Error']['Code']
 if error_code == '404':
 return False
 # Re-raise non-404 errors (e.g., permission issues)
 raise

# Usage example
if check_s3_object_exists('my-bucket', 'path/to/file.json'):
 print("Object exists")
else:
 print("Object does not exist")

Key Points

head_object retrieves only metadata without downloading the object content
Non-404 errors (e.g., 403: Access Denied) should be re-raised for proper handling
Wrapping in a function makes it easier to test

Using boto3.resource

If you prefer an object-oriented approach, this method also works:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


import boto3
from botocore.exceptions import ClientError

def check_s3_object_exists_resource(bucket_name: str, object_key: str) -> bool:
 """Check if an S3 object exists (resource version)"""
 s3 = boto3.resource('s3')
 try:
 s3.Object(bucket_name, object_key).load()
 return True
 except ClientError as e:
 error_code = e.response['Error']['Code']
 if error_code == '404':
 return False
 raise

Common Pitfalls

1. Confusing Permission Errors with 404

If you lack permissions to access the S3 bucket, you’ll get a 403 error. Catching only 404 and treating it as “doesn’t exist” will mask permission issues.

1
2
3
4
5
6
7
8
9


# BAD: Swallowing all errors
except ClientError:
 return False # Permission errors also treated as "doesn't exist"

# GOOD: Re-raise non-404 errors
except ClientError as e:
 if e.response['Error']['Code'] == '404':
 return False
 raise # Let caller handle permission errors

2. Using list_objects is Inefficient

You could use list_objects with prefix search, but it becomes slow with many objects. For single object existence checks, use head_object.

Summary

Use head_object to check S3 object existence
Handle non-404 errors properly (they may indicate permission issues)
Both resource and client work, but client is simpler

References

Building a Cleaning Reminder Bot with AWS Lambda + LINE
Setting Up a Local Environment Using Pyenv and venv (Python environment setup)
Setting Up a Python Development Environment on Mac with UV (modern Python environment setup)

Integrating AWS API Gateway with Lambda

Sat, 13 Jan 2024 18:06:52 +0900

Overview

Integrating AWS API Gateway with Lambda enables you to call Lambda functions from API Gateway. This article introduces how to integrate AWS API Gateway with Lambda.

Prerequisites

It is assumed that the Lambda function has already been created. If not, please refer to the following article for creation:

Creating an AWS Lambda function

Deciding How to Integrate API Gateway with Lambda

When integrating API Gateway with Lambda, you need to consider the following two points:

The request format for API Gateway
Whether to use Proxy Integration or Non-Proxy Integration

Request Format for API Gateway

You can choose from the following formats:

REST API
HTTP API
WebSocket API

If you choose to use the REST API format, you will need to decide between REST API and HTTP API. While REST API has more features, it is more expensive than HTTP API. HTTP API is a good choice for simpler requirements.

For a detailed comparison, please refer to the official documentation.

Proxy vs. Non-Proxy Integration

Using Proxy Integration standardizes the format of the response returned from Lambda. It is generally recommended to use Proxy Integration.

Setting up Lambda Proxy Integrations in API Gateway

Configuration

After creating the Lambda function, select “Add Trigger.”

Choose API Gateway.

Configure the trigger addition as shown below.

Once configured successfully, the screen should look like this.

Access the API endpoint listed with a tool like curl to execute the Lambda function.

1
2
3


$ curl https://xxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/default/apigateway-get-sample

"Hello from Lambda!"%

Summary

This article introduced how to integrate AWS API Gateway with Lambda. By integrating with API Gateway, you can externally invoke Lambda functions at any time.

How to Create a Product Strategy

Mon, 08 Jan 2024 21:55:15 +0900

Overview

This article summarizes how to create a good product strategy, based on the book “Good Strategy Bad Strategy.”

Background

As a product owner in my job, I took over a product from the previous owner. The product lacked a clear strategy and direction, prompting me to develop a new product strategy.

While developing the strategy, I realized that the term “strategy” is interpreted in various ways by different people, often used loosely in many contexts. To understand what constitutes a good strategy, I referred to “Good Strategy Bad Strategy.”

What is a Good Strategy?

A good strategy identifies critical points where concerted efforts can significantly boost the effect of one’s actions. A strategy should show the direction for an organization to move forward.

A good strategy has three basic structures:

Diagnosis
Guiding Policy
Actions

Diagnosis

Diagnosis involves assessing the situation to identify the key challenges to address. A good diagnosis separates crucial issues from the complex mix of problems and simplifies them.

Most of the strategy work lies in figuring out what is happening. Gathering information is crucial. Although the book criticizes consultants’ frameworks, I find them useful in organizing information after thorough collection.

In creating my product strategy, I used SWOT analysis and Impact Mapping to organize the current situation and the impact of existing strategies.

For more on these methods, refer to the following resources:

Guiding Policy

The guiding policy outlines how to approach the challenges identified in the diagnosis. A good policy is not about goals or visions but about how to face challenges and exclude other options. It focuses efforts on a decisive point to achieve a significant effect.

A good strategy clearly shows where resources will be allocated according to the strategy.

In my case, I defined the product direction based on the current situation and vision, focusing on specific target segments and values to offer.

Actions

Actions are a coherent set of steps designed to execute the guiding policy. A strategy coordinates all actions to implement the policy effectively.

A good strategy includes guidelines for implementing actions.

My Strategy

Based on the above, I developed the following strategy (partially obscured as it was for my company):

1

Reduce the new user cost of using xx function.

This simple strategy meets the three basic structures of a good strategy:

Diagnosis
- Identified increasing new users of xx as critical for the company’s benefit.
Guiding Policy
- Decided to lower the onboarding cost to acquire new users.
Actions
- Prioritized several approaches to realize the policy.
- Although it could have included more action-oriented words, I decided on the above.

What is a Bad Strategy

Finally, let’s touch on common patterns of bad strategies:

Characteristics of a Bad Strategy:

Vague
- Uses jargon or industry terms to obscure simple facts.
Avoids Significant Problems
- Strategy should overcome difficult challenges and obstacles.
- Strategies focusing only on attainability are bad.
Confuses Goals with Strategy
- For instance, a 10% revenue increase is a goal, not a strategy.
Sets Wrong Strategic Objectives
- Strategies set without sufficient investigation of causes and surroundings.

Summary

This article summarized how to create a good product strategy based on “Good Strategy Bad Strategy.” Creating a good strategy might not happen at once. However, not having a strategy is like running blindly, unable to judge whether actions are successful or not.

It’s important to start with a basic strategy, constantly observe surroundings, and update the strategy to define the direction for your product and organization.

Setting Up Twitter Social Cards

Sat, 06 Jan 2024 21:45:12 +0900

Overview

This article explains how to set up Twitter Social Cards for a blog created with Hugo.

A Twitter Social Card is an image that is displayed when an article is shared on Twitter. The image below is an example of a Twitter Social Card.

There are several types of Twitter Social Cards, including:

Summary Card
Summary Card with Large Image
App Card
Player Card

For sharing blog posts, the most commonly used types are Summary Card and Summary Card with Large Image.

For more details on each card type, refer to Twitter’s official documentation.

There are two main methods to set up Twitter Social Cards:

Setting up through the theme
Setting up independently of the theme

Setting Up Through the Theme

Some themes allow you to set up Twitter Social Cards directly.

For this example, I’ll use the Stack theme that I’m using. In Stack, you can configure Twitter Social Cards in config.toml as follows:

1
2
3
4
5
6
7
8


[opengraph.twitter]
site = ""
card = "summary" # summary or summary_large_image

[defaultImage.opengraph]
enabled = true
local = false
src = "/images/share.webp" # Path to the default image you want to set

Setting Up Independently of the Theme

If your theme doesn’t support Twitter Social Card settings, you will need to implement it yourself.

Hugo’s official template for implementation is available, which you can use for an easy setup.

Troubleshooting

If the settings don’t seem to work, it might be due to incorrect implementation or the meta tags not being properly set. In such cases, use the debugging tool provided by Twitter to check if the settings have been correctly applied.

Summary

This article explained how to set up Twitter Social Cards for a blog created with Hugo. Setting up Social Cards can enhance the visibility of your shared articles on Twitter, potentially attracting more readers, so it’s definitely worth doing.

Publishing a Blog with Hugo + Netlify + Github - Blog initial setup
Using Google Search Console to Make Your Blog Searchable on Google - SEO setup

Using ChatGPT to Make a Hugo Blog Multilingual

Sun, 31 Dec 2023 20:46:36 +0900

Overview

This article explains how to use ChatGPT to make a blog created with Hugo multilingual.

Translating Articles into English with ChatGPT

You can translate articles written in Markdown into English using ChatGPT. When doing so, use the following prompt to ensure that the format remains intact:

1
2
3


Please translate this Markdown into English without altering its format.
Ensure that no extraneous output is included.
Present the translated content in a format that can be easily copied.

Paste your article in Japanese, and ChatGPT will output the translated Markdown. You can copy the output directly by clicking the copy button at the bottom left of the output.

It is highly recommended to use GPT-4 instead of GPT-3.5, despite the subscription cost. GPT-4 significantly outperforms GPT-3.5 in providing accurate responses, making it useful for purposes beyond just translation.

Making Hugo Multilingual

Here are the steps to make your Hugo site multilingual.

Creating Configuration Files

Add the following settings to your config.toml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


# Set the default language, without this setting English is assumed default.
defaultContentLanguage = "jp"

[languages]
 # Set configurations for each language
 [languages.jp]
 title = "Bossagyu Blog"
 languageName = "ja-jp 🇯🇵"
 LanguageCode = "ja-jp"
 contentDir = "content" # Directory for Japanese blog articles
 [languages.jp.params]

 [languages.en]
 title = "Bossagyu Blog"
 languageName = "en-US 🇺🇸"
 LanguageCode = "en-US"
 contentDir = "content.en" # Directory for English blog articles
 [languages.en.params]

With the above settings, you can write Japanese articles in the content directory and English articles in the content.en directory to support multiple languages.

The final directory structure will look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


project/
├── content/
│ ├── index.md
│ └── blog/
│ ├── index.md
│ └── article1.md
└── content.en/
 ├── index.md
 └── blog/
 ├── index.md
 └── article1.en.md

In the English directory, naming files as article-name.en.md identifies them as English versions of the default language articles, and language switch icons will be added to the articles. For the articles, just copy and paste the translations from ChatGPT.

Summary

This article explained how to make a Hugo blog multilingual using ChatGPT. Utilizing ChatGPT simplifies the translation process. Multilingual support can help reach audiences outside Japan, increasing readership.

Since it’s low-cost and efficient, it’s worth giving it a try.

Publishing a Blog with Hugo + Netlify + Github - Blog initial setup

Creating and Displaying a Favicon with Hugo

Sun, 24 Dec 2023 22:14:39 +0900

Overview

This article explains a simple method to create a favicon. It also covers how to display a favicon in Hugo.

What is a Favicon?

A favicon is an icon that appears in bookmarks, tabs, and home screens for websites. Google has published guidelines for favicons that appear in search results. Adhering to these guidelines can help your favicon appear in search results.

Creating a Favicon

To create a favicon, use the following site:

Favicon.ico & App Icon Generator

When you visit the site, you’ll see a screen like this.

Enter the URL of the site you want a favicon for and click on Generate Favicon. Then, on the displayed screen, click the ‘Download the generated favicon’ link to download the favicon.

Displaying a Favicon in Hugo

To display a favicon in Hugo, the process varies depending on the theme. For the bearcub theme, you can simply set it in the toml like this:

1
2


[params]
 favicon = "images/favicon.ico"

Summary

This article explained how to create a favicon and display it in Hugo. Favicons are displayed in bookmarks, tabs, and home screens, so it’s a good idea to create one for your blog.

Introduction to Using Lighthouse

Fri, 22 Dec 2023 23:08:00 +0900

Overview

This article explains how to measure your blog’s performance using Lighthouse.

What is Lighthouse?

Lighthouse is a performance measurement tool for websites provided by Google. It’s available as a Google Chrome extension and can be used by installing the plugin.

Installing Lighthouse

Install Lighthouse from the Chrome Web Store.

Open the site you want to analyze and click on the Lighthouse icon.

Click on Generate report to start the analysis.
This time, I ran it on my blog page.

The results are displayed as follows, taking about 1 minute to complete.

Interpreting the Results

Performance

Evaluates web performance, like page loading and image display speeds. Clicking the See calculator link takes you to more details.

Accessibility

Checks whether all users can access content and navigate efficiently within the site. Scrolling down shows areas flagged by Accessibility.

It points out weak color contrast in the code snippets and missing descriptions in links.

However, the flagged content is not from my writing but depends on the template, so to fix this, it would be necessary to override the Hugo template.

Best Practices

Tests the integrity of web pages. You can view the testing items in the results.

SEO

You can check if the page is optimized for search engine result rankings.

Progressive Web App

Checks if the loading speed of web pages on smartphones is optimized and if it’s suitable for PWAs. This wasn’t checked in this case.

Summary

The article explained how to measure the performance of a blog using Lighthouse. Especially for SEO, as it affects visibility in Google search results, it’s important to address these issues adequately.

Publishing a Blog with Hugo + Netlify + Github - Blog initial setup
How to Set Up Google Analytics with Hugo - Adding analytics

How to Schedule Lambda Functions with AWS EventBridge

Thu, 21 Dec 2023 23:03:13 +0900

Overview

This article explains how to use AWS EventBridge to schedule Lambda functions for periodic execution.

I used this approach to implement scheduled reminder notifications for a LINE Bot. For example, you can send cleaning reminders every morning at 7 AM without managing any servers.

What is AWS EventBridge?

AWS EventBridge is a service that facilitates event passing between AWS services. It has two main use cases:

Use Case	Description
Scheduled Execution	Run Lambda periodically using cron or rate expressions
Event-Driven	Trigger Lambda on events like S3 file uploads

This article focuses on scheduled execution.

Real-World Use Cases

EventBridge + Lambda scheduled execution is useful for:

Reminder notifications: Send daily notifications to Slack or LINE at fixed times
Batch processing: Aggregate data hourly and save to database
Health checks: Monitor external API availability every 5 minutes
Cleanup tasks: Delete old logs every night

I use this for my Cleaning Reminder Bot to check notification conditions every hour.

Prerequisites

This guide assumes you have already created a Lambda function. For instructions on creating Lambda functions, see the AWS Lambda Getting Started guide.

Steps

1. Add a Trigger

Select the Lambda function you want to schedule with EventBridge and click “Add trigger”.

2. Select EventBridge

Choose “EventBridge (CloudWatch Events)” from the trigger options.

3. Configure the Schedule

After selecting EventBridge, you’ll be prompted to create a rule. Configure the schedule as needed.

4. Setup Complete

Once configured, EventBridge will appear as a trigger in the Lambda function diagram.

Cron Expression Syntax

EventBridge cron expressions consist of 6 fields:

1

cron(minute hour day month day-of-week year)

Common Patterns

Timing	Cron Expression
Daily at 9 AM (UTC)	`cron(0 9 * * ? *)`
Daily at 9 AM (JST = UTC+9)	`cron(0 0 * * ? *)`
Every hour at minute 0	`cron(0 * * * ? *)`
Every 5 minutes	`cron(0/5 * * * ? *)`
Weekdays only at 9 AM (UTC)	`cron(0 9 ? * MON-FRI *)`
First day of each month at midnight (UTC)	`cron(0 0 1 * ? *)`

Note: Timezone is UTC

EventBridge cron expressions use UTC timezone. To schedule in JST (Japan Standard Time), subtract 9 hours.

Example: Run at 7 AM JST → cron(0 22 * * ? *) (10 PM UTC the previous day)

Testing the Setup

I set up a function to send messages to LINE. The setup now sends notifications every 5 minutes:

For testing, set a short interval (like 5 minutes) to verify it works, then change to an appropriate interval for production.

Cost

The EventBridge + Lambda combination is very cost-effective:

Service	Free Tier	Cost After Free Tier
EventBridge	Free	Schedule rules are free
Lambda	1M requests/month free	$0.20 per 1M requests

For personal projects like reminder bots, you can run nearly for free.

Common Pitfalls

1. Timezone Confusion

As mentioned, EventBridge uses UTC. A common mistake is setting “9 AM” and finding it runs at 6 PM instead.

2. Initial Execution Timing

After setting up a cron expression, you need to wait for the next scheduled time. To test immediately, use Lambda’s built-in test feature.

3. Don’t Forget to Delete

Test EventBridge rules left running will keep triggering Lambda. Always delete rules you no longer need.

Summary

This article covered how to schedule Lambda functions with AWS EventBridge.

EventBridge supports both scheduled execution and event-driven triggers
Cron expressions use UTC timezone - subtract 9 hours for JST
Running costs are nearly free for personal projects
Don’t forget to delete unused rules

Serverless scheduled execution is great for reminder bots and batch processing. Give it a try!

Using Google Search Console to Make Your Blog Searchable on Google

Mon, 18 Dec 2023 19:10:04 +0900

Overview

To appear in Google searches, it’s not enough just to apply SEO strategies; your site must first be recognized by Google. This article explains how to make your own custom domain blog appear in Google search results using Google Search Console.

Steps to Implementation

Registering with Google Search Console
Verifying Domain Ownership
Registering the Sitemap
Requesting Index Registration
Summary

Registering with Google Search Console

Choose your domain and enter the URL.

Verifying Domain Ownership

A screen like the following will appear to verify DNS ownership.
(The TXT record content is blacked out for privacy.)

You can verify ownership by adding a string specified by Google to your domain’s TXT record. Go to your domain’s DNS settings and add a TXT record.

In my case, I acquired the domain through Netlify, so I went to Netlify’s DNS settings. Navigate to Domains -> Domain Settings -> DNS Records and add the TXT record.

Copy the content displayed on Google Search Console and paste it into the Value field.
(The Value part is blacked out for privacy.)

Wait for DNS updates, which can take a few hours depending on the provider.

You can check DNS propagation from the command line.

1

dig -t txt bossagyu.com

Afterward, press the verify ownership button on Google Search Console.
This completes the verification process, and your domain will be registered with Google Search Console.

Registering the Sitemap

Registering a sitemap informs Google about the structure of your site, facilitating the crawling process. For blogs created with Hugo, the sitemap is available at /sitemap.xml, which you should register.

From the left menu of Google Search Console, select ‘Sitemaps’ and add your sitemap.

Requesting Index Registration

Even if your site is registered in the sitemap, it can take time for Google to crawl and index it. In my case, I requested index registration after waiting several days without being indexed.

Search for the URL you want to register in Google Search Console, and click on ‘Request Indexing’ found on the right side of the search result.

This requests index registration. It took a few hours for the index to be registered after clicking.

Summary

This article explained how to make your custom domain blog searchable using Google Search Console.
It’s a waste not to have your blog appear in Google searches after all the effort of creating it, so give it a try.

Publishing a Blog with Hugo + Netlify + Github - Blog initial setup
How to Set Up Google Analytics with Hugo - Adding analytics
Introduction to Using Lighthouse - Performance measurement

Efficient Lambda Development with AWS Toolkit in IntelliJ

Tue, 12 Dec 2023 22:40:05 +0900

Overview

This article explains how to efficiently develop Lambda functions using the AWS Toolkit in IntelliJ.

Steps to Implementation

Preliminary Preparation
Installing AWS Toolkit
Configuring AWS Toolkit
Developing Lambda
Executing Lambda Locally
Summary

Preliminary Preparation

Installing Docker

AWS Toolkit in IntelliJ uses Docker to run Lambda.
Prior to proceeding, please install Docker by referring to these instructions.

Installing AWS CLI

Install AWS CLI (SAM).
For installation, refer to these instructions.

In IntelliJ, set the path for SAM CLI executable under File -> Settings -> Tools -> AWS Toolkit.
In my case, as I installed it through brew, I set the following path.

Installing AWS Toolkit

Install the AWS Toolkit via IntelliJ plugins. Refer to this guide for plugin installation.

Configuring AWS Toolkit

To use AWS Toolkit, you need to set up your AWS credentials.

Set up AWS credentials through AWS Explorer.

Obtain and configure your Access Key ID and Secret Access Key from the AWS console. Once configured, AWS resources should appear in AWS Explorer.

Note: In this image, the region is set to us-east-1. Please adjust according to the region where you intend to create your Lambda.

Developing Lambda

Create a code snippet like the following.

lambda-sample.py

1
2
3


def lambda_handler(event, context):
 print("Hello World")
 return "Hello World!"

Create a Lambda through AWS Explorer.

Select Create Lambda Function and input the necessary values.
For the Handler, enter <filename>.<function name> from your code snippet.

This completes the creation of your Lambda.

Executing Lambda Locally

The Toolkit also allows you to execute Lambda locally.

Selecting Run will execute the Lambda locally.

Summary

This article provided a guide on efficiently developing Lambda using AWS Toolkit in IntelliJ. Developing in IntelliJ and executing locally can significantly improve development efficiency.

How to Use GitHub Copilot in IntelliJ

Mon, 11 Dec 2023 22:45:40 +0900

Overview

This article explains how to use GitHub Copilot in IntelliJ. Additionally, a cheat sheet of shortcuts is provided.

Steps to Implementation

Register for GitHub Copilot
Configure IntelliJ
Use GitHub Copilot
Summary

Registering for GitHub Copilot

Configuring IntelliJ

Install the GitHub Copilot plugin from IntelliJ plugins.

Once installed, restart IntelliJ.

Using GitHub Copilot

When you write code in IntelliJ, GitHub Copilot will assist with code completion.

Here is a list of shortcuts for Mac:

Shortcut	Function
`tab`	Complete the code
`Option` + `]`	Show the next completion suggestion
`Option` + `[`	Show the previous completion suggestion
`Command` + `→`	Accept only the next word of the suggestion

Summary

This article explained how to use GitHub Copilot in IntelliJ. It’s worth noting that this article was written using GitHub Copilot, and it significantly assists with blog creation in Markdown, so those interested should give it a try.

Setting Up a Local Environment Using Pyenv and venv

Sun, 10 Dec 2023 23:19:33 +0900

Overview

This article documents how to set up a Python environment for development on a Mac local environment.
In this case, we will use two systems to manage different versions of Python and virtual environments:

pyenv
- Used to handle multiple versions of Python.
venv
- Used to separate environments for each project.

For explanations on the differences and the necessity of each, this article is a helpful reference.

Installing Python

First, install Pyenv on your local environment to use a specific version of Python.

Install pyenv.

1

brew install pyenv

Check the installed version of pyenv.

1
2


pyenv --version
pyenv 2.3.35

Add settings to zsh.

1
2
3


echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.zshrc

Reload .zshrc.

1

source ~/.zshrc

Display a list of installable Python versions.

1

pyenv install --list

Install the specified version.

1

pyenv install 3.11.7

Use the specified Python version in your project folder.

1
2
3


cd <created project folder>
pyenv local 3.11.7
pyenv versions

If global, it will be applied to the entire system.

1

pyenv global 3.11.7

Check the version of Python being executed.

1

python -V

Creating a Virtual Environment with venv

Create a virtual environment in the project directory.

1
2


# python -m venv <virtual environment name>
python -m venv venv

Activate the virtual environment.

1

source venv/bin/activate

To deactivate, execute the following command.

1

deactivate

How to Update a venv

If you need to refresh dependencies, activate the virtual environment and run the following. When using requirements.txt, update the file first and then reinstall.

1
2
3


source venv/bin/activate
pip install --upgrade pip
pip install -U -r requirements.txt

When upgrading Python to a new minor version, install the new version with pyenv, recreate the virtual environment, and reinstall dependencies to avoid mismatches.

1
2
3
4
5
6


pyenv install 3.12.2
pyenv local 3.12.2
rm -rf venv
python -m venv venv
source venv/bin/activate
pip install -U -r requirements.txt

How to Delete a venv

When the virtual environment is no longer needed, delete the directory. Confirm you are in the correct working directory before running the command.

1

rm -rf venv

This completes the setup of the local environment.

Setting Up a Python Development Environment on Mac with UV (as a modern alternative)

How to Set Up Google Analytics with Hugo

Sat, 09 Dec 2023 18:09:42 +0900

Overview

This article briefly explains how to set up Google Analytics with Hugo.

Steps to Implementation

Register with Google Analytics
Obtain the Tracking ID
Add the Tracking ID to Hugo’s configuration

Registering with Google Analytics

Follow the instructions on Setting up a new website or app with GA4 to register.
When you add a data stream, you will get a Tracking ID, so make a note of it. ※ The Tracking ID may be displayed as “Measurement ID” due to translation.

Adding Tracking ID to Hugo’s Configuration

Add settings in toml

Add googleAnalytics = Tracking ID to your config.toml.

1
2
3
4
5
6


baseURL = 'https://bossagyu.com'
languageCode = 'ja-jp'
title = 'Bossagyu Blog'
theme = 'hugo-bearcub'
googleAnalytics = "G-1234ABCDEF"
# ↑ Add this line, replace the Tracking ID with your own.

Embedding the Tracking Code

Some templates might read settings from the toml file, but the bearcub template I use does not support this, so I added the tracking code to the header myself.

For the code snippet, I referred to Makumaku Hugo Notes.

Create layouts/partials/analytics.html to load the tracking code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


{{ if not .Site.IsServer }}
{{ with .Site.GoogleAnalytics }}
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id={{ . }}"></script>
<script>
 window.dataLayer = window.dataLayer || [];
 function gtag(){dataLayer.push(arguments);}
 gtag('js', new Date());

 gtag('config', '{{ . }}');
</script>
{{ end }}
{{ end }}

Make the page header load analytics.html.

1
2
3


# Copy the content of the template to override it
cp themes/hugo-bearcub/layouts/_default/baseof.html layouts/_default/baseof.html
vim layouts/_default/baseof.html

Add {{- partial "analytics" . -}} to baseof.html.

1
2
3
4
5
6


<!DOCTYPE html>
<html lang="{{ with .Site.LanguageCode }}{{ . }}{{ else }}en-US{{ end }}">

<head>
 {{- partial "analytics" . -}}
 <meta http-equiv="X-Clacks-Overhead" content="GNU Terry Pratchett" /

After adding the above source code and rebuilding, data will be sent to Google Analytics.

Tips

If it seems like data is not being transmitted to Google Analytics despite these steps, it’s possible that the tags have not been added correctly.
To troubleshoot, first check if the tracking is included in the HTML by using Google Developer Tools.

Publishing a Blog with Hugo + Netlify + Github - Blog initial setup
Using Google Search Console to Make Your Blog Searchable on Google - SEO setup

Registering and Using the LINE Messaging API

Thu, 07 Dec 2023 09:37:00 +0900

Overview

I thought of creating an application using LINE’s Bot, so first, I will make the Bot usable.
This page introduces how to register for the LINE Messaging API and how to send messages from the command line using curl.

Using the Messaging API

A provider is (Explanation)

1

On the LINE Developers site, a service provider refers to individuals, companies, or organizations that provide services and obtain user information (service proprietor in LINE Mini Apps).

So, you can enter any string you like.

Then, create a new channel.

Clicking the create button will establish a new channel.

Posting from the Command Line

Add friends by reading the QR code in the Messaging API settings.

Obtain the ‘Channel Access Token (Long-lived)’ from the Messaging API settings. Get ‘Your User ID’ from the channel basic settings.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


TOKEN="<Channel Access Token (Long-lived)>"
ID="<Your User ID>"
UUID=$(uuidgen | tr "[:upper:]" "[:lower:]")

curl -v -X POST https://api.line.me/v2/bot/message/push \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer ${TOKEN}" \
-H "X-Line-Retry-Key: " \
-d "{
 \"to\": \"${ID}\",
 \"messages\":[
 {
 \"type\":\"text\",
 \"text\":\"Hello, world1\"
 }
 ]
}"

If a response is returned and you see a post from the Bot in your LINE chat, it’s a success!

Cleaning Reminder LINE Bot (a practical LINE Bot project)

Publishing a Blog with Hugo + Netlify + Github

Sat, 02 Dec 2023 00:59:37 +0900

Overview

This document describes the steps to create a site with Hugo, manage it with Github, and build it with Netlify from scratch. With this method, you can easily publish by just pushing your Markdown-written blog to Github.

This guide also covers how to set up a favicon.

Process

Generate a site with Hugo
Push to Github
Deploy with Netlify
Set up Favicon

Generating a Static Site with Hugo

First, install Hugo.

1

brew install hugo

Create a template for the blog.

1

hugo new site my-blog

Add a theme suitable for the blog as a submodule.

1
2
3
4
5


cd my-blog
git init

# Add the theme as a submodule from Github
git submodule add https://github.com/theNewDynamic/gohugo-theme-ananke.git themes/ananke

Apply the theme by adding it to hugo.toml.

1

echo "theme = 'ananke'" >> config.toml

Start the server.

1

hugo server

Access the URL like http://localhost:51517/ shown in the startup log Web Server is available at http://localhost:51517/ (bind address 127.0.0.1) to view the locally launched static site.

Tips

If you want to change the Hugo theme, please choose your favorite one from Hugo Themes.
- It’s recommended to run through until you build with Netlify first, as this can be changed later.
The way to write Toml files is described in Configure Hugo.

Push to Github

Create a repository on Github.
After creation, execute the following commands to push your site.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


cd my-blog
echo .hugo_build.lock >> .gitignore
git add .
git commit -m "first commit"
git branch -M main

# Replace <user name> with your own username.
# This is an example of creating a repository called my-blog.
git remote add origin git@github.com:<user name>/my-blog
git push -u origin main

Once the push is complete, the source code becomes viewable on the Github UI.

Deploy with Netlify

Access Netlify and perform deployment.
There are instructions on Hugo’s official website, so refer to them for integration.

Follow the instructions to complete the deployment, and the result of the Deploy will be shown as published.

Click on the URL displayed on the site to access the deployed site.

This completes the deployment process. After this, any changes made and pushed to main will automatically trigger deployment, updating the site content.

Setting Up Favicon

A favicon is an icon displayed in bookmarks, browser tabs, and home screens. Google has published favicon guidelines for appearing in search results. Following these guidelines will help your favicon appear in search results.

Creating a Favicon

To create a favicon, use the following website:

Favicon.ico & App Icon Generator

Access the site, upload the image you want to use as a favicon, and click “Generate Favicon”. Then click “Download the generated favicon” to download your favicon.

Displaying Favicon in Hugo

To display a favicon in Hugo, the method varies by theme, but most themes allow you to simply add the following to your toml configuration:

1
2


[params]
 favicon = "images/favicon.ico"

Place the downloaded favicon.ico in the static/images/ directory and add the above configuration.

Summary

This guide covered how to publish a blog with Hugo + Netlify + Github.

Hugo: Generate static site
Github: Manage source code
Netlify: Automatic deployment
Favicon: Branding configuration

With this setup, you just write articles in Markdown and push to Github for automatic site updates.

How to Set Up Google Analytics with Hugo - Adding analytics
How to Add Your Blog to Google Search Using Google Search Console - SEO setup
Introduction to Using Lighthouse - Performance measurement
Using ChatGPT to Make a Hugo Blog Multilingual - Multilingual support
Setting Up Twitter Social Cards - Social media optimization

Blogs on Bossagyu Blog

Three Models of Technical Proficiency: Dreyfus, Shu-Ha-Ri, and the Four Stages of Competence

Overview

Why Understand the Stages of Proficiency

The Dreyfus Model

Origin and Background

Five Stages

Cognitive Shifts Between Stages

Shu-Ha-Ri

Origin and Background

Three Stages

Differences from Western Models

Modern Applications

The Four Stages of Competence

Origin and Background

Four Stages

Psychological Significance and a Fifth Stage

Comparing the Three Frameworks

Commonalities

Differences

Which to Use When

Applications: Learning and Product Design

1. Designing Your Own Learning

2. Mentoring and Supporting Others

3. Product and App Design

Summary

References

Getting Started with FastAPI: Building a CRUD API Tutorial

Overview

Prerequisites

Environment Setup

Hello World

Swagger UI

Implementing the CRUD API

Defining Models

Create (POST)

List (GET)

Get by ID (GET)

Update (PUT)

Delete (DELETE)

Complete Code

Testing

1. Create a TODO

2. List TODOs

3. Update

4. Delete

Summary

Related Articles

Designing an Event-Driven Backend to Sync GCS with Gemini File Search API

Overview

System Architecture

Component Roles

Event-Driven Architecture Design

Why Event-Driven?

Event Routing with Eventarc

File Upload Processing Flow

File Deletion Processing Flow

Integration with Gemini File Search API

Unified GCP Ecosystem

File Search Store Overview

Metadata-Based Management

Design Patterns for Extensibility

Path-Based Metadata Extraction

Guaranteeing State Synchronization Between GCS and Gemini File Search API

Local Download Design

Error Handling and Operational Design

Always Return 200 OK

Error Monitoring via Slack Notifications

Lazy Initialization for Client Management

Summary

Related Articles

Building a Cleaning Reminder Bot with AWS Lambda + LINE

Overview

Architecture

Services Used

Why This Architecture

Project Structure

Implementation Details

1. Lambda Entry Points

2. Task Deadline Management