RAG is a technique that augments the power of generative models by incorporating a retrieval step. Instead of relying solely on the model’s internal knowledge, we can retrieve relevant documents or data from an external source and use that information to produce more accurate, context-aware responses. This approach is particularly useful when building applications such as question-answering systems, chatbots, or any scenario where up-to-date or domain-specific knowledge is needed.

Setting Up our Ent schema​

Let's begin our tutorial by initializing the Go module which we will be using for our project:

go mod init github.com/rotemtam/entrag # Feel free to replace the module path with your own

In this project we will use Ent, an entity framework for Go, to define our database schema. The database will store the documents we want to retrieve (chunked to a fixed size) and the vectors representing each chunk. Initialize the Ent project by running the following command:

go run -mod=mod entgo.io/ent/cmd/ent new Embedding Chunk

This command creates placeholders for our data models. Our project should look like this:

├── ent
│   ├── generate.go
│   └── schema
│       ├── chunk.go
│       └── embedding.go
├── go.mod
└── go.sum

Next, let's define the schema for the Chunk model. Open the ent/schema/chunk.go file and define the schema as follows:

package schema

import (
    "entgo.io/ent"
    "entgo.io/ent/schema/edge"
    "entgo.io/ent/schema/field"
)

// Chunk holds the schema definition for the Chunk entity.
type Chunk struct {
    ent.Schema
}

// Fields of the Chunk.
func (Chunk) Fields() []ent.Field {
    return []ent.Field{
        field.String("path"),
        field.Int("nchunk"),
        field.Text("data"),
    }
}

// Edges of the Chunk.
func (Chunk) Edges() []ent.Edge {
    return []ent.Edge{
        edge.To("embedding", Embedding.Type).StorageKey(edge.Column("chunk_id")).Unique(),
    }
}

This schema defines a Chunk entity with three fields: path, nchunk, and data. The path field stores the path of the document, nchunk stores the chunk number, and data stores the chunked text data. We also define an edge to the Embedding entity, which will store the vector representation of the chunk.

Before we proceed, let's install the pgvector package. pgvector is a PostgreSQL extension that provides support for vector operations and similarity search. We will need it to store and retrieve the vector representations of our chunks.

go get github.com/pgvector/pgvector-go

Next, let's define the schema for the Embedding model. Open the ent/schema/embedding.go file and define the schema as follows:

package schema

import (
    "entgo.io/ent"
    "entgo.io/ent/dialect"
    "entgo.io/ent/dialect/entsql"
    "entgo.io/ent/schema/edge"
    "entgo.io/ent/schema/field"
    "entgo.io/ent/schema/index"
    "github.com/pgvector/pgvector-go"
)

// Embedding holds the schema definition for the Embedding entity.
type Embedding struct {
    ent.Schema
}

// Fields of the Embedding.
func (Embedding) Fields() []ent.Field {
    return []ent.Field{
        field.Other("embedding", pgvector.Vector{}).
            SchemaType(map[string]string{
                dialect.Postgres: "vector(1536)",
            }),
    }
}

// Edges of the Embedding.
func (Embedding) Edges() []ent.Edge {
    return []ent.Edge{
        edge.From("chunk", Chunk.Type).Ref("embedding").Unique().Required(),
    }
}

func (Embedding) Indexes() []ent.Index {
    return []ent.Index{
        index.Fields("embedding").
            Annotations(
                entsql.IndexType("hnsw"),
                entsql.OpClass("vector_l2_ops"),
            ),
    }
}

This schema defines an Embedding entity with a single field embedding of type pgvector.Vector. The embedding field stores the vector representation of the chunk. We also define an edge to the Chunk entity and an index on the embedding field using the hnsw index type and vector_l2_ops operator class. This index will enable us to perform efficient similarity searches on the embeddings.

Finally, let's generate the Ent code by running the following commands:

go mod tidy
go generate ./...

Ent will generate the necessary code for our models based on the schema definitions.

Setting Up the database​

Next, let's set up the PostgreSQL database. We will use Docker to run a PostgreSQL instance locally. As we need the pgvector extension, we will use the pgvector/pgvector:pg17 Docker image, which comes with the extension pre-installed.

docker run --rm --name postgres -e POSTGRES_PASSWORD=pass -p 5432:5432 -d pgvector/pgvector:pg17

We will be using Atlas, a database schema-as-code tool that integrates with Ent, to manage our database schema. Install Atlas by running the following command:

curl -sSfL https://atlasgo.io/install.sh | sh

For other installation options, see the Atlas installation docs.

As we are going to managing extensions, we need an Atlas Pro account. You can sign up for a free trial by running:

atlas login

Now, let's create our base configuration base.pg.hcl which provides the vector extension for the public schema:

schema "public" {
}

extension "vector" {
  schema = schema.public
}

Now, let's create our Atlas configuration which composes the base.pg.hcl file with the Ent schema:

data "composite_schema" "schema" {
  schema {
    url = "file://base.pg.hcl"
  }
  schema "public" {
    url = "ent://ent/schema"
  }
}

env "local" {
  url = getenv("DB_URL")
  schema {
    src = data.composite_schema.schema.url
  }
  dev = "docker://pgvector/pg17/dev"
}

This configuration defines a composite schema that includes the base.pg.hcl file and the Ent schema. We also define an environment named local that uses the composite schema which we will use for local development. The dev field specifies the Dev Database URL, which is used by Atlas to normalize schemas and make various calculations.

Next, let's apply the schema to the database by running the following command:

export DB_URL='postgresql://postgres:pass@localhost:5432/postgres?sslmode=disable'
atlas schema apply --env local

Atlas will load the desired state of the database from our configuration, compare it to the current state of the database, and create a migration plan to bring the database to the desired state:

Planning migration statements (5 in total):

  -- create extension "vector":
    -> CREATE EXTENSION "vector" WITH SCHEMA "public" VERSION "0.8.0";
  -- create "chunks" table:
    -> CREATE TABLE "public"."chunks" (
         "id" bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY,
         "path" character varying NOT NULL,
         "nchunk" bigint NOT NULL,
         "data" text NOT NULL,
         PRIMARY KEY ("id")
       );
  -- create "embeddings" table:
    -> CREATE TABLE "public"."embeddings" (
         "id" bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY,
         "embedding" public.vector(1536) NOT NULL,
         "chunk_id" bigint NOT NULL,
         PRIMARY KEY ("id"),
         CONSTRAINT "embeddings_chunks_embedding" FOREIGN KEY ("chunk_id") REFERENCES "public"."chunks" ("id") ON UPDATE NO ACTION ON DELETE NO ACTION
       );
  -- create index "embedding_embedding" to table: "embeddings":
    -> CREATE INDEX "embedding_embedding" ON "public"."embeddings" USING hnsw ("embedding" vector_l2_ops);
  -- create index "embeddings_chunk_id_key" to table: "embeddings":
    -> CREATE UNIQUE INDEX "embeddings_chunk_id_key" ON "public"."embeddings" ("chunk_id");

-------------------------------------------

Analyzing planned statements (5 in total):

  -- non-optimal columns alignment:
    -- L4: Table "chunks" has 8 redundant bytes of padding per row. To reduce disk space,
       the optimal order of the columns is as follows: "id", "nchunk", "path",
       "data" https://atlasgo.io/lint/analyzers#PG110
  -- ok (370.25µs)

  -------------------------
  -- 114.306667ms
  -- 5 schema changes
  -- 1 diagnostic

-------------------------------------------

? Approve or abort the plan:
  ▸ Approve and apply
    Abort

In addition to planning the change, Atlas will also provide diagnostics and suggestions for optimizing the schema. In this case it suggests reordering the columns in the chunks table to reduce disk space. As we are not concerned with disk space in this tutorial, we can proceed with the migration by selecting Approve and apply.

Finally, we can verify that our schema was applied successfully, we can re-run the atlas schema apply command. Atlas will output:

Schema is synced, no changes to be made

Scaffolding the CLI​

Now that our database schema is set up, let's scaffold our CLI application. For this tutorial, we will be using the alecthomas/kong library to build a small app that can load, index and query the documents in our database.

First, install the kong library:

go get github.com/alecthomas/kong

Next, create a new file named cmd/entrag/main.go and define the CLI application as follows:

package main

import (
    "fmt"
    "os"

    "github.com/alecthomas/kong"
)

// CLI holds global options and subcommands.
type CLI struct {
    // DBURL is read from the environment variable DB_URL.
    DBURL     string `kong:"env='DB_URL',help='Database URL for the application.'"`
    OpenAIKey string `kong:"env='OPENAI_KEY',help='OpenAI API key for the application.'"`

    // Subcommands
    Load  *LoadCmd  `kong:"cmd,help='Load command that accepts a path.'"`
    Index *IndexCmd `kong:"cmd,help='Create embeddings for any chunks that do not have one.'"`
    Ask   *AskCmd   `kong:"cmd,help='Ask a question about the indexed documents'"`
}

func main() {
    var cli CLI
    app := kong.Parse(&cli,
        kong.Name("entrag"),
        kong.Description("Ask questions about markdown files."),
        kong.UsageOnError(),
    )
    if err := app.Run(&cli); err != nil {
        fmt.Fprintf(os.Stderr, "Error: %s\n", err)
        os.Exit(1)
    }
}

Create an additional file named cmd/entrag/rag.go with the following content:

package main

type (
    // LoadCmd loads the markdown files into the database.
    LoadCmd struct {
        Path string `help:"path to dir with markdown files" type:"existingdir" required:""`
    }
    // IndexCmd creates the embedding index on the database.
    IndexCmd struct {
    }
    // AskCmd is another leaf command.
    AskCmd struct {
        // Text is the positional argument for the ask command.
        Text string `kong:"arg,required,help='Text for the ask command.'"`
    }
)

Verify our scaffolded CLI application works by running:

go run ./cmd/entrag --help

If everything is set up correctly, you should see the help output for the CLI application:

Usage: entrag <command> [flags]

Ask questions about markdown files.

Flags:
  -h, --help                  Show context-sensitive help.
      --dburl=STRING          Database URL for the application ($DB_URL).
      --open-ai-key=STRING    OpenAI API key for the application ($OPENAI_KEY).

Commands:
  load --path=STRING [flags]
    Load command that accepts a path.

  index [flags]
    Create embeddings for any chunks that do not have one.

  ask <text> [flags]
    Ask a question about the indexed documents

Run "entrag <command> --help" for more information on a command.

Load the documents into the database​

Next, we need some markdown files to load into the database. Create a directory named data and add some markdown files to it. For this example, I downloaded the ent/ent repository and used the docs directory as the source of markdown files.

Now, let's implement the LoadCmd command to load the markdown files into the database. Open the cmd/entrag/rag.go file and add the following code:

const (
    tokenEncoding = "cl100k_base"
    chunkSize     = 1000
)

// Run is the method called when the "load" command is executed.
func (cmd *LoadCmd) Run(ctx *CLI) error {
    client, err := ctx.entClient()
    if err != nil {
        return fmt.Errorf("failed opening connection to postgres: %w", err)
    }
    tokTotal := 0
    return filepath.WalkDir(ctx.Load.Path, func(path string, d fs.DirEntry, err error) error {
        if filepath.Ext(path) == ".mdx" || filepath.Ext(path) == ".md" {
            chunks := breakToChunks(path)
            for i, chunk := range chunks {
                tokTotal += len(chunk)
                client.Chunk.Create().
                    SetData(chunk).
                    SetPath(path).
                    SetNchunk(i).
                    SaveX(context.Background())
            }
        }
        return nil
    })
}

func (c *CLI) entClient() (*ent.Client, error) {
    return ent.Open("postgres", c.DBURL)
}

This code defines the Run method for the LoadCmd command. The method reads the markdown files from the specified path, breaks them into chunks of 1000 tokens each, and saves them to the database. We use the entClient method to create a new Ent client using the database URL specified in the CLI options.

For the implementation of breakToChunks, see the full code in the entrag repository, which is based almost entirely on Eli Bendersky's intro to RAG in Go.

Finally, let's run the load command to load the markdown files into the database:

go run ./cmd/entrag load --path=data

After the command completes, you should see the chunks loaded into the database. To verify run:

docker exec -it postgres psql -U postgres -d postgres -c "SELECT COUNT(*) FROM chunks;"

You should see something similar to:

  count
-------
   276
(1 row)

Indexing the embeddings​

Now that we have loaded the documents into the database, we need to create embeddings for each chunk. We will use the OpenAI API to generate embeddings for the chunks. To do this, we need to install the openai package:

go get github.com/sashabaranov/go-openai

If you do not have an OpenAI API key, you can sign up for an account on the OpenAI Platform and generate an API key.

We will be reading this key from the environment variable OPENAI_KEY, so let's set it:

export OPENAI_KEY=<your OpenAI API key>

Next, let's implement the IndexCmd command to create embeddings for the chunks. Open the cmd/entrag/rag.go file and add the following code:

// Run is the method called when the "index" command is executed.
func (cmd *IndexCmd) Run(cli *CLI) error {
    client, err := cli.entClient()
    if err != nil {
        return fmt.Errorf("failed opening connection to postgres: %w", err)
    }
    ctx := context.Background()
    chunks := client.Chunk.Query().
        Where(
            chunk.Not(
                chunk.HasEmbedding(),
            ),
        ).
        Order(ent.Asc(chunk.FieldID)).
        AllX(ctx)
    for _, ch := range chunks {
        log.Println("Created embedding for chunk", ch.Path, ch.Nchunk)
        embedding := getEmbedding(ch.Data)
        _, err := client.Embedding.Create().
            SetEmbedding(pgvector.NewVector(embedding)).
            SetChunk(ch).
            Save(ctx)
        if err != nil {
            return fmt.Errorf("error creating embedding: %v", err)
        }
    }
    return nil
}

// getEmbedding invokes the OpenAI embedding API to calculate the embedding
// for the given string. It returns the embedding.
func getEmbedding(data string) []float32 {
    client := openai.NewClient(os.Getenv("OPENAI_KEY"))
    queryReq := openai.EmbeddingRequest{
        Input: []string{data},
        Model: openai.AdaEmbeddingV2,
    }
    queryResponse, err := client.CreateEmbeddings(context.Background(), queryReq)
    if err != nil {
        log.Fatalf("Error getting embedding: %v", err)
    }
    return queryResponse.Data[0].Embedding
}

We have defined the Run method for the IndexCmd command. The method queries the database for chunks that do not have embeddings, generates embeddings for each chunk using the OpenAI API, and saves the embeddings to the database.

Finally, let's run the index command to create embeddings for the chunks:

go run ./cmd/entrag index

You should see logs similar to:

2025/02/13 13:04:42 Created embedding for chunk /Users/home/entr/data/md/aggregate.md 0
2025/02/13 13:04:43 Created embedding for chunk /Users/home/entr/data/md/ci.mdx 0
2025/02/13 13:04:44 Created embedding for chunk /Users/home/entr/data/md/ci.mdx 1
2025/02/13 13:04:45 Created embedding for chunk /Users/home/entr/data/md/ci.mdx 2
2025/02/13 13:04:46 Created embedding for chunk /Users/home/entr/data/md/code-gen.md 0
2025/02/13 13:04:47 Created embedding for chunk /Users/home/entr/data/md/code-gen.md 1

Asking questions​

Now that we have loaded the documents and created embeddings for the chunks, we can implement the AskCmd command to ask questions about the indexed documents. Open the cmd/entrag/rag.go file and add the following code:

// Run is the method called when the "ask" command is executed.
func (cmd *AskCmd) Run(ctx *CLI) error {
    client, err := ctx.entClient()
    if err != nil {
        return fmt.Errorf("failed opening connection to postgres: %w", err)
    }
    question := cmd.Text
    emb := getEmbedding(question)
    embVec := pgvector.NewVector(emb)
    embs := client.Embedding.
        Query().
        Order(func(s *sql.Selector) {
            s.OrderExpr(sql.ExprP("embedding <-> $1", embVec))
        }).
        WithChunk().
        Limit(5).
        AllX(context.Background())
    b := strings.Builder{}
    for _, e := range embs {
        chnk := e.Edges.Chunk
        b.WriteString(fmt.Sprintf("From file: %v\n", chnk.Path))
        b.WriteString(chnk.Data)
    }
    query := fmt.Sprintf(`Use the below information from the ent docs to answer the subsequent question.
Information:
%v

Question: %v`, b.String(), question)
    oac := openai.NewClient(ctx.OpenAIKey)
    resp, err := oac.CreateChatCompletion(
        context.Background(),
        openai.ChatCompletionRequest{
            Model: openai.GPT4o,
            Messages: []openai.ChatCompletionMessage{

                {
                    Role:    openai.ChatMessageRoleUser,
                    Content: query,
                },
            },
        },
    )
    if err != nil {
        return fmt.Errorf("error creating chat completion: %v", err)
    }
    choice := resp.Choices[0]
    out, err := glamour.Render(choice.Message.Content, "dark")
    fmt.Print(out)
    return nil
}

This is where all of the parts come together. After preparing our database with the documents and their embeddings, we can now ask questions about them. Let's break down the AskCmd command:

emb := getEmbedding(question)
embVec := pgvector.NewVector(emb)
embs := client.Embedding.
    Query().
    Order(func(s *sql.Selector) {
        s.OrderExpr(sql.ExprP("embedding <-> $1", embVec))
    }).
    WithChunk().
    Limit(5).
    AllX(context.Background())

We begin by transforming the user's question into a vector using the OpenAI API. Using this vector we would like to find the most similar embeddings in our database. We query the database for the embeddings, order them by similarity using pgvector's <-> operator, and limit the results to the top 5.

for _, e := range embs {
        chnk := e.Edges.Chunk
        b.WriteString(fmt.Sprintf("From file: %v\n", chnk.Path))
        b.WriteString(chnk.Data)
    }
    query := fmt.Sprintf(`Use the below information from the ent docs to answer the subsequent question.
Information:
%v

Question: %v`, b.String(), question)

Next, we prepare the information from the top 5 chunks to be used as context for the question. We then format the question and the context into a single string.

oac := openai.NewClient(ctx.OpenAIKey)
resp, err := oac.CreateChatCompletion(
    context.Background(),
    openai.ChatCompletionRequest{
        Model: openai.GPT4o,
        Messages: []openai.ChatCompletionMessage{

            {
                Role:    openai.ChatMessageRoleUser,
                Content: query,
            },
        },
    },
)
if err != nil {
    return fmt.Errorf("error creating chat completion: %v", err)
}
choice := resp.Choices[0]
out, err := glamour.Render(choice.Message.Content, "dark")
fmt.Print(out)

Then, we use the OpenAI API to generate a response to the question. We pass the question and context to the API and receive a response. We then render the response using the glamour package to display it in the terminal.

Before running the ask command, let's install the glamour package:

go get github.com/charmbracelet/glamour

Finally, let's run the ask command to ask a question about the indexed documents:

go run ./cmd/entrag ask "tl;dr What is Ent?"

And our RAG system responds:

  Ent is an open-source entity framework (ORM) for the Go programming language. It
  allows developers to define data models or graph-structures in Go code. Ent
  emphasizes principles such as schema as code, a statically typed and explicit
  API generated through codegen, simple queries and graph traversals, statically
  typed predicates, and storage agnosticism. It supports various databases,
  including MySQL, MariaDB, PostgreSQL, SQLite, and Gremlin-based graph databases,
  and aims to enhance productivity in Go development.

Amazing! We have successfully built a RAG system using Ent, Atlas, and pgvector. We can now ask questions about the documents we loaded into the database and receive context-aware responses.

Here are some more Q&As with our nifty RAG system:

Wrapping up​

In this blog post, we explored how to build a RAG system using Ent, Atlas, and pgvector. Special thanks to Eli Bendersky for the informative blog post and for his great Go writing over the years!

Create a visualization of your Ent schema with one command:

atlas schema inspect \
  -u ent://ent/schema \
  --dev-url "sqlite://demo?mode=memory&_fk=1" \
  --visualize

Hi Everyone!

A few months ago, we shared entviz, a cool tool that enables you to visualize your Ent schemas. Due to its success and popularity, we decided to integrate it directly into Atlas, the migration engine that Ent uses.

Since the release of v0.13.0 of Atlas, you can now visualize your Ent schemas directly from Atlas without needing to install an additional tool.

Private vs. Public Visualizations​

Previously, you could only share a visualization of your schema to the Atlas Public Playground. While this is convenient for sharing your schema with others, it is not acceptable for many teams who maintain schemas that themselves are sensitive and cannot be shared publicly.

With this new release, you can easily publish your schema directly to your private workspace on Atlas Cloud. This means that only you and your team can access the visualization of your schema.

Visualizing your Ent Schema with Atlas​

To visualize your Ent schema with Atlas, first install its latest version:

curl -sSfL https://atlasgo.io/install.sh | sh

For other installation options, see the Atlas installation docs.

Next, run the following command to generate a visualization of your Ent schema:

atlas schema inspect \
  -u ent://ent/schema \
  --dev-url "sqlite://demo?mode=memory&_fk=1" \
  --visualize

Let's break this command down:

• atlas schema inspect - this command can be used to inspect schemas from a variety of sources and outputs them in various formats. In this case, we are using it to inspect an Ent schema.
• -u ent://ent/schema - this is the URL to the Ent schema we want to inspect. In this case, we are using the ent:// schema loader to point to a local Ent schema in the ./ent/schema directory.
• --dev-url "sqlite://demo?mode=memory&_fk=1" - Atlas relies on having an empty database called the Dev Database to normalize schemas and make various calculations. In this case, we are using an in memory SQLite database; but, if you are using a different driver, you can use docker://mysql/8/dev (for MySQL) or docker://postgres/15/?search_path=public (for PostgreSQL).

Once you run this command, you should see the following output:

Use the arrow keys to navigate: ↓ ↑ → ←
? Where would you like to share your schema visualization?:
  ▸ Publicly (gh.atlasgo.cloud)
    Your personal workspace (requires 'atlas login')

If you want to share your schema publicly, you can select the first option. If you want to share it privately, you can select the second option and then run atlas login to log in to your (free) Atlas account.

Wrapping up​

In this post, we showed how you can easily visualize your Ent schema with Atlas. We hope you find this feature useful and we look forward to hearing your feedback!

This tutorial is intended for Ent and Go novices who want to start by building a simple project: a very minimal content management system.

Over the last few years, Ent has become one of the fastest growing ORMs in Go:

Some of Ent's most cited features are:

• A type-safe Go API for working with your database. Forget about using interface{} or reflection to work with your database. Use pure Go that your editor understands and your compiler enforces.

• Model your data in graph semantics - Ent uses graph semantics to model your application's data. This makes it very easy to traverse complex datasets in a simple API.

  Let’s say we want to get all users that are in groups that are about dogs. Here are two ways to write something like this with Ent:

  // Start traversing from the topic.
  client.Topic.Query().
      Where(topic.Name("dogs")).
      QueryGroups().
      QueryUsers().
      All(ctx)
  
  // OR: Start traversing from the users and filter.
  client.User.Query().
      Where(      
          user.HasGroupsWith(  
              group.HasTopicsWith(  
                  topic.Name("dogs"),  
              ),  
          ),
      ).
      All(ctx)
  

• Automatically generate servers - whether you need GraphQL, gRPC or an OpenAPI compliant API layer, Ent can generate the necessary code you need to create a performant server on top of your database. Ent will generate both the third-party schemas (GraphQL types, Protobuf messages, etc.) and optimized code for the repetitive tasks for reading and writing from the database.
• Bundled with Atlas - Ent is built with a rich integration with Atlas, a robust schema management tool with many advanced capabilities. Atlas can automatically plan schema migrations for you as well as verify them in CI or deploy them to production for you. (Full disclosure: Ariel and I are the creators and maintainers)

Prerequisites​

• Install Go
• Install Docker

Step 1: Setting up the database schema​

You can find the code described in this step in this commit.

Let's start by initializing our project using go mod init:

go mod init github.com/rotemtam/ent-blog-example

Go confirms our new module was created:

go: creating new go.mod: module github.com/rotemtam/ent-blog-example

The first thing we will handle in our demo project will be to setup our database. We create our application data model using Ent. Let's fetch it using go get:

go get -u entgo.io/ent@master

Once installed, we can use the Ent CLI to initialize the models for the two types of entities we will be dealing with in this tutorial: the User and the Post.

go run -mod=mod entgo.io/ent/cmd/ent new User Post

Notice that a few files are created:

.
`-- ent
    |-- generate.go
    `-- schema
        |-- post.go
        `-- user.go

2 directories, 3 files

Ent created the basic structure for our project:

• generate.go - we will see in a bit how this file is used to invoke Ent's code-generation engine.
• The schema directory, with a bare ent.Schema for each of the entities we requested.

Let's continue by defining the schema for our entities. This is the schema definition for User:

// Fields of the User.  
func (User) Fields() []ent.Field {  
   return []ent.Field{  
      field.String("name"),  
      field.String("email").
            Unique(),  
      field.Time("created_at").
            Default(time.Now),  
   }  
}  
  
// Edges of the User.  
func (User) Edges() []ent.Edge {  
   return []ent.Edge{  
      edge.To("posts", Post.Type),  
   }  
}

Observe that we defined three fields, name, email and created_at (which takes the default value of time.Now()). Since we expect emails to be unique in our system we added that constraint on the email field. In addition, we defined an edge named posts to the Post type. Edges are used in Ent to define relationships between entities. When working with a relational database, edges are translated into foreign keys and association tables.

// Post holds the schema definition for the Post entity.
type Post struct {  
   ent.Schema  
}  
  
// Fields of the Post.  
func (Post) Fields() []ent.Field {  
   return []ent.Field{  
      field.String("title"),  
      field.Text("body"),  
      field.Time("created_at").
            Default(time.Now),  
   }  
}  
  
// Edges of the Post.
func (Post) Edges() []ent.Edge {  
   return []ent.Edge{  
      edge.From("author", User.Type).  
         Unique().  
         Ref("posts"),  
   }  
}

On the Post schema, we defined three fields as well: title, body and created_at. In addition, we defined an edge named author from Post to the User entity. We marked this edge as Unique because in our budding system, each post can have only one author. We used Ref to tell Ent that this edge's back reference is the posts edge on the User.

Ent's power stems from it's code-generation engine. When developing with Ent, whenever we make any change to our application schema, we must invoke Ent's code-gen engine to regenerate our database access code. This is what allows Ent to maintain a type-safe and efficient Go API for us.

Let's see this in action, run:

go generate ./...

Observe that a whole lot of new Go files were created for us:

.
`-- ent
    |-- client.go
    |-- context.go
    |-- ent.go
    |-- enttest
    |   `-- enttest.go
/// .. Truncated for brevity 
    |-- user_query.go
    `-- user_update.go

9 directories, 29 files

go run -mod=mod ariga.io/entviz ./ent/schema

Once we have our data model defined, let's create the database schema for it.

To install the latest release of Atlas, simply run one of the following commands in your terminal, or check out the Atlas website:

curl -sSf https://atlasgo.sh | sh

brew install ariga/tap/atlas

go install ariga.io/atlas/cmd/atlas@master

docker pull arigaio/atlas
docker run --rm arigaio/atlas --help

docker run --rm --net=host \
  -v $(pwd)/migrations:/migrations \
  arigaio/atlas migrate apply
  --url "mysql://root:pass@:3306/test"

With Atlas installed, we can create the initial migration script:

atlas migrate diff add_users_posts \
  --dir "file://ent/migrate/migrations" \
  --to "ent://ent/schema" \
  --dev-url "docker://mysql/8/ent"

Observe that two new files were created:

ent/migrate/migrations
|-- 20230226150934_add_users_posts.sql
`-- atlas.sum

The SQL file (the actual file name will vary on your machine depending on the timestamp in which you run atlas migrate diff) contains the SQL DDL statements required to set up the database schema on an empty MySQL database:

-- create "users" table  
CREATE TABLE `users` (`id` bigint NOT NULL AUTO_INCREMENT, `name` varchar(255) NOT NULL, `email` varchar(255) NOT NULL, `created_at` timestamp NOT NULL, PRIMARY KEY (`id`), UNIQUE INDEX `email` (`email`)) CHARSET utf8mb4 COLLATE utf8mb4_bin;  
-- create "posts" table  
CREATE TABLE `posts` (`id` bigint NOT NULL AUTO_INCREMENT, `title` varchar(255) NOT NULL, `body` longtext NOT NULL, `created_at` timestamp NOT NULL, `user_posts` bigint NULL, PRIMARY KEY (`id`), INDEX `posts_users_posts` (`user_posts`), CONSTRAINT `posts_users_posts` FOREIGN KEY (`user_posts`) REFERENCES `users` (`id`) ON UPDATE NO ACTION ON DELETE SET NULL) CHARSET utf8mb4 COLLATE utf8mb4_bin;

To setup our development environment, let's use Docker to run a local mysql container:

docker run --rm --name entdb -d -p 3306:3306 -e MYSQL_DATABASE=ent -e MYSQL_ROOT_PASSWORD=pass mysql:8

Finally, let's run the migration script on our local database:

atlas migrate apply --dir file://ent/migrate/migrations \
  --url mysql://root:pass@localhost:3306/ent

Atlas reports that it successfully created the tables:

Migrating to version 20230220115943 (1 migrations in total):

  -- migrating version 20230220115943
    -> CREATE TABLE `users` (`id` bigint NOT NULL AUTO_INCREMENT, `name` varchar(255) NOT NULL, `email` varchar(255) NOT NULL, `created_at` timestamp NOT NULL, PRIMARY KEY (`id`), UNIQUE INDEX `email` (`email`)) CHARSET utf8mb4 COLLATE utf8mb4_bin;
    -> CREATE TABLE `posts` (`id` bigint NOT NULL AUTO_INCREMENT, `title` varchar(255) NOT NULL, `body` longtext NOT NULL, `created_at` timestamp NOT NULL, `post_author` bigint NULL, PRIMARY KEY (`id`), INDEX `posts_users_author` (`post_author`), CONSTRAINT `posts_users_author` FOREIGN KEY (`post_author`) REFERENCES `users` (`id`) ON UPDATE NO ACTION ON DELETE SET NULL) CHARSET utf8mb4 COLLATE utf8mb4_bin;
  -- ok (55.972329ms)

  -------------------------
  -- 67.18167ms
  -- 1 migrations 
  -- 2 sql statements

Step 2: Seeding our database​

While we are developing our content management system, it would be sad to load a web page for our system and not see content for it. Let's start by seeding data into our database and learn some Ent concepts.

To access our local MySQL database, we need a driver for it, use go get to fetch it:

go get -u github.com/go-sql-driver/mysql

Create a file named main.go and add this basic seeding script.

package main

import (
    "context"
    "flag"
    "fmt"
    "log"

    "github.com/rotemtam/ent-blog-example/ent"

    _ "github.com/go-sql-driver/mysql"
    "github.com/rotemtam/ent-blog-example/ent/user"
)

func main() {
    // Read the connection string to the database from a CLI flag.
    var dsn string
    flag.StringVar(&dsn, "dsn", "", "database DSN")
    flag.Parse()

    // Instantiate the Ent client.
    client, err := ent.Open("mysql", dsn)
    if err != nil {
        log.Fatalf("failed connecting to mysql: %v", err)
    }
    defer client.Close()

    ctx := context.Background()
    // If we don't have any posts yet, seed the database.
    if !client.Post.Query().ExistX(ctx) {
        if err := seed(ctx, client); err != nil {
            log.Fatalf("failed seeding the database: %v", err)
        }
    }
    // ... Continue with server start.
}

func seed(ctx context.Context, client *ent.Client) error {
    // Check if the user "rotemtam" already exists.
    r, err := client.User.Query().
        Where(
            user.Name("rotemtam"),
        ).
        Only(ctx)
    switch {
    // If not, create the user.
    case ent.IsNotFound(err):
        r, err = client.User.Create().
            SetName("rotemtam").
            SetEmail("r@hello.world").
            Save(ctx)
        if err != nil {
            return fmt.Errorf("failed creating user: %v", err)
        }
    case err != nil:
        return fmt.Errorf("failed querying user: %v", err)
    }
    // Finally, create a "Hello, world" blogpost.
    return client.Post.Create().
        SetTitle("Hello, World!").
        SetBody("This is my first post").
        SetAuthor(r).
        Exec(ctx)
}

As you can see, this program first checks if any Post entity exists in the database, if it does not it invokes the seed function. This function uses Ent to retrieve the user named rotemtam from the database and in case it does not exist, tries to create it. Finally, the function creates a blog post with this user as its author.

Run it:

 go run main.go -dsn "root:pass@tcp(localhost:3306)/ent?parseTime=true"

Step 3: Creating the home page​

Let's now create the home page of the blog. This will consist of a few parts:

1. The view - this is a Go html/template that renders the actual HTML the user will see.
2. The server code - this contains the HTTP request handlers that our users' browsers will communicate with and will render our templates with data they retrieve from the database.
3. The router - registers different paths to handlers.
4. A unit test - to verify our server behaves correctly.

The view​

Go has an excellent templating engine that comes in two flavors: text/template for rendering general purpose text and html/template which had some extra security features to prevent code injection when working with HTML documents. Read more about it here .

Let's create our first template that will be used to display a list of blog posts. Create a new file named templates/list.tmpl:

<html>
<head>
    <title>My Blog</title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/css/bootstrap.min.css" rel="stylesheet"
          integrity="sha384-GLhlTQ8iRABdZLl6O3oVMWSktQOp6b7In1Zl3/Jr59b6EGGoI1aFkw7cmDA6j6gD" crossorigin="anonymous">

</head>
<body>
<div class="col-lg-8 mx-auto p-4 py-md-5">
    <header class="d-flex align-items-center pb-3 mb-5 border-bottom">
        <a href="/" class="d-flex align-items-center text-dark text-decoration-none">
            <span class="fs-4">Ent Blog Demo</span>
        </a>
    </header>

    <main>
        <div class="row g-5">
            <div class="col-md-12">
                {{- range . }}
                    <h2>{{ .Title }}</h2>
                    <p>
                        {{ .CreatedAt.Format "2006-01-02" }} by {{ .Edges.Author.Name }}
                    </p>
                    <p>
                        {{ .Body }}
                    </p>
                {{- end }}
            </div>

        </div>
    </main>
    <footer class="pt-5 my-5 text-muted border-top">
        <p>
            This is the Ent Blog Demo. It is a simple blog application built with Ent and Go. Get started:
        </p>
        <pre>go get entgo.io/ent</pre>
    </footer>
</div>

<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha1/dist/js/bootstrap.bundle.min.js"
        integrity="sha384-w76AqPfDkMBDXo30jS1Sgez6pr3x5MlQ1ZAGC+nuZB+EYdgRZgiwxhTBTkF7CXvN"
        crossorigin="anonymous"></script>
</body>
</html>

Here we are using a modified version of the Bootstrap Starter Template as the basis of our UI. Let's highlight the important parts. As you will see below, in our index handler, we will pass this template a slice of Post objects.

Inside the Go-template, whatever we pass to it as data is available as ".", this explains this line, where we use range to iterate over each post:

{{- range . }}

Next, we print the title, creation time and the author name, via the Author edge:

<h2>{{ .Title }}</h2>
<p>
    {{ .CreatedAt.Format "2006-01-02" }} by {{ .Edges.Author.Name }}
</p>

Finally, we print the post body and close the loop.

    <p>
        {{ .Body }}
    </p>
{{- end }}

After defining the template, we need to make it available to our program. We embed this template into our binary using the embed package (docs):

var (  
   //go:embed templates/*   
   resources embed.FS  
   tmpl      = template.Must(template.ParseFS(resources, "templates/*"))  
)

Server code​

We continue by defining a type named server and a constructor for it, newServer. This struct will have receiver methods for each HTTP handler we create and binds the Ent client we created at init to the server code.

type server struct {
    client *ent.Client
}

func newServer(client *ent.Client) *server {
    return &server{client: client}
}

Next, let's define the handler for our blog home page. This page should contain a list of all available blog posts:

// index serves the blog home page
func (s *server) index(w http.ResponseWriter, r *http.Request) {
    posts, err := s.client.Post.
        Query().
        WithAuthor().
        All(r.Context())
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    if err := tmpl.Execute(w, posts); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
    }
}

Let's zoom in on the Ent code here that is used to retrieve the posts from the database:

// s.client.Post contains methods for interacting with Post entities
s.client.Post.
    // Begin a query.
    Query().
    // Retrieve the entities using the `Author` edge. (a `User` instance)
    WithAuthor().
    // Run the query against the database using the request context.
    All(r.Context())

The router​

To manage the routes for our application, let's use go-chi, a popular routing library for Go.

go get -u github.com/go-chi/chi/v5

We define the newRouter function that sets up our router:

// newRouter creates a new router with the blog handlers mounted.
func newRouter(srv *server) chi.Router {
    r := chi.NewRouter()
    r.Use(middleware.Logger)
    r.Use(middleware.Recoverer)
    r.Get("/", srv.index)
    return r
}

In this function, we first instantiate a new chi.Router, then register two middlewares:

• middleware.Logger is a basic access logger that prints out some information on every request our server handles.
• middleware.Recoverer recovers from when our handlers panic, preventing a case where our entire server will crash because of an application error.

Finally, we register the index function of the server struct to handle GET requests to the / path of our server.

A unit test​

Before wiring everything together, let's write a simple unit test to check that our code works as expected.

To simplify our tests we will install the SQLite driver for Go which allows us to use an in-memory database:

go get -u github.com/mattn/go-sqlite3

Next, we install testify, a utility library that is commonly used for writing assertions in tests.

go get github.com/stretchr/testify 

With these dependencies installed, create a new file named main_test.go:

package main

import (
    "context"
    "io"
    "net/http"
    "net/http/httptest"
    "testing"

    _ "github.com/mattn/go-sqlite3"
    "github.com/rotemtam/ent-blog-example/ent/enttest"
    "github.com/stretchr/testify/require"
)

func TestIndex(t *testing.T) {
    // Initialize an Ent client that uses an in memory SQLite db.
    client := enttest.Open(t, "sqlite3", "file:ent?mode=memory&cache=shared&_fk=1")
    defer client.Close()
    
    // seed the database with our "Hello, world" post and user.
    err := seed(context.Background(), client)
    require.NoError(t, err)

    // Initialize a server and router.
    srv := newServer(client)
    r := newRouter(srv)

    // Create a test server using the `httptest` package.
    ts := httptest.NewServer(r)
    defer ts.Close()

    // Make a GET request to the server root path.
    resp, err := ts.Client().Get(ts.URL)

    // Assert we get a 200 OK status code.
    require.NoError(t, err)
    require.Equal(t, http.StatusOK, resp.StatusCode)

    // Read the response body and assert it contains "Hello, world!"
    body, err := io.ReadAll(resp.Body)
    require.NoError(t, err)
    require.Contains(t, string(body), "Hello, World!")
}

Run the test to verify our server works correctly:

go test ./...

Observe our test passes:

ok      github.com/rotemtam/ent-blog-example    0.719s
?       github.com/rotemtam/ent-blog-example/ent        [no test files]
?       github.com/rotemtam/ent-blog-example/ent/enttest        [no test files]
?       github.com/rotemtam/ent-blog-example/ent/hook   [no test files]
?       github.com/rotemtam/ent-blog-example/ent/migrate        [no test files]
?       github.com/rotemtam/ent-blog-example/ent/post   [no test files]
?       github.com/rotemtam/ent-blog-example/ent/predicate      [no test files]
?       github.com/rotemtam/ent-blog-example/ent/runtime        [no test files]
?       github.com/rotemtam/ent-blog-example/ent/schema [no test files]
?       github.com/rotemtam/ent-blog-example/ent/user   [no test files]

Putting everything together​

Finally, let's update our main function to put everything together:

func main() {  
   // Read the connection string to the database from a CLI flag.  
   var dsn string  
   flag.StringVar(&dsn, "dsn", "", "database DSN")  
   flag.Parse()  
  
   // Instantiate the Ent client.  
   client, err := ent.Open("mysql", dsn)  
   if err != nil {  
      log.Fatalf("failed connecting to mysql: %v", err)  
   }  
   defer client.Close()  
  
   ctx := context.Background()  
   // If we don't have any posts yet, seed the database.  
   if !client.Post.Query().ExistX(ctx) {  
      if err := seed(ctx, client); err != nil {  
         log.Fatalf("failed seeding the database: %v", err)  
      }  
   }  
   srv := newServer(client)  
   r := newRouter(srv)  
   log.Fatal(http.ListenAndServe(":8080", r))  
}

We can now run our application and stand amazed at our achievement: a working blog front page!

 go run main.go -dsn "root:pass@tcp(localhost:3306)/test?parseTime=true"

Step 4: Adding content​

No content management system would be complete without the ability, well, to manage content. Let's demonstrate how we can add support for publishing new posts on our blog.

Let's start by creating the backend handler:

// add creates a new blog post.
func (s *server) add(w http.ResponseWriter, r *http.Request) {
    author, err := s.client.User.Query().Only(r.Context())
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    if err := s.client.Post.Create().
        SetTitle(r.FormValue("title")).
        SetBody(r.FormValue("body")).
        SetAuthor(author).
        Exec(r.Context()); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
    }
    http.Redirect(w, r, "/", http.StatusFound)
}

As you can see, the handler currently loads the only user from the users table (since we have yet to create a user management system or login capabilities). Only will fail unless exactly one result is retrieved from the database.

Next, our handler creates a new post, by setting the title and body fields to values retrieved from r.FormValue. This is where Go stores all of the form input passed to an HTTP request.

After creating the handler, we should wire it to our router:

// newRouter creates a new router with the blog handlers mounted.
func newRouter(srv *server) chi.Router {
    r := chi.NewRouter()
    r.Use(middleware.Logger)
    r.Use(middleware.Recoverer)
    r.Get("/", srv.index)
    r.Post("/add", srv.add)
    return r
}

Next, we can add an HTML <form> component that will be used by our user to write their content:

<div class="col-md-12">
    <hr/>
    <h2>Create a new post</h2>
    <form action="/add" method="post">
        <div class="mb-3">
            <label for="title" class="form-label">Title</label>
            <input name="title" type="text" class="form-control" id="title" placeholder="Once upon a time..">
        </div>
        <div class="mb-3">
            <label for="body" class="form-label">Body</label>
            <textarea name="body" class="form-control" id="body" rows="8"></textarea>
        </div>
        <div class="mb-3">
            <button type="submit" class="btn btn-primary mb-3">Post</button>
        </div>
    </form>
</div>