Parsing user-agent strings and go:embed

2024-12-13 798 words 4 minutes

Contents

I’m currently in the process of writing a back-end for a small web application that I plan to deploy in my local environment. The application isn’t really relevant. The important bit is a new trick I learned while working on it.

Within the back-end, at some point, I’m parsing an user-agent string. These strings are standardised and should follow the format described by HTTP Semantics.

For convenience, I’ve chosen to go with uap-go to parse these strings and here lies the catch.

This library relies on a file regexes.yaml containing regular expressions to match user agent strings. The file is provided in uap-core repository. So, in order to use the library, I need the regex file as well. Since I’m planning on dockerising the back-end anyway, this shouldn’t be that much of a problem right?

Such approach, is a bit of a compromise - source code would become part of the deployment. I’d have to put uap-core or regexes.yaml itself into my image and that’s not really what I want.

go:embed to the rescue

go:embed is a perfect solution for this kind of problems. Let’s start with a simple example. I’m gonna create a small repo with a user-agent parser and add uap-go to it:

1
2


go mod init gitlab.com/twdev_projects/goembed
go get github.com/ua-parser/uap-go/uaparser

Additionally, I’m gonna add uap-core as a submodule to have the regexes.yaml file.

1
2


mkdir 3rd
git submodule add https://github.com/ua-parser/uap-core 3rd/uap-core

Finally, my test application’s code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


package main

import (
        "flag"
        "fmt"
        "os"

        "github.com/ua-parser/uap-go/uaparser"
)

func main() {
        userAgentFlag := flag.String("useragent", "", "user agent string to parse")

        flag.Parse()

        userAgent := *userAgentFlag

        if len(userAgent) == 0 {
                fmt.Println("User agent string can't be empty")
                os.Exit(1)
        }

        parser, err := uaparser.New("3rd/uap-core/regexes.yaml")
        if err != nil {
                fmt.Println("Unable to instantiate user-agent parser:", err)
                os.Exit(1)
        }

        client := parser.Parse(userAgent)

        fmt.Printf("UserAgent's Browser Family: %s, Major Version: %s\n",
                client.UserAgent.Family,
                client.UserAgent.Major)

        fmt.Printf("UserAgent's Browser Os: %s\n", client.Os.Family)
}

The path to regexes.yaml is hard-coded. This poses a problem. The path has to be either configurable, the regexes.yaml has to be installed in a fixed, well known location or it has to be baked into docker image, in case the application is meant to be dockerised.

But, with go:embed, this can be solved quite elegantly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


diff --git a/uapparser.go b/uapparser.go
index 8ba4029..2c67070 100644
--- a/uapparser.go
+++ b/uapparser.go
@@ -1,6 +1,7 @@
 package main

 import (
+       _ "embed"
        "flag"
        "fmt"
        "os"
@@ -8,6 +9,9 @@ import (
        "github.com/ua-parser/uap-go/uaparser"
 )

+//go:embed 3rd/uap-core/regexes.yaml
+var regexes []byte
+
 func main() {
        userAgentFlag := flag.String("useragent", "", "user agent string to parse")

@@ -20,7 +24,7 @@ func main() {
                os.Exit(1)
        }

-       parser, err := uaparser.New("3rd/uap-core/regexes.yaml")
+       parser, err := uaparser.NewFromBytes(regexes)
        if err != nil {
                fmt.Println("Unable to instantiate user-agent parser:", err)
                os.Exit(1)

The contents of the file become embedded in the application itself. The path is known and fixed at built time.

Embedding directories

In my case, embedding a single file was enough to solve the problem but it’s possible to embed entire directories equally easy. Let’s make a small change to the example program to show that:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


-//go:embed 3rd/uap-core/regexes.yaml
-var regexes []byte
+//go:embed 3rd/uap-core/*.yaml
+//go:embed 3rd/uap-core/*.md
+var uapCore embed.FS

 func main() {
        userAgentFlag := flag.String("useragent", "", "user agent string to parse")
@@ -24,6 +27,12 @@ func main() {
                os.Exit(1)
        }

+       regexes, err := uapCore.ReadFile("3rd/uap-core/regexes.yaml")
+       if err != nil {
+               fmt.Println("Couldn't open regexes.yaml:", err)
+               os.Exit(1)
+       }
+
        parser, err := uaparser.NewFromBytes(regexes)
        if err != nil {
                fmt.Println("Unable to instantiate user-agent parser:", err)
@@ -37,4 +46,14 @@ func main() {
                client.UserAgent.Major)

        fmt.Printf("UserAgent's Browser Os: %s\n", client.Os.Family)
+
+       // reading other files
+       readmeFile, err := uapCore.Open("3rd/uap-core/README.md")
+       if err != nil {
+               fmt.Println("Couldn't open README.md:", err)
+               os.Exit(1)
+       }
+       defer readmeFile.Close()
+       io.Copy(os.Stdout, readmeFile)
+
 }

This is perfect for embedding HTML templates or database migration files.

Conclusion

go:embed is a perfect solution for including application assets that are difficult to handle outside of docker giving a fixed, deterministic access and a guarantee that all needed files are part of the deployment, regardless of the environment.

It’s perfect for embedding source code dependencies which really shouldn’t be a part of docker images.

Example code is available here.