clamav-api-go

command module
v0.2.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 2, 2024 License: GPL-3.0 Imports: 15 Imported by: 0

README

clamav-api-go

build and test Kubernetes Run e2e tests Release

go skaffold

Simple REST API wrapper for ClamAV written in Go.

Description

This is a REST API wrapper server with support for basic INSTREAM scanning, VERSIONCOMMANDS, STATS, SHUTDOWN, VERSION, RELOAD and PING command.

The Clamd tcp protocol is explained here: http://linux.die.net/man/8/clamd

Requirements

  • Golang 1.19 or higher

Getting started

Prebuilt binaries can be downloaded from the GitHub Releases section, or using a Docker image from the Github Container Registry. See here

Building clamav-api-go 💿
From source with Go

You need a working go toolchain (It has been developped and tested with go 1.20 and should work with go >= 1.20). Refer to the official documentation for more information (or from your Linux/Mac/Windows distribution documentation to install it from your favorite package manager).

# Clone this repository
git clone https://github.com/lescactus/clamav-api-go.git && cd clamav-api-go/

# Build from sources. Use the '-o' flag to change the compiled binary name
go build

# Default compiled binary is clamav-api-go
# You can optionnaly move it somewhere in your $PATH to access it shell wide
./clamav-api-go
From source with docker

If you don't have go installed but have docker, run the following command to build inside a docker container:

# Build from sources inside a docker container. Use the '-o' flag to change the compiled binary name
# Warning: the compiled binary belongs to root:root
docker run --rm -it -v "$PWD":/app -w /app golang:1.20 go build -buildvcs=false

# Default compiled binary is clamav-api-go
# You can optionnaly move it somewhere in your $PATH to access it shell wide
./clamav-api-go

The server is accessible at http://127.0.0.1:8080

With Docker

clamav-api-go comes with a Dockerfile. To build the image:

docker build -t clamav-api .

docker run -d -p 8080:8080 --restart="always" --name clamav-api-go clamav-api-go

The server is accessible at http://127.0.0.1:8080

Running with Docker 🐓
docker run -d -p 8080:8080 --restart="always" --name clamav-api-go ghcr.io/lescactus/clamav-api-go

The server is accessible at http://127.0.0.1:8080

Running with Docker Compose 🌵
docker compose up

The server is accessible at http://127.0.0.1:8080

Running in Kubernetes 🎯
With skaffold

Ensure you have a properly working and accessible Kubernetes cluster with a valid ~/.kube/config. This project is using Skaffold v2.2.0 to deploy to a local Kubernetes cluster, such as Minikube or KinD. You can dowload skaffold here. It is assumed that skaffold is installed.

To deploy to a local Kubernetes cluster, simply run skaffold run:

$ skaffold run
Generating tags...
 - clamav-api -> clamav-api:2023-07-09_01-24-29.9_CEST
Checking cache...
 - clamav-api: Not found. Building
Starting build...
Found [k3d-k3s-default] context, using local docker daemon.
Building [clamav-api]...
Target platforms: [linux/amd64]
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 256B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 122B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/golang:1.20
#3 DONE 0.0s

#4 [builder 1/6] FROM docker.io/library/golang:1.20
#4 DONE 0.0s

#5 [internal] load build context
#5 transferring context: 10.14kB done
#5 DONE 0.0s

#6 [builder 2/6] WORKDIR /app
#6 CACHED

#7 [builder 3/6] COPY go.* ./
#7 CACHED

#8 [builder 4/6] RUN go mod download
#8 CACHED

#9 [builder 5/6] COPY . .
#9 DONE 0.1s

#10 [builder 6/6] RUN CGO_ENABLED=0 go build -ldflags '-d -w -s' -o main
#10 DONE 6.5s

#11 [stage-1 1/1] COPY --from=builder /app/main /
#11 CACHED

#12 exporting to image
#12 exporting layers done
#12 writing image sha256:c505b5f5ed48c79ec01a85bfb8827e036f211d17296e7b34aa94f9bb5e16a83d done
#12 naming to docker.io/library/clamav-api:2023-07-09_01-24-29.9_CEST done
#12 DONE 0.0s
Build [clamav-api] succeeded
Starting test...
Testing images...
Running custom test command: "go test ./..."
?   	github.com/lescactus/clamav-api-go	[no test files]
ok  	github.com/lescactus/clamav-api-go/internal/clamav	(cached)
?   	github.com/lescactus/clamav-api-go/internal/config	[no test files]
ok  	github.com/lescactus/clamav-api-go/internal/controllers	(cached)
ok  	github.com/lescactus/clamav-api-go/internal/logger	(cached)
Command finished successfully.
Tags used in deployment:
 - clamav-api -> clamav-api:c505b5f5ed48c79ec01a85bfb8827e036f211d17296e7b34aa94f9bb5e16a83d
Starting deploy...
Loading images into k3d cluster nodes...
 - clamav-api:c505b5f5ed48c79ec01a85bfb8827e036f211d17296e7b34aa94f9bb5e16a83d -> Found
Images loaded in 68.063635ms
 - configmap/clamav created
 - deployment.apps/clamav-api created
 - service/clamav-api created
 - serviceaccount/clamav-api created
Waiting for deployments to stabilize...
 - deployment/clamav-api: waiting for rollout to finish: 0 of 1 updated replicas are available...
 - deployment/clamav-api is ready.
Deployments stabilized in 31.094 seconds
You can also run [skaffold run --tail] to get the logs

The following happened:

  • Skaffold will generate a docker tag based on the current timestamp.

  • If the image doesn't exist locally, Skaffold will build it.

  • Once the docker image is built, Skaffold will substitute the raw image defined in deploy/k8s/deployment.yaml with the image just built.

  • Skaffold will apply the manifests in deploy/k8s/.

  • Skaffold will wait for the clamav-api deployment to be ready.

For more informations about Skaffold and what it can do, visit the project [documentation](Without Skaffold).

Without Skaffold

To deploy to a Kubernetes cluster without Skaffold, simply build & push the docker image to an external registry. Then change the docker image name to include the registry in the deploy/k8s/deployment.yaml manifest.

Note: You can change and customize the clamd.conf in deploy/k8s/configmap.yaml.

Specifications 🌊

GET /rest/v1/ping will send the PING command to Clamd

GET /rest/v1/version will send the VERSION command to Clamd

GET /rest/v1/stats will send the STATS command to Clamd

GET /rest/v1/versioncommands will send the VERSIONCOMMANDs command to Clamd

POST /rest/v1/reload will send the RELOAD command to Clamd

POST /rest/v1/shutdown will send the SHUTDOWN command to Clamd

POST /rest/v1/scan (with a form in the request body) will send the INSTREAM command to Clamd and stream the form for Clamd to scan. Note: this endpoint expects a multipart/form-data. See Examples below.

Configuration 🌳

clamav-api-go is a 12-factor compliant app using Viper as a configuration manager. It can read configuration from either config files or environment variables. Available configuration files are:

  • config.json
  • config.yaml
  • config.env
config.json

Example

{
    "server_addr": ":8080",
    "server_read_timeout": "30s",
    "server_read_header_timeout": "10s",
    "server_write_timeout": "30s",
    "server_max_request_size": 10485760,
    "logger_log_level": "debug",
    "logger_duration_field_unit": "ms",
    "logger_format": "console",
    "clamav_addr": "127.0.0.1:3310",
    "clamav_network": "tcp",
    "clamav_timeout": "30s",
    "clamav_keepalive": "30s"
}
config.yaml

Example

server_addr: :8080
server_read_timeout: 30s
server_read_header_timeout: 10s
server_write_timeout: 30s
server_max_request_size: 10485760
logger_log_level: debug
logger_duration_field_unit: ms
logger_format: console
clamav_addr: 127.0.0.1:3310
clamav_network: tcp
clamav_timeout: 30s
clamav_keepalive: 30s
config.env

Example

SERVER_ADDR=:8080
SERVER_READ_TIMEOUT=30s
SERVER_READ_HEADER_TIMEOUT=10s
SERVER_WRITE_TIMEOUT=30s
SERVER_MAX_REQUEST_SIZE=10485760
LOGGER_LOG_LEVEL=debug
LOGGER_DURATION_FIELD_UNIT=s
LOGGER_FORMAT=console
CLAMAV_ADDR=127.0.0.1:3310
CLAMAV_NETWORK=tcp
CLAMAV_TIMEOUT=30s
CLAMAV_KEEPALIVE=300s
From environment variables

It is the same variables as in the config.env

Variable Default value Description
SERVER_ADDR :8080 Define the TCP address for the server to listen on, in the form "host:port"
SERVER_READ_TIMEOUT 30s Maximum duration for the http server to read the entire request, including the body. A zero or negative value means there will be no timeout.
SERVER_READ_HEADER_TIMEOUT 10s Amount of time the http server allow to read request headers. If the value is zero, the value of SERVER_READ_TIMEOUT is used. If both are zero, there is no timeout
SERVER_WRITE_TIMEOUT 30s Maximum duration before the http server times out writes of the response. A zero or negative value means there will be no timeout
SERVER_MAX_REQUEST_SIZE 10485760 (10MiB) Maximum size of a client request, including headers and body
LOGGER_LOG_LEVEL info Log level. Available: trace, debug, info, warn, error, fatal and panic. Ref
LOGGER_DURATION_FIELD_UNIT ms Defines the unit for time.Duration type fields in the logger. Available: ms, millisecond, s, second
LOGGER_FORMAT json Format of the logs. Can be either json or console
CLAMAV_ADDR 127.0.0.0:3310 Network address of the Clamav server
CLAMAV_NETWORK tcp Define the named network of the Clamav server. Example: tcp, tcp4, tcp6, unix, etc ... See the Dial() documentation for more details
CLAMAV_TIMEOUT 30s Maximum amount of time a dial to the Clamav server will wait for a connect to complete
CLAMAV_KEEPALIVE 30s Specifies the interval between keep-alive probes for an active connection to the Clamav server. If negative, keep-alive probes are disabled

Examples 📻

$ curl 127.0.0.1:8080/rest/v1/ping
{"ping":"PONG"}

$ curl 127.0.0.1:8080/rest/v1/version
{"clamav_version":"ClamAV 1.0.0/26734/Mon Nov 28 08:17:05 2022"}

$ curl 127.0.0.1:8080/rest/v1/stats
{"pools":1,"state":"VALID PRIMARY","threads":"live 1  idle 0 max 10 idle-timeout 30","queue":"0 items\n\tSTATS 0.000179 ","memstats":"heap N/A mmap N/A used N/A free N/A releasable N/A pools 1 pools_used 1260.177M pools_total 1260.222M"} 

$ curl 127.0.0.1:8080/rest/v1/versioncommands
{"clamav_version":"ClamAV 1.0.0/26734/Mon Nov 28 08:17:05 2022","commands":["SCAN","QUIT","RELOAD","PING","CONTSCAN","VERSIONCOMMANDS","VERSION","END","SHUTDOWN","MULTISCAN","FILDES","STATS","IDSESSION","INSTREAM","DETSTATSCLEAR","DETSTATS","ALLMATCHSCAN"]}

$ curl 127.0.0.1:8080/rest/v1/reload -XPOST
{"status":"RELOADING"}

$ curl 127.0.0.1:8080/rest/v1/shutdown -XPOST
{"status":"Shutting down"}

# Download the EICAR anti malware test file in /tmp/eicar.txt
$ wget https://secure.eicar.org/eicar.com.txt -O /tmp/eicar.txt

# Generate a 1M file with random content
$ dd if=/dev/urandom of=/tmp/test.txt bs=1M count=1

$ curl 127.0.0.1:8080/rest/v1/scan -F "file=@/tmp/test.txt" -v | jq ''
*   Trying 127.0.0.1:8080...
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> POST /rest/v1/scan HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.81.0
> Accept: */*
> Content-Length: 1048762
> Content-Type: multipart/form-data; boundary=------------------------b6b6ef4a0ac767d7
> Expect: 100-continue
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/json
< X-Request-Id: cikv9kqrnmmc73e13940
< Date: Sat, 08 Jul 2023 23:44:19 GMT
< Content-Length: 74
< 
* Connection #0 to host 127.0.0.1 left intact
{
  "status": "noerror",
  "msg": "stream: OK",
  "signature": "",
  "virus_found": false
}

$ curl 127.0.0.1:8080/rest/v1/scan -F "file=@/tmp/eicar.txt" -v | jq ''
*   Trying 127.0.0.1:8080...
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> POST /rest/v1/scan HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.81.0
> Accept: */*
> Content-Length: 255
> Content-Type: multipart/form-data; boundary=------------------------2c5ea3b07f1f700d
> 
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/json
< X-Request-Id: cikv9oirnmmc73e1394g
< Date: Sat, 08 Jul 2023 23:44:34 GMT
< Content-Length: 110
< 
* Connection #0 to host 127.0.0.1 left intact
{
  "status": "error",
  "msg": "file contains potential virus",
  "signature": "Win.Test.EICAR_HDB-1",
  "virus_found": true
}

Development

Live reloading with air

Use air for live code reloading:

# Install air
$ go install github.com/cosmtrek/air@latest

$ air
  __    _   ___  
 / /\  | | | |_) 
/_/--\ |_| |_| \_ , built with Go 

watching .
watching deploy
watching deploy/k8s
watching dist
watching dist/clamav-api-go_darwin_amd64_v1
watching dist/clamav-api-go_darwin_arm64
watching dist/clamav-api-go_linux_386
watching dist/clamav-api-go_linux_amd64_v1
watching dist/clamav-api-go_linux_arm64
watching dist/clamav-api-go_windows_386
watching dist/clamav-api-go_windows_amd64_v1
watching dist/clamav-api-go_windows_arm64
watching internal
watching internal/clamav
watching internal/config
watching internal/controllers
watching internal/logger
!exclude tmp
building...
running...
main.go has changed
building...
main.go has changed
running...
Use Skaffold with Kubernetes

Use skaffold dev with a local k8s cluster, such as minikube, k3d or kind:

# Example with k3d
# Create a local k8s cluster
$ k3d cluster create

# Run skaffold in dev mode, with live reloading, log tailing and port-forwarding
$ skaffold dev --tail --port-forward
Generating tags...
 - clamav-api -> clamav-api:2023-07-09_12-17-11.852_CEST
Checking cache...
 - clamav-api: Not found. Building
Starting build...
Found [k3d-k3s-default] context, using local docker daemon.
Building [clamav-api]...
Target platforms: [linux/amd64]
...
Build [clamav-api] succeeded
Starting test...
Testing images...
Running custom test command: "go test ./..."
?   	github.com/lescactus/clamav-api-go	[no test files]
ok  	github.com/lescactus/clamav-api-go/internal/clamav	(cached)
?   	github.com/lescactus/clamav-api-go/internal/config	[no test files]
ok  	github.com/lescactus/clamav-api-go/internal/controllers	(cached)
ok  	github.com/lescactus/clamav-api-go/internal/logger	(cached)
Command finished successfully.
Tags used in deployment:
 - clamav-api -> clamav-api:c505b5f5ed48c79ec01a85bfb8827e036f211d17296e7b34aa94f9bb5e16a83d
Starting deploy...
Loading images into k3d cluster nodes...
 - clamav-api:c505b5f5ed48c79ec01a85bfb8827e036f211d17296e7b34aa94f9bb5e16a83d -> Found
Images loaded in 69.747973ms
 - configmap/clamav created
 - deployment.apps/clamav-api created
 - service/clamav-api created
 - serviceaccount/clamav-api created
Waiting for deployments to stabilize...
 - deployment/clamav-api: waiting for rollout to finish: 0 of 1 updated replicas are available...
 - deployment/clamav-api is ready.
Deployments stabilized in 31.091 seconds
Port forwarding service/clamav-api in namespace default, remote port 80 -> http://127.0.0.1:8080
Listing files to watch...
 - clamav-api
Press Ctrl+C to exit
Watching for changes...
[clamav-api] {"level":"error","svc":"clamav-api-go","req_id":"cil8ifirnmmc73a3q740","time":"2023-07-09T10:17:34Z","message":"error while sending ping command: dial tcp 127.0.0.0:3310: connect: connection refused"}
[clamav-api] {"level":"info","svc":"clamav-api-go","remote_client":"10.42.0.1:45986","user_agent":"kube-probe/1.26","req_id":"cil8ifirnmmc73a3q740","method":"GET","url":"/rest/v1/ping","status":502,"size":83,"duration":0.204641,"time":"2023-07-09T10:17:34Z"}
[clamav-api] {"level":"debug","svc":"clamav-api-go","req_id":"cil8ijarnmmc73a3q74g","time":"2023-07-09T10:17:49Z","message":"ping command sent successfully"}
[clamav-api] {"level":"info","svc":"clamav-api-go","remote_client":"10.42.0.1:42870","user_agent":"kube-probe/1.26","req_id":"cil8ijarnmmc73a3q74g","method":"GET","url":"/rest/v1/ping","status":200,"size":15,"duration":0.511549,"time":"2023-07-09T10:17:49Z"}
[clamav-api] {"level":"debug","svc":"clamav-api-go","req_id":"cil8ijarnmmc73a3q750","time":"2023-07-09T10:17:49Z","message":"ping command sent successfully"}
[clamav-api] {"level":"info","svc":"clamav-api-go","remote_client":"10.42.0.1:42876","user_agent":"kube-probe/1.26","req_id":"cil8ijarnmmc73a3q750","method":"GET","url":"/rest/v1/ping","status":200,"size":15,"duration":0.375841,"time":"2023-07-09T10:17:49Z"}
[clamav] Starting Freshclamd
[clamav] Starting ClamAV
Socket for clamd not found yet, retrying (0/1800) ...ClamAV update process started at Sun Jul  9 10:17:19 2023
...
[clamav] Sun Jul  9 10:17:36 2023 -> Set stacksize to 1048576
[clamav] socket found, clamd started.
[clamav-api] {"level":"debug","svc":"clamav-api-go","req_id":"cil8ilirnmmc73a3q75g","file_name":"test.txt","file_size":1048576,"time":"2023-07-09T10:17:58Z","message":"multipart file read successfully"}
[clamav-api] {"level":"debug","svc":"clamav-api-go","req_id":"cil8ilirnmmc73a3q75g","time":"2023-07-09T10:17:58Z","message":"file scanned successfully"}
[clamav-api] {"level":"info","svc":"clamav-api-go","remote_client":"127.0.0.1:44044","user_agent":"curl/7.81.0","req_id":"cil8ilirnmmc73a3q75g","method":"POST","url":"/rest/v1/scan","status":200,"size":74,"duration":61.101685,"time":"2023-07-09T10:17:58Z"}
[clamav-api] {"level":"debug","svc":"clamav-api-go","req_id":"cil8ilqrnmmc73a3q760","time":"2023-07-09T10:17:59Z","message":"ping command sent successfully"}
[clamav-api] {"level":"info","svc":"clamav-api-go","remote_client":"10.42.0.1:44084","user_agent":"kube-probe/1.26","req_id":"cil8ilqrnmmc73a3q760","method":"GET","url":"/rest/v1/ping","status":200,"size":15,"duration":0.522211,"time":"2023-07-09T10:17:59Z"}
[clamav-api] {"level":"debug","svc":"clamav-api-go","req_id":"cil8ioarnmmc73a3q76g","time":"2023-07-09T10:18:09Z","message":"ping command sent successfully"}
[clamav-api] {"level":"info","svc":"clamav-api-go","remote_client":"10.42.0.1:56626","user_agent":"kube-probe/1.26","req_id":"cil8ioarnmmc73a3q76g","method":"GET","url":"/rest/v1/ping","status":200,"size":15,"duration":0.373957,"time":"2023-07-09T10:18:09Z"}
Unit tests

To run the test suite, run the following commands:

# Run the unit tests. Remove the '-v' flag to reduce verbosity
go test -v ./... 

# Get coverage to html format
go test -coverprofile /tmp/cover.out ./... -v
go tool cover -html=/tmp/cover.out
End to End tests

This project uses ovh/venom to run a e2e test suite against all the endpoints. Refer to the documentation to install the cli.

# Optionnaly, start docker compose
docker compose up -d --wait

# Execute the test suite against http://127.0.0.1:8080
venom run -vv e2e/venom.e2e.yaml

# To override the default "http://127.0.0.1:8080" protocol://host:port, pass the "--var" flag
# to the venom cli:
venom run -vv --var=baseuri=https://clamav.api.com:443  e2e/venom.e2e.yaml

Documentation

The Go Gopher

There is no documentation for this package.

Source Files

View all Source files

Directories

Path Synopsis
internal
clamav
config
controllers
logger

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.