ulist

package module
v0.14.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 23, 2024 License: GPL-3.0 Imports: 29 Imported by: 0

README

ulist

A mailing list service that keeps it simple. An alternative to mailman in some use cases.

Build

go build ./cmd/...

Arch Linux users can install ulist from the AUR.

Integration

  • Email submission: ulist listens to an LMTP socket
  • Email delivery to system's MTA: ulist executes /usr/sbin/sendmail
  • Web UI: ulist listens to a port or a unix socket
  • Web UI authentication: against a local SQLite database or an SMTP server

See docs/integration for examples.

Features

  • single binary
  • nice web interface
  • works with SPF, DKIM etc. out of the box
  • SMTP authentication
  • probably GDPR compliant
  • appends a footer with an unsubscribe link
  • socketmap server for postfix

Design Choices

  • Email delivery via the sendmail interface
    • no recipient limit
    • when running in a jail, you need access to /etc/postfix/main.cf and /var/spool/postfix/maildrop
    • easier than SMTP delivery (localhost:25 usually accepts mail for localhost only and might drop emails for other recipients, localhost:587 usually requires authentication and SSL/TLS)
  • From-Munging
    • If a forwarded email is not modified, DKIM will pass but SPF checks might fail. We could predict the consequences by checking the sender's DMARC policy. But for the sake of consistence, let's rewrite all From headers to the mailing list address and remove existing DKIM signatures.
  • Modifying emails
    • As original DKIM signatures are removed, we can modify parts of the email. We can prepend the list name to the subject header and add an unsubscribe footer to the message body.
  • Subscribe and unsubscribe
    • Issue: emails (like "subscribe" or "unsubscribe" instructions) can be spoofed
    • Issue: opt-in backscatter spam is an issue rather with web forms (trade a http request for an email) than email (trade an email for an email)
    • Issue: individual unsubscribe links in the footer will fall into others hands, as people will forward or full-quote emails
    • Decision: signup web form must be protected against spam bots and use rate limiting
    • Decision: subscribe/unsubscribe requesters get an email with a confirmation link
    • Terms
      1. user: ask (via web or email with special subject)
      2. server: checkback (send email with link)
      3. user: confirm (click link)
      4. server: sign off (send welcome or goodbye email)
  • Memory consumption
    • Issue: some people use email aliases and don't remember which address they subscribed
    • Issue: individual list emails consume much memory, e.g. 1000 recipients × 10 MB message = 10 GB
    • Decision: notification emails (checkback, sign-off, moderation) are individual
    • Decision: list emails are not individual, MTA gets one email with many recipients (envelope-to)
    • List receivers must maintain an overview over their email aliases or check the Delivered-To header line.

Security Considerations

  • We can't hide the existence of a list. Maybe in the web interface, but not via SMTP.

TODO

  • fail2ban pattern
  • LDAP authenticator
  • more unit tests
  • GDPR: require opt-in after n days or member won't get mails any more
  • more sophisticated bounce processing
  • web UI: list creation permissions per domain
  • remove IP address of sender (or check that removal works)
  • ensure that the sender is not leaked if HideFrom is true, e.g. by removing Delivered-To headers?
  • ability to block people (maybe keep membership and set optInExpiry timestamp or so to -1)
  • maybe issue with Apple Mail: two line breaks after header

Omitted features

  • Archive

Known issues

  • Email addresses like alice@example.com <alice@example.com> are not RFC 5322 compliant, use alice <alice@example.com> or "alice@example.com" <alice@example.com>

Documentation

Index

Constants

View Source 
const BounceAddressSuffix = "+bounces"
View Source 
const WebBatchLimit = 1000

Variables

View Source 
var ErrLink = errors.New("link is invalid or expired") // HMAC related, don't leak the reason
View Source 
var ErrUnknownActionString = errors.New("unknown action string")
View Source 
var SMTPErrUserNotExist = SMTPErrorf(550, "user not found")

Functions

func NewLMTPServer added in v0.13.0 

func NewLMTPServer(ul *Ulist) *smtp.Server

func SMTPErrorf 

func SMTPErrorf(code int, format string, a ...interface{}) *smtp.SMTPError

Types

type Action added in v0.13.0 

type Action int
const (
	// ordered, order is required for list.GetAction
	Reject Action = iota
	Mod
	Pass
)

func ParseAction added in v0.13.0 

func ParseAction(s string) (Action, error)

func (Action) EqualsMod added in v0.13.0 

func (a Action) EqualsMod() bool

func (Action) EqualsPass added in v0.13.0 

func (a Action) EqualsPass() bool

func (Action) EqualsReject added in v0.13.0 

func (a Action) EqualsReject() bool

func (*Action) Scan added in v0.13.0 

func (a *Action) Scan(value interface{}) (err error)

implement sql.Scanner

func (Action) String added in v0.13.0 

func (a Action) String() string

func (Action) Value added in v0.13.0 

func (a Action) Value() (driver.Value, error)

implement sql/driver.Valuer

type Addr added in v0.13.0 

type Addr = mailutil.Addr

type LMTPBackend 

type LMTPBackend struct {
	Ulist *Ulist
}

func (LMTPBackend) NewSession added in v0.14.3 

func (lb LMTPBackend) NewSession(_ *smtp.Conn) (smtp.Session, error)

type List added in v0.13.0 

type List struct {
	ListInfo
	HMACKey       []byte // [32]byte would require check when reading from database
	PublicSignup  bool   // default: false
	HideFrom      bool   // default: false
	ActionMod     Action
	ActionMember  Action
	ActionKnown   Action
	ActionUnknown Action
}

func (*List) CreateHMAC added in v0.13.0 

func (list *List) CreateHMAC(addr *Addr) (int64, string, error)

CreateHMAC creates an HMAC with a given user email address and the current time. The HMAC is returned as a base64 RawURLEncoding string.

func (*List) SignoffLeaveMessage added in v0.13.0 

func (list *List) SignoffLeaveMessage() ([]byte, error)

func (*List) ValidateHMAC added in v0.13.0 

func (list *List) ValidateHMAC(inputHMAC []byte, addr *Addr, timestamp int64, maxAgeDays int) error

ValidateHMAC validates an HMAC. If the given timestamp is older than maxAgeDays, then ErrLink is returned.

type ListInfo added in v0.13.0 

type ListInfo struct {
	ID int
	Addr
}

func (*ListInfo) BounceAddress added in v0.13.0 

func (li *ListInfo) BounceAddress() string

func (*ListInfo) NewMessageId added in v0.13.0 

func (li *ListInfo) NewMessageId() string

NewMessageId creates a new RFC5322 compliant Message-Id with the list domain as "id-right".

func (*ListInfo) PrefixSubject added in v0.13.0 

func (li *ListInfo) PrefixSubject(subject string) string

type ListRepo added in v0.13.0 

type ListRepo interface {
	AddKnowns(list *List, addrs []*Addr) ([]*Addr, error)
	AddMembers(list *List, addrs []*Addr, receive, moderate, notify, admin, bounces bool) ([]*Addr, error)
	Admins(list *List) ([]string, error)
	AllLists() ([]ListInfo, error)
	BounceNotifieds(list *List) ([]string, error)
	Create(address, name string) (*List, error)
	Delete(list *List) error
	GetList(list *Addr) (*List, error)
	Members(list *List) ([]Membership, error)
	GetMembership(list *List, user *Addr) (Membership, error)
	IsList(addr Addr) (bool, error)
	IsMember(list *List, addr *Addr) (bool, error)
	IsKnown(list *List, rawAddress string) (bool, error)
	Knowns(list *List) ([]string, error)
	Memberships(member *Addr) ([]Membership, error)
	Notifieds(list *List) ([]string, error)
	PublicLists() ([]ListInfo, error)
	Receivers(list *List) ([]string, error)
	RemoveKnowns(list *List, addrs []*Addr) ([]*mailutil.Addr, error)
	RemoveMembers(list *List, addrs []*Addr) ([]*Addr, error)
	Update(list *List, display string, publicSignup, hideFrom bool, actionMod, actionMember, actionKnown, actionUnknown Action) error
	UpdateMember(list *List, rawAddress string, receive, moderate, notify, admin, bounces bool) error
}

type Logger added in v0.13.0 

type Logger interface {
	Printf(format string, v ...interface{}) error
}

type Membership added in v0.13.0 

type Membership struct {
	ListInfo      // not List because we had to fetch all of them from the database in Memberships()
	Member        bool
	MemberAddress string
	Receive       bool
	Moderate      bool
	Notify        bool
	Admin         bool
	Bounces       bool
}

type Status added in v0.13.0 

type Status int
const (
	Known Status = iota
	Member
	Moderator
)

func (Status) String added in v0.13.0 

func (s Status) String() string

type Ulist added in v0.13.0 

type Ulist struct {
	DummyMode     bool
	GDPRLogger    Logger
	Lists         ListRepo
	LMTPSock      string
	MTA           mailutil.MTA
	SocketmapSock string
	SpoolDir      string
	Superadmin    string       // RFC5322 AddrSpec, can create new mailing lists and modify all mailing lists
	Web           WebInterface // if nil, users won't be able to checkback join and leave, and moderators won't be able to moderate

	LastLogID uint32
	Waiting   sync.WaitGroup
}

func (*Ulist) AddMembers added in v0.13.0 

func (u *Ulist) AddMembers(list *List, sendWelcome bool, addrs []*Addr, receive, moderate, notify, admin, bounces bool, reason string) (int, []error)

func (*Ulist) CheckbackJoinUrl added in v0.13.0 

func (u *Ulist) CheckbackJoinUrl(list *List, recipient *Addr) (string, error)

func (*Ulist) CheckbackLeaveUrl added in v0.13.0 

func (u *Ulist) CheckbackLeaveUrl(list *List, recipient *Addr) (string, error)

func (*Ulist) CreateList added in v0.13.0 

func (u *Ulist) CreateList(address, name, rawAdminMods string, reason string) (*List, int, []error)

func (*Ulist) DeleteModeratedMail added in v0.13.0 

func (u *Ulist) DeleteModeratedMail(list *List, filename string) error

func (*Ulist) Forward added in v0.13.0 

func (u *Ulist) Forward(list *List, m *mailutil.Message) error

Forwards a message over the given mailing list. This is the main job of this software.

func (*Ulist) GetAction added in v0.13.0 

func (u *Ulist) GetAction(list *List, header mail.Header, froms []*Addr) (Action, string, error)

GetAction determines the maximum action of an email by the "From" addresses and possible spam headers. It also returns a human-readable reason for the decision.

The SMTP envelope sender is ignored, because it's actually something different and a case for the spam filtering system. (Mailman incorporates it last, which is probably never, because each email must have a From header: https://mail.python.org/pipermail/mailman-users/2017-January/081797.html)

func (*Ulist) GetRoles added in v0.13.0 

func (u *Ulist) GetRoles(list *List, addr *Addr) ([]Status, error)

func (*Ulist) ListenAndServe added in v0.13.8 

func (u *Ulist) ListenAndServe() error

func (*Ulist) Notify added in v0.13.0 

func (u *Ulist) Notify(list *List, recipient string, subject string, body io.Reader) error

Notify notifies recipients about something related to the list.

func (*Ulist) NotifyMods added in v0.13.0 

func (u *Ulist) NotifyMods(list *List, mods []string) error

appends a footer

func (*Ulist) Open added in v0.13.0 

func (u *Ulist) Open(list *List, filename string) (*os.File, error)

caller must close the returned file

func (*Ulist) ReadHeader added in v0.13.0 

func (u *Ulist) ReadHeader(list *List, filename string) (mail.Header, error)

func (*Ulist) ReadMessage added in v0.13.0 

func (u *Ulist) ReadMessage(list *List, filename string) (*mailutil.Message, error)

func (*Ulist) RemoveMembers added in v0.13.0 

func (u *Ulist) RemoveMembers(list *List, sendGoodbye bool, addrs []*Addr, reason string) (int, []error)

func (*Ulist) Save added in v0.13.0 

func (u *Ulist) Save(list *List, m *mailutil.Message) error

Saves the message into an eml file with a unique name within the storage folder. The filename is not returned.

func (*Ulist) SendJoinCheckback added in v0.13.0 

func (u *Ulist) SendJoinCheckback(list *List, recipient *Addr) error

SendJoinCheckback does not check the authorization of the asking person. This must be done by the caller.

func (*Ulist) SendLeaveCheckback added in v0.13.0 

func (u *Ulist) SendLeaveCheckback(list *List, user *Addr) (bool, error)

SendLeaveCheckback sends a checkback email if the user is a member of the list.

If the user is not a member, the returned error is nil, so it doesn't reveal about the membership. However both timing and other errors can still reveal it.

The returned bool value indicates whether the email was sent.

func (*Ulist) SignoffJoinMessage added in v0.13.0 

func (u *Ulist) SignoffJoinMessage(list *List, member *Addr) (*bytes.Buffer, error)

func (*Ulist) StorageFolder added in v0.13.0 

func (u *Ulist) StorageFolder(li ListInfo) string

type WebInterface added in v0.13.8 

type WebInterface interface {
	AskLeaveUrl(list *List) string
	AuthenticationAvailable() bool
	CheckbackJoinUrl(list *List, timestamp int64, hmac string, recipient *Addr) string
	CheckbackLeaveUrl(list *List, timestamp int64, hmac string, recipient *Addr) string
	FooterHTML(list *List) string
	FooterPlain(list *List) string
	ListenAndServe() error
	ModUrl(list *List) string
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
ulist
repo
sqlite
netstring
web
auth
captcha
html
static

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.