Documentation
¶
Overview ¶
Package server_utils shares utility functions used across multiple server pacakges (origin, cache, registry, director).
It should only import lower level packages (config, param, etc), or server_structs package. It should never import any server pacakges (origin, cache, registry, director) or upeer level packages (launcher_utils, cmd, etc).
For structs used across multiple server pacakges, put them in common package instead
Index ¶
- Variables
- func CheckOriginSentinelLocations(exports []OriginExport) (ok bool, err error)
- func GetJWKSFromIssUrl(issuer string) (*jwk.Set, error)
- func GetJWKSURLFromIssuerURL(issuerUrl string) (string, error)
- func GetNSIssuerURL(prefix string) (string, error)
- func GetServerIssuerURL() (*url.URL, error)
- func HandleDirectorTestResponse(ctx *gin.Context, nChan chan bool)
- func LaunchPeriodicDirectorTimeout(ctx context.Context, egrp *errgroup.Group, nChan chan bool)
- func LaunchWatcherMaintenance(ctx context.Context, dirPaths []string, description string, ...)
- func RegisterOIDCAPI(engine *gin.Engine)
- func ResetOriginExports()
- func StringListToCapsHookFunc() mapstructure.DecodeHookFuncType
- func WaitUntilWorking(ctx context.Context, method, reqUrl, server string, expectedStatus int, ...) error
- type OriginExport
- type OriginStorageType
- type TestFileTransfer
- type TestFileTransferImpl
- type TestType
Constants ¶
This section is empty.
Variables ¶
var ( ErrUnknownOriginStorageType = errors.New("unknown origin storage type") ErrInvalidOriginConfig = errors.New("invalid origin configuration") )
Functions ¶
func CheckOriginSentinelLocations ¶
func CheckOriginSentinelLocations(exports []OriginExport) (ok bool, err error)
Check the sentinel files from Origin.Exports
func GetJWKSFromIssUrl ¶
Given an issuer URL, get the JWKS from the issuer's JWKS URL
func GetJWKSURLFromIssuerURL ¶
Given an issuer url, lookup the JWKS URL from the openid-configuration For example, if the issuer URL is https://registry.com:8446/api/v1.0/registry/test-namespace, this function will return the key indicated by the openid-configuration JSON hosted at https://registry.com:8446/api/v1.0/registry/test-namespace/.well-known/openid-configuration.
func GetNSIssuerURL ¶
For a given prefix, get the prefix's issuer URL, where we consider that the openid endpoint we use to look up a key location. Note that this is NOT the same as the issuer key -- to find that, follow openid-style discovery using the issuer URL as a base.
func GetServerIssuerURL ¶
For calling from within the server. Returns the server's issuer URL/port
func HandleDirectorTestResponse ¶
The director periodically uploads/downloads files to/from all online origins for testing. It sends a request reporting the status of the test result to this endpoint, and we will update origin internal health status metric by what director returns.
func LaunchPeriodicDirectorTimeout ¶
Launch a go routine in errorgroup to report timeout if director-based health test response was not sent within the defined time limit
func LaunchWatcherMaintenance ¶
func LaunchWatcherMaintenance(ctx context.Context, dirPaths []string, description string, sleepTime time.Duration, maintenanceFunc func(notifyEvent bool) error)
Launch a maintenance goroutine. The maintenance routine will watch the directory `dirPath`, invoking `maintenanceFunc` whenever an event occurs in the directory. Note the behavior of directory watching differs across platforms; for example, an atomic rename might be one or two events for the destination file depending on Mac OS X or Linux.
Even if the filesystem watcher fails, this will invoke `maintenanceFunc` every `sleepTime` duration. The maintenance function will be called with `true` if invoked due to a directory change, false otherwise When generating error messages, `description` will be used to describe the task.
func RegisterOIDCAPI ¶
func ResetOriginExports ¶
func ResetOriginExports()
func StringListToCapsHookFunc ¶
func StringListToCapsHookFunc() mapstructure.DecodeHookFuncType
A decoder hook we can pass to viper.Unmarshal to convert a list of strings to a struct with boolean fields. In this case, we're converting a string slice (flow) from yaml:
Exports: Capabilities: ["PublicReads", "Writes"]
to a struct like:
ExportCapabilities{
PublicReads: true,
Writes: true,
Listings: false,
}
Here's a helpful tutorial on how to write these: https://sagikazarmark.hu/blog/decoding-custom-formats-with-viper/
func WaitUntilWorking ¶
func WaitUntilWorking(ctx context.Context, method, reqUrl, server string, expectedStatus int, statusMismatch bool) error
Wait until given `reqUrl` returns a HTTP 200. Logging messages emitted will refer to `server` (e.g., origin, cache, director) Pass true to statusMismatch to allow a mismatch of expected status code and what's returned not fail immediately
Types ¶
type OriginExport ¶
type OriginExport struct {
StoragePrefix string `json:"storage_prefix"`
FederationPrefix string `json:"federation_prefix"`
// Export fields specific to S3. Other things like
// S3ServiceUrl, S3Region, etc are kept top-level in the config
S3Bucket string `json:"s3_bucket"`
S3AccessKeyfile string `json:"s3_access_keyfile"`
S3SecretKeyfile string `json:"s3_secret_keyfile"`
// Capabilities for the export
Capabilities server_structs.Capabilities `json:"capabilities"`
SentinelLocation string `json:"sentinel_location"`
}
func GetOriginExports ¶
func GetOriginExports() ([]OriginExport, error)
GetOriginExports is used to parse the config yaml and return a list of OriginExports. It should only touch the yaml the first time it's called, and then return the cached value on subsequent calls. When the configuration is set up using the older single-prefix style of configuration, the function will convert those values (such as Origin.FederationPrefix, Origin.StoragePrefix, etc.) into the OriginExports struct and return a list of one. Otherwise, we'll base things off the list of exports and ignore the single-prefix style of configuration.
type OriginStorageType ¶
type OriginStorageType string
const ( OriginStoragePosix OriginStorageType = "posix" OriginStorageS3 OriginStorageType = "s3" OriginStorageHTTPS OriginStorageType = "https" )
func ParseOriginStorageType ¶
func ParseOriginStorageType(storageType string) (ost OriginStorageType, err error)
Convert a string to an OriginStorageType
type TestFileTransfer ¶
type TestFileTransferImpl ¶
type TestFileTransferImpl struct {
// contains filtered or unexported fields
}
func (TestFileTransferImpl) RunTests ¶
func (t TestFileTransferImpl) RunTests(ctx context.Context, baseUrl, audienceUrl, issuerUrl string, testType TestType) (bool, error)
Run a file transfer test suite with upload/download/delete a test file from the server and a xrootd service. It expects `baseUrl` to be the url to the xrootd endpoint, `issuerUrl` be the url to issue scitoken for file transfer, and the test file content/name be based on `testType`
Note that for this test to work, you need to have the `issuerUrl` registered in your xrootd as a list of trusted token issuers and the issuer is expected to follow WLCG rules for issuer metadata discovery and public key access
Read more: https://github.com/WLCG-AuthZ-WG/common-jwt-profile/blob/master/profile.md#token-verification
func (TestFileTransferImpl) TestCacheDownload ¶
func (t TestFileTransferImpl) TestCacheDownload(ctx context.Context, cacheUrl, issuerUrl string, filePath string, body string) (bool, error)
Run a file transfer test to download a test file from the server and a xrootd service. It expects `cacheUrl` to be the url to the xrootd cache, `issuerUrl` be the url to issue a scitoken for file transfer, `filePath“ to be the namespace and file name of the test file, and the test file to contain the string `body`
Note that for this test to work, you need to have the `issuerUrl` registered in your xrootd as a list of trusted token issuers and the issuer is expected to follow WLCG rules for issuer metadata discovery and public key access
Read more: https://github.com/WLCG-AuthZ-WG/common-jwt-profile/blob/master/profile.md#token-verification
Source Files