vssh

package module
v0.0.0-...-bfa903e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 22, 2020 License: Apache-2.0 Imports: 18 Imported by: 6

README

vSSH

Go library to handle tens of thousands SSH connections and execute the command(s) with higher-level API for building network device / server automation. Documentation and examples are available via godoc.

Test Status Go Report Card Coverage Status GoDoc PkgGoDev

Alt text

Features

  • Connect to multiple remote machines concurrently
  • Persistent SSH connection
  • DSL query based on the labels
  • Manage number of sessions per SSH connection
  • Limit amount of stdout and stderr data in bytes
  • Higher-level API for building automation
Sample query with label
labels := map[string]string {
  "POP" : "LAX",
  "OS" : "JUNOS",
}
// sets labels to a client
vs.AddClient(addr, config, vssh.SetLabels(labels))
// query with label
vs.RunWithLabel(ctx, cmd, timeout, "(POP == LAX || POP == DCA) && OS == JUNOS")
Basic example
vs := vssh.New().Start()
config := vssh.GetConfigUserPass("vssh", "vssh")
for _, addr := range []string{"54.193.17.197:22", "192.168.2.19:22"} {
  vs.AddClient(addr, config, vssh.SetMaxSessions(4))
}
vs.Wait()

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

cmd:= "ping -c 4 192.168.55.10"
timeout, _ := time.ParseDuration("6s")
respChan := vs.Run(ctx, cmd, timeout)

for resp := range respChan {
  if err := resp.Err(); err != nil {
    log.Println(err)
      continue
    }

  outTxt, errTxt, _ := resp.GetText(vs)
  fmt.Println(outTxt, errTxt, resp.ExitStatus())
}
Stream example
vs := vssh.New().Start()
config, _ := vssh.GetConfigPEM("vssh", "mypem.pem")
vs.AddClient("54.193.17.197:22", config, vssh.SetMaxSessions(4))
vs.Wait()

ctx, cancel := context.WithCancel(context.Background())
defer cancel()

cmd:= "ping -c 4 192.168.55.10"
timeout, _ := time.ParseDuration("6s")
respChan := vs.Run(ctx, cmd, timeout)

resp := <- respChan
if err := resp.Err(); err != nil {
  log.Fatal(err)
}

stream := resp.GetStream()
defer stream.Close()

for stream.ScanStdout() {
  txt := stream.TextStdout()
  fmt.Println(txt)
}

Supported platform

  • Linux
  • Windows
  • Darwin
  • BSD
  • Solaris

License

Code is licensed under the Apache License, Version 2.0 (the "License"). Content is licensed under the CC BY 4.0 license. Terms available at https://creativecommons.org/licenses/by/4.0/.

Contribute

Welcomes any kind of contribution, please follow the next steps:

  • Fork the project on github.com.
  • Create a new branch.
  • Commit changes to the new branch.
  • Send a pull request.

Documentation

Overview

Package vssh is a Go library to handle tens of thousands SSH connections and execute the command with higher-level API for building network device / server automation. 

run(ctx, command, timeout)
runWithLabel(ctx, command, timeout, "OS == Ubuntu && POP == LAX")

By calling the run method vssh sends the given command to all available clients or based on your query it runs the command on the specific clients and the results of the ran command can be received in two options, streaming or final result.In streaming you can get line by line from command’s stdout / stderr in real time or in case of non-real time you can get the whole of the lines together.

Example (Cloud)

This example demonstrates integration vSSH with AWS EC2 

package main

import (
	"context"
	"fmt"
	"log"
	"net"
	"time"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/credentials"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/ec2"
	"github.com/yahoo/vssh"
)

func main() {
	vs := vssh.New().Start()
	config, _ := vssh.GetConfigPEM("ubuntu", "aws.pem")

	// AWS EC2 Golang SDK
	// Please check their website for more information
	// https://docs.aws.amazon.com/sdk-for-go/
	awsConfig := &aws.Config{
		Region:      aws.String("us-west-1"),
		Credentials: credentials.NewStaticCredentials("YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", ""),
	}
	sess, err := session.NewSession(awsConfig)
	if err != nil {
		fmt.Println("error creating new session:", err.Error())
		log.Fatal(err)
	}
	ec2svc := ec2.New(sess)
	params := &ec2.DescribeInstancesInput{
		// filter running instances at us-west-1
		Filters: []*ec2.Filter{
			{
				Name:   aws.String("instance-state-name"),
				Values: []*string{aws.String("running")},
			},
		},
	}
	resp, err := ec2svc.DescribeInstances(params)
	if err != nil {
		fmt.Println("there was an error listing instances in", err.Error())
		log.Fatal(err.Error())
	}

	// iterate over the EC2 running instances and add to vssh
	for idx := range resp.Reservations {
		for _, inst := range resp.Reservations[idx].Instances {
			labels := make(map[string]string)
			for _, tag := range inst.Tags {
				labels[*tag.Key] = *tag.Value
			}
			addr := net.JoinHostPort(*inst.PublicIpAddress, "22")
			vs.AddClient(addr, config, vssh.SetLabels(labels))
		}
	}

	vs.Wait()

	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	cmd := "uname -a"
	timeout, _ := time.ParseDuration("5s")
	respChan := vs.Run(ctx, cmd, timeout)

	for resp := range respChan {
		// in case of the connectivity issue to client
		if err := resp.Err(); err != nil {
			log.Println(err)
			continue
		}

		// get the returned data from client
		outTxt, errTxt, err := resp.GetText(vs)

		// check the error like timeout but still
		// we can have data on outTxt and errTxt
		if err != nil {
			log.Println(err)
		}

		// print command's stdout
		fmt.Println(outTxt)

		// print command's stderr
		fmt.Println(errTxt)

		// print exit status of the remote command
		fmt.Println(resp.ExitStatus())
	}
}

Output:
Example (Stream) 
package main

import (
	"context"
	"fmt"
	"log"
	"sync"
	"time"

	"github.com/yahoo/vssh"
)

func main() {
	var wg sync.WaitGroup

	vs := vssh.New().Start()
	config, err := vssh.GetConfigPEM("ubuntu", "myaws.pem")
	if err != nil {
		log.Fatal(err)
	}

	for _, addr := range []string{"3.101.78.17:22", "13.57.12.15:22"} {
		vs.AddClient(addr, config, vssh.SetMaxSessions(4))
	}

	vs.Wait()

	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	cmd := "uname -a"
	timeout, _ := time.ParseDuration("6s")
	respChan := vs.Run(ctx, cmd, timeout)

	for resp := range respChan {
		if err := resp.Err(); err != nil {
			log.Println("error", err)
			continue
		}

		wg.Add(1)
		go func(resp *vssh.Response) {
			defer wg.Done()
			handler(resp)
		}(resp)
	}

	wg.Wait()
}

func handler(resp *vssh.Response) {
	stream := resp.GetStream()
	defer stream.Close()

	for stream.ScanStdout() {
		txt := stream.TextStdout()
		fmt.Println(resp.ID(), txt)
	}

	if err := stream.Err(); err != nil {
		log.Println("error", err)
	}
}

Output:

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetConfigPEM 

func GetConfigPEM(user, keyFile string) (*ssh.ClientConfig, error)

GetConfigPEM returns SSH configuration that uses the given private key. the keyfile should be unencrypted PEM-encoded private key file.

func GetConfigUserPass 

func GetConfigUserPass(user, password string) *ssh.ClientConfig

GetConfigUserPass returns SSH configuration that uses the given username and password.

func SetClientsShardNumber 

func SetClientsShardNumber(n int)

SetClientsShardNumber sets the clients shard number.

vSSH uses map data structure to keep the clients data in the memory. Sharding helps to have better performance on write/read with mutex. This setting can be tuned if needed.

Types

type ClientOption 

type ClientOption func(c *clientAttr)

ClientOption represents client optional parameters.

func DisableRequestPty 

func DisableRequestPty() ClientOption

DisableRequestPty disables the pty.

func RequestPty 

func RequestPty(term string, h, w uint, modes ssh.TerminalModes) ClientOption

RequestPty sets the pty parameters.

func SetLabels 

func SetLabels(labels map[string]string) ClientOption

SetLabels sets labels for a client.

func SetMaxSessions 

func SetMaxSessions(n int) ClientOption

SetMaxSessions sets maximum sessions for given client.

type MaxSessionsError 

type MaxSessionsError struct {
	// contains filtered or unexported fields
}

MaxSessionsError represents max sessions error.

type Response 

type Response struct {
	// contains filtered or unexported fields
}

Response represents the response for given session.

Example

This example demonstrates the use of GetText() for two hosts. 

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/yahoo/vssh"
)

func main() {
	// construct and start the vssh
	vs := vssh.New().Start()

	// create ssh configuration with user/pass
	// you can create this configuration by golang ssh package
	config := vssh.GetConfigUserPass("vssh", "vssh")

	// add clients to vssh with one option: max session
	// there are other options that you can add to this method
	for _, addr := range []string{"54.193.17.197:22", "192.168.2.19:22"} {
		vs.AddClient(addr, config, vssh.SetMaxSessions(4))
	}

	// wait until vssh connected to all the clients
	vs.Wait()

	// create a context with cancel
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// send the ping command to clients with 6 seconds timeout
	cmd := "ping -c 4 192.168.55.10"
	timeout, _ := time.ParseDuration("6s")
	respChan := vs.Run(ctx, cmd, timeout)

	// get the resp channel for each client
	for resp := range respChan {
		// in case of the connectivity issue to client
		if err := resp.Err(); err != nil {
			log.Println(err)
			continue
		}

		// get the returned data from client
		outTxt, errTxt, err := resp.GetText(vs)

		// check the error like timeout but still
		// we can have data on outTxt and errTxt
		if err != nil {
			log.Println(err)
		}

		// print command's stdout
		fmt.Println(outTxt)

		// print command's stderr
		fmt.Println(errTxt)

		// print exit status of the remote command
		fmt.Println(resp.ExitStatus())
	}
}

Output:

func (*Response) Err 

func (r *Response) Err() error

Err returns response error.

func (*Response) ExitStatus 

func (r *Response) ExitStatus() int

ExitStatus returns the exit status of the remote command.

func (*Response) GetStream 

func (r *Response) GetStream() *Stream

GetStream constructs a new stream from a response.

func (*Response) GetText 

func (r *Response) GetText(v *VSSH) (string, string, error)

GetText gets the final result of the given response.

func (*Response) ID 

func (r *Response) ID() string

ID returns response identification.

type RunOption 

type RunOption func(q *query)

RunOption represents run optional parameters.

func SetLimitReaderStderr 

func SetLimitReaderStderr(n int64) RunOption

SetLimitReaderStderr sets limit for stderr reader.

func SetLimitReaderStdout 

func SetLimitReaderStdout(n int64) RunOption

SetLimitReaderStdout sets limit for stdout reader. 

respChan := vs.Run(ctx, cmd, timeout, vssh.SetLimitReaderStdout(1024))
Example

This example demonstrates how to set limit the amount of returned data 

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/yahoo/vssh"
)

func main() {

	// construct and start the vssh
	vs := vssh.New().Start()

	// create ssh configuration
	// you can create this configuration by golang ssh package
	config, _ := vssh.GetConfigPEM("ubuntu", "aws.pem")

	// add client to vssh with one option: max-sessions
	// there are other options that you can add to this method
	vs.AddClient("54.215.209.152:22", config, vssh.SetMaxSessions(2))

	// wait until vssh connected to the client
	vs.Wait()

	// create a context with cancel
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	timeout, _ := time.ParseDuration("6s")

	// run dmesg command with limit the amounth of returned data to 512 bytes
	respChan := vs.Run(ctx, "dmesg", timeout, vssh.SetLimitReaderStdout(1024))

	// get the resp
	resp := <-respChan
	// in case of the connectivity issue to client
	if err := resp.Err(); err != nil {
		log.Fatal(err)
	}

	outTxt, errTxt, err := resp.GetText(vs)
	// check the error like timeout but still
	// we can have data on outTxt and errTxt
	if err != nil {
		log.Println(err)
	}

	fmt.Println(outTxt)
	fmt.Println(errTxt)
}

Output:

type Stream 

type Stream struct {
	// contains filtered or unexported fields
}

Stream represents data stream for given response. It provides convenient interfaces to get the returned data real-time.

Example

This example demonstrates the use of stream 

package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/yahoo/vssh"
)

func main() {
	vs := vssh.New().Start()
	config, _ := vssh.GetConfigPEM("ubuntu", "aws.pem")
	vs.AddClient("54.193.17.197:22", config, vssh.SetMaxSessions(4))
	vs.Wait()

	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	cmd := "uname -a"
	timeout, _ := time.ParseDuration("5s")
	respChan := vs.Run(ctx, cmd, timeout)

	resp := <-respChan
	if err := resp.Err(); err != nil {
		log.Fatal(err)
	}

	stream := resp.GetStream()
	defer stream.Close()

	for stream.ScanStdout() {
		txt := stream.TextStdout()
		fmt.Println(txt)
	}

	if err := stream.Err(); err != nil {
		log.Fatal(err)
	}
}

Output:

func (*Stream) BytesStderr 

func (s *Stream) BytesStderr() []byte

BytesStderr returns the most recent data scanned by ScanStderr as bytes.

func (*Stream) BytesStdout 

func (s *Stream) BytesStdout() []byte

BytesStdout returns the most recent data scanned by ScanStdout as bytes.

func (*Stream) Close 

func (s *Stream) Close() error

Close cleans up the stream's response.

func (*Stream) Err 

func (s *Stream) Err() error

Err returns stream response error.

func (*Stream) Input 

func (s *Stream) Input(in io.Reader)

Input writes the given reader to remote command's standard input when the command starts.

func (*Stream) ScanStderr 

func (s *Stream) ScanStderr() bool

ScanStderr provides a convenient interface for reading stderr which it connected to remote host. It reads a line and buffers it. The TextStdout() or BytesStdout() methods return the buffer in string or bytes.

func (*Stream) ScanStdout 

func (s *Stream) ScanStdout() bool

ScanStdout provides a convenient interface for reading stdout which it connected to remote host. It reads a line and buffers it. The TextStdout() or BytesStdout() methods return the buffer in string or bytes.

func (*Stream) Signal 

func (s *Stream) Signal(sig ssh.Signal)

Signal sends the given signal to remote process.

func (*Stream) TextStderr 

func (s *Stream) TextStderr() string

TextStderr returns the most recent data scanned by ScanStderr as string.

func (*Stream) TextStdout 

func (s *Stream) TextStdout() string

TextStdout returns the most recent data scanned by ScanStdout as string.

type TimeoutError 

type TimeoutError struct {
	// contains filtered or unexported fields
}

TimeoutError represents timeout error.

type VSSH 

type VSSH struct {
	// contains filtered or unexported fields
}

VSSH represents VSSH instance.

func New 

func New() *VSSH

New constructs a new VSSH instance.

func (*VSSH) AddClient 

func (v *VSSH) AddClient(addr string, config *ssh.ClientConfig, opts ...ClientOption) error

AddClient adds a new SSH client to VSSH.

func (*VSSH) CurrentProc 

func (v *VSSH) CurrentProc() uint64

CurrentProc returns number of running processes / workers.

func (*VSSH) DecreaseProc 

func (v *VSSH) DecreaseProc(n ...int)

DecreaseProc destroys the idle processes / workers.

func (*VSSH) ForceReConn 

func (v *VSSH) ForceReConn(addr string) error

ForceReConn reconnects the client immediately.

func (*VSSH) IncreaseProc 

func (v *VSSH) IncreaseProc(n ...int)

IncreaseProc adds more processes / workers.

func (*VSSH) OnDemand 

func (v *VSSH) OnDemand() *VSSH

OnDemand changes VSSH connection behavior. By default VSSH connects to all of the clients before any run request and it maintains the authenticated SSH connection to all clients. We can call this "persistent SSH connection" but with OnDemand it tries to connect to clients once the run requested and it closes the appropriate connection once the response data returned.

func (*VSSH) Run 

func (v *VSSH) Run(ctx context.Context, cmd string, timeout time.Duration, opts ...RunOption) chan *Response

Run sends a new run query with given context, command and timeout.

timeout allows you to set a limit on the length of time the command will run for. You can cancel the running command by context.WithCancel.

func (*VSSH) RunWithLabel 

func (v *VSSH) RunWithLabel(ctx context.Context, cmd, queryStmt string, timeout time.Duration, opts ...RunOption) (chan *Response, error)

RunWithLabel runs the command on the specific clients which they matched with given query statement. 

	labels := map[string]string {
 	"POP" : "LAX",
 	"OS" : "JUNOS",
	}
	// sets labels to a client
	vs.AddClient(addr, config, vssh.SetLabels(labels))
	// run the command with label
	vs.RunWithLabel(ctx, cmd, timeout, "POP == LAX || POP == DCA) && OS == JUNOS")

func (*VSSH) SetInitNumProc 

func (v *VSSH) SetInitNumProc(n int)

SetInitNumProc sets the initial number of processes / workers.

You need to set this number right after creating vssh. 

vs := vssh.New()
vs.SetInitNumProc(200)
vs.Start()

There are two other methods in case you need to change the settings in the middle of your code. 

IncreaseProc(n int)
DecreaseProc(n int)

func (*VSSH) SetLogger 

func (v *VSSH) SetLogger(l *log.Logger)

SetLogger sets external logger.

func (*VSSH) Start 

func (v *VSSH) Start() *VSSH

Start starts vSSH, including action queue and re-connect procedures. You can construct and start the vssh like below: 

vs := vssh.New().Start()

func (*VSSH) StartWithContext 

func (v *VSSH) StartWithContext(ctx context.Context) *VSSH

StartWithContext is same as Run but it accepts external context.

func (*VSSH) Wait 

func (v *VSSH) Wait(p ...int) (float64, error)

Wait stands by until percentage of the clients have been processed. An optional percentage can be passed as an argument - otherwise the default value of 100% is used.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.