awesome-github

Awesome GitHub

Explore your favorite Awesome GitHub repositories via the command-line tool awg!

[ Chinese | 中文 ]

Awesome Lists

What is Awesome Lists？

Such as famous awesome-go, which is a member of Awesome Lists, where we can quickly find frameworks, libraries, software, and other resources related to Go.

Furthermore, we can find more awesome things from Awesome Lists.

Command-line Tool awg

At the moment, Awesome List usually exhibit lots of GitHub repositories. For example, awesome-go contains thousands of GitHub repositories. However, an Awesome List doesn't include information like the count of stars or the last commit date for those repositories. In many cases, we demand that information and have to manually open links to see the them.

The command-line tool, awg, helps us dig a little deeper into the Awesome List to find out more about all the GitHub repositories on it!

awg will fetch information of GitHub repositories listed on an Awesome List, and output the data to a file of your choice. You can use awg to generate a browser page to view the data later, or use your favorite tools like jq or python to analyze.

The final output:

{
  "data": {
    "Command Line": [
      {
        "id": {
          "owner": "urfave",
          "name": "cli"
        },
        "owner": "urfave",
        "awesome_name": "urfave/cli",
        "link": "https://github.com/urfave/cli",
        "watch": 295,
        "star": 14171,
        "fork": 1134,
        "last_commit": "2020-07-12T13:32:01Z",
        "description": "A simple, fast, and fun package for building command line apps in Go",
        "awesome_description": "urfave/cli - Simple, fast, and fun package for building command line apps in Go (formerly codegangsta/cli)."
      },
      // ...
    ]
    // ...
  }
}

Data Analysis

We can use any tool to analyze the obtained data file.

For example, after getting data file awg.json, which is concerned with awesome-go

View in Browser

The page effect is shown below

By runing this command

awg view awg.json

It will be running a simple web server locally, listening at 127.0.0.1:3000 by default. Open this page with your browser to view. Can use --listen to specify other address.

Note: This does not mean that it can be viewed offline. An Internet connection is necessary.

To replace the implementation for viewing, you can replace the embedded JS script address via --script, which supports local path.

View in Terminal

By using the popular command line tool jq, we can:

View Command Line section of awesome-go, sorted by the count of stars

cat awg.json | jq '.data | ."Command Line" | sort_by(.star)'

Installation

Get command-line tool awg

go get github.com/rydesun/awesome-github/cmd/awg

Fetch Data

First prepare the following before running awg:

• GitHub personal access token
• awg configuration file

Access Token

awg fetch information of GitHub repository by calling the GitHub GraphQL API. This official API requires your personal access token to be verified before you can use it. So, you need to provide awg with a GitHub personal access token.

If you do not have the token, please view the article Creating a personal access token。

awg does not need the token to have any scopes or permissions. Therefore, do not grant the token any scopes or permissions.

Configuration

A configuration file is necessary. See configuration file template in the directory configs as a reference.

awg will read the personal access token from the environment variable GITHUB_ACCESS_TOKEN first. So it is not necessary to store this value in the configuration file.

Increasing the number of concurrent query requests will increase the speed of the query, but the number should not be too large (the current recommended value is 3), otherwise this behavior will be viewed as abusing, and then blocked.

All relative paths in a configuration file are relative to awg's current working directory, not to the directory where the configuration file is located. Absolute paths are encouraged.

Run

Fetch JSON data

awg fetch --config path/to/config.yaml

(Recommended) Specify the GitHub Personal Access Token from an environment variable

GITHUB_ACCESS_TOKEN=<Your Token> awg fetch --config path/to/config.yaml

Note the rate limit, which is not identical to the number of concurrent requests. Currently awg can query up to 5000 GitHub repositories per hour. If there are too many queries, GitHub will impose limits on requests and responses. See GitHub Resource limitations for more.

Notes

This project is currently only testing awesome-go's lists, other testing for Awesome List results are pending.

awg does not support Windows platforms.