carbon

ZEO.ORG - Carbon API

Carbon aims to find related results for the given URLs or Keywords.
It's mostly used to find alternatives for 404 pages or SERP operations.
It exports data in Excel or Google Sheets.

The API is served at AWS Lambda.

Features

• Supports URL and Keyword option.
  • URL option; finds related 3 URLs.
    Mostly used to find alternative URLs for 404 pages.
  • Keyword option; finds related 10 results.
    The result includes title, url, and description for each keywords.
    Mostly used for SERP.
• Shows fails with a reason in a sperated sheet.
• Automatically trims duplicated inputs.
• Supports country and language specification.
• Supports 2 export options; Excel and Google Sheets.
  • For URL option, makes a suggestion that is most similar with the input.
• Supports internal accounts with limitation.
  • For non-login users, the limit is 100 URLs.
• Supports 2 resources to take SERP data.

Endpoint

URL: /

Request:

• Type: POST
• Params:
  • type must
    options: keyword or url.
    note: keyword option is only available for internal users.
  • format must
    options: excel or sheet.
  • country must
    options: all countries supported by Google.
  • langauge must
    options: all languages supported by Google.
  • accountName
  • accountPassword
• Header:
  • Accept must
    If the format is excel,
    you need to set Accept: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
• Body:
  • Raw Data
    • As a JSON value,
      For example;

      {
          "values": [
            {
              "value": "https://tools.zeo.org/carbon"
            },
            {
              "value": "https://seo.do/pricing/"
            },
            ...
          ]
      }
      

Response:

Status;

• Type: 201
  • That means the data is created.
• Type: 400
  • That means the inputs are not valid.
    Check the error message.
• Type: 401
  • That means auth is not successful.
• Type: 405
  • That means the method is forbidden.
    Use POST method.
• Type: 500
  • That means internal error occurs while creating the data.
• Type: 503
  • That means the service is not available.
    Try later.

Header and body;

• For excel;
  • Header

    Content-Disposition: attachment; filename="result.xlsx"
    Content-Length: 453646
    Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
    

  • Body
    file
• For sheet;
  • Body

     {
         "sheetURL": "https://docs.google.com/spreadsheets/d/...",
     }
    

Development

Requirements

• SERP API Credantials. (to access credantials contact with zeo.org)
• Google Credantials. (have to enabled Drive API V3 on the account)
• Google Access Token. (to access to Google Drive as an user)
• Go v1.15

How to set up

• Copy env.sample to .env.
• Update secret values in .env.

Usage at local

There is an easy way to test lambda projects at the local, lambci.

go build -o carbon && docker run --rm -v "$PWD":/var/task:ro,delegated lambci/lambda:go1.x carbon '{"HTTPMethod": "POST", "QueryStringParameters": {"type": "url", "format": "sheet", "country": "tr", "language": "tr"},"Body": "{\"values\": [{\"value\": \"https://boratanrikulu.dev\/contact\"}] }"}' && rm carbon

Run tests

To run all tests;

go test ./...

To run specific test;

go test ./services -run TestConvertURLResultToExcel -v 

How to deploy to AWS Lambda

Build.

go build -o carbon && zip deploy.zip carbon

Create the function.

aws lambda create-function --function-name CarbonLambda \
    --handler carbon --runtime go1.x \
    --role  arn:aws:iam::<account-id>:role/<role> \
    --zip-file fileb://./deploy.zip \
    --tracing-config Mode=Active

If you need to update the function, take a build and run this command.

aws lambda update-function-code --function-name CarbonLambda --zip-file fileb://./deploy.zip

Note: To increase API Gateway's timeout limit, check this repo: Timeout Increaser

Credits

T. Mert Azizoğlu Idea By	Bora Tanrıkulu Developed By