mdcode

ｍｄｃｏｄｅ

Markdown code block authoring tool

The mdcode command-line tool allows code blocks embedded in a markdown document to be developed in the usual way. During the development of the code blocks, the usual tools and methods can be used. This makes the embedded codes testable, which is especially important for example codes. There is no worse developer experience than a faulty sample code.

Here is a simple example code for a factorial calculation:

go

func factorial(n uint64) uint64 {
	if n > 1 {
		return n * factorial(n-1)
	}

	return 1
}

JavaScript

function factorial(n) {
    if (n > 1) {
        return n * factorial(n - 1)
    }

    return 1
}

At first glance, there is nothing special about this. However, these code blocks are testable!

This document includes the necessary code for testing within invisible code blocks. By examining the source of this document, you can observe how effortlessly code blocks, even invisible ones, can be embedded using mdcode.

Code blocks embedded in this document can be saved to files using the mdcode extract command. A README_test.go and a README.test.js file will be created in the current directory. After modification, the code blocks can be updated from these files to the document using the mdcode update command.

After the modification, it is advisable to test the above examples using the following commands:

go test ./...
node --test

Since the above code block has a name (test), it can also be run with the mdcode run command:

mdcode run -n test

More examples can be found in the examples directory and in the tutorial.

Features

• include source files as code blocks in the markdown document
• update the code blocks in the markdown document
• save markdown code blocks to source files
• supports source file fragments using #region comments
• supports invisible (not rendered) code blocks
• allows you to add metadata to code blocks
• programming language agnostic
• dump code blocks as tar archive

Use Cases

Develop the example/tutorial codes as you would any other code

• use your favorite IDE, toolchain
• use any test framework for testing
• integrate example code testing into the build process
• use mdcode update to update example code in markdown documents

Write example code directly in the markdown documents

• use mdcode extract to extract code blocks and save them to files
• use any test framework for testing
• integrate example code testing into the build process

Create a self-contained markdown tutorial document

• use mdcode update to embed source fragments
• use mdcode update to embed additional files (package.json, go.mod, etc.) as invisible code blocks
• use mdcode extract to extract working examples from the markdown documemt

Save all examples for later use

• use mdcode dump to create tar archive from code blocks

Install

Precompiled binaries can be downloaded and installed from the Releases page.

If you have a go development environment, the installation can also be done with the following command:

go install github.com/szkiba/mdcode@latest

It can even be run without installation using the following command:

go run github.com/szkiba/mdcode@latest

Usage

Check CLI Reference section for detailed command line usage.

Concepts

Metadata

Metadata can be specified for code blocks. These metadata can be used to modify the operation of the subcommands (for example, they can be used for filtering).

The CommonMark specification allows the use of a so-called info-string in the fenced code block. The first word of the info-string typically indicates the programming language, the meaning of the remaining part is not defined by the specification.

mdcode uses the part after the first word of the info-string to specify metadata. The metadata can be entered in JSON format and in a simple, space-separated name="value" format list (where the use of quotation marks is only necessary for values containing spaces). The latter form is more readable, but the JSON format is more portable.

Example name="value" list metadata:

```js file=sample.js region=factorial

```

Example JSON metadata:

```js {"file":"sample.js","region":"factorial"}

```

Metadata used by mdcode:

The only mandatory metadata is file.

Filtering

By default, mdcode work with all code blocks in a markdown document. It is possible to filter code blocks based on programming language or metadata. In this case, mdcode ignores code blocks that do not meet the filter criteria.

A language filter pattern can be specified using the --lang flag. Then only code blocks with a language matching the pattern will be processed. For example, filtering for code blocks containing JavaScript code:

mdcode --lang js

A file name filtering pattern can be specified using the --file flag. Then only code blocks with file metadata matching the pattern will be processed. For example, filtering for code blocks containing the file named examples/foo.js (or parts of it):

mdcode --file examples/foo.js

The --meta flag can be used to specify an arbitrary metadata filtering pattern. Then only code blocks with metadata matching the pattern are processed. For example, filtering for code blocks that have metadata named name and its value is simple:

mdcode --meta name=simple

Specifying several different filter criteria (e.g. language and metadata, or two different metadata) each criteria must be met (and relation).

Standard glob patterns can be used in programming language and metadata filter criteria.

Examples of filter pattern use:

mdcode extract --meta file='examples/**/*.go'
mdcode extract --lang '{go,js}'

Filtering with frequently used metadata can also be done using dedicated flags.

Regions

In addition to embedding entire files, mdcode supports the use of file regions. Named regions can be used in the source code of any programming language. The beginning of the region is marked by a comment line with the content #region name and the end by a comment line with the content #endregion.

For example, in the case of programming languages using C-style line comments (C, C++, Java, JavaScript, go, etc.):

// #region common

// #endregion

In the case of programming languages using shell-style line comments (Python, sh, bash, etc.):

# #region common

# #endregion

Or if only block comments can be used (CSS):

/* #region common */

/* #endregion */

Regions marked in this way are used by IDEs to collapse parts of the source code.

In the case of mdcode, regions can be referenced with the region metadata. If a region is specified for a code block, the subcommand (update or extract) applies only to the specified region of the file. That is, the update command only embeds the specified region from the file to the markdown document, and the extract command overwrites only the specified region in the file.

mdcode can handle regions in any programming language, the only requirement is that the comment indicating the beginning and end of the region is placed in a separate line containing only the given comment.

Invisible

It is possible to use invisible code blocks. This is useful, for embedding test code or additional files in the markdown document. The invisible code block is also useful if you want to embed the entire file, but only want to display certain parts of it.

A markdown document can contain HTML elements. Unknown or unsupported HTML elements are usually not rendered by markdown renderers. Taking advantage of this, mdcode supports hiding code blocks using the standard <script> HTML element:

<script type="text/markdown">
```js file=sample.js region=factorial

```
</script>

Unfortunately, the GitHub markdown renderer renders the content of unsupported HTML elements as text. Therefore, mdcode also supports the use of a <script> element surrounded by an HTML comment to hide a code block.

<!--<script type="text/markdown">
```js file=sample.js region=factorial

```
</script>-->

It is important to note that the opening character of the comment and the opening tag of the script element must be placed on the same line. Similarly, the closing tag of the script element and the closing tag of the comment must also be placed on the same line.

Highlighting invisible code block

To highlight markdown within an HTML script element, the Markdown Script Tag extension can be used in Visual Studio Code.

Outline

When using regions, only parts of the source file are embedded in the markdown document. If we want to create a self-contained markdown document, the true value of the outline metadata can be used for this purpose.

In this case, only parts of the source file other than the region comments are embedded in the markdown document (and the empty region comments).

The outline flag is typically used in an invisible code block preceding the visible regions. Since the mdcode extract command processes the code blocks sequentially, the code block marked with an outline first overwrites the file, then the code blocks containing the named regions are inserted in their place.

Here is the invisible outline code block at the beginning of this document:

<!--<script type="text/markdown">
```go file=README_test.go outline=true
package main

import "testing"

// #region function
// #endregion

func Test_factorial(t *testing.T) {
  t.Parallel()

  testvect := []uint64{1, 1, 2, 6, 24, 120, 720, 5040, 40320, 362880, 3628800}

  for idx, expected := range testvect {
    if actual := factorial(uint64(idx)); actual != expected {
      t.Errorf("factorial(%d) should be %d but got %d", idx, expected, actual)
    }
  }
}
```
</script>-->

And here is the code block containing the corresponding region reference:

```go file=README_test.go region=function
func factorial(n uint64) uint64 {
  if n > 1 {
    return n * factorial(n-1)
    }

    return 1
}
```

Development

Tasks

This section contains a description of the tasks performed during development. If you have the xc (Markdown defined task runner) command-line tool, individual tasks can be executed simply by using the xc task-name command.

golangci-lint run

go test -count 1 -race -coverprofile=build/coverage.txt ./...

go tool cover -html=build/coverage.txt

go build -ldflags="-w -s" -o build/mdcode .

goreleaser build --snapshot --clean --single-target -o build/mdcode

go generate

go generate
go test ./...
node --test

rm -rf build

CLI Reference

This chapter contains the reference documentation for the command line interface. In addition to the commands described here, additional help topics are available from the command line. To view them, the topic name must be specified as a subcommand.

Additional help topics:

• mdcode filtering - Pattern based filtering
• mdcode invisible - Invisible code blocks
• mdcode metadata - Code block metadata
• mdcode outline - Embedding the file structure
• mdcode regions - Handling file regions

mdcode

Markdown code block authoring tool

Synopsis

Lists the code blocks (with file metadata) from the markdown document.

The optional argument of the mdcode command is the name of the markdown file. If it is missing, the README.md file in the current directory (if it exists) is processed.

mdcode [flags] [filename]

Flags

  -f, --file strings          file filter (default [?*])
  -h, --help                  help for mdcode
      --json                  generate JSON output
  -l, --lang strings          language filter (default [?*])
  -m, --meta stringToString   metadata filter (default [])
  -o, --output string         output file (default: standard output)

SEE ALSO

• mdcode dump - Dump markdown code blocks
• mdcode extract - Extract markdown code blocks to the file system
• mdcode run - Run shell commands on markdown code blocks
• mdcode update - Update markdown code blocks from the file system

mdcode dump

Dump markdown code blocks

Synopsis

Dump markdown code blocks as tar archive

Creating a tar format archive from code blocks that meet the filtering criteria. By default, it writes to standard output, but it can also be directed to file with the --output flag.

A base directory can be specified with the --dir flag, all files will be created under this directory.

The optional argument of the mdcode dump command is the name of the markdown file. If it is missing, the README.md file in the current directory (if it exists) is processed.

mdcode dump  [flags] [filename]

Flags

  -d, --dir string      base directory name (default ".")
  -h, --help            help for dump
  -o, --output string   output file (default: standard output)
  -q, --quiet           suppress the status output

Global Flags

  -f, --file strings          file filter (default [?*])
  -l, --lang strings          language filter (default [?*])
  -m, --meta stringToString   metadata filter (default [])

SEE ALSO

• mdcode - Markdown code block authoring tool

mdcode extract

Extract markdown code blocks to the file system

Synopsis

Writing all code blocks matching the filter criteria to the file system

The code blocks are written to the file named in the file metadata. The file name is relative to the current directory or to the directory specified with the --dir flag.

The code block may include region metadata, which contains the name of the region. In this case, the code block is written to the appropriate part of the file marked with the #region comment.

The optional argument of the mdcode extract command is the name of the markdown file. If it is missing, the README.md file in the current directory (if it exists) is processed.

mdcode extract [flags] [filename]

Flags

  -d, --dir string   base directory name (default ".")
  -h, --help         help for extract
  -q, --quiet        suppress the status output

Global Flags

  -f, --file strings          file filter (default [?*])
  -l, --lang strings          language filter (default [?*])
  -m, --meta stringToString   metadata filter (default [])

SEE ALSO

• mdcode - Markdown code block authoring tool

mdcode run

Run shell commands on markdown code blocks

Synopsis

Extract code blocks to the file system and run shell commands on them

The code blocks are written to the file named in the file metadata.

The code block may include region metadata, which contains the name of the region. In this case, the code block is written to the appropriate part of the file marked with the #region comment.

The optional argument of the mdcode run command is the name of the markdown file. If it is missing, the README.md file in the current directory (if it exists) is processed.

This can be followed by a double dash (--) and then the shell command line to be executed (even a complex command, such as for).

Alternatively, the commands to be executed can be embedded in a code block in the document. In this case, the language must be sh and it is necessary to name the code block with the metadata name. The name of the code block containing the commands can be specified with the --name flag (if not, the first code block containing the sh language and name metadata will be executed).

Code blocks are extracted to a temporary directory. This directory will be the current directory when running the commands. The temporary directory is deleted after executing the commands (deletion can be prevented by using the --keep flag). Instead of a temporary directory, the name of the directory to be used can be specified with the --dir flag. In this case, of course, the directory is not deleted after executing the commands.

mdcode run [flags] [filename] [-- commands]

Flags

  -d, --dir string    base directory name (default ".")
  -h, --help          help for run
  -k, --keep          don't remove temporary directory
  -n, --name string   code block name contains commands
  -q, --quiet         suppress the status output

Global Flags

  -f, --file strings          file filter (default [?*])
  -l, --lang strings          language filter (default [?*])
  -m, --meta stringToString   metadata filter (default [])

SEE ALSO

• mdcode - Markdown code block authoring tool

mdcode update

Update markdown code blocks from the file system

Synopsis

Update all code blocks that meet the filter criteria from the file system

The code blocks are read from the file named in the file metadata. The file name is relative to the current directory or to the directory specified with the --dir flag.

The code block may include region metadata, which contains the name of the region. In this case, the code block is read from the appropriate part of the file marked with the #region comment.

The optional argument of the mdcode update command is the name of the markdown file. If it is missing, the README.md file in the current directory (if it exists) is processed.

mdcode update [flags] [filename]

Flags

  -d, --dir string   base directory name (default ".")
  -h, --help         help for update
  -q, --quiet        suppress the status output

Global Flags

  -f, --file strings          file filter (default [?*])
  -l, --lang strings          language filter (default [?*])
  -m, --meta stringToString   metadata filter (default [])

SEE ALSO

• mdcode - Markdown code block authoring tool