README
¶
🔍 Lens
Search engine for the distributed web
Lens is an opt-in search engine and data collection tool to aid content discovery of the distributed web. It exposes a simple, minimal API for intelligently indexing and querying content on IPFS.
Features and Usage
Initially integrated with Temporal, Lens will allow users to optionally have the data they upload be searched and indexed and be awarded with RTC for participating in the data collection process. Users can then search for content using a simple-to-use API.
Searching through Lens will be facilitated through Temporal web. Optionally, we will have a service independent from Temporal which users can submit content to have it be indexed. This however, is not compensated with RTC. In order to receive the RTC, you must participate through Lens indexing within the Temporal web interface.
API
Lens exposes a simple API via gRPC. The definitions are in
RTradeLtd/grpc.
The Lens API, summarized, currently consists of three core RPCs:
service LensV2 {
rpc Index(IndexReq) returns (IndexResp) {}
rpc Search(SearchReq) returns (SearchResp) {}
rpc Remove(RemoveReq) returns (RemoveResp) {}
}
Golang bindings for the Lens API can be found in
RTradeLtd/grpc.
Supported Formats
Only IPFS CIDs are supported, and they must be plaintext files. We attempt to determine the content type via mime type sniffing, and use that to determine whether or not we can analyze the content.
Please see the following table for supported content types that we can index.
Note if the type is listed as <type>/* it means that any "sub type" of that
mime type is supported.
| Mime Type | Support Level | Tested Types |
|---|---|---|
text/* |
Beta | text/plain, text/html |
image/* |
Beta | image/jpeg |
application/pdf |
Beta | application/pdf |
Deployment
The recommended way to deploy a Lens instance is via the
rtradetech/lens
Docker image.
$> docker pull rtradetech/lens:latest
A docker-compose configuration
is available that also starts up other prerequisites:
$> wget -O lens.yml https://raw.githubusercontent.com/RTradeLtd/Lens/master/lens.yml
$> LENS=latest BASE=/my/dir docker-compose -f lens.yml up
Development
This project requires:
To fetch the codebase, use go get:
$> go get github.com/RTradeLtd/Lens
A rudimentary Makefile target make dep
is available for installing the required dependencies.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type V2 ¶ added in v0.2.0
type V2 struct {
// contains filtered or unexported fields
}
V2 is the new Lens API, and implements the LensV2 gRPC interface directly.
func NewV2 ¶ added in v0.2.0
func NewV2( opts V2Options, ipfs rtfs.Manager, ia images.TensorflowAnalyzer, logger *zap.SugaredLogger, ) (*V2, error)
NewV2 instantiates a new V2 API
func NewV2WithEngine ¶ added in v0.2.0
func NewV2WithEngine( opts V2Options, ipfs rtfs.Manager, ia images.TensorflowAnalyzer, se engine.Searcher, logger *zap.SugaredLogger, ) *V2
NewV2WithEngine instantiates a Lens V2 service with the given engine
Source Files
Directories