Denoflow
Table of Contents
About
Deno script based yaml
workflow file.
WIP. Use with care.
Ideal result is having a apps schemas, with a web gui to generate yaml configs.
Now we can write yaml by ourself, and actually, it's not that hard.
Getting Started
mkdir workflows
Fetch Example
Fetch from Hacker News API to webhook.
touch workflows/fetch.yml
sources:
- use: fetch
args:
- https://test.owenyoung.com/slim.json
run: return ctx.result.json()
itemsPath: hits
key: objectID
format: return ctx.item.title
limit: 1
steps:
- use: fetch
args:
- https://enyvb91j5zjv9.x.pipedream.net/
- method: POST
headers:
'Content-Type': 'application/json'
body: ${{JSON.stringify(ctx.item)}}
Open: https://requestbin.com/r/enyvb91j5zjv9/23eNPamD4DK4YK1rfEB1FAQOKIj , See live webhook request.
deno run --allow-read --allow-net --allow-write --allow-env --allow-run --unstable https://denopkg.com/denoflow/denoflow@main/cli.ts run
It will scan the
workflows
directory and run all valid.yml
files.
RSS Feed to Discord Webhook Message
touch workflows/rss.yml
sources:
- use: fetch
args:
- https://actionsflow.github.io/test-page/hn-rss.xml
run: |
const rss = await import("https://deno.land/x/rss/mod.ts");
const xml = await ctx.result.text();
const feed = await rss.parseFeed(xml);
return feed.entries;
steps:
- use: fetch
args:
- <your discord webhook url>
- method: POST
headers:
'Content-Type': 'application/json'
body: ${{ JSON.stringify({content:ctx.item.title.value}) }}
deno run --allow-read --allow-net --allow-write --allow-run --allow-env --unstable https://denopkg.com/denoflow/denoflow@main/cli.ts run
Life Cycle
on?
: when to run the workflow,Record<EventType,EventConfig>
, can beschedule
orhttp
,always
, default isalways
.sources?
: where to fetch the data,Source[]
, can be one or more sources. Every source should return an array of items.from
?: import ts/js script fromurl
orfile path
use
?: runmoduleName
from abovefrom
, or iffrom
is not provided, runglobalFunction
likefetch
,args
will be passed to the function, the return value will be attached toctx.result
andctx.sources[index].result
run
?: run ts/js code, you can handleuse
result here. Return a result that can be stringified to json. The return value will be attached toctx.result
andctx.sources[index].result
itemsPath
?: the path to the items in the result, likehits
inhttps://test.owenyoung.com/slim.json
key
?: the key to identify the item, likeobjectID
inhttps://test.owenyoung.com/slim.json
, if not provided, will useid
, denoflow will hash the id, then the same item withid
will be skipped.limit
,number
limit the number of items of this source.format
,string
, every item will be callformat
functioncmd
?:string
, exec a shell command after all other task, the return value will be attached toctx.cmdResult
andctx.sources[index].cmdResult
filter
? filter from all sources items, expected return a new items array. The result will be attached to thectx.items
from
?: import ts/js script fromurl
orfile path
use
?: runmoduleName
from abovefrom
, or iffrom
is not provided, runglobalFunction
likefetch
,args
will be passed to the function, the return value will be attached toctx.result
andfilter.result
run
?: run ts/js code, you can handleuse
result here. Return a result that can be stringified to json. the return value will be attached toctx.result
andctx.filter.result
limit
?, limit the number of itemscmd
?:string
, exec a shell command after all other task, the return value will be attached toctx.cmdResult
andfilter.cmdResult
steps
? the steps to run,Step[]
, can be one or more steps.from
?: import script fromurl
orfile path
use
?: runmoduleName
from abovefrom
, or iffrom
is not provided, runglobalFunction
likefetch
,args
will be passed to the functionrun
?: run ts/js code, you can handleuse
result here. Return a result that can be stringified to json. the result will be attached to thectx.steps[index].result
cmd
?: exec shell commands, will be run afterrun
, the result will be attached to thectx.steps[index].cmdResult
### Prerequisites
Install [Deno](https://deno.land/#installation) first.
### Installing
```bash
deno install -n denoflow --allow-read --allow-net --allow-write --allow-env https://denopkg.com/denoflow/denoflow@main/cli.ts
Then, you can run it with denoflow run
, or denoflow run <files>
Usage
Usage:
$ denoflow run [...files]
Options:
--force Force run workflow files (default: false)
--max-items max items for workflow every runs
-h, --help Display this message
YAML Syntax
You can use ${{variable}}
in any fields to inject variables into your workflow, we inject ctx
variable in template and script. For example:
Expressions
steps:
- if: ${{ctx.items.lengh>10}}
run: console.log(ctx.item);
All ctx
see [Context] in the following doc.
State
You can simply use ctx.state
to get or set state, for example:
let currentState = ctx.state || {};
let sent = ctx.state.sent || [];
if(sent.includes(ctx.item.id)){
sent.push(ctx.item.id);
}
ctx.state = {
sent
};
// deno flow will save the state for you , next you can read it.
return;
The state will be saved to data
folder in json
format. If needed, we'll add sqlite or postgres etc as the key value database.
Syntax
All workflow syntax:
// WorkflowOptions File Structure
export interface WorkflowOptions {
general?: GeneralOptions;
env?: Record<string, string | undefined>;
// default: always
on?: Record<EventType, EventOptions>;
sources?: SourceOptions[];
filter?: FilterOptions;
steps?: StepOptions[];
}
// general: General Options
export interface GeneralOptions {
sleep?: string | number;
debug?: boolean;
}
// on: Event Options
type EventOptions = ScheduleOptions | HttpOptions;
enum EventType {
Schedule = "schedule",
Http = "http",
Always = "always", // default
}
// sources: Source Options
export interface SourceOptions extends FilterOptions {
itemsPath?: string;
limit?: number;
key?: string;
force?: boolean;
format?: string;
}
// filter: FilterOptions Options
export interface FilterOptions extends StepOptions {
limit?: number;
}
// step: StepOptionss Options
export interface StepOptions extends GeneralOptions {
id?: string;
from?: string;
use?: string;
args?: unknown[];
run?: string;
if?: string | boolean;
env?: Record<string, string | undefined>;
// run shell command
cmd?: string;
continueOnError?: boolean;
}
export interface StepResponse {
result: unknown;
ok: boolean;
isRealOk: boolean;
error?: unknown;
cmdResult?: string;
cmdCode?: number;
cmdOk?: boolean;
cmdError?: string;
}
// ctx: all ctx you may need
export interface PublicContext {
env: Record<string, string | undefined>; // env vars
cwd: string; // current working directory
workflowPath: string; // workflowfile absolute path
workflowRelativePath: string; // workflow file path relative to cwd
workflowCwd: string; // workflow cwd, absolute path
options?: GeneralOptions; // workflow general options, formated by getDefaultWorkflowOptionsOptions
result?: unknown; // last step result
error?: unknown; // last step error
ok?: boolean; // last step state, true if no error
isRealOk?: boolean; // last step real state, true if no error, when continueOnError is true, and step is error, it will be false, but ok will be true
state: unknown; // workflow state , write/read, change this value, can be persisted
items: unknown[]; // sources/filter result items
item?: unknown; // current item that being step handled
itemIndex?: number; // current item index that being step handled
itemKey?: string; // current item unique key that being step handled
sourceIndex?: number; // current source index , used in sources
filter?: StepResponse; // filter result
sources: Record<string | number, StepResponse>; // sources result
steps: Record<string | number, StepResponse>; // steps results
stepIndex?: number; // current step index
cmdResult?: string;
cmdCode?: number;
cmdOk?: boolean;
cmdError?: string;
}
// run workflow options
export interface RunWorkflowOptions extends GeneralOptions {
force?: boolean;
limit?: number;
files?: string[];
}
// schedule options
export interface ScheduleOptions {
every?: string;
}
// http options
export interface HttpOptions {
resStatusCode?: number;
resContentType?: string;
resBody?: string;
}
Todo
-
- Support Sleep Option
-
- Support GUI generated workflow
-
- Support
on
options,schedule
andhttp
- Support