DELETED .deepsource.toml Index: .deepsource.toml ================================================================== --- .deepsource.toml +++ /dev/null @@ -1,8 +0,0 @@ -version = 1 - -[[analyzers]] -name = "go" -enabled = true - - [analyzers.meta] -import_paths = ["github.com/zettelstore/zettelstore"] Index: LICENSE.txt ================================================================== --- LICENSE.txt +++ LICENSE.txt @@ -1,6 +1,6 @@ -Copyright (c) 2020-2021 Detlef Stern +Copyright (c) 2020-present Detlef Stern Licensed under the EUPL Zettelstore is licensed under the European Union Public License, version 1.2 or later (EUPL v. 1.2). The license is available in the official languages of the Index: Makefile ================================================================== --- Makefile +++ Makefile @@ -1,25 +1,31 @@ -## Copyright (c) 2020-2021 Detlef Stern +## Copyright (c) 2020-present Detlef Stern ## -## This file is part of zettelstore. +## This file is part of Zettelstore. ## ## Zettelstore is licensed under the latest version of the EUPL (European Union ## Public License). Please see file LICENSE.txt for your rights and obligations ## under this license. -.PHONY: check build release clean +.PHONY: check relcheck api version build release clean check: - go run tools/build.go check + go run tools/check/check.go + +relcheck: + go run tools/check/check.go -r + +api: + go run tools/testapi/testapi.go version: - @echo $(shell go run tools/build.go version) + @echo $(shell go run tools/build/build.go version) build: - go run tools/build.go build + go run tools/build/build.go build release: - go run tools/build.go release + go run tools/build/build.go release clean: - go run tools/build.go clean + go run tools/clean/clean.go Index: README.md ================================================================== --- README.md +++ README.md @@ -10,11 +10,17 @@ “Zettelstore”. To get an initial impression, take a look at the [manual](https://zettelstore.de/manual/). It is a live example of the zettelstore software, running in read-only mode. + +[Zettelstore Client](https://t73f.de/r/zsc) provides client software to access +Zettelstore via its API more easily, [Zettelstore +Contrib](https://zettelstore.de/contrib) contains contributed software, which +often connects to Zettelstore via its API. Some of the software packages may be +experimental. The software, including the manual, is licensed under the [European Union Public License 1.2 (or later)](https://zettelstore.de/home/file?name=LICENSE.txt&ci=trunk). -[Stay tuned](https://twitter.com/zettelstore)… +[Stay tuned](https://mastodon.social/tags/Zettelstore) … Index: VERSION ================================================================== --- VERSION +++ VERSION @@ -1,1 +1,1 @@ -0.0.13 +0.22.0-dev DELETED ast/ast.go Index: ast/ast.go ================================================================== --- ast/ast.go +++ /dev/null @@ -1,94 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast - -import ( - "net/url" - - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -// ZettelNode is the root node of the abstract syntax tree. -// It is *not* part of the visitor pattern. -type ZettelNode struct { - // Zettel domain.Zettel - Meta *meta.Meta // Original metadata - Content domain.Content // Original content - Zid id.Zid // Zettel identification. - InhMeta *meta.Meta // Metadata of the zettel, with inherited values. - Ast BlockSlice // Zettel abstract syntax tree is a sequence of block nodes. -} - -// Node is the interface, all nodes must implement. -type Node interface { - Accept(v Visitor) -} - -// BlockNode is the interface that all block nodes must implement. -type BlockNode interface { - Node - blockNode() -} - -// BlockSlice is a slice of BlockNodes. -type BlockSlice []BlockNode - -// ItemNode is a node that can occur as a list item. -type ItemNode interface { - BlockNode - itemNode() -} - -// ItemSlice is a slice of ItemNodes. -type ItemSlice []ItemNode - -// DescriptionNode is a node that contains just textual description. -type DescriptionNode interface { - ItemNode - descriptionNode() -} - -// DescriptionSlice is a slice of DescriptionNodes. -type DescriptionSlice []DescriptionNode - -// InlineNode is the interface that all inline nodes must implement. -type InlineNode interface { - Node - inlineNode() -} - -// InlineSlice is a slice of InlineNodes. -type InlineSlice []InlineNode - -// Reference is a reference to external or internal material. -type Reference struct { - URL *url.URL - Value string - State RefState -} - -// RefState indicates the state of the reference. -type RefState int - -// Constants for RefState -const ( - RefStateInvalid RefState = iota // Invalid Reference - RefStateZettel // Reference to an internal zettel - RefStateSelf // Reference to same zettel with a fragment - RefStateFound // Reference to an existing internal zettel - RefStateBroken // Reference to a non-existing internal zettel - RefStateHosted // Reference to local hosted non-Zettel, without URL change - RefStateBased // Reference to local non-Zettel, to be prefixed - RefStateExternal // Reference to external material -) DELETED ast/attr.go Index: ast/attr.go ================================================================== --- ast/attr.go +++ /dev/null @@ -1,103 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast - -import ( - "strings" -) - -// Attributes store additional information about some node types. -type Attributes struct { - Attrs map[string]string -} - -// HasDefault returns true, if the default attribute "-" has been set. -func (a *Attributes) HasDefault() bool { - if a != nil { - _, ok := a.Attrs["-"] - return ok - } - return false -} - -// RemoveDefault removes the default attribute -func (a *Attributes) RemoveDefault() { - a.Remove("-") -} - -// Get returns the attribute value of the given key and a succes value. -func (a *Attributes) Get(key string) (string, bool) { - if a != nil { - value, ok := a.Attrs[key] - return value, ok - } - return "", false -} - -// Clone returns a duplicate of the attribute. -func (a *Attributes) Clone() *Attributes { - if a == nil { - return nil - } - attrs := make(map[string]string, len(a.Attrs)) - for k, v := range a.Attrs { - attrs[k] = v - } - return &Attributes{attrs} -} - -// Set changes the attribute that a given key has now a given value. -func (a *Attributes) Set(key, value string) *Attributes { - if a == nil { - return &Attributes{map[string]string{key: value}} - } - if a.Attrs == nil { - a.Attrs = make(map[string]string) - } - a.Attrs[key] = value - return a -} - -// Remove the key from the attributes. -func (a *Attributes) Remove(key string) { - if a != nil { - delete(a.Attrs, key) - } -} - -// AddClass adds a value to the class attribute. -func (a *Attributes) AddClass(class string) *Attributes { - if a == nil { - return &Attributes{map[string]string{"class": class}} - } - classes := a.GetClasses() - for _, cls := range classes { - if cls == class { - return a - } - } - classes = append(classes, class) - a.Attrs["class"] = strings.Join(classes, " ") - return a -} - -// GetClasses returns the class values as a string slice -func (a *Attributes) GetClasses() []string { - if a == nil { - return nil - } - classes, ok := a.Attrs["class"] - if !ok { - return nil - } - return strings.Fields(classes) -} DELETED ast/attr_test.go Index: ast/attr_test.go ================================================================== --- ast/attr_test.go +++ /dev/null @@ -1,48 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast_test - -import ( - "testing" - - "zettelstore.de/z/ast" -) - -func TestHasDefault(t *testing.T) { - attr := &ast.Attributes{} - if attr.HasDefault() { - t.Error("Should not have default attr") - } - attr = &ast.Attributes{Attrs: map[string]string{"-": "value"}} - if !attr.HasDefault() { - t.Error("Should have default attr") - } -} - -func TestAttrClone(t *testing.T) { - orig := &ast.Attributes{} - clone := orig.Clone() - if len(clone.Attrs) > 0 { - t.Error("Attrs must be empty") - } - - orig = &ast.Attributes{Attrs: map[string]string{"": "0", "-": "1", "a": "b"}} - clone = orig.Clone() - m := clone.Attrs - if m[""] != "0" || m["-"] != "1" || m["a"] != "b" || len(m) != len(orig.Attrs) { - t.Error("Wrong cloned map") - } - m["a"] = "c" - if orig.Attrs["a"] != "b" { - t.Error("Aliased map") - } -} DELETED ast/block.go Index: ast/block.go ================================================================== --- ast/block.go +++ /dev/null @@ -1,204 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast - -// Definition of Block nodes. - -// ParaNode contains just a sequence of inline elements. -// Another name is "paragraph". -type ParaNode struct { - Inlines InlineSlice -} - -func (pn *ParaNode) blockNode() {} -func (pn *ParaNode) itemNode() {} -func (pn *ParaNode) descriptionNode() {} - -// Accept a visitor and visit the node. -func (pn *ParaNode) Accept(v Visitor) { v.VisitPara(pn) } - -//-------------------------------------------------------------------------- - -// VerbatimNode contains lines of uninterpreted text -type VerbatimNode struct { - Code VerbatimCode - Attrs *Attributes - Lines []string -} - -// VerbatimCode specifies the format that is applied to code inline nodes. -type VerbatimCode int - -// Constants for VerbatimCode -const ( - _ VerbatimCode = iota - VerbatimProg // Program code. - VerbatimComment // Block comment - VerbatimHTML // Block HTML, e.g. for Markdown -) - -func (vn *VerbatimNode) blockNode() {} -func (vn *VerbatimNode) itemNode() {} - -// Accept a visitor an visit the node. -func (vn *VerbatimNode) Accept(v Visitor) { v.VisitVerbatim(vn) } - -//-------------------------------------------------------------------------- - -// RegionNode encapsulates a region of block nodes. -type RegionNode struct { - Code RegionCode - Attrs *Attributes - Blocks BlockSlice - Inlines InlineSlice // Additional text at the end of the region -} - -// RegionCode specifies the actual region type. -type RegionCode int - -// Values for RegionCode -const ( - _ RegionCode = iota - RegionSpan // Just a span of blocks - RegionQuote // A longer quotation - RegionVerse // Line breaks matter -) - -func (rn *RegionNode) blockNode() {} -func (rn *RegionNode) itemNode() {} - -// Accept a visitor and visit the node. -func (rn *RegionNode) Accept(v Visitor) { v.VisitRegion(rn) } - -//-------------------------------------------------------------------------- - -// HeadingNode stores the heading text and level. -type HeadingNode struct { - Level int - Inlines InlineSlice // Heading text, possibly formatted - Slug string // Heading text, suitable to be used as an URL fragment - Attrs *Attributes -} - -func (hn *HeadingNode) blockNode() {} -func (hn *HeadingNode) itemNode() {} - -// Accept a visitor and visit the node. -func (hn *HeadingNode) Accept(v Visitor) { v.VisitHeading(hn) } - -//-------------------------------------------------------------------------- - -// HRuleNode specifies a horizontal rule. -type HRuleNode struct { - Attrs *Attributes -} - -func (hn *HRuleNode) blockNode() {} -func (hn *HRuleNode) itemNode() {} - -// Accept a visitor and visit the node. -func (hn *HRuleNode) Accept(v Visitor) { v.VisitHRule(hn) } - -//-------------------------------------------------------------------------- - -// NestedListNode specifies a nestable list, either ordered or unordered. -type NestedListNode struct { - Code NestedListCode - Items []ItemSlice - Attrs *Attributes -} - -// NestedListCode specifies the actual list type. -type NestedListCode int - -// Values for ListCode -const ( - _ NestedListCode = iota - NestedListOrdered // Ordered list. - NestedListUnordered // Unordered list. - NestedListQuote // Quote list. -) - -func (ln *NestedListNode) blockNode() {} -func (ln *NestedListNode) itemNode() {} - -// Accept a visitor and visit the node. -func (ln *NestedListNode) Accept(v Visitor) { v.VisitNestedList(ln) } - -//-------------------------------------------------------------------------- - -// DescriptionListNode specifies a description list. -type DescriptionListNode struct { - Descriptions []Description -} - -// Description is one element of a description list. -type Description struct { - Term InlineSlice - Descriptions []DescriptionSlice -} - -func (dn *DescriptionListNode) blockNode() {} - -// Accept a visitor and visit the node. -func (dn *DescriptionListNode) Accept(v Visitor) { v.VisitDescriptionList(dn) } - -//-------------------------------------------------------------------------- - -// TableNode specifies a full table -type TableNode struct { - Header TableRow // The header row - Align []Alignment // Default column alignment - Rows []TableRow // The slice of cell rows -} - -// TableCell contains the data for one table cell -type TableCell struct { - Align Alignment // Cell alignment - Inlines InlineSlice // Cell content -} - -// TableRow is a slice of cells. -type TableRow []*TableCell - -// Alignment specifies text alignment. -// Currently only for tables. -type Alignment int - -// Constants for Alignment. -const ( - _ Alignment = iota - AlignDefault // Default alignment, inherited - AlignLeft // Left alignment - AlignCenter // Center the content - AlignRight // Right alignment -) - -func (tn *TableNode) blockNode() {} - -// Accept a visitor and visit the node. -func (tn *TableNode) Accept(v Visitor) { v.VisitTable(tn) } - -//-------------------------------------------------------------------------- - -// BLOBNode contains just binary data that must be interpreted according to -// a syntax. -type BLOBNode struct { - Title string - Syntax string - Blob []byte -} - -func (bn *BLOBNode) blockNode() {} - -// Accept a visitor and visit the node. -func (bn *BLOBNode) Accept(v Visitor) { v.VisitBLOB(bn) } DELETED ast/inline.go Index: ast/inline.go ================================================================== --- ast/inline.go +++ /dev/null @@ -1,196 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast - -// Definitions of inline nodes. - -// TextNode just contains some text. -type TextNode struct { - Text string // The text itself. -} - -func (tn *TextNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (tn *TextNode) Accept(v Visitor) { v.VisitText(tn) } - -// -------------------------------------------------------------------------- - -// TagNode contains a tag. -type TagNode struct { - Tag string // The text itself. -} - -func (tn *TagNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (tn *TagNode) Accept(v Visitor) { v.VisitTag(tn) } - -// -------------------------------------------------------------------------- - -// SpaceNode tracks inter-word space characters. -type SpaceNode struct { - Lexeme string -} - -func (sn *SpaceNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (sn *SpaceNode) Accept(v Visitor) { v.VisitSpace(sn) } - -// -------------------------------------------------------------------------- - -// BreakNode signals a new line that must / should be interpreted as a new line break. -type BreakNode struct { - Hard bool // Hard line break? -} - -func (bn *BreakNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (bn *BreakNode) Accept(v Visitor) { v.VisitBreak(bn) } - -// -------------------------------------------------------------------------- - -// LinkNode contains the specified link. -type LinkNode struct { - Ref *Reference - Inlines InlineSlice // The text associated with the link. - OnlyRef bool // True if no text was specified. - Attrs *Attributes // Optional attributes -} - -func (ln *LinkNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (ln *LinkNode) Accept(v Visitor) { v.VisitLink(ln) } - -// -------------------------------------------------------------------------- - -// ImageNode contains the specified image reference. -type ImageNode struct { - Ref *Reference // Reference to image - Blob []byte // BLOB data of the image, as an alternative to Ref. - Syntax string // Syntax of Blob - Inlines InlineSlice // The text associated with the image. - Attrs *Attributes // Optional attributes -} - -func (in *ImageNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (in *ImageNode) Accept(v Visitor) { v.VisitImage(in) } - -// -------------------------------------------------------------------------- - -// CiteNode contains the specified citation. -type CiteNode struct { - Key string // The citation key - Inlines InlineSlice // The text associated with the citation. - Attrs *Attributes // Optional attributes -} - -func (cn *CiteNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (cn *CiteNode) Accept(v Visitor) { v.VisitCite(cn) } - -// -------------------------------------------------------------------------- - -// MarkNode contains the specified merked position. -// It is a BlockNode too, because although it is typically parsed during inline -// mode, it is moved into block mode afterwards. -type MarkNode struct { - Text string -} - -func (mn *MarkNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (mn *MarkNode) Accept(v Visitor) { v.VisitMark(mn) } - -// -------------------------------------------------------------------------- - -// FootnoteNode contains the specified footnote. -type FootnoteNode struct { - Inlines InlineSlice // The footnote text. - Attrs *Attributes // Optional attributes -} - -func (fn *FootnoteNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (fn *FootnoteNode) Accept(v Visitor) { v.VisitFootnote(fn) } - -// -------------------------------------------------------------------------- - -// FormatNode specifies some inline formatting. -type FormatNode struct { - Code FormatCode - Attrs *Attributes // Optional attributes. - Inlines InlineSlice -} - -// FormatCode specifies the format that is applied to the inline nodes. -type FormatCode int - -// Constants for FormatCode -const ( - _ FormatCode = iota - FormatItalic // Italic text. - FormatEmph // Semantically emphasized text. - FormatBold // Bold text. - FormatStrong // Semantically strongly emphasized text. - FormatUnder // Underlined text. - FormatInsert // Inserted text. - FormatStrike // Text that is no longer relevant or no longer accurate. - FormatDelete // Deleted text. - FormatSuper // Superscripted text. - FormatSub // SubscriptedText. - FormatQuote // Quoted text. - FormatQuotation // Quotation text. - FormatSmall // Smaller text. - FormatSpan // Generic inline container. - FormatMonospace // Monospaced text. -) - -func (fn *FormatNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (fn *FormatNode) Accept(v Visitor) { v.VisitFormat(fn) } - -// -------------------------------------------------------------------------- - -// LiteralNode specifies some uninterpreted text. -type LiteralNode struct { - Code LiteralCode - Attrs *Attributes // Optional attributes. - Text string -} - -// LiteralCode specifies the format that is applied to code inline nodes. -type LiteralCode int - -// Constants for LiteralCode -const ( - _ LiteralCode = iota - LiteralProg // Inline program code. - LiteralKeyb // Keyboard strokes. - LiteralOutput // Sample output. - LiteralComment // Inline comment - LiteralHTML // Inline HTML, e.g. for Markdown -) - -func (rn *LiteralNode) inlineNode() {} - -// Accept a visitor and visit the node. -func (rn *LiteralNode) Accept(v Visitor) { v.VisitLiteral(rn) } DELETED ast/ref.go Index: ast/ref.go ================================================================== --- ast/ref.go +++ /dev/null @@ -1,92 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast - -import ( - "net/url" - - "zettelstore.de/z/domain/id" -) - -// ParseReference parses a string and returns a reference. -func ParseReference(s string) *Reference { - switch s { - case "", "00000000000000": - return &Reference{URL: nil, Value: s, State: RefStateInvalid} - } - if state, ok := localState(s); ok { - if state == RefStateBased { - s = s[1:] - } - u, err := url.Parse(s) - if err == nil { - return &Reference{URL: u, Value: s, State: state} - } - } - u, err := url.Parse(s) - if err != nil { - return &Reference{URL: nil, Value: s, State: RefStateInvalid} - } - if len(u.Scheme)+len(u.Opaque)+len(u.Host) == 0 && u.User == nil { - if _, err := id.Parse(u.Path); err == nil { - return &Reference{URL: u, Value: s, State: RefStateZettel} - } - if u.Path == "" && u.Fragment != "" { - return &Reference{URL: u, Value: s, State: RefStateSelf} - } - } - return &Reference{URL: u, Value: s, State: RefStateExternal} -} - -func localState(path string) (RefState, bool) { - if len(path) > 0 && path[0] == '/' { - if len(path) > 1 && path[1] == '/' { - return RefStateBased, true - } - return RefStateHosted, true - } - if len(path) > 1 && path[0] == '.' { - if len(path) > 2 && path[1] == '.' && path[2] == '/' { - return RefStateHosted, true - } - return RefStateHosted, path[1] == '/' - } - return RefStateInvalid, false -} - -// String returns the string representation of a reference. -func (r Reference) String() string { - if r.URL != nil { - return r.URL.String() - } - return r.Value -} - -// IsValid returns true if reference is valid -func (r *Reference) IsValid() bool { return r.State != RefStateInvalid } - -// IsZettel returns true if it is a referencen to a local zettel. -func (r *Reference) IsZettel() bool { - switch r.State { - case RefStateZettel, RefStateSelf, RefStateFound, RefStateBroken: - return true - } - return false -} - -// IsLocal returns true if reference is local -func (r *Reference) IsLocal() bool { - return r.State == RefStateHosted || r.State == RefStateBased -} - -// IsExternal returns true if it is a referencen to external material. -func (r *Reference) IsExternal() bool { return r.State == RefStateExternal } DELETED ast/ref_test.go Index: ast/ref_test.go ================================================================== --- ast/ref_test.go +++ /dev/null @@ -1,94 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast_test provides the tests for the abstract syntax tree. -package ast_test - -import ( - "testing" - - "zettelstore.de/z/ast" -) - -func TestParseReference(t *testing.T) { - testcases := []struct { - link string - err bool - exp string - }{ - {"", true, ""}, - {"123", false, "123"}, - {",://", true, ""}, - } - - for i, tc := range testcases { - got := ast.ParseReference(tc.link) - if got.IsValid() == tc.err { - t.Errorf( - "TC=%d, expected parse error of %q: %v, but got %q", i, tc.link, tc.err, got) - } - if got.IsValid() && got.String() != tc.exp { - t.Errorf("TC=%d, Reference of %q is %q, but got %q", i, tc.link, tc.exp, got) - } - } -} - -func TestReferenceIsZettelMaterial(t *testing.T) { - testcases := []struct { - link string - isZettel bool - isExternal bool - isLocal bool - }{ - {"", false, false, false}, - {"00000000000000", false, false, false}, - {"http://zettelstore.de/z/ast", false, true, false}, - {"12345678901234", true, false, false}, - {"12345678901234#local", true, false, false}, - {"http://12345678901234", false, true, false}, - {"http://zettelstore.de/z/12345678901234", false, true, false}, - {"http://zettelstore.de/12345678901234", false, true, false}, - {"/12345678901234", false, false, true}, - {"//12345678901234", false, false, true}, - {"./12345678901234", false, false, true}, - {"../12345678901234", false, false, true}, - {".../12345678901234", false, true, false}, - } - - for i, tc := range testcases { - ref := ast.ParseReference(tc.link) - isZettel := ref.IsZettel() - if isZettel != tc.isZettel { - t.Errorf( - "TC=%d, Reference %q isZettel=%v expected, but got %v", - i, - tc.link, - tc.isZettel, - isZettel) - } - isLocal := ref.IsLocal() - if isLocal != tc.isLocal { - t.Errorf( - "TC=%d, Reference %q isLocal=%v expected, but got %v", - i, - tc.link, - tc.isLocal, isLocal) - } - isExternal := ref.IsExternal() - if isExternal != tc.isExternal { - t.Errorf( - "TC=%d, Reference %q isExternal=%v expected, but got %v", - i, - tc.link, - tc.isExternal, - isExternal) - } - } -} DELETED ast/traverser.go Index: ast/traverser.go ================================================================== --- ast/traverser.go +++ /dev/null @@ -1,161 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast - -// A traverser is a Visitor that just traverses the AST and delegates node -// spacific actions to a Visitor. This Visitor should not traverse the AST. - -// TopDownTraverser visits first the node and then the children nodes. -type TopDownTraverser struct { - v Visitor -} - -// NewTopDownTraverser creates a new traverser. -func NewTopDownTraverser(visitor Visitor) TopDownTraverser { - return TopDownTraverser{visitor} -} - -// VisitVerbatim has nothing to traverse. -func (t TopDownTraverser) VisitVerbatim(vn *VerbatimNode) { t.v.VisitVerbatim(vn) } - -// VisitRegion traverses the content and the additional text. -func (t TopDownTraverser) VisitRegion(rn *RegionNode) { - t.v.VisitRegion(rn) - t.VisitBlockSlice(rn.Blocks) - t.VisitInlineSlice(rn.Inlines) -} - -// VisitHeading traverses the heading. -func (t TopDownTraverser) VisitHeading(hn *HeadingNode) { - t.v.VisitHeading(hn) - t.VisitInlineSlice(hn.Inlines) -} - -// VisitHRule traverses nothing. -func (t TopDownTraverser) VisitHRule(hn *HRuleNode) { t.v.VisitHRule(hn) } - -// VisitNestedList traverses all nested list elements. -func (t TopDownTraverser) VisitNestedList(ln *NestedListNode) { - t.v.VisitNestedList(ln) - for _, item := range ln.Items { - t.visitItemSlice(item) - } -} - -// VisitDescriptionList traverses all description terms and their associated -// descriptions. -func (t TopDownTraverser) VisitDescriptionList(dn *DescriptionListNode) { - t.v.VisitDescriptionList(dn) - for _, defs := range dn.Descriptions { - t.VisitInlineSlice(defs.Term) - for _, descr := range defs.Descriptions { - t.visitDescriptionSlice(descr) - } - } -} - -// VisitPara traverses the inlines of a paragraph. -func (t TopDownTraverser) VisitPara(pn *ParaNode) { - t.v.VisitPara(pn) - t.VisitInlineSlice(pn.Inlines) -} - -// VisitTable traverses all cells of the header and then row-wise all cells of -// the table body. -func (t TopDownTraverser) VisitTable(tn *TableNode) { - t.v.VisitTable(tn) - for _, col := range tn.Header { - t.VisitInlineSlice(col.Inlines) - } - for _, row := range tn.Rows { - for _, col := range row { - t.VisitInlineSlice(col.Inlines) - } - } -} - -// VisitBLOB traverses nothing. -func (t TopDownTraverser) VisitBLOB(bn *BLOBNode) { t.v.VisitBLOB(bn) } - -// VisitText traverses nothing. -func (t TopDownTraverser) VisitText(tn *TextNode) { t.v.VisitText(tn) } - -// VisitTag traverses nothing. -func (t TopDownTraverser) VisitTag(tn *TagNode) { t.v.VisitTag(tn) } - -// VisitSpace traverses nothing. -func (t TopDownTraverser) VisitSpace(sn *SpaceNode) { t.v.VisitSpace(sn) } - -// VisitBreak traverses nothing. -func (t TopDownTraverser) VisitBreak(bn *BreakNode) { t.v.VisitBreak(bn) } - -// VisitLink traverses the link text. -func (t TopDownTraverser) VisitLink(ln *LinkNode) { - t.v.VisitLink(ln) - t.VisitInlineSlice(ln.Inlines) -} - -// VisitImage traverses the image text. -func (t TopDownTraverser) VisitImage(in *ImageNode) { - t.v.VisitImage(in) - t.VisitInlineSlice(in.Inlines) -} - -// VisitCite traverses the cite text. -func (t TopDownTraverser) VisitCite(cn *CiteNode) { - t.v.VisitCite(cn) - t.VisitInlineSlice(cn.Inlines) -} - -// VisitFootnote traverses the footnote text. -func (t TopDownTraverser) VisitFootnote(fn *FootnoteNode) { - t.v.VisitFootnote(fn) - t.VisitInlineSlice(fn.Inlines) -} - -// VisitMark traverses nothing. -func (t TopDownTraverser) VisitMark(mn *MarkNode) { t.v.VisitMark(mn) } - -// VisitFormat traverses the formatted text. -func (t TopDownTraverser) VisitFormat(fn *FormatNode) { - t.v.VisitFormat(fn) - t.VisitInlineSlice(fn.Inlines) -} - -// VisitLiteral traverses nothing. -func (t TopDownTraverser) VisitLiteral(ln *LiteralNode) { t.v.VisitLiteral(ln) } - -// VisitBlockSlice traverses a block slice. -func (t TopDownTraverser) VisitBlockSlice(bns BlockSlice) { - for _, bn := range bns { - bn.Accept(t) - } -} - -func (t TopDownTraverser) visitItemSlice(ins ItemSlice) { - for _, in := range ins { - in.Accept(t) - } -} - -func (t TopDownTraverser) visitDescriptionSlice(dns DescriptionSlice) { - for _, dn := range dns { - dn.Accept(t) - } -} - -// VisitInlineSlice traverses a block slice. -func (t TopDownTraverser) VisitInlineSlice(ins InlineSlice) { - for _, in := range ins { - in.Accept(t) - } -} DELETED ast/visitor.go Index: ast/visitor.go ================================================================== --- ast/visitor.go +++ /dev/null @@ -1,39 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package ast provides the abstract syntax tree. -package ast - -// Visitor is the interface all visitors must implement. -type Visitor interface { - // Block nodes - VisitVerbatim(vn *VerbatimNode) - VisitRegion(rn *RegionNode) - VisitHeading(hn *HeadingNode) - VisitHRule(hn *HRuleNode) - VisitNestedList(ln *NestedListNode) - VisitDescriptionList(dn *DescriptionListNode) - VisitPara(pn *ParaNode) - VisitTable(tn *TableNode) - VisitBLOB(bn *BLOBNode) - - // Inline nodes - VisitText(tn *TextNode) - VisitTag(tn *TagNode) - VisitSpace(sn *SpaceNode) - VisitBreak(bn *BreakNode) - VisitLink(ln *LinkNode) - VisitImage(in *ImageNode) - VisitCite(cn *CiteNode) - VisitFootnote(fn *FootnoteNode) - VisitMark(mn *MarkNode) - VisitFormat(fn *FormatNode) - VisitLiteral(ln *LiteralNode) -} DELETED auth/auth.go Index: auth/auth.go ================================================================== --- auth/auth.go +++ /dev/null @@ -1,101 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package auth provides services for authentification / authorization. -package auth - -import ( - "time" - - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/web/server" -) - -// BaseManager allows to check some base auth modes. -type BaseManager interface { - // IsReadonly returns true, if the systems is configured to run in read-only-mode. - IsReadonly() bool -} - -// TokenManager provides methods to create authentication -type TokenManager interface { - - // GetToken produces a authentication token. - GetToken(ident *meta.Meta, d time.Duration, kind TokenKind) ([]byte, error) - - // CheckToken checks the validity of the token and returns relevant data. - CheckToken(token []byte, k TokenKind) (TokenData, error) -} - -// TokenKind specifies for which application / usage a token is/was requested. -type TokenKind int - -// Allowed values of token kind -const ( - _ TokenKind = iota - KindJSON - KindHTML -) - -// TokenData contains some important elements from a token. -type TokenData struct { - Token []byte - Now time.Time - Issued time.Time - Expires time.Time - Ident string - Zid id.Zid -} - -// AuthzManager provides methods for authorization. -type AuthzManager interface { - BaseManager - - // Owner returns the zettel identifier of the owner. - Owner() id.Zid - - // IsOwner returns true, if the given zettel identifier is that of the owner. - IsOwner(zid id.Zid) bool - - // Returns true if authentication is enabled. - WithAuth() bool - - // GetUserRole role returns the user role of the given user zettel. - GetUserRole(user *meta.Meta) meta.UserRole -} - -// Manager is the main interface for providing the service. -type Manager interface { - TokenManager - AuthzManager - - PlaceWithPolicy(auth server.Auth, unprotectedPlace place.Place, rtConfig config.Config) (place.Place, Policy) -} - -// Policy is an interface for checking access authorization. -type Policy interface { - // User is allowed to create a new zettel. - CanCreate(user, newMeta *meta.Meta) bool - - // User is allowed to read zettel - CanRead(user, m *meta.Meta) bool - - // User is allowed to write zettel. - CanWrite(user, oldMeta, newMeta *meta.Meta) bool - - // User is allowed to rename zettel - CanRename(user, m *meta.Meta) bool - - // User is allowed to delete zettel - CanDelete(user, m *meta.Meta) bool -} DELETED auth/cred/cred.go Index: auth/cred/cred.go ================================================================== --- auth/cred/cred.go +++ /dev/null @@ -1,53 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package cred provides some function for handling credentials. -package cred - -import ( - "bytes" - - "golang.org/x/crypto/bcrypt" - "zettelstore.de/z/domain/id" -) - -// HashCredential returns a hashed vesion of the given credential -func HashCredential(zid id.Zid, ident, credential string) (string, error) { - fullCredential := createFullCredential(zid, ident, credential) - res, err := bcrypt.GenerateFromPassword(fullCredential, bcrypt.DefaultCost) - if err != nil { - return "", err - } - return string(res), nil -} - -// CompareHashAndCredential checks, whether the hashed credential is a possible -// value when hashing the credential. -func CompareHashAndCredential(hashed string, zid id.Zid, ident, credential string) (bool, error) { - fullCredential := createFullCredential(zid, ident, credential) - err := bcrypt.CompareHashAndPassword([]byte(hashed), fullCredential) - if err == nil { - return true, nil - } - if err == bcrypt.ErrMismatchedHashAndPassword { - return false, nil - } - return false, err -} - -func createFullCredential(zid id.Zid, ident, credential string) []byte { - var buf bytes.Buffer - buf.WriteString(zid.String()) - buf.WriteByte(' ') - buf.WriteString(ident) - buf.WriteByte(' ') - buf.WriteString(credential) - return buf.Bytes() -} DELETED auth/impl/impl.go Index: auth/impl/impl.go ================================================================== --- auth/impl/impl.go +++ /dev/null @@ -1,184 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package impl provides services for authentification / authorization. -package impl - -import ( - "errors" - "hash/fnv" - "io" - "time" - - "github.com/pascaldekloe/jwt" - - "zettelstore.de/z/auth" - "zettelstore.de/z/auth/policy" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/kernel" - "zettelstore.de/z/place" - "zettelstore.de/z/web/server" -) - -type myAuth struct { - readonly bool - owner id.Zid - secret []byte -} - -// New creates a new auth object. -func New(readonly bool, owner id.Zid, extSecret string) auth.Manager { - return &myAuth{ - readonly: readonly, - owner: owner, - secret: calcSecret(extSecret), - } -} - -var configKeys = []string{ - kernel.CoreProgname, - kernel.CoreGoVersion, - kernel.CoreHostname, - kernel.CoreGoOS, - kernel.CoreGoArch, - kernel.CoreVersion, -} - -func calcSecret(extSecret string) []byte { - h := fnv.New128() - if extSecret != "" { - io.WriteString(h, extSecret) - } - for _, key := range configKeys { - io.WriteString(h, kernel.Main.GetConfig(kernel.CoreService, key).(string)) - } - return h.Sum(nil) -} - -// IsReadonly returns true, if the systems is configured to run in read-only-mode. -func (a *myAuth) IsReadonly() bool { return a.readonly } - -const reqHash = jwt.HS512 - -// ErrNoUser signals that the meta data has no role value 'user'. -var ErrNoUser = errors.New("auth: meta is no user") - -// ErrNoIdent signals that the 'ident' key is missing. -var ErrNoIdent = errors.New("auth: missing ident") - -// ErrOtherKind signals that the token was defined for another token kind. -var ErrOtherKind = errors.New("auth: wrong token kind") - -// ErrNoZid signals that the 'zid' key is missing. -var ErrNoZid = errors.New("auth: missing zettel id") - -// GetToken returns a token to be used for authentification. -func (a *myAuth) GetToken(ident *meta.Meta, d time.Duration, kind auth.TokenKind) ([]byte, error) { - if role, ok := ident.Get(meta.KeyRole); !ok || role != meta.ValueRoleUser { - return nil, ErrNoUser - } - subject, ok := ident.Get(meta.KeyUserID) - if !ok || subject == "" { - return nil, ErrNoIdent - } - - now := time.Now().Round(time.Second) - claims := jwt.Claims{ - Registered: jwt.Registered{ - Subject: subject, - Expires: jwt.NewNumericTime(now.Add(d)), - Issued: jwt.NewNumericTime(now), - }, - Set: map[string]interface{}{ - "zid": ident.Zid.String(), - "_tk": int(kind), - }, - } - token, err := claims.HMACSign(reqHash, a.secret) - if err != nil { - return nil, err - } - return token, nil -} - -// ErrTokenExpired signals an exired token -var ErrTokenExpired = errors.New("auth: token expired") - -// CheckToken checks the validity of the token and returns relevant data. -func (a *myAuth) CheckToken(token []byte, k auth.TokenKind) (auth.TokenData, error) { - h, err := jwt.NewHMAC(reqHash, a.secret) - if err != nil { - return auth.TokenData{}, err - } - claims, err := h.Check(token) - if err != nil { - return auth.TokenData{}, err - } - now := time.Now().Round(time.Second) - expires := claims.Expires.Time() - if expires.Before(now) { - return auth.TokenData{}, ErrTokenExpired - } - ident := claims.Subject - if ident == "" { - return auth.TokenData{}, ErrNoIdent - } - if zidS, ok := claims.Set["zid"].(string); ok { - if zid, err := id.Parse(zidS); err == nil { - if kind, ok := claims.Set["_tk"].(float64); ok { - if auth.TokenKind(kind) == k { - return auth.TokenData{ - Token: token, - Now: now, - Issued: claims.Issued.Time(), - Expires: expires, - Ident: ident, - Zid: zid, - }, nil - } - } - return auth.TokenData{}, ErrOtherKind - } - } - return auth.TokenData{}, ErrNoZid -} - -func (a *myAuth) Owner() id.Zid { return a.owner } - -func (a *myAuth) IsOwner(zid id.Zid) bool { - return zid.IsValid() && zid == a.owner -} - -func (a *myAuth) WithAuth() bool { return a.owner != id.Invalid } - -// GetUserRole role returns the user role of the given user zettel. -func (a *myAuth) GetUserRole(user *meta.Meta) meta.UserRole { - if user == nil { - if a.WithAuth() { - return meta.UserRoleUnknown - } - return meta.UserRoleOwner - } - if a.IsOwner(user.Zid) { - return meta.UserRoleOwner - } - if val, ok := user.Get(meta.KeyUserRole); ok { - if ur := meta.GetUserRole(val); ur != meta.UserRoleUnknown { - return ur - } - } - return meta.UserRoleReader -} - -func (a *myAuth) PlaceWithPolicy(auth server.Auth, unprotectedPlace place.Place, rtConfig config.Config) (place.Place, auth.Policy) { - return policy.PlaceWithPolicy(auth, a, unprotectedPlace, rtConfig) -} DELETED auth/policy/anon.go Index: auth/policy/anon.go ================================================================== --- auth/policy/anon.go +++ /dev/null @@ -1,50 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package policy provides some interfaces and implementation for authorization policies. -package policy - -import ( - "zettelstore.de/z/auth" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/meta" -) - -type anonPolicy struct { - authConfig config.AuthConfig - pre auth.Policy -} - -func (ap *anonPolicy) CanCreate(user, newMeta *meta.Meta) bool { - return ap.pre.CanCreate(user, newMeta) -} - -func (ap *anonPolicy) CanRead(user, m *meta.Meta) bool { - return ap.pre.CanRead(user, m) && ap.checkVisibility(m) -} - -func (ap *anonPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool { - return ap.pre.CanWrite(user, oldMeta, newMeta) && ap.checkVisibility(oldMeta) -} - -func (ap *anonPolicy) CanRename(user, m *meta.Meta) bool { - return ap.pre.CanRename(user, m) && ap.checkVisibility(m) -} - -func (ap *anonPolicy) CanDelete(user, m *meta.Meta) bool { - return ap.pre.CanDelete(user, m) && ap.checkVisibility(m) -} - -func (ap *anonPolicy) checkVisibility(m *meta.Meta) bool { - if ap.authConfig.GetVisibility(m) == meta.VisibilityExpert { - return ap.authConfig.GetExpertMode() - } - return true -} DELETED auth/policy/default.go Index: auth/policy/default.go ================================================================== --- auth/policy/default.go +++ /dev/null @@ -1,55 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package policy provides some interfaces and implementation for authorizsation policies. -package policy - -import ( - "zettelstore.de/z/auth" - "zettelstore.de/z/domain/meta" -) - -type defaultPolicy struct { - manager auth.AuthzManager -} - -func (d *defaultPolicy) CanCreate(user, newMeta *meta.Meta) bool { return true } -func (d *defaultPolicy) CanRead(user, m *meta.Meta) bool { return true } -func (d *defaultPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool { - return d.canChange(user, oldMeta) -} -func (d *defaultPolicy) CanRename(user, m *meta.Meta) bool { return d.canChange(user, m) } -func (d *defaultPolicy) CanDelete(user, m *meta.Meta) bool { return d.canChange(user, m) } - -func (d *defaultPolicy) canChange(user, m *meta.Meta) bool { - metaRo, ok := m.Get(meta.KeyReadOnly) - if !ok { - return true - } - if user == nil { - // If we are here, there is no authentication. - // See owner.go:CanWrite. - - // No authentication: check for owner-like restriction, because the user - // acts as an owner - return metaRo != meta.ValueUserRoleOwner && !meta.BoolValue(metaRo) - } - - userRole := d.manager.GetUserRole(user) - switch metaRo { - case meta.ValueUserRoleReader: - return userRole > meta.UserRoleReader - case meta.ValueUserRoleWriter: - return userRole > meta.UserRoleWriter - case meta.ValueUserRoleOwner: - return userRole > meta.UserRoleOwner - } - return !meta.BoolValue(metaRo) -} DELETED auth/policy/owner.go Index: auth/policy/owner.go ================================================================== --- auth/policy/owner.go +++ /dev/null @@ -1,145 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package policy provides some interfaces and implementation for authorizsation policies. -package policy - -import ( - "zettelstore.de/z/auth" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/meta" -) - -type ownerPolicy struct { - manager auth.AuthzManager - authConfig config.AuthConfig - pre auth.Policy -} - -func (o *ownerPolicy) CanCreate(user, newMeta *meta.Meta) bool { - if user == nil || !o.pre.CanCreate(user, newMeta) { - return false - } - return o.userIsOwner(user) || o.userCanCreate(user, newMeta) -} - -func (o *ownerPolicy) userCanCreate(user, newMeta *meta.Meta) bool { - if o.manager.GetUserRole(user) == meta.UserRoleReader { - return false - } - if role, ok := newMeta.Get(meta.KeyRole); ok && role == meta.ValueRoleUser { - return false - } - return true -} - -func (o *ownerPolicy) CanRead(user, m *meta.Meta) bool { - // No need to call o.pre.CanRead(user, meta), because it will always return true. - // Both the default and the readonly policy allow to read a zettel. - vis := o.authConfig.GetVisibility(m) - if res, ok := o.checkVisibility(user, vis); ok { - return res - } - return o.userIsOwner(user) || o.userCanRead(user, m, vis) -} - -func (o *ownerPolicy) userCanRead(user, m *meta.Meta, vis meta.Visibility) bool { - switch vis { - case meta.VisibilityOwner, meta.VisibilityExpert: - return false - case meta.VisibilityPublic: - return true - } - if user == nil { - return false - } - if role, ok := m.Get(meta.KeyRole); ok && role == meta.ValueRoleUser { - // Only the user can read its own zettel - return user.Zid == m.Zid - } - return true -} - -var noChangeUser = []string{ - meta.KeyID, - meta.KeyRole, - meta.KeyUserID, - meta.KeyUserRole, -} - -func (o *ownerPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool { - if user == nil || !o.pre.CanWrite(user, oldMeta, newMeta) { - return false - } - vis := o.authConfig.GetVisibility(oldMeta) - if res, ok := o.checkVisibility(user, vis); ok { - return res - } - if o.userIsOwner(user) { - return true - } - if !o.userCanRead(user, oldMeta, vis) { - return false - } - if role, ok := oldMeta.Get(meta.KeyRole); ok && role == meta.ValueRoleUser { - // Here we know, that user.Zid == newMeta.Zid (because of userCanRead) and - // user.Zid == newMeta.Zid (because oldMeta.Zid == newMeta.Zid) - for _, key := range noChangeUser { - if oldMeta.GetDefault(key, "") != newMeta.GetDefault(key, "") { - return false - } - } - return true - } - if o.manager.GetUserRole(user) == meta.UserRoleReader { - return false - } - return o.userCanCreate(user, newMeta) -} - -func (o *ownerPolicy) CanRename(user, m *meta.Meta) bool { - if user == nil || !o.pre.CanRename(user, m) { - return false - } - if res, ok := o.checkVisibility(user, o.authConfig.GetVisibility(m)); ok { - return res - } - return o.userIsOwner(user) -} - -func (o *ownerPolicy) CanDelete(user, m *meta.Meta) bool { - if user == nil || !o.pre.CanDelete(user, m) { - return false - } - if res, ok := o.checkVisibility(user, o.authConfig.GetVisibility(m)); ok { - return res - } - return o.userIsOwner(user) -} - -func (o *ownerPolicy) checkVisibility(user *meta.Meta, vis meta.Visibility) (bool, bool) { - if vis == meta.VisibilityExpert { - return o.userIsOwner(user) && o.authConfig.GetExpertMode(), true - } - return false, false -} - -func (o *ownerPolicy) userIsOwner(user *meta.Meta) bool { - if user == nil { - return false - } - if o.manager.IsOwner(user.Zid) { - return true - } - if val, ok := user.Get(meta.KeyUserRole); ok && val == meta.ValueUserRoleOwner { - return true - } - return false -} DELETED auth/policy/place.go Index: auth/policy/place.go ================================================================== --- auth/policy/place.go +++ /dev/null @@ -1,165 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package policy provides some interfaces and implementation for authorizsation policies. -package policy - -import ( - "context" - "io" - - "zettelstore.de/z/auth" - "zettelstore.de/z/config" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/search" - "zettelstore.de/z/web/server" -) - -// PlaceWithPolicy wraps the given place inside a policy place. -func PlaceWithPolicy( - auth server.Auth, - manager auth.AuthzManager, - place place.Place, - authConfig config.AuthConfig, -) (place.Place, auth.Policy) { - pol := newPolicy(manager, authConfig) - return newPlace(auth, place, pol), pol -} - -// polPlace implements a policy place. -type polPlace struct { - auth server.Auth - place place.Place - policy auth.Policy -} - -// newPlace creates a new policy place. -func newPlace(auth server.Auth, place place.Place, policy auth.Policy) place.Place { - return &polPlace{ - auth: auth, - place: place, - policy: policy, - } -} - -func (pp *polPlace) Location() string { - return pp.place.Location() -} - -func (pp *polPlace) CanCreateZettel(ctx context.Context) bool { - return pp.place.CanCreateZettel(ctx) -} - -func (pp *polPlace) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) { - user := pp.auth.GetUser(ctx) - if pp.policy.CanCreate(user, zettel.Meta) { - return pp.place.CreateZettel(ctx, zettel) - } - return id.Invalid, place.NewErrNotAllowed("Create", user, id.Invalid) -} - -func (pp *polPlace) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) { - zettel, err := pp.place.GetZettel(ctx, zid) - if err != nil { - return domain.Zettel{}, err - } - user := pp.auth.GetUser(ctx) - if pp.policy.CanRead(user, zettel.Meta) { - return zettel, nil - } - return domain.Zettel{}, place.NewErrNotAllowed("GetZettel", user, zid) -} - -func (pp *polPlace) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) { - m, err := pp.place.GetMeta(ctx, zid) - if err != nil { - return nil, err - } - user := pp.auth.GetUser(ctx) - if pp.policy.CanRead(user, m) { - return m, nil - } - return nil, place.NewErrNotAllowed("GetMeta", user, zid) -} - -func (pp *polPlace) FetchZids(ctx context.Context) (id.Set, error) { - return nil, place.NewErrNotAllowed("fetch-zids", pp.auth.GetUser(ctx), id.Invalid) -} - -func (pp *polPlace) SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) { - user := pp.auth.GetUser(ctx) - canRead := pp.policy.CanRead - s = s.AddPreMatch(func(m *meta.Meta) bool { return canRead(user, m) }) - return pp.place.SelectMeta(ctx, s) -} - -func (pp *polPlace) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool { - return pp.place.CanUpdateZettel(ctx, zettel) -} - -func (pp *polPlace) UpdateZettel(ctx context.Context, zettel domain.Zettel) error { - zid := zettel.Meta.Zid - user := pp.auth.GetUser(ctx) - if !zid.IsValid() { - return &place.ErrInvalidID{Zid: zid} - } - // Write existing zettel - oldMeta, err := pp.place.GetMeta(ctx, zid) - if err != nil { - return err - } - if pp.policy.CanWrite(user, oldMeta, zettel.Meta) { - return pp.place.UpdateZettel(ctx, zettel) - } - return place.NewErrNotAllowed("Write", user, zid) -} - -func (pp *polPlace) AllowRenameZettel(ctx context.Context, zid id.Zid) bool { - return pp.place.AllowRenameZettel(ctx, zid) -} - -func (pp *polPlace) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error { - meta, err := pp.place.GetMeta(ctx, curZid) - if err != nil { - return err - } - user := pp.auth.GetUser(ctx) - if pp.policy.CanRename(user, meta) { - return pp.place.RenameZettel(ctx, curZid, newZid) - } - return place.NewErrNotAllowed("Rename", user, curZid) -} - -func (pp *polPlace) CanDeleteZettel(ctx context.Context, zid id.Zid) bool { - return pp.place.CanDeleteZettel(ctx, zid) -} - -func (pp *polPlace) DeleteZettel(ctx context.Context, zid id.Zid) error { - meta, err := pp.place.GetMeta(ctx, zid) - if err != nil { - return err - } - user := pp.auth.GetUser(ctx) - if pp.policy.CanDelete(user, meta) { - return pp.place.DeleteZettel(ctx, zid) - } - return place.NewErrNotAllowed("Delete", user, zid) -} - -func (pp *polPlace) ReadStats(st *place.Stats) { - pp.place.ReadStats(st) -} - -func (pp *polPlace) Dump(w io.Writer) { - pp.place.Dump(w) -} DELETED auth/policy/policy.go Index: auth/policy/policy.go ================================================================== --- auth/policy/policy.go +++ /dev/null @@ -1,66 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package policy provides some interfaces and implementation for authorizsation policies. -package policy - -import ( - "zettelstore.de/z/auth" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/meta" -) - -// newPolicy creates a policy based on given constraints. -func newPolicy(manager auth.AuthzManager, authConfig config.AuthConfig) auth.Policy { - var pol auth.Policy - if manager.IsReadonly() { - pol = &roPolicy{} - } else { - pol = &defaultPolicy{manager} - } - if manager.WithAuth() { - pol = &ownerPolicy{ - manager: manager, - authConfig: authConfig, - pre: pol, - } - } else { - pol = &anonPolicy{ - authConfig: authConfig, - pre: pol, - } - } - return &prePolicy{pol} -} - -type prePolicy struct { - post auth.Policy -} - -func (p *prePolicy) CanCreate(user, newMeta *meta.Meta) bool { - return newMeta != nil && p.post.CanCreate(user, newMeta) -} - -func (p *prePolicy) CanRead(user, m *meta.Meta) bool { - return m != nil && p.post.CanRead(user, m) -} - -func (p *prePolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool { - return oldMeta != nil && newMeta != nil && oldMeta.Zid == newMeta.Zid && - p.post.CanWrite(user, oldMeta, newMeta) -} - -func (p *prePolicy) CanRename(user, m *meta.Meta) bool { - return m != nil && p.post.CanRename(user, m) -} - -func (p *prePolicy) CanDelete(user, m *meta.Meta) bool { - return m != nil && p.post.CanDelete(user, m) -} DELETED auth/policy/policy_test.go Index: auth/policy/policy_test.go ================================================================== --- auth/policy/policy_test.go +++ /dev/null @@ -1,620 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package policy provides some interfaces and implementation for authorizsation policies. -package policy - -import ( - "fmt" - "testing" - - "zettelstore.de/z/auth" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -func TestPolicies(t *testing.T) { - testScene := []struct { - readonly bool - withAuth bool - expert bool - }{ - {true, true, true}, - {true, true, false}, - {true, false, true}, - {true, false, false}, - {false, true, true}, - {false, true, false}, - {false, false, true}, - {false, false, false}, - } - for _, ts := range testScene { - authzManager := &testAuthzManager{ - readOnly: ts.readonly, - withAuth: ts.withAuth, - } - pol := newPolicy(authzManager, &authConfig{ts.expert}) - name := fmt.Sprintf("readonly=%v/withauth=%v/expert=%v", - ts.readonly, ts.withAuth, ts.expert) - t.Run(name, func(tt *testing.T) { - testCreate(tt, pol, ts.withAuth, ts.readonly, ts.expert) - testRead(tt, pol, ts.withAuth, ts.readonly, ts.expert) - testWrite(tt, pol, ts.withAuth, ts.readonly, ts.expert) - testRename(tt, pol, ts.withAuth, ts.readonly, ts.expert) - testDelete(tt, pol, ts.withAuth, ts.readonly, ts.expert) - }) - } -} - -type testAuthzManager struct { - readOnly bool - withAuth bool -} - -func (a *testAuthzManager) IsReadonly() bool { return a.readOnly } -func (a *testAuthzManager) Owner() id.Zid { return ownerZid } -func (a *testAuthzManager) IsOwner(zid id.Zid) bool { return zid == ownerZid } - -func (a *testAuthzManager) WithAuth() bool { return a.withAuth } - -func (a *testAuthzManager) GetUserRole(user *meta.Meta) meta.UserRole { - if user == nil { - if a.WithAuth() { - return meta.UserRoleUnknown - } - return meta.UserRoleOwner - } - if a.IsOwner(user.Zid) { - return meta.UserRoleOwner - } - if val, ok := user.Get(meta.KeyUserRole); ok { - if ur := meta.GetUserRole(val); ur != meta.UserRoleUnknown { - return ur - } - } - return meta.UserRoleReader -} - -type authConfig struct{ expert bool } - -func (ac *authConfig) GetExpertMode() bool { return ac.expert } - -func (ac *authConfig) GetVisibility(m *meta.Meta) meta.Visibility { - if vis, ok := m.Get(meta.KeyVisibility); ok { - switch vis { - case meta.ValueVisibilityPublic: - return meta.VisibilityPublic - case meta.ValueVisibilityOwner: - return meta.VisibilityOwner - case meta.ValueVisibilityExpert: - return meta.VisibilityExpert - } - } - return meta.VisibilityLogin -} - -func testCreate(t *testing.T, pol auth.Policy, withAuth, readonly, isExpert bool) { - t.Helper() - anonUser := newAnon() - reader := newReader() - writer := newWriter() - owner := newOwner() - owner2 := newOwner2() - zettel := newZettel() - userZettel := newUserZettel() - testCases := []struct { - user *meta.Meta - meta *meta.Meta - exp bool - }{ - // No meta - {anonUser, nil, false}, - {reader, nil, false}, - {writer, nil, false}, - {owner, nil, false}, - {owner2, nil, false}, - // Ordinary zettel - {anonUser, zettel, !withAuth && !readonly}, - {reader, zettel, !withAuth && !readonly}, - {writer, zettel, !readonly}, - {owner, zettel, !readonly}, - {owner2, zettel, !readonly}, - // User zettel - {anonUser, userZettel, !withAuth && !readonly}, - {reader, userZettel, !withAuth && !readonly}, - {writer, userZettel, !withAuth && !readonly}, - {owner, userZettel, !readonly}, - {owner2, userZettel, !readonly}, - } - for _, tc := range testCases { - t.Run("Create", func(tt *testing.T) { - got := pol.CanCreate(tc.user, tc.meta) - if tc.exp != got { - tt.Errorf("exp=%v, but got=%v", tc.exp, got) - } - }) - } -} - -func testRead(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) { - t.Helper() - anonUser := newAnon() - reader := newReader() - writer := newWriter() - owner := newOwner() - owner2 := newOwner2() - zettel := newZettel() - publicZettel := newPublicZettel() - loginZettel := newLoginZettel() - ownerZettel := newOwnerZettel() - expertZettel := newExpertZettel() - userZettel := newUserZettel() - testCases := []struct { - user *meta.Meta - meta *meta.Meta - exp bool - }{ - // No meta - {anonUser, nil, false}, - {reader, nil, false}, - {writer, nil, false}, - {owner, nil, false}, - {owner2, nil, false}, - // Ordinary zettel - {anonUser, zettel, !withAuth}, - {reader, zettel, true}, - {writer, zettel, true}, - {owner, zettel, true}, - {owner2, zettel, true}, - // Public zettel - {anonUser, publicZettel, true}, - {reader, publicZettel, true}, - {writer, publicZettel, true}, - {owner, publicZettel, true}, - {owner2, publicZettel, true}, - // Login zettel - {anonUser, loginZettel, !withAuth}, - {reader, loginZettel, true}, - {writer, loginZettel, true}, - {owner, loginZettel, true}, - {owner2, loginZettel, true}, - // Owner zettel - {anonUser, ownerZettel, !withAuth}, - {reader, ownerZettel, !withAuth}, - {writer, ownerZettel, !withAuth}, - {owner, ownerZettel, true}, - {owner2, ownerZettel, true}, - // Expert zettel - {anonUser, expertZettel, !withAuth && expert}, - {reader, expertZettel, !withAuth && expert}, - {writer, expertZettel, !withAuth && expert}, - {owner, expertZettel, expert}, - {owner2, expertZettel, expert}, - // Other user zettel - {anonUser, userZettel, !withAuth}, - {reader, userZettel, !withAuth}, - {writer, userZettel, !withAuth}, - {owner, userZettel, true}, - {owner2, userZettel, true}, - // Own user zettel - {reader, reader, true}, - {writer, writer, true}, - {owner, owner, true}, - {owner, owner2, true}, - {owner2, owner, true}, - {owner2, owner2, true}, - } - for _, tc := range testCases { - t.Run("Read", func(tt *testing.T) { - got := pol.CanRead(tc.user, tc.meta) - if tc.exp != got { - tt.Errorf("exp=%v, but got=%v", tc.exp, got) - } - }) - } -} - -func testWrite(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) { - t.Helper() - anonUser := newAnon() - reader := newReader() - writer := newWriter() - owner := newOwner() - owner2 := newOwner2() - zettel := newZettel() - publicZettel := newPublicZettel() - loginZettel := newLoginZettel() - ownerZettel := newOwnerZettel() - expertZettel := newExpertZettel() - userZettel := newUserZettel() - writerNew := writer.Clone() - writerNew.Set(meta.KeyUserRole, owner.GetDefault(meta.KeyUserRole, "")) - roFalse := newRoFalseZettel() - roTrue := newRoTrueZettel() - roReader := newRoReaderZettel() - roWriter := newRoWriterZettel() - roOwner := newRoOwnerZettel() - notAuthNotReadonly := !withAuth && !readonly - testCases := []struct { - user *meta.Meta - old *meta.Meta - new *meta.Meta - exp bool - }{ - // No old and new meta - {anonUser, nil, nil, false}, - {reader, nil, nil, false}, - {writer, nil, nil, false}, - {owner, nil, nil, false}, - {owner2, nil, nil, false}, - // No old meta - {anonUser, nil, zettel, false}, - {reader, nil, zettel, false}, - {writer, nil, zettel, false}, - {owner, nil, zettel, false}, - {owner2, nil, zettel, false}, - // No new meta - {anonUser, zettel, nil, false}, - {reader, zettel, nil, false}, - {writer, zettel, nil, false}, - {owner, zettel, nil, false}, - {owner2, zettel, nil, false}, - // Old an new zettel have different zettel identifier - {anonUser, zettel, publicZettel, false}, - {reader, zettel, publicZettel, false}, - {writer, zettel, publicZettel, false}, - {owner, zettel, publicZettel, false}, - {owner2, zettel, publicZettel, false}, - // Overwrite a normal zettel - {anonUser, zettel, zettel, notAuthNotReadonly}, - {reader, zettel, zettel, notAuthNotReadonly}, - {writer, zettel, zettel, !readonly}, - {owner, zettel, zettel, !readonly}, - {owner2, zettel, zettel, !readonly}, - // Public zettel - {anonUser, publicZettel, publicZettel, notAuthNotReadonly}, - {reader, publicZettel, publicZettel, notAuthNotReadonly}, - {writer, publicZettel, publicZettel, !readonly}, - {owner, publicZettel, publicZettel, !readonly}, - {owner2, publicZettel, publicZettel, !readonly}, - // Login zettel - {anonUser, loginZettel, loginZettel, notAuthNotReadonly}, - {reader, loginZettel, loginZettel, notAuthNotReadonly}, - {writer, loginZettel, loginZettel, !readonly}, - {owner, loginZettel, loginZettel, !readonly}, - {owner2, loginZettel, loginZettel, !readonly}, - // Owner zettel - {anonUser, ownerZettel, ownerZettel, notAuthNotReadonly}, - {reader, ownerZettel, ownerZettel, notAuthNotReadonly}, - {writer, ownerZettel, ownerZettel, notAuthNotReadonly}, - {owner, ownerZettel, ownerZettel, !readonly}, - {owner2, ownerZettel, ownerZettel, !readonly}, - // Expert zettel - {anonUser, expertZettel, expertZettel, notAuthNotReadonly && expert}, - {reader, expertZettel, expertZettel, notAuthNotReadonly && expert}, - {writer, expertZettel, expertZettel, notAuthNotReadonly && expert}, - {owner, expertZettel, expertZettel, !readonly && expert}, - {owner2, expertZettel, expertZettel, !readonly && expert}, - // Other user zettel - {anonUser, userZettel, userZettel, notAuthNotReadonly}, - {reader, userZettel, userZettel, notAuthNotReadonly}, - {writer, userZettel, userZettel, notAuthNotReadonly}, - {owner, userZettel, userZettel, !readonly}, - {owner2, userZettel, userZettel, !readonly}, - // Own user zettel - {reader, reader, reader, !readonly}, - {writer, writer, writer, !readonly}, - {owner, owner, owner, !readonly}, - {owner2, owner2, owner2, !readonly}, - // Writer cannot change importand metadata of its own user zettel - {writer, writer, writerNew, notAuthNotReadonly}, - // No r/o zettel - {anonUser, roFalse, roFalse, notAuthNotReadonly}, - {reader, roFalse, roFalse, notAuthNotReadonly}, - {writer, roFalse, roFalse, !readonly}, - {owner, roFalse, roFalse, !readonly}, - {owner2, roFalse, roFalse, !readonly}, - // Reader r/o zettel - {anonUser, roReader, roReader, false}, - {reader, roReader, roReader, false}, - {writer, roReader, roReader, !readonly}, - {owner, roReader, roReader, !readonly}, - {owner2, roReader, roReader, !readonly}, - // Writer r/o zettel - {anonUser, roWriter, roWriter, false}, - {reader, roWriter, roWriter, false}, - {writer, roWriter, roWriter, false}, - {owner, roWriter, roWriter, !readonly}, - {owner2, roWriter, roWriter, !readonly}, - // Owner r/o zettel - {anonUser, roOwner, roOwner, false}, - {reader, roOwner, roOwner, false}, - {writer, roOwner, roOwner, false}, - {owner, roOwner, roOwner, false}, - {owner2, roOwner, roOwner, false}, - // r/o = true zettel - {anonUser, roTrue, roTrue, false}, - {reader, roTrue, roTrue, false}, - {writer, roTrue, roTrue, false}, - {owner, roTrue, roTrue, false}, - {owner2, roTrue, roTrue, false}, - } - for _, tc := range testCases { - t.Run("Write", func(tt *testing.T) { - got := pol.CanWrite(tc.user, tc.old, tc.new) - if tc.exp != got { - tt.Errorf("exp=%v, but got=%v", tc.exp, got) - } - }) - } -} - -func testRename(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) { - t.Helper() - anonUser := newAnon() - reader := newReader() - writer := newWriter() - owner := newOwner() - owner2 := newOwner2() - zettel := newZettel() - expertZettel := newExpertZettel() - roFalse := newRoFalseZettel() - roTrue := newRoTrueZettel() - roReader := newRoReaderZettel() - roWriter := newRoWriterZettel() - roOwner := newRoOwnerZettel() - notAuthNotReadonly := !withAuth && !readonly - testCases := []struct { - user *meta.Meta - meta *meta.Meta - exp bool - }{ - // No meta - {anonUser, nil, false}, - {reader, nil, false}, - {writer, nil, false}, - {owner, nil, false}, - {owner2, nil, false}, - // Any zettel - {anonUser, zettel, notAuthNotReadonly}, - {reader, zettel, notAuthNotReadonly}, - {writer, zettel, notAuthNotReadonly}, - {owner, zettel, !readonly}, - {owner2, zettel, !readonly}, - // Expert zettel - {anonUser, expertZettel, notAuthNotReadonly && expert}, - {reader, expertZettel, notAuthNotReadonly && expert}, - {writer, expertZettel, notAuthNotReadonly && expert}, - {owner, expertZettel, !readonly && expert}, - {owner2, expertZettel, !readonly && expert}, - // No r/o zettel - {anonUser, roFalse, notAuthNotReadonly}, - {reader, roFalse, notAuthNotReadonly}, - {writer, roFalse, notAuthNotReadonly}, - {owner, roFalse, !readonly}, - {owner2, roFalse, !readonly}, - // Reader r/o zettel - {anonUser, roReader, false}, - {reader, roReader, false}, - {writer, roReader, notAuthNotReadonly}, - {owner, roReader, !readonly}, - {owner2, roReader, !readonly}, - // Writer r/o zettel - {anonUser, roWriter, false}, - {reader, roWriter, false}, - {writer, roWriter, false}, - {owner, roWriter, !readonly}, - {owner2, roWriter, !readonly}, - // Owner r/o zettel - {anonUser, roOwner, false}, - {reader, roOwner, false}, - {writer, roOwner, false}, - {owner, roOwner, false}, - {owner2, roOwner, false}, - // r/o = true zettel - {anonUser, roTrue, false}, - {reader, roTrue, false}, - {writer, roTrue, false}, - {owner, roTrue, false}, - {owner2, roTrue, false}, - } - for _, tc := range testCases { - t.Run("Rename", func(tt *testing.T) { - got := pol.CanRename(tc.user, tc.meta) - if tc.exp != got { - tt.Errorf("exp=%v, but got=%v", tc.exp, got) - } - }) - } -} - -func testDelete(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) { - t.Helper() - anonUser := newAnon() - reader := newReader() - writer := newWriter() - owner := newOwner() - owner2 := newOwner2() - zettel := newZettel() - expertZettel := newExpertZettel() - roFalse := newRoFalseZettel() - roTrue := newRoTrueZettel() - roReader := newRoReaderZettel() - roWriter := newRoWriterZettel() - roOwner := newRoOwnerZettel() - notAuthNotReadonly := !withAuth && !readonly - testCases := []struct { - user *meta.Meta - meta *meta.Meta - exp bool - }{ - // No meta - {anonUser, nil, false}, - {reader, nil, false}, - {writer, nil, false}, - {owner, nil, false}, - {owner2, nil, false}, - // Any zettel - {anonUser, zettel, notAuthNotReadonly}, - {reader, zettel, notAuthNotReadonly}, - {writer, zettel, notAuthNotReadonly}, - {owner, zettel, !readonly}, - {owner2, zettel, !readonly}, - // Expert zettel - {anonUser, expertZettel, notAuthNotReadonly && expert}, - {reader, expertZettel, notAuthNotReadonly && expert}, - {writer, expertZettel, notAuthNotReadonly && expert}, - {owner, expertZettel, !readonly && expert}, - {owner2, expertZettel, !readonly && expert}, - // No r/o zettel - {anonUser, roFalse, notAuthNotReadonly}, - {reader, roFalse, notAuthNotReadonly}, - {writer, roFalse, notAuthNotReadonly}, - {owner, roFalse, !readonly}, - {owner2, roFalse, !readonly}, - // Reader r/o zettel - {anonUser, roReader, false}, - {reader, roReader, false}, - {writer, roReader, notAuthNotReadonly}, - {owner, roReader, !readonly}, - {owner2, roReader, !readonly}, - // Writer r/o zettel - {anonUser, roWriter, false}, - {reader, roWriter, false}, - {writer, roWriter, false}, - {owner, roWriter, !readonly}, - {owner2, roWriter, !readonly}, - // Owner r/o zettel - {anonUser, roOwner, false}, - {reader, roOwner, false}, - {writer, roOwner, false}, - {owner, roOwner, false}, - {owner2, roOwner, false}, - // r/o = true zettel - {anonUser, roTrue, false}, - {reader, roTrue, false}, - {writer, roTrue, false}, - {owner, roTrue, false}, - {owner2, roTrue, false}, - } - for _, tc := range testCases { - t.Run("Delete", func(tt *testing.T) { - got := pol.CanDelete(tc.user, tc.meta) - if tc.exp != got { - tt.Errorf("exp=%v, but got=%v", tc.exp, got) - } - }) - } -} - -const ( - readerZid = id.Zid(1013) - writerZid = id.Zid(1015) - ownerZid = id.Zid(1017) - owner2Zid = id.Zid(1019) - zettelZid = id.Zid(1021) - visZid = id.Zid(1023) - userZid = id.Zid(1025) -) - -func newAnon() *meta.Meta { return nil } -func newReader() *meta.Meta { - user := meta.New(readerZid) - user.Set(meta.KeyTitle, "Reader") - user.Set(meta.KeyRole, meta.ValueRoleUser) - user.Set(meta.KeyUserRole, meta.ValueUserRoleReader) - return user -} -func newWriter() *meta.Meta { - user := meta.New(writerZid) - user.Set(meta.KeyTitle, "Writer") - user.Set(meta.KeyRole, meta.ValueRoleUser) - user.Set(meta.KeyUserRole, meta.ValueUserRoleWriter) - return user -} -func newOwner() *meta.Meta { - user := meta.New(ownerZid) - user.Set(meta.KeyTitle, "Owner") - user.Set(meta.KeyRole, meta.ValueRoleUser) - user.Set(meta.KeyUserRole, meta.ValueUserRoleOwner) - return user -} -func newOwner2() *meta.Meta { - user := meta.New(owner2Zid) - user.Set(meta.KeyTitle, "Owner 2") - user.Set(meta.KeyRole, meta.ValueRoleUser) - user.Set(meta.KeyUserRole, meta.ValueUserRoleOwner) - return user -} -func newZettel() *meta.Meta { - m := meta.New(zettelZid) - m.Set(meta.KeyTitle, "Any Zettel") - return m -} -func newPublicZettel() *meta.Meta { - m := meta.New(visZid) - m.Set(meta.KeyTitle, "Public Zettel") - m.Set(meta.KeyVisibility, meta.ValueVisibilityPublic) - return m -} -func newLoginZettel() *meta.Meta { - m := meta.New(visZid) - m.Set(meta.KeyTitle, "Login Zettel") - m.Set(meta.KeyVisibility, meta.ValueVisibilityLogin) - return m -} -func newOwnerZettel() *meta.Meta { - m := meta.New(visZid) - m.Set(meta.KeyTitle, "Owner Zettel") - m.Set(meta.KeyVisibility, meta.ValueVisibilityOwner) - return m -} -func newExpertZettel() *meta.Meta { - m := meta.New(visZid) - m.Set(meta.KeyTitle, "Expert Zettel") - m.Set(meta.KeyVisibility, meta.ValueVisibilityExpert) - return m -} -func newRoFalseZettel() *meta.Meta { - m := meta.New(zettelZid) - m.Set(meta.KeyTitle, "No r/o Zettel") - m.Set(meta.KeyReadOnly, "false") - return m -} -func newRoTrueZettel() *meta.Meta { - m := meta.New(zettelZid) - m.Set(meta.KeyTitle, "A r/o Zettel") - m.Set(meta.KeyReadOnly, "true") - return m -} -func newRoReaderZettel() *meta.Meta { - m := meta.New(zettelZid) - m.Set(meta.KeyTitle, "Reader r/o Zettel") - m.Set(meta.KeyReadOnly, meta.ValueUserRoleReader) - return m -} -func newRoWriterZettel() *meta.Meta { - m := meta.New(zettelZid) - m.Set(meta.KeyTitle, "Writer r/o Zettel") - m.Set(meta.KeyReadOnly, meta.ValueUserRoleWriter) - return m -} -func newRoOwnerZettel() *meta.Meta { - m := meta.New(zettelZid) - m.Set(meta.KeyTitle, "Owner r/o Zettel") - m.Set(meta.KeyReadOnly, meta.ValueUserRoleOwner) - return m -} -func newUserZettel() *meta.Meta { - m := meta.New(userZid) - m.Set(meta.KeyTitle, "Any User") - m.Set(meta.KeyRole, meta.ValueRoleUser) - return m -} DELETED auth/policy/readonly.go Index: auth/policy/readonly.go ================================================================== --- auth/policy/readonly.go +++ /dev/null @@ -1,22 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package policy provides some interfaces and implementation for authorization policies. -package policy - -import "zettelstore.de/z/domain/meta" - -type roPolicy struct{} - -func (p *roPolicy) CanCreate(user, newMeta *meta.Meta) bool { return false } -func (p *roPolicy) CanRead(user, m *meta.Meta) bool { return true } -func (p *roPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool { return false } -func (p *roPolicy) CanRename(user, m *meta.Meta) bool { return false } -func (p *roPolicy) CanDelete(user, m *meta.Meta) bool { return false } Index: cmd/cmd_file.go ================================================================== --- cmd/cmd_file.go +++ cmd/cmd_file.go @@ -1,53 +1,62 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- package cmd import ( + "context" "flag" "fmt" "io" "os" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/input" - "zettelstore.de/z/parser" + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/domain/id" + "t73f.de/r/zsc/domain/meta" + "t73f.de/r/zsx/input" + + "zettelstore.de/z/internal/encoder" + "zettelstore.de/z/internal/parser" + "zettelstore.de/z/internal/zettel" ) // ---------- Subcommand: file ----------------------------------------------- -func cmdFile(fs *flag.FlagSet, cfg *meta.Meta) (int, error) { - format := fs.Lookup("t").Value.String() +func cmdFile(fs *flag.FlagSet) (int, error) { + enc := fs.Lookup("t").Value.String() m, inp, err := getInput(fs.Args()) if m == nil { return 2, err } z := parser.ParseZettel( - domain.Zettel{ + context.Background(), + zettel.Zettel{ Meta: m, - Content: domain.NewContent(inp.Src[inp.Pos:]), + Content: zettel.NewContent(inp.Src[inp.Pos:]), }, - m.GetDefault(meta.KeySyntax, meta.ValueSyntaxZmk), + string(m.GetDefault(meta.KeySyntax, meta.DefaultSyntax)), nil, ) - enc := encoder.Create(format, &encoder.Environment{Lang: m.GetDefault(meta.KeyLang, meta.ValueLangEN)}) - if enc == nil { - fmt.Fprintf(os.Stderr, "Unknown format %q\n", format) + encdr := encoder.Create( + api.Encoder(enc), + &encoder.CreateParameter{Lang: string(m.GetDefault(meta.KeyLang, meta.ValueLangEN))}) + if encdr == nil { + fmt.Fprintf(os.Stderr, "Unknown format %q\n", enc) return 2, nil } - _, err = enc.WriteZettel(os.Stdout, z, format != "raw") + _, err = encdr.WriteZettel(os.Stdout, z) if err != nil { return 2, err } fmt.Println() @@ -58,26 +67,26 @@ if len(args) < 1 { src, err := io.ReadAll(os.Stdin) if err != nil { return nil, nil, err } - inp := input.NewInput(string(src)) + inp := input.NewInput(src) m := meta.NewFromInput(id.New(true), inp) return m, inp, nil } src, err := os.ReadFile(args[0]) if err != nil { return nil, nil, err } - inp := input.NewInput(string(src)) + inp := input.NewInput(src) m := meta.NewFromInput(id.New(true), inp) if len(args) > 1 { - src, err := os.ReadFile(args[1]) + src, err = os.ReadFile(args[1]) if err != nil { return nil, nil, err } - inp = input.NewInput(string(src)) + inp = input.NewInput(src) } return m, inp, nil } Index: cmd/cmd_password.go ================================================================== --- cmd/cmd_password.go +++ cmd/cmd_password.go @@ -1,13 +1,16 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- package cmd import ( @@ -15,18 +18,19 @@ "fmt" "os" "golang.org/x/term" - "zettelstore.de/z/auth/cred" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" + "t73f.de/r/zsc/domain/id" + "t73f.de/r/zsc/domain/meta" + + "zettelstore.de/z/internal/auth/cred" ) // ---------- Subcommand: password ------------------------------------------- -func cmdPassword(fs *flag.FlagSet, cfg *meta.Meta) (int, error) { +func cmdPassword(fs *flag.FlagSet) (int, error) { if fs.NArg() == 0 { fmt.Fprintln(os.Stderr, "User name and user zettel identification missing") return 2, nil } if fs.NArg() == 1 { Index: cmd/cmd_run.go ================================================================== --- cmd/cmd_run.go +++ cmd/cmd_run.go @@ -1,126 +1,140 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- package cmd import ( + "context" "flag" "net/http" - "zettelstore.de/z/auth" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/kernel" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" - "zettelstore.de/z/web/adapter/api" - "zettelstore.de/z/web/adapter/webui" - "zettelstore.de/z/web/server" + "t73f.de/r/zsc/domain/meta" + + "zettelstore.de/z/internal/auth" + "zettelstore.de/z/internal/auth/user" + "zettelstore.de/z/internal/box" + "zettelstore.de/z/internal/config" + "zettelstore.de/z/internal/kernel" + "zettelstore.de/z/internal/usecase" + "zettelstore.de/z/internal/web/adapter/api" + "zettelstore.de/z/internal/web/adapter/webui" + "zettelstore.de/z/internal/web/server" ) // ---------- Subcommand: run ------------------------------------------------ func flgRun(fs *flag.FlagSet) { - fs.String("c", defConfigfile, "configuration file") + fs.String("c", "", "configuration file") fs.Uint("a", 0, "port number kernel service (0=disable)") fs.Uint("p", 23123, "port number web service") fs.String("d", "", "zettel directory") fs.Bool("r", false, "system-wide read-only mode") fs.Bool("v", false, "verbose mode") fs.Bool("debug", false, "debug mode") } -func withDebug(fs *flag.FlagSet) bool { - dbg := fs.Lookup("debug") - return dbg != nil && dbg.Value.String() == "true" -} - -func runFunc(fs *flag.FlagSet, cfg *meta.Meta) (int, error) { - exitCode, err := doRun(withDebug(fs)) +func runFunc(*flag.FlagSet) (int, error) { + var exitCode int + err := kernel.Main.StartService(kernel.WebService) + if err != nil { + exitCode = 1 + } kernel.Main.WaitForShutdown() return exitCode, err } -func doRun(debug bool) (int, error) { - kern := kernel.Main - kern.SetDebug(debug) - if err := kern.StartService(kernel.WebService); err != nil { - return 1, err - } - return 0, nil -} - -func setupRouting(webSrv server.Server, placeManager place.Manager, authManager auth.Manager, rtConfig config.Config) { - protectedPlaceManager, authPolicy := authManager.PlaceWithPolicy(webSrv, placeManager, rtConfig) - api := api.New(webSrv, authManager, authManager, webSrv, rtConfig) - wui := webui.New(webSrv, authManager, rtConfig, authManager, placeManager, authPolicy) - - ucAuthenticate := usecase.NewAuthenticate(authManager, authManager, placeManager) - ucCreateZettel := usecase.NewCreateZettel(rtConfig, protectedPlaceManager) - ucGetMeta := usecase.NewGetMeta(protectedPlaceManager) - ucGetZettel := usecase.NewGetZettel(protectedPlaceManager) - ucParseZettel := usecase.NewParseZettel(rtConfig, ucGetZettel) - ucListMeta := usecase.NewListMeta(protectedPlaceManager) - ucListRoles := usecase.NewListRole(protectedPlaceManager) - ucListTags := usecase.NewListTags(protectedPlaceManager) - ucZettelContext := usecase.NewZettelContext(protectedPlaceManager) - - webSrv.Handle("/", wui.MakeGetRootHandler(protectedPlaceManager)) - webSrv.AddListRoute('a', http.MethodGet, wui.MakeGetLoginHandler()) - webSrv.AddListRoute('a', http.MethodPost, adapter.MakePostLoginHandler( - api.MakePostLoginHandlerAPI(ucAuthenticate), - wui.MakePostLoginHandlerHTML(ucAuthenticate))) - webSrv.AddListRoute('a', http.MethodPut, api.MakeRenewAuthHandler()) - webSrv.AddZettelRoute('a', http.MethodGet, wui.MakeGetLogoutHandler()) - if !authManager.IsReadonly() { - webSrv.AddZettelRoute('b', http.MethodGet, wui.MakeGetRenameZettelHandler(ucGetMeta)) - webSrv.AddZettelRoute('b', http.MethodPost, wui.MakePostRenameZettelHandler( - usecase.NewRenameZettel(protectedPlaceManager))) - webSrv.AddZettelRoute('c', http.MethodGet, wui.MakeGetCopyZettelHandler( - ucGetZettel, usecase.NewCopyZettel())) - webSrv.AddZettelRoute('c', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel)) - webSrv.AddZettelRoute('d', http.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetZettel)) - webSrv.AddZettelRoute('d', http.MethodPost, wui.MakePostDeleteZettelHandler( - usecase.NewDeleteZettel(protectedPlaceManager))) - webSrv.AddZettelRoute('e', http.MethodGet, wui.MakeEditGetZettelHandler(ucGetZettel)) - webSrv.AddZettelRoute('e', http.MethodPost, wui.MakeEditSetZettelHandler( - usecase.NewUpdateZettel(protectedPlaceManager))) - webSrv.AddZettelRoute('f', http.MethodGet, wui.MakeGetFolgeZettelHandler( - ucGetZettel, usecase.NewFolgeZettel(rtConfig))) - webSrv.AddZettelRoute('f', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel)) - webSrv.AddZettelRoute('g', http.MethodGet, wui.MakeGetNewZettelHandler( - ucGetZettel, usecase.NewNewZettel())) - webSrv.AddZettelRoute('g', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel)) - } - webSrv.AddListRoute('f', http.MethodGet, wui.MakeSearchHandler( - usecase.NewSearch(protectedPlaceManager), ucGetMeta, ucGetZettel)) - webSrv.AddListRoute('h', http.MethodGet, wui.MakeListHTMLMetaHandler( - ucListMeta, ucListRoles, ucListTags)) - webSrv.AddZettelRoute('h', http.MethodGet, wui.MakeGetHTMLZettelHandler( - ucParseZettel, ucGetMeta)) - webSrv.AddZettelRoute('i', http.MethodGet, wui.MakeGetInfoHandler(ucParseZettel, ucGetMeta)) - webSrv.AddZettelRoute('j', http.MethodGet, wui.MakeZettelContextHandler(ucZettelContext)) - - webSrv.AddZettelRoute('l', http.MethodGet, api.MakeGetLinksHandler(ucParseZettel)) - webSrv.AddZettelRoute('o', http.MethodGet, api.MakeGetOrderHandler( - usecase.NewZettelOrder(protectedPlaceManager, ucParseZettel))) - webSrv.AddListRoute('r', http.MethodGet, api.MakeListRoleHandler(ucListRoles)) - webSrv.AddListRoute('t', http.MethodGet, api.MakeListTagsHandler(ucListTags)) - webSrv.AddZettelRoute('y', http.MethodGet, api.MakeZettelContextHandler(ucZettelContext)) - webSrv.AddListRoute('z', http.MethodGet, api.MakeListMetaHandler( - usecase.NewListMeta(protectedPlaceManager), ucGetMeta, ucParseZettel)) - webSrv.AddZettelRoute('z', http.MethodGet, api.MakeGetZettelHandler( - ucParseZettel, ucGetMeta)) +func setupRouting(webSrv server.Server, boxManager box.Manager, authManager auth.Manager, rtConfig config.Config) { + protectedBoxManager, authPolicy := authManager.BoxWithPolicy(boxManager, rtConfig) + kern := kernel.Main + webLogger := kern.GetLogger(kernel.WebService) + + var getUser getUserImpl + authLogger := kern.GetLogger(kernel.AuthService) + ucLogger := kern.GetLogger(kernel.CoreService) + ucGetUser := usecase.NewGetUser(authManager, boxManager) + ucAuthenticate := usecase.NewAuthenticate(authLogger, authManager, &ucGetUser) + ucIsAuth := usecase.NewIsAuthenticated(ucLogger, &getUser, authManager) + ucCreateZettel := usecase.NewCreateZettel(ucLogger, rtConfig, protectedBoxManager) + ucGetAllZettel := usecase.NewGetAllZettel(protectedBoxManager) + ucGetZettel := usecase.NewGetZettel(protectedBoxManager) + ucParseZettel := usecase.NewParseZettel(rtConfig, ucGetZettel) + ucGetReferences := usecase.NewGetReferences() + ucQuery := usecase.NewQuery(protectedBoxManager) + ucEvaluate := usecase.NewEvaluate(rtConfig, &ucGetZettel, &ucQuery) + ucQuery.SetEvaluate(&ucEvaluate) + ucTagZettel := usecase.NewTagZettel(protectedBoxManager, &ucQuery) + ucRoleZettel := usecase.NewRoleZettel(protectedBoxManager, &ucQuery) + ucListSyntax := usecase.NewListSyntax(protectedBoxManager) + ucListRoles := usecase.NewListRoles(protectedBoxManager) + ucDelete := usecase.NewDeleteZettel(ucLogger, protectedBoxManager) + ucUpdate := usecase.NewUpdateZettel(ucLogger, protectedBoxManager) + ucRefresh := usecase.NewRefresh(ucLogger, protectedBoxManager) + ucReIndex := usecase.NewReIndex(ucLogger, protectedBoxManager) + ucVersion := usecase.NewVersion(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string)) + + a := api.New( + webLogger.With("system", "WEBAPI"), + webSrv, authManager, authManager, rtConfig, authPolicy) + wui := webui.New( + webLogger.With("system", "WEBUI"), + webSrv, authManager, rtConfig, authManager, boxManager, authPolicy, &ucEvaluate) + + webSrv.Handle("/", wui.MakeGetRootHandler(protectedBoxManager)) + if assetDir := kern.GetConfig(kernel.WebService, kernel.WebAssetDir).(string); assetDir != "" { + const assetPrefix = "/assets/" + webSrv.Handle(assetPrefix, http.StripPrefix(assetPrefix, http.FileServer(http.Dir(assetDir)))) + webSrv.Handle("/favicon.ico", wui.MakeFaviconHandler(assetDir)) + } + + // Web user interface + if !authManager.IsReadonly() { + webSrv.AddListRoute('c', server.MethodGet, wui.MakeGetZettelFromListHandler(&ucQuery, &ucEvaluate, ucListRoles, ucListSyntax)) + webSrv.AddListRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel)) + webSrv.AddZettelRoute('c', server.MethodGet, wui.MakeGetCreateZettelHandler( + ucGetZettel, &ucCreateZettel, ucListRoles, ucListSyntax)) + webSrv.AddZettelRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel)) + webSrv.AddZettelRoute('d', server.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetZettel, ucGetAllZettel)) + webSrv.AddZettelRoute('d', server.MethodPost, wui.MakePostDeleteZettelHandler(&ucDelete)) + webSrv.AddZettelRoute('e', server.MethodGet, wui.MakeEditGetZettelHandler(ucGetZettel, ucListRoles, ucListSyntax)) + webSrv.AddZettelRoute('e', server.MethodPost, wui.MakeEditSetZettelHandler(&ucUpdate)) + } + webSrv.AddListRoute('g', server.MethodGet, wui.MakeGetGoActionHandler(&ucRefresh)) + webSrv.AddListRoute('h', server.MethodGet, wui.MakeListHTMLMetaHandler(&ucQuery, &ucTagZettel, &ucRoleZettel, &ucReIndex)) + webSrv.AddZettelRoute('h', server.MethodGet, wui.MakeGetHTMLZettelHandler(&ucEvaluate, ucGetZettel)) + webSrv.AddListRoute('i', server.MethodGet, wui.MakeGetLoginOutHandler()) + webSrv.AddListRoute('i', server.MethodPost, wui.MakePostLoginHandler(&ucAuthenticate)) + webSrv.AddZettelRoute('i', server.MethodGet, wui.MakeGetInfoHandler( + ucParseZettel, ucGetReferences, &ucEvaluate, ucGetZettel, ucGetAllZettel, &ucQuery)) + + // API + webSrv.AddListRoute('a', server.MethodPost, a.MakePostLoginHandler(&ucAuthenticate)) + webSrv.AddListRoute('a', server.MethodPut, a.MakeRenewAuthHandler()) + webSrv.AddZettelRoute('r', server.MethodGet, a.MakeGetReferencesHandler(ucParseZettel, ucGetReferences)) + webSrv.AddListRoute('x', server.MethodGet, a.MakeGetDataHandler(ucVersion)) + webSrv.AddListRoute('x', server.MethodPost, a.MakePostCommandHandler(&ucIsAuth, &ucRefresh)) + webSrv.AddListRoute('z', server.MethodGet, a.MakeQueryHandler(&ucQuery, &ucTagZettel, &ucRoleZettel, &ucReIndex)) + webSrv.AddZettelRoute('z', server.MethodGet, a.MakeGetZettelHandler(ucGetZettel, ucParseZettel, ucEvaluate)) + if !authManager.IsReadonly() { + webSrv.AddListRoute('z', server.MethodPost, a.MakePostCreateZettelHandler(&ucCreateZettel)) + webSrv.AddZettelRoute('z', server.MethodPut, a.MakeUpdateZettelHandler(&ucUpdate)) + webSrv.AddZettelRoute('z', server.MethodDelete, a.MakeDeleteZettelHandler(&ucDelete)) + } if authManager.WithAuth() { - webSrv.SetUserRetriever(usecase.NewGetUserByZid(placeManager)) + webSrv.SetUserRetriever(usecase.NewGetUserByZid(boxManager)) } } + +type getUserImpl struct{} + +func (*getUserImpl) GetCurrentUser(ctx context.Context) *meta.Meta { return user.GetCurrentUser(ctx) } DELETED cmd/cmd_run_simple.go Index: cmd/cmd_run_simple.go ================================================================== --- cmd/cmd_run_simple.go +++ /dev/null @@ -1,52 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -package cmd - -import ( - "flag" - "fmt" - "os" - "strings" - - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/kernel" -) - -func flgSimpleRun(fs *flag.FlagSet) { - fs.String("d", "", "zettel directory") -} - -func runSimpleFunc(fs *flag.FlagSet, cfg *meta.Meta) (int, error) { - kern := kernel.Main - listenAddr := kern.GetConfig(kernel.WebService, kernel.WebListenAddress).(string) - exitCode, err := doRun(false) - if idx := strings.LastIndexByte(listenAddr, ':'); idx >= 0 { - kern.Log() - kern.Log("--------------------------") - kern.Log("Open your browser and enter the following URL:") - kern.Log() - kern.Log(fmt.Sprintf(" http://localhost%v", listenAddr[idx:])) - kern.Log() - } - kern.WaitForShutdown() - return exitCode, err -} - -// runSimple is called, when the user just starts the software via a double click -// or via a simple call ``./zettelstore`` on the command line. -func runSimple() int { - dir := "./zettel" - if err := os.MkdirAll(dir, 0750); err != nil { - fmt.Fprintf(os.Stderr, "Unable to create zettel directory %q (%s)\n", dir, err) - os.Exit(1) - } - return executeCommand("run-simple", "-d", dir) -} Index: cmd/command.go ================================================================== --- cmd/command.go +++ cmd/command.go @@ -1,39 +1,43 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- package cmd import ( "flag" - "sort" - - "zettelstore.de/z/domain/meta" + "log/slog" + "maps" + "slices" ) // Command stores information about commands / sub-commands. type Command struct { - Name string // command name as it appears on the command line - Func CommandFunc // function that executes a command - Places bool // if true then places will be set up - Header bool // Print a heading on startup - Flags func(*flag.FlagSet) // function to set up flag.FlagSet - flags *flag.FlagSet // flags that belong to the command - + Name string // command name as it appears on the command line + Func CommandFunc // function that executes a command + Simple bool // Operate in simple-mode + Boxes bool // if true then boxes will be set up + Header bool // Print a heading on startup + LineServer bool // Start admin line server + SetFlags func(*flag.FlagSet) // function to set up flag.FlagSet + flags *flag.FlagSet // flags that belong to the command } // CommandFunc is the function that executes the command. // It accepts the parsed command line parameters. // It returns the exit code and an error. -type CommandFunc func(*flag.FlagSet, *meta.Meta) (int, error) +type CommandFunc func(*flag.FlagSet) (int, error) // GetFlags return the flag.FlagSet defined for the command. func (c *Command) GetFlags() *flag.FlagSet { return c.flags } var commands = make(map[string]Command) @@ -45,12 +49,14 @@ } if _, ok := commands[cmd.Name]; ok { panic("Command already registered: " + cmd.Name) } cmd.flags = flag.NewFlagSet(cmd.Name, flag.ExitOnError) - if cmd.Flags != nil { - cmd.Flags(cmd.flags) + cmd.flags.String("l", slog.LevelInfo.String(), "log level specification") + + if cmd.SetFlags != nil { + cmd.SetFlags(cmd.flags) } commands[cmd.Name] = cmd } // Get returns the command identified by the given name and a bool to signal success. @@ -58,13 +64,6 @@ cmd, ok := commands[name] return cmd, ok } // List returns a sorted list of all registered command names. -func List() []string { - result := make([]string, 0, len(commands)) - for name := range commands { - result = append(result, name) - } - sort.Strings(result) - return result -} +func List() []string { return slices.Sorted(maps.Keys(commands)) } DELETED cmd/fd_limit.go Index: cmd/fd_limit.go ================================================================== --- cmd/fd_limit.go +++ /dev/null @@ -1,15 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// +build !darwin - -package cmd - -func raiseFdLimit() error { return nil } DELETED cmd/fd_limit_raise.go Index: cmd/fd_limit_raise.go ================================================================== --- cmd/fd_limit_raise.go +++ /dev/null @@ -1,47 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// +build darwin - -package cmd - -import ( - "log" - "syscall" -) - -const minFiles = 1048576 - -func raiseFdLimit() error { - var rLimit syscall.Rlimit - err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rLimit) - if err != nil { - return err - } - if rLimit.Cur >= minFiles { - return nil - } - rLimit.Cur = minFiles - if rLimit.Cur > rLimit.Max { - rLimit.Cur = rLimit.Max - } - err = syscall.Setrlimit(syscall.RLIMIT_NOFILE, &rLimit) - if err != nil { - return err - } - err = syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rLimit) - if err != nil { - return err - } - if rLimit.Cur < minFiles { - log.Printf("Make sure you have no more than %d files in all your places if you enabled notification\n", rLimit.Cur) - } - return nil -} Index: cmd/main.go ================================================================== --- cmd/main.go +++ cmd/main.go @@ -1,230 +1,263 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- +// Package cmd provides the commands to call Zettelstore from the command line. package cmd import ( - "errors" + "crypto/sha256" "flag" "fmt" + "log/slog" "net" "net/url" "os" + "runtime/debug" "strconv" "strings" - - "zettelstore.de/z/auth" - "zettelstore.de/z/auth/impl" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/input" - "zettelstore.de/z/kernel" - "zettelstore.de/z/place" - "zettelstore.de/z/place/manager" - "zettelstore.de/z/place/progplace" - "zettelstore.de/z/web/server" -) - -const ( - defConfigfile = ".zscfg" -) + "time" + + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/domain/id" + "t73f.de/r/zsc/domain/meta" + "t73f.de/r/zsx/input" + + "zettelstore.de/z/internal/auth" + "zettelstore.de/z/internal/auth/impl" + "zettelstore.de/z/internal/box" + "zettelstore.de/z/internal/box/compbox" + "zettelstore.de/z/internal/box/manager" + "zettelstore.de/z/internal/config" + "zettelstore.de/z/internal/kernel" + "zettelstore.de/z/internal/logging" + "zettelstore.de/z/internal/web/server" +) + +const strRunSimple = "run-simple" func init() { RegisterCommand(Command{ Name: "help", - Func: func(*flag.FlagSet, *meta.Meta) (int, error) { + Func: func(*flag.FlagSet) (int, error) { fmt.Println("Available commands:") for _, name := range List() { fmt.Printf("- %q\n", name) } return 0, nil }, }) RegisterCommand(Command{ Name: "version", - Func: func(*flag.FlagSet, *meta.Meta) (int, error) { return 0, nil }, - Header: true, - }) - RegisterCommand(Command{ - Name: "run", - Func: runFunc, - Places: true, - Header: true, - Flags: flgRun, - }) - RegisterCommand(Command{ - Name: "run-simple", - Func: runSimpleFunc, - Places: true, - Header: true, - Flags: flgSimpleRun, + Func: func(*flag.FlagSet) (int, error) { return 0, nil }, + Header: true, + }) + RegisterCommand(Command{ + Name: "run", + Func: runFunc, + Boxes: true, + Header: true, + LineServer: true, + SetFlags: flgRun, + }) + RegisterCommand(Command{ + Name: strRunSimple, + Func: runFunc, + Simple: true, + Boxes: true, + Header: true, + // LineServer: true, + SetFlags: func(fs *flag.FlagSet) { + // fs.Uint("a", 0, "port number kernel service (0=disable)") + fs.String("d", "", "zettel directory") + }, }) RegisterCommand(Command{ Name: "file", Func: cmdFile, - Flags: func(fs *flag.FlagSet) { - fs.String("t", "html", "target output format") + SetFlags: func(fs *flag.FlagSet) { + fs.String("t", api.EncoderHTML.String(), "target output encoding") }, }) RegisterCommand(Command{ Name: "password", Func: cmdPassword, }) } -func readConfig(fs *flag.FlagSet) (cfg *meta.Meta) { - var configFile string +func fetchStartupConfiguration(fs *flag.FlagSet) (string, *meta.Meta) { if configFlag := fs.Lookup("c"); configFlag != nil { - configFile = configFlag.Value.String() - } else { - configFile = defConfigfile + if filename := configFlag.Value.String(); filename != "" { + content, err := readConfiguration(filename) + return filename, createConfiguration(content, err) + } } - content, err := os.ReadFile(configFile) + filename, content, err := searchAndReadConfiguration() + return filename, createConfiguration(content, err) +} + +func createConfiguration(content []byte, err error) *meta.Meta { if err != nil { return meta.New(id.Invalid) } - return meta.NewFromInput(id.Invalid, input.NewInput(string(content))) + return meta.NewFromInput(id.Invalid, input.NewInput(content)) +} + +func readConfiguration(filename string) ([]byte, error) { return os.ReadFile(filename) } + +func searchAndReadConfiguration() (string, []byte, error) { + for _, filename := range []string{"zettelstore.cfg", "zsconfig.txt", "zscfg.txt", "_zscfg", ".zscfg"} { + if content, err := readConfiguration(filename); err == nil { + return filename, content, nil + } + } + return "", nil, os.ErrNotExist } -func getConfig(fs *flag.FlagSet) *meta.Meta { - cfg := readConfig(fs) +func getConfig(fs *flag.FlagSet) (string, *meta.Meta) { + filename, cfg := fetchStartupConfiguration(fs) fs.Visit(func(flg *flag.Flag) { switch flg.Name { case "p": - if portStr, err := parsePort(flg.Value.String()); err == nil { - cfg.Set(keyListenAddr, net.JoinHostPort("127.0.0.1", portStr)) - } + cfg.Set(keyListenAddr, meta.Value(net.JoinHostPort("127.0.0.1", flg.Value.String()))) case "a": - if portStr, err := parsePort(flg.Value.String()); err == nil { - cfg.Set(keyAdminPort, portStr) - } + cfg.Set(keyAdminPort, meta.Value(flg.Value.String())) case "d": val := flg.Value.String() if strings.HasPrefix(val, "/") { val = "dir://" + val } else { val = "dir:" + val } - cfg.Set(keyPlaceOneURI, val) + deleteConfiguredBoxes(cfg) + cfg.Set(keyBoxOneURI, meta.Value(val)) + case "l": + cfg.Set(keyLogLevel, meta.Value(flg.Value.String())) + case "debug": + cfg.Set(keyDebug, meta.Value(flg.Value.String())) case "r": - cfg.Set(keyReadOnly, flg.Value.String()) + cfg.Set(keyReadOnly, meta.Value(flg.Value.String())) case "v": - cfg.Set(keyVerbose, flg.Value.String()) + cfg.Set(keyVerbose, meta.Value(flg.Value.String())) } }) - return cfg + return filename, cfg } -func parsePort(s string) (string, error) { - port, err := net.LookupPort("tcp", s) - if err != nil { - fmt.Fprintf(os.Stderr, "Wrong port specification: %q", s) - return "", err +func deleteConfiguredBoxes(cfg *meta.Meta) { + for key := range cfg.Rest() { + if strings.HasPrefix(key, kernel.BoxURIs) { + cfg.Delete(key) + } } - return strconv.Itoa(port), nil } const ( - keyAdminPort = "admin-port" - keyDefaultDirPlaceType = "default-dir-place-type" - keyInsecureCookie = "insecure-cookie" - keyListenAddr = "listen-addr" - keyOwner = "owner" - keyPersistentCookie = "persistent-cookie" - keyPlaceOneURI = kernel.PlaceURIs + "1" - keyReadOnly = "read-only-mode" - keyTokenLifetimeHTML = "token-lifetime-html" - keyTokenLifetimeAPI = "token-lifetime-api" - keyURLPrefix = "url-prefix" - keyVerbose = "verbose" -) - -func setServiceConfig(cfg *meta.Meta) error { - ok := setConfigValue(true, kernel.CoreService, kernel.CoreVerbose, cfg.GetBool(keyVerbose)) - if val, found := cfg.Get(keyAdminPort); found { - ok = setConfigValue(ok, kernel.CoreService, kernel.CorePort, val) - } - - ok = setConfigValue(ok, kernel.AuthService, kernel.AuthOwner, cfg.GetDefault(keyOwner, "")) - ok = setConfigValue(ok, kernel.AuthService, kernel.AuthReadonly, cfg.GetBool(keyReadOnly)) - - ok = setConfigValue( - ok, kernel.PlaceService, kernel.PlaceDefaultDirType, - cfg.GetDefault(keyDefaultDirPlaceType, kernel.PlaceDirTypeNotify)) - ok = setConfigValue(ok, kernel.PlaceService, kernel.PlaceURIs+"1", "dir:./zettel") - format := kernel.PlaceURIs + "%v" - for i := 1; ; i++ { - key := fmt.Sprintf(format, i) + keyAdminPort = "admin-port" + keyAssetDir = "asset-dir" + keyBaseURL = "base-url" + keyBoxOneURI = kernel.BoxURIs + "1" + keyDebug = "debug-mode" + keyDefaultDirBoxType = "default-dir-box-type" + keyInsecureCookie = "insecure-cookie" + keyInsecureHTML = "insecure-html" + keyListenAddr = "listen-addr" + keyLogLevel = "log-level" + keyLoopbackIdent = "loopback-ident" + keyLoopbackZid = "loopback-zid" + keyMaxRequestSize = "max-request-size" + keyOwner = "owner" + keyPersistentCookie = "persistent-cookie" + keyReadOnly = "read-only-mode" + keyRuntimeProfiling = "runtime-profiling" + keySxNesting = "sx-max-nesting" + keyTokenLifetimeHTML = "token-lifetime-html" + keyTokenLifetimeAPI = "token-lifetime-api" + keyURLPrefix = "url-prefix" + keyVerbose = "verbose-mode" +) + +func setServiceConfig(cfg *meta.Meta) bool { + debugMode := cfg.GetBool(keyDebug) + if debugMode && kernel.Main.GetKernelLogLevel() > slog.LevelDebug { + kernel.Main.SetLogLevel(logging.LevelString(slog.LevelDebug)) + } + if logLevel, found := cfg.Get(keyLogLevel); found { + kernel.Main.SetLogLevel(string(logLevel)) + } + err := setConfigValue(nil, kernel.CoreService, kernel.CoreDebug, debugMode) + err = setConfigValue(err, kernel.CoreService, kernel.CoreVerbose, cfg.GetBool(keyVerbose)) + if val, found := cfg.Get(keyAdminPort); found { + err = setConfigValue(err, kernel.CoreService, kernel.CorePort, val) + } + + err = setConfigValue(err, kernel.AuthService, kernel.AuthOwner, cfg.GetDefault(keyOwner, "")) + err = setConfigValue(err, kernel.AuthService, kernel.AuthReadonly, cfg.GetBool(keyReadOnly)) + + err = setConfigValue( + err, kernel.BoxService, kernel.BoxDefaultDirType, + cfg.GetDefault(keyDefaultDirBoxType, kernel.BoxDirTypeNotify)) + err = setConfigValue(err, kernel.BoxService, kernel.BoxURIs+"1", "dir:./zettel") + for i := 1; ; i++ { + key := kernel.BoxURIs + strconv.Itoa(i) val, found := cfg.Get(key) if !found { break } - ok = setConfigValue(ok, kernel.PlaceService, key, val) - } - - ok = setConfigValue( - ok, kernel.WebService, kernel.WebListenAddress, - cfg.GetDefault(keyListenAddr, "127.0.0.1:23123")) - ok = setConfigValue(ok, kernel.WebService, kernel.WebURLPrefix, cfg.GetDefault(keyURLPrefix, "/")) - ok = setConfigValue(ok, kernel.WebService, kernel.WebSecureCookie, !cfg.GetBool(keyInsecureCookie)) - ok = setConfigValue(ok, kernel.WebService, kernel.WebPersistentCookie, cfg.GetBool(keyPersistentCookie)) - ok = setConfigValue( - ok, kernel.WebService, kernel.WebTokenLifetimeAPI, cfg.GetDefault(keyTokenLifetimeAPI, "")) - ok = setConfigValue( - ok, kernel.WebService, kernel.WebTokenLifetimeHTML, cfg.GetDefault(keyTokenLifetimeHTML, "")) - - if !ok { - return errors.New("unable to set configuration") - } - return nil -} - -func setConfigValue(ok bool, subsys kernel.Service, key string, val interface{}) bool { - done := kernel.Main.SetConfig(subsys, key, fmt.Sprintf("%v", val)) - if !done { - kernel.Main.Log("unable to set configuration:", key, val) - } - return ok && done -} - -func setupOperations(cfg *meta.Meta, withPlaces bool) { - var createManager kernel.CreatePlaceManagerFunc - if withPlaces { - err := raiseFdLimit() - if err != nil { - srvm := kernel.Main - srvm.Log("Raising some limitions did not work:", err) - srvm.Log("Prepare to encounter errors. Most of them can be mitigated. See the manual for details") - srvm.SetConfig(kernel.PlaceService, kernel.PlaceDefaultDirType, kernel.PlaceDirTypeSimple) - } - createManager = func(placeURIs []*url.URL, authManager auth.Manager, rtConfig config.Config) (place.Manager, error) { - progplace.Setup(cfg) - return manager.New(placeURIs, authManager, rtConfig) - } - } else { - createManager = func([]*url.URL, auth.Manager, config.Config) (place.Manager, error) { return nil, nil } - } - - kernel.Main.SetCreators( - func(readonly bool, owner id.Zid) (auth.Manager, error) { - return impl.New(readonly, owner, cfg.GetDefault("secret", "")), nil - }, - createManager, - func(srv server.Server, plMgr place.Manager, authMgr auth.Manager, rtConfig config.Config) error { - setupRouting(srv, plMgr, authMgr, rtConfig) - return nil - }, - ) + err = setConfigValue(err, kernel.BoxService, key, val) + } + + err = setConfigValue( + err, kernel.ConfigService, kernel.ConfigInsecureHTML, cfg.GetDefault(keyInsecureHTML, kernel.ConfigSecureHTML)) + + err = setConfigValue( + err, kernel.WebService, kernel.WebListenAddress, cfg.GetDefault(keyListenAddr, "127.0.0.1:23123")) + err = setConfigValue(err, kernel.WebService, kernel.WebLoopbackIdent, cfg.GetDefault(keyLoopbackIdent, "")) + err = setConfigValue(err, kernel.WebService, kernel.WebLoopbackZid, cfg.GetDefault(keyLoopbackZid, "")) + if val, found := cfg.Get(keyBaseURL); found { + err = setConfigValue(err, kernel.WebService, kernel.WebBaseURL, val) + } + if val, found := cfg.Get(keyURLPrefix); found { + err = setConfigValue(err, kernel.WebService, kernel.WebURLPrefix, val) + } + err = setConfigValue(err, kernel.WebService, kernel.WebSecureCookie, !cfg.GetBool(keyInsecureCookie)) + err = setConfigValue(err, kernel.WebService, kernel.WebPersistentCookie, cfg.GetBool(keyPersistentCookie)) + if val, found := cfg.Get(keyMaxRequestSize); found { + err = setConfigValue(err, kernel.WebService, kernel.WebMaxRequestSize, val) + } + err = setConfigValue( + err, kernel.WebService, kernel.WebTokenLifetimeAPI, cfg.GetDefault(keyTokenLifetimeAPI, "")) + err = setConfigValue( + err, kernel.WebService, kernel.WebTokenLifetimeHTML, cfg.GetDefault(keyTokenLifetimeHTML, "")) + err = setConfigValue(err, kernel.WebService, kernel.WebProfiling, debugMode || cfg.GetBool(keyRuntimeProfiling)) + if val, found := cfg.Get(keyAssetDir); found { + err = setConfigValue(err, kernel.WebService, kernel.WebAssetDir, val) + } + if val, found := cfg.Get(keySxNesting); found { + err = setConfigValue(err, kernel.WebService, kernel.WebSxMaxNesting, val) + } + return err == nil +} + +func setConfigValue(err error, subsys kernel.Service, key string, val any) error { + if err == nil { + if err = kernel.Main.SetConfig(subsys, key, fmt.Sprint(val)); err != nil { + kernel.Main.GetKernelLogger().Error("Unable to set configuration", + "key", key, "value", val, "err", err) + } + } + return err } func executeCommand(name string, args ...string) int { command, ok := Get(name) if !ok { @@ -234,34 +267,139 @@ fs := command.GetFlags() if err := fs.Parse(args); err != nil { fmt.Fprintf(os.Stderr, "%s: unable to parse flags: %v %v\n", name, args, err) return 1 } - cfg := getConfig(fs) - if err := setServiceConfig(cfg); err != nil { - fmt.Fprintf(os.Stderr, "%s: %v\n", name, err) + filename, cfg := getConfig(fs) + if !setServiceConfig(cfg) { + fs.Usage() + return 2 + } + + kern := kernel.Main + var createManager kernel.CreateBoxManagerFunc + if command.Boxes { + createManager = func(boxURIs []*url.URL, authManager auth.Manager, rtConfig config.Config) (box.Manager, error) { + compbox.Setup(cfg) + return manager.New(boxURIs, authManager, rtConfig) + } + } else { + createManager = func([]*url.URL, auth.Manager, config.Config) (box.Manager, error) { return nil, nil } + } + + secret := cfg.GetDefault("secret", "") + if len(secret) < 16 && cfg.GetDefault(keyOwner, "") != "" { + fmt.Fprintf(os.Stderr, "secret must have at least length 16 when authentication is enabled, but is %q\n", secret) return 2 } - setupOperations(cfg, command.Places) - kernel.Main.Start(command.Header) - exitCode, err := command.Func(fs, cfg) + cfg.Delete("secret") + secretHash := fmt.Sprintf("%x", sha256.Sum256([]byte(string(secret)))) + + kern.SetCreators( + func(readonly bool, owner id.Zid) (auth.Manager, error) { + return impl.New(readonly, owner, secretHash), nil + }, + createManager, + func(srv server.Server, plMgr box.Manager, authMgr auth.Manager, rtConfig config.Config) error { + setupRouting(srv, plMgr, authMgr, rtConfig) + return nil + }, + ) + + if command.Simple { + if err := kern.SetConfig(kernel.ConfigService, kernel.ConfigSimpleMode, "true"); err != nil { + kern.GetKernelLogger().Error("unable to set simple-mode", "err", err) + return 1 + } + } + kern.Start(command.Header, command.LineServer, filename) + exitCode, err := command.Func(fs) if err != nil { fmt.Fprintf(os.Stderr, "%s: %v\n", name, err) } - kernel.Main.Shutdown(true) + kern.Shutdown(true) return exitCode } +// runSimple is called, when the user just starts the software via a double click +// or via a simple call “./zettelstore“ on the command line. +func runSimple() int { + if _, _, err := searchAndReadConfiguration(); err == nil { + return executeCommand(strRunSimple) + } + dir := "./zettel" + if err := os.MkdirAll(dir, 0750); err != nil { + fmt.Fprintf(os.Stderr, "Unable to create zettel directory %q (%s)\n", dir, err) + return 1 + } + return executeCommand(strRunSimple, "-d", dir) +} + +var cpuprofile = flag.String("cpuprofile", "", "write cpu profile to `file`") +var memprofile = flag.String("memprofile", "", "write memory profile to `file`") + // Main is the real entrypoint of the zettelstore. -func Main(progName, buildVersion string) { - kernel.Main.SetConfig(kernel.CoreService, kernel.CoreProgname, progName) - kernel.Main.SetConfig(kernel.CoreService, kernel.CoreVersion, buildVersion) - var exitCode int - if len(os.Args) <= 1 { - exitCode = runSimple() - } else { - exitCode = executeCommand(os.Args[1], os.Args[2:]...) - } - if exitCode != 0 { - os.Exit(exitCode) - } +func Main(progName, buildVersion string) int { + info := retrieveVCSInfo(buildVersion) + fullVersion := info.revision + if info.dirty { + fullVersion += "-dirty" + } + kernel.Main.Setup(progName, fullVersion, info.time) + flag.Parse() + if *cpuprofile != "" || *memprofile != "" { + var err error + if *cpuprofile != "" { + err = kernel.Main.StartProfiling(kernel.ProfileCPU, *cpuprofile) + } else { + err = kernel.Main.StartProfiling(kernel.ProfileHead, *memprofile) + } + if err != nil { + kernel.Main.GetKernelLogger().Error("start profiling", "err", err) + return 1 + } + defer func() { + if err = kernel.Main.StopProfiling(); err != nil { + kernel.Main.GetKernelLogger().Error("stop profiling", "err", err) + } + }() + } + args := flag.Args() + if len(args) == 0 { + return runSimple() + } + return executeCommand(args[0], args[1:]...) +} + +type vcsInfo struct { + revision string + dirty bool + time time.Time +} + +func retrieveVCSInfo(version string) vcsInfo { + buildTime := time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC) + info, ok := debug.ReadBuildInfo() + if !ok { + return vcsInfo{revision: version, dirty: false, time: buildTime} + } + result := vcsInfo{time: buildTime} + for _, kv := range info.Settings { + switch kv.Key { + case "vcs.revision": + revision := "+" + kv.Value + if len(revision) > 11 { + revision = revision[:11] + } + result.revision = version + revision + case "vcs.modified": + if kv.Value == "true" { + result.dirty = true + } + case "vcs.time": + if t, err := time.Parse(time.RFC3339, kv.Value); err == nil { + result.time = t + } + } + } + return result } Index: cmd/register.go ================================================================== --- cmd/register.go +++ cmd/register.go @@ -1,33 +1,23 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- -// Package cmd provides command generic functions. package cmd -// Mention all needed encoders, parsers and stores to have them registered. +// Mention all needed boxes, encoders, and parsers to have them registered. import ( - _ "zettelstore.de/z/encoder/htmlenc" // Allow to use HTML encoder. - _ "zettelstore.de/z/encoder/jsonenc" // Allow to use JSON encoder. - _ "zettelstore.de/z/encoder/nativeenc" // Allow to use native encoder. - _ "zettelstore.de/z/encoder/rawenc" // Allow to use raw encoder. - _ "zettelstore.de/z/encoder/textenc" // Allow to use text encoder. - _ "zettelstore.de/z/encoder/zmkenc" // Allow to use zmk encoder. - _ "zettelstore.de/z/kernel/impl" // Allow kernel implementation to create itself - _ "zettelstore.de/z/parser/blob" // Allow to use BLOB parser. - _ "zettelstore.de/z/parser/markdown" // Allow to use markdown parser. - _ "zettelstore.de/z/parser/none" // Allow to use none parser. - _ "zettelstore.de/z/parser/plain" // Allow to use plain parser. - _ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser. - _ "zettelstore.de/z/place/constplace" // Allow to use global internal place. - _ "zettelstore.de/z/place/dirplace" // Allow to use directory place. - _ "zettelstore.de/z/place/fileplace" // Allow to use file place. - _ "zettelstore.de/z/place/memplace" // Allow to use memory place. - _ "zettelstore.de/z/place/progplace" // Allow to use computed place. + _ "zettelstore.de/z/internal/box/compbox" // Allow to use computed box. + _ "zettelstore.de/z/internal/box/constbox" // Allow to use global internal box. + _ "zettelstore.de/z/internal/box/dirbox" // Allow to use directory box. + _ "zettelstore.de/z/internal/box/filebox" // Allow to use file box. + _ "zettelstore.de/z/internal/box/membox" // Allow to use in-memory box. ) Index: cmd/zettelstore/main.go ================================================================== --- cmd/zettelstore/main.go +++ cmd/zettelstore/main.go @@ -1,21 +1,29 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- // Package main is the starting point for the zettelstore command. package main -import "zettelstore.de/z/cmd" +import ( + "os" + + "zettelstore.de/z/cmd" +) // Version variable. Will be filled by build process. -var version string = "" +var version string func main() { - cmd.Main("Zettelstore", version) + exitCode := cmd.Main("Zettelstore", version) + os.Exit(exitCode) } DELETED collect/collect.go Index: collect/collect.go ================================================================== --- collect/collect.go +++ /dev/null @@ -1,103 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package collect provides functions to collect items from a syntax tree. -package collect - -import ( - "zettelstore.de/z/ast" -) - -// Summary stores the relevant parts of the syntax tree -type Summary struct { - Links []*ast.Reference // list of all referenced links - Images []*ast.Reference // list of all referenced images - Cites []*ast.CiteNode // list of all referenced citations -} - -// References returns all references mentioned in the given zettel. This also -// includes references to images. -func References(zn *ast.ZettelNode) Summary { - lv := linkVisitor{} - ast.NewTopDownTraverser(&lv).VisitBlockSlice(zn.Ast) - return lv.summary -} - -type linkVisitor struct { - summary Summary -} - -// VisitVerbatim does nothing. -func (lv *linkVisitor) VisitVerbatim(vn *ast.VerbatimNode) {} - -// VisitRegion does nothing. -func (lv *linkVisitor) VisitRegion(rn *ast.RegionNode) {} - -// VisitHeading does nothing. -func (lv *linkVisitor) VisitHeading(hn *ast.HeadingNode) {} - -// VisitHRule does nothing. -func (lv *linkVisitor) VisitHRule(hn *ast.HRuleNode) {} - -// VisitList does nothing. -func (lv *linkVisitor) VisitNestedList(ln *ast.NestedListNode) {} - -// VisitDescriptionList does nothing. -func (lv *linkVisitor) VisitDescriptionList(dn *ast.DescriptionListNode) {} - -// VisitPara does nothing. -func (lv *linkVisitor) VisitPara(pn *ast.ParaNode) {} - -// VisitTable does nothing. -func (lv *linkVisitor) VisitTable(tn *ast.TableNode) {} - -// VisitBLOB does nothing. -func (lv *linkVisitor) VisitBLOB(bn *ast.BLOBNode) {} - -// VisitText does nothing. -func (lv *linkVisitor) VisitText(tn *ast.TextNode) {} - -// VisitTag does nothing. -func (lv *linkVisitor) VisitTag(tn *ast.TagNode) {} - -// VisitSpace does nothing. -func (lv *linkVisitor) VisitSpace(sn *ast.SpaceNode) {} - -// VisitBreak does nothing. -func (lv *linkVisitor) VisitBreak(bn *ast.BreakNode) {} - -// VisitLink collects the given link as a reference. -func (lv *linkVisitor) VisitLink(ln *ast.LinkNode) { - lv.summary.Links = append(lv.summary.Links, ln.Ref) -} - -// VisitImage collects the image links as a reference. -func (lv *linkVisitor) VisitImage(in *ast.ImageNode) { - if in.Ref != nil { - lv.summary.Images = append(lv.summary.Images, in.Ref) - } -} - -// VisitCite collects the citation. -func (lv *linkVisitor) VisitCite(cn *ast.CiteNode) { - lv.summary.Cites = append(lv.summary.Cites, cn) -} - -// VisitFootnote does nothing. -func (lv *linkVisitor) VisitFootnote(fn *ast.FootnoteNode) {} - -// VisitMark does nothing. -func (lv *linkVisitor) VisitMark(mn *ast.MarkNode) {} - -// VisitFormat does nothing. -func (lv *linkVisitor) VisitFormat(fn *ast.FormatNode) {} - -// VisitLiteral does nothing. -func (lv *linkVisitor) VisitLiteral(ln *ast.LiteralNode) {} DELETED collect/collect_test.go Index: collect/collect_test.go ================================================================== --- collect/collect_test.go +++ /dev/null @@ -1,70 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package collect_test provides some unit test for collectors. -package collect_test - -import ( - "testing" - - "zettelstore.de/z/ast" - "zettelstore.de/z/collect" -) - -func parseRef(s string) *ast.Reference { - r := ast.ParseReference(s) - if !r.IsValid() { - panic(s) - } - return r -} - -func TestLinks(t *testing.T) { - zn := &ast.ZettelNode{} - summary := collect.References(zn) - if summary.Links != nil || summary.Images != nil { - t.Error("No links/images expected, but got:", summary.Links, "and", summary.Images) - } - - intNode := &ast.LinkNode{Ref: parseRef("01234567890123")} - para := &ast.ParaNode{ - Inlines: ast.InlineSlice{ - intNode, - &ast.LinkNode{Ref: parseRef("https://zettelstore.de/z")}, - }, - } - zn.Ast = ast.BlockSlice{para} - summary = collect.References(zn) - if summary.Links == nil || summary.Images != nil { - t.Error("Links expected, and no images, but got:", summary.Links, "and", summary.Images) - } - - para.Inlines = append(para.Inlines, intNode) - summary = collect.References(zn) - if cnt := len(summary.Links); cnt != 3 { - t.Error("Link count does not work. Expected: 3, got", summary.Links) - } -} - -func TestImage(t *testing.T) { - zn := &ast.ZettelNode{ - Ast: ast.BlockSlice{ - &ast.ParaNode{ - Inlines: ast.InlineSlice{ - &ast.ImageNode{Ref: parseRef("12345678901234")}, - }, - }, - }, - } - summary := collect.References(zn) - if summary.Images == nil { - t.Error("Only image expected, but got: ", summary.Images) - } -} DELETED collect/order.go Index: collect/order.go ================================================================== --- collect/order.go +++ /dev/null @@ -1,69 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package collect provides functions to collect items from a syntax tree. -package collect - -import "zettelstore.de/z/ast" - -// Order of internal reference within the given zettel. -func Order(zn *ast.ZettelNode) (result []*ast.Reference) { - for _, bn := range zn.Ast { - if ln, ok := bn.(*ast.NestedListNode); ok { - switch ln.Code { - case ast.NestedListOrdered, ast.NestedListUnordered: - for _, is := range ln.Items { - if ref := firstItemZettelReference(is); ref != nil { - result = append(result, ref) - } - } - } - } - } - return result -} - -func firstItemZettelReference(is ast.ItemSlice) *ast.Reference { - for _, in := range is { - if pn, ok := in.(*ast.ParaNode); ok { - if ref := firstInlineZettelReference(pn.Inlines); ref != nil { - return ref - } - } - } - return nil -} - -func firstInlineZettelReference(ins ast.InlineSlice) (result *ast.Reference) { - for _, inl := range ins { - switch in := inl.(type) { - case *ast.LinkNode: - if ref := in.Ref; ref.IsZettel() { - return ref - } - result = firstInlineZettelReference(in.Inlines) - case *ast.ImageNode: - result = firstInlineZettelReference(in.Inlines) - case *ast.CiteNode: - result = firstInlineZettelReference(in.Inlines) - case *ast.FootnoteNode: - // Ignore references in footnotes - continue - case *ast.FormatNode: - result = firstInlineZettelReference(in.Inlines) - default: - continue - } - if result != nil { - return result - } - } - return nil -} DELETED collect/split.go Index: collect/split.go ================================================================== --- collect/split.go +++ /dev/null @@ -1,47 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package collect provides functions to collect items from a syntax tree. -package collect - -import "zettelstore.de/z/ast" - -// DivideReferences divides the given list of rederences into zettel, local, and external References. -func DivideReferences(all []*ast.Reference) (zettel, local, external []*ast.Reference) { - if len(all) == 0 { - return nil, nil, nil - } - - mapZettel := make(map[string]bool) - mapLocal := make(map[string]bool) - mapExternal := make(map[string]bool) - for _, ref := range all { - if ref.State == ast.RefStateSelf { - continue - } - if ref.IsZettel() { - zettel = appendRefToList(zettel, mapZettel, ref) - } else if ref.IsExternal() { - external = appendRefToList(external, mapExternal, ref) - } else { - local = appendRefToList(local, mapLocal, ref) - } - } - return zettel, local, external -} - -func appendRefToList(reflist []*ast.Reference, refSet map[string]bool, ref *ast.Reference) []*ast.Reference { - s := ref.String() - if _, ok := refSet[s]; !ok { - reflist = append(reflist, ref) - refSet[s] = true - } - return reflist -} DELETED config/config.go Index: config/config.go ================================================================== --- config/config.go +++ /dev/null @@ -1,108 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package config provides functions to retrieve runtime configuration data. -package config - -import ( - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -// Config allows to retrieve all defined configuration values that can be changed during runtime. -type Config interface { - AuthConfig - - // AddDefaultValues enriches the given meta data with its default values. - AddDefaultValues(m *meta.Meta) *meta.Meta - - // GetDefaultTitle returns the current value of the "default-title" key. - GetDefaultTitle() string - - // GetDefaultRole returns the current value of the "default-role" key. - GetDefaultRole() string - - // GetDefaultSyntax returns the current value of the "default-syntax" key. - GetDefaultSyntax() string - - // GetDefaultLang returns the current value of the "default-lang" key. - GetDefaultLang() string - - // GetSiteName returns the current value of the "site-name" key. - GetSiteName() string - - // GetHomeZettel returns the value of the "home-zettel" key. - GetHomeZettel() id.Zid - - // GetDefaultVisibility returns the default value for zettel visibility. - GetDefaultVisibility() meta.Visibility - - // GetYAMLHeader returns the current value of the "yaml-header" key. - GetYAMLHeader() bool - - // GetZettelFileSyntax returns the current value of the "zettel-file-syntax" key. - GetZettelFileSyntax() []string - - // GetMarkerExternal returns the current value of the "marker-external" key. - GetMarkerExternal() string - - // GetFooterHTML returns HTML code that should be embedded into the footer - // of each WebUI page. - GetFooterHTML() string - - // GetListPageSize returns the maximum length of a list to be returned in WebUI. - // A value less or equal to zero signals no limit. - GetListPageSize() int -} - -// AuthConfig are relevant configuration values for authentication. -type AuthConfig interface { - // GetExpertMode returns the current value of the "expert-mode" key - GetExpertMode() bool - - // GetVisibility returns the visibility value of the metadata. - GetVisibility(m *meta.Meta) meta.Visibility -} - -// GetTitle returns the value of the "title" key of the given meta. If there -// is no such value, GetDefaultTitle is returned. -func GetTitle(m *meta.Meta, cfg Config) string { - if val, ok := m.Get(meta.KeyTitle); ok { - return val - } - return cfg.GetDefaultTitle() -} - -// GetRole returns the value of the "role" key of the given meta. If there -// is no such value, GetDefaultRole is returned. -func GetRole(m *meta.Meta, cfg Config) string { - if val, ok := m.Get(meta.KeyRole); ok { - return val - } - return cfg.GetDefaultRole() -} - -// GetSyntax returns the value of the "syntax" key of the given meta. If there -// is no such value, GetDefaultSyntax is returned. -func GetSyntax(m *meta.Meta, cfg Config) string { - if val, ok := m.Get(meta.KeySyntax); ok { - return val - } - return cfg.GetDefaultSyntax() -} - -// GetLang returns the value of the "lang" key of the given meta. If there is -// no such value, GetDefaultLang is returned. -func GetLang(m *meta.Meta, cfg Config) string { - if val, ok := m.Get(meta.KeyLang); ok { - return val - } - return cfg.GetDefaultLang() -} ADDED docs/development/00010000000000.zettel Index: docs/development/00010000000000.zettel ================================================================== --- /dev/null +++ docs/development/00010000000000.zettel @@ -0,0 +1,11 @@ +id: 00010000000000 +title: Developments Notes +role: zettel +syntax: zmk +created: 00010101000000 +modified: 20231218182020 + +* [[Required Software|20210916193200]] +* [[Fuzzing tests|20221026184300]] +* [[Checklist for Release|20210916194900]] +* [[Development tools|20231218181900]] ADDED docs/development/20210916193200.zettel Index: docs/development/20210916193200.zettel ================================================================== --- /dev/null +++ docs/development/20210916193200.zettel @@ -0,0 +1,29 @@ +id: 20210916193200 +title: Required Software +role: zettel +syntax: zmk +created: 20210916193200 +modified: 20241213124936 + +The following software must be installed: + +* A current, supported [[release of Go|https://go.dev/doc/devel/release]], +* [[Fossil|https://fossil-scm.org/]], +* [[Git|https://git-scm.org/]] (most dependencies are accessible via Git only). + +Make sure that the software is in your path, e.g. via: +```sh +export PATH=$PATH:/usr/local/go/bin +export PATH=$PATH:$(go env GOPATH)/bin +``` + +The internal build tool needs the following software tools. +They can be installed / updated via the build tool itself: ``go run tools/devtools/devtools.go``. + +Otherwise you can install the software by hand: + +* [[shadow|https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/shadow]] via ``go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest``, +* [[staticcheck|https://staticcheck.io/]] via ``go install honnef.co/go/tools/cmd/staticcheck@latest``, +* [[unparam|https://mvdan.cc/unparam]][^[[GitHub|https://github.com/mvdan/unparam]]] via ``go install mvdan.cc/unparam@latest``, +* [[revive|https://revive.run]] via ``go install github.com/mgechev/revive@vlatest``, +* [[govulncheck|https://golang.org/x/vuln/cmd/govulncheck]] via ``go install golang.org/x/vuln/cmd/govulncheck@latest``, ADDED docs/development/20210916194900.zettel Index: docs/development/20210916194900.zettel ================================================================== --- /dev/null +++ docs/development/20210916194900.zettel @@ -0,0 +1,62 @@ +id: 20210916194900 +title: Checklist for Release +role: zettel +syntax: zmk +created: 20210916194900 +modified: 20250702180238 + +# Sync with the official repository: +#* ``fossil sync -u`` +# Make sure that there is no workspace defined: +#* ``ls ..`` must not have a file ''go.work'', in no parent folder. +# Make sure that all dependencies are up-to-date: +#* ``cat go.mod`` +# Ensure clean indirect dependencies: +#* ``go mod graph | modgraphviz | dot -Tpdf -o /tmp/zsgraph.pdf`` +#* Grey boxes signal outdated dependencies +# Clean up your Go workspace: +#* ``go run tools/clean/clean.go`` (alternatively: ``make clean``) +# All internal tests must succeed: +#* ``go run tools/check/check.go -r`` (alternatively: ``make relcheck``) +# The API tests must succeed on every development platform: +#* ``go run tools/testapi/testapi.go`` (alternatively: ``make api``) +# Run [[linkchecker|https://linkchecker.github.io/linkchecker/]] with the manual: +#* ``go run -race cmd/zettelstore/main.go run -d docs/manual`` +#* ``linkchecker http://127.0.0.1:23123 2>&1 | tee lc.txt`` +#* Check all ""Error: 404 Not Found"" +#* Check all ""Error: 403 Forbidden"": allowed for endpoint ''/z'' for those zettel that are accessible only in ''expert-mode'' +#* Try to resolve other error messages and warnings +#* Warnings about empty content can be ignored +# On every development platform, the box with 10.000 zettel must run, with ''-race'' enabled: +#* ``go run -race cmd/zettelstore/main.go run -d DIR`` +# Create a development release: +#* ``go run tools/build.go release`` (alternatively: ``make release``) +# On every platform (esp. macOS), the box with 10.000 zettel must run properly: +#* ``./zettelstore -d DIR`` +# Update files in directory ''www'': +#* ''index.wiki'' +#* ''download.wiki'' +#* ''changes.wiki'' +#* ''plan.wiki'' +# Set file ''VERSION'' to the new release version. + It **must** consists of three numbers: ''MAJOR.MINOR.PATCH'', even if ''PATCH'' is zero. +# Disable Fossil autosync mode: +#* ``fossil setting autosync off`` +# Commit the new release version: +#* ``fossil commit --tag release --tag vVERSION -m "Version VERSION"`` +#* **Important:** the tag must follow the given pattern, e.g. ''v0.0.15''. + Otherwise client software will not be able to import ''zettelstore.de/z''. +# Clean up your Go workspace: +#* ``go run tools/clean/clean.go`` (alternatively: ``make clean``) +# Create the release: +#* ``go run tools/build/build.go release`` (alternatively: ``make release``) +# Remove previous executables: +#* ``fossil uv remove --glob '*-PREVVERSION*'`` +# Add executables for release: +#* ``cd releases`` +#* ``fossil uv add *.zip`` +#* ``cd ..`` +#* Synchronize with main repository: +#* ``fossil sync -u`` +# Enable autosync: +#* ``fossil setting autosync on`` ADDED docs/development/20221026184300.zettel Index: docs/development/20221026184300.zettel ================================================================== --- /dev/null +++ docs/development/20221026184300.zettel @@ -0,0 +1,13 @@ +id: 20221026184300 +title: Fuzzing Tests +role: zettel +syntax: zmk +created: 20221026184320 +modified: 20221102140156 + +The source code contains some simple [[fuzzing tests|https://go.dev/security/fuzz/]]. +You should call them regularly to make sure that the software will cope with unusual input. + +```sh +go test -fuzz=FuzzParseZmk zettelstore.de/z/internal/parser/ +``` ADDED docs/development/20231218181900.zettel Index: docs/development/20231218181900.zettel ================================================================== --- /dev/null +++ docs/development/20231218181900.zettel @@ -0,0 +1,116 @@ +id: 20231218181900 +title: Development tools +role: zettel +syntax: zmk +created: 20231218181956 +modified: 20231218184500 + +The source code contains some tools to assist the development of Zettelstore. +These are located in the ''tools'' directory. + +Most tool support the generic option ``-v``, which log internal activities. + +Some of the tools can be called easier by using ``make``, that reads in a provided ''Makefile''. + +=== Check +The ""check"" tool automates some testing activities. +It is called via the command line: +``` +# go run tools/check/check.go +``` +There is an additional option ``-r`` to check in advance of a release. + +The following checks are executed: +* Execution of unit tests, like ``go test ./...`` +* Analyze the source code for general problems, as in ``go vet ./...`` +* Tries to find shadowed variable, via ``shadow ./...`` +* Performs some additional checks on the source code, via ``staticcheck ./...`` +* Checks the usage of function parameters and usage of return values, via ``unparam ./...``. + In case the option ''-r'' is set, the check includes exported functions and internal tests. +* In case option ''-r'' is set, the source code is checked against the vulnerability database, via ``govulncheck ./...`` + +Please note, that most of the tools above are not automatically installed in a standard Go distribution. +Use the command ""devtools"" to install them. + +=== Devtools +The following command installs all needed tools: +``` +# go run tooles/devtools/devtools.go +``` +It will also automatically update these tools. + +=== TestAPI +The following command will perform some high-level tests: +```sh +# go run tools/testapi/testapi.go +``` +Basically, a Zettelstore will be started and then API calls will be made to simulate some typical activities with the Zettelstore. + +If a Zettelstore is already running on port 23123, this Zettelstore will be used instead. +Even if the API test should clean up later, some zettel might stay created if a test fails. +This feature is used, if you want to have more control on the running Zettelstore. +You should start it with the following command: +```sh +# go run -race cmd/zettelstore/main.go run -c testdata/testbox/19700101000000.zettel +``` +This allows you to debug failing API tests. + +=== HTMLlint +The following command will check the generated HTML code for validity: +```sh +# go run tools/htmllint/htmllint.go +``` +In addition, you might specify the URL od a running Zettelstore. +Otherwise ''http://localhost:23123'' is used. + +This command fetches first the list of all zettel. +This list is used to check the generated HTML code (''ZID'' is the paceholder for the zettel identification): + +* Check all zettel HTML encodings, via the path ''/z/ZID?enc=html&part=zettel'' +* Check all zettel web views, via the path ''/h/ZID'' +* The info page of all zettel is checked, via path ''/i/ZID'' +* A subset of max. 100 zettel will be checked for the validity of their edit page, via ''/e/ZID'' +* 10 random zettel are checked for a valid create form, via ''/c/ZID'' +* A maximum of 200 random zettel are checked for a valid delete dialog, via ''/d/ZID'' + +Depending on the selected Zettelstore, the command might take a long time. + +You can shorten the time, if you disable any zettel query in the footer. + +=== Build +The ""build"" tool allows to build the software, either for tests or for a release. + +The following command will create a Zettelstore executable for the architecture of the current computer: +```sh +# go tools/build/build.go build +``` +You will find the executable in the ''bin'' directory. + +A full release will be build in the directory ''releases'', containing ZIP files for the computer architectures ""Linux/amd64"", ""Linux/arm"", ""MacOS/arm64"", ""MacOS/amd64"", and ""Windows/amd64"". +In addition, the manual is also build as a ZIP file: +```sh +# go run tools/build/build.go release +``` + +If you just want the ZIP file with the manual, please use: +```sh +# go run tools/build/build.go manual +``` + +In case you want to check the version of the Zettelstore to be build, use: +```sh +# go run tools/build/build.go version +``` + +=== Clean +To remove the directories ''bin'' and ''releases'', as well as all cached Go libraries used by Zettelstore, execute: +```sh +# go run tools/clean/clean.go +``` + +Internally, the following commands are executed +```sh +# rm -rf bin releases +# go clean ./... +# go clean -cache -modcache -testcache +``` Index: docs/manual/00000000000100.zettel ================================================================== --- docs/manual/00000000000100.zettel +++ docs/manual/00000000000100.zettel @@ -1,13 +1,14 @@ id: 00000000000100 title: Zettelstore Runtime Configuration role: configuration syntax: none -default-copyright: (c) 2020-2021 by Detlef Stern +created: 20210126175322 +default-copyright: (c) 2020-present by Detlef Stern default-license: EUPL-1.2-or-later default-visibility: public -footer-html:

Imprint / Privacy

+footer-zettel: 00001000000100 home-zettel: 00001000000000 -no-index: true +modified: 20221205173642 site-name: Zettelstore Manual visibility: owner ADDED docs/manual/00000000025001 Index: docs/manual/00000000025001 ================================================================== --- /dev/null +++ docs/manual/00000000025001 @@ -0,0 +1,7 @@ +id: 00000000025001 +title: Zettelstore User CSS +role: configuration +syntax: css +created: 20210622110143 +modified: 20220926183101 +visibility: public ADDED docs/manual/00000000025001.css Index: docs/manual/00000000025001.css ================================================================== --- /dev/null +++ docs/manual/00000000025001.css @@ -0,0 +1,2 @@ +/* User-defined CSS */ +.example { border-style: dotted !important } Index: docs/manual/00001000000000.zettel ================================================================== --- docs/manual/00001000000000.zettel +++ docs/manual/00001000000000.zettel @@ -1,10 +1,13 @@ id: 00001000000000 title: Zettelstore Manual role: manual tags: #manual #zettelstore syntax: zmk +created: 20210126175322 +modified: 20241128141924 +show-back-links: false * [[Introduction|00001001000000]] * [[Design goals|00001002000000]] * [[Installation|00001003000000]] * [[Configuration|00001004000000]] @@ -13,9 +16,12 @@ * [[Zettelmarkup|00001007000000]] * [[Other markup languages|00001008000000]] * [[Security|00001010000000]] * [[API|00001012000000]] * [[Web user interface|00001014000000]] -* Troubleshooting +* [[Tips and Tricks|00001017000000]] +* [[Troubleshooting|00001018000000]] * Frequently asked questions + +Version: {{00001000000001}} Licensed under the EUPL-1.2-or-later. ADDED docs/manual/00001000000001.zettel Index: docs/manual/00001000000001.zettel ================================================================== --- /dev/null +++ docs/manual/00001000000001.zettel @@ -0,0 +1,8 @@ +id: 00001000000001 +title: Manual Version +role: configuration +syntax: zmk +created: 20231002142915 +modified: 20231002142948 + +To be set by build tool. ADDED docs/manual/00001000000002.zettel Index: docs/manual/00001000000002.zettel ================================================================== --- /dev/null +++ docs/manual/00001000000002.zettel @@ -0,0 +1,7 @@ +id: 00001000000002 +title: manual +role: role +syntax: zmk +created: 20231128184200 + +Zettel with the role ""manual"" contain the manual of the zettelstore. ADDED docs/manual/00001000000100.zettel Index: docs/manual/00001000000100.zettel ================================================================== --- /dev/null +++ docs/manual/00001000000100.zettel @@ -0,0 +1,8 @@ +id: 00001000000100 +title: Footer Zettel +role: configuration +syntax: zmk +created: 20221205173520 +modified: 20221207175927 + +[[Imprint / Privacy|/home/doc/trunk/www/impri.wiki]] Index: docs/manual/00001001000000.zettel ================================================================== --- docs/manual/00001001000000.zettel +++ docs/manual/00001001000000.zettel @@ -1,25 +1,17 @@ id: 00001001000000 title: Introduction to the Zettelstore role: manual tags: #introduction #manual #zettelstore syntax: zmk - -[[Personal knowledge -management|https://en.wikipedia.org/wiki/Personal_knowledge_management]] is -about collecting, classifying, storing, searching, retrieving, assessing, -evaluating, and sharing knowledge as a daily activity. Personal knowledge -management is done by most people, not necessarily as part of their main -business. It is essential for knowledge workers, like students, researchers, -lecturers, software developers, scientists, engineers, architects, to name -a few. Many hobbyists build up a significant amount of knowledge, even if the -do not need to think for a living. Personal knowledge management can be seen as -a prerequisite for many kinds of collaboration. - -Zettelstore is a software that collects and relates your notes (""zettel"") -to represent and enhance your knowledge. It helps with many tasks of personal -knowledge management by explicitly supporting the ""[[Zettelkasten -method|https://en.wikipedia.org/wiki/Zettelkasten]]"". The method is based on -creating many individual notes, each with one idea or information, that are -related to each other. Since knowledge is typically build up gradually, one -major focus is a long-term store of these notes, hence the name -""Zettelstore"". +created: 20210126175322 +modified: 20250102181246 + +[[Personal knowledge management|https://en.wikipedia.org/wiki/Personal_knowledge_management]] involves collecting, classifying, storing, searching, retrieving, assessing, evaluating, and sharing knowledge as a daily activity. +It's done by most individuals, not necessarily as part of their main business. +It's essential for knowledge workers, such as students, researchers, lecturers, software developers, scientists, engineers, architects, etc. +Many hobbyists build up a significant amount of knowledge, even if they do not need to think for a living. +Personal knowledge management can be seen as a prerequisite for many kinds of collaboration. + +Zettelstore is software that collects and relates your notes (""zettel"") to represent and enhance your knowledge, supporting the ""[[Zettelkasten method|https://en.wikipedia.org/wiki/Zettelkasten]]"". +The method is based on creating many individual notes, each containing one idea or piece of information, which are related to each other. +Since knowledge is typically built up gradually, one major focus is a long-term store of these notes, hence the name ""Zettelstore"". Index: docs/manual/00001002000000.zettel ================================================================== --- docs/manual/00001002000000.zettel +++ docs/manual/00001002000000.zettel @@ -1,31 +1,43 @@ id: 00001002000000 title: Design goals for the Zettelstore +role: manual tags: #design #goal #manual #zettelstore syntax: zmk -role: manual +created: 20210126175322 +modified: 20250602181324 Zettelstore supports the following design goals: ; Longevity of stored notes / zettel : Every zettel you create should be readable without the help of any tool, even without Zettelstore. -: It should be not hard to write other software that works with your zettel. +: It should not be hard to write other software that works with your zettel. +: Normal zettel should be stored in a single file. + If this is not possible: at most in two files: one for the metadata, one for the content. + The only exceptions are [[predefined zettel|00001005090000]] stored in the Zettelstore executable. +: There is no additional database. ; Single user : All zettel belong to you, only to you. Zettelstore provides its services only to one person: you. - If your device is securely configured, there should be no risk that others are able to read or update your zettel. + If the computer running Zettelstore is securely configured, there should be no risk that others are able to read or update your zettel. : If you want, you can customize Zettelstore in a way that some specific or all persons are able to read some of your zettel. ; Ease of installation -: If you want to use the Zettelstore software, all you need is to copy the executable to an appropriate place and start working. +: If you want to use the Zettelstore software, all you need is to copy the executable to an appropriate file directory and start working. : Upgrading the software is done just by replacing the executable with a newer one. ; Ease of operation -: There is only one executable for Zettelstore and one directory, where your zettel are placed. +: There is only one executable for Zettelstore and one directory, where your zettel are stored. : If you decide to use multiple directories, you are free to configure Zettelstore appropriately. ; Multiple modes of operation : You can use Zettelstore as a standalone software on your device, but you are not restricted to it. -: You can install the software on a central server, or you can install it on all your devices with no restrictions how to synchronize your zettel. +: You can install the software on a central server, or you can install it on all your devices with no restrictions on how to synchronize your zettel. ; Multiple user interfaces -: Zettelstore provides a default web-based user interface. - Anybody can provide alternative user interfaces, e.g. for special purposes. +: Zettelstore provides a default [[web-based user interface|00001014000000]]. + Anyone can provide alternative user interfaces, e.g. for special purposes. ; Simple service : The purpose of Zettelstore is to safely store your zettel and to provide some initial relations between them. : External software can be written to deeply analyze your zettel and the structures they form. +; Security by default +: Without any customization, Zettelstore provides its services in a safe and secure manner and does not expose you (or other users) to security risks. +: If you know what you are doing, Zettelstore allows you to relax some security-related preferences. + However, even in this case, the more secure way is chosen. +: Zettelstore features a minimal design and relies on external software only when absolutely necessary. +: There will be no plugin mechanism, which allows external software to control the inner workings of the Zettelstore software. Index: docs/manual/00001003000000.zettel ================================================================== --- docs/manual/00001003000000.zettel +++ docs/manual/00001003000000.zettel @@ -1,72 +1,33 @@ id: 00001003000000 title: Installation of the Zettelstore software role: manual tags: #installation #manual #zettelstore syntax: zmk +created: 20210126175322 +modified: 20250415170240 === The curious user You just want to check out the Zettelstore software -* Grab the appropriate executable and copy it into any directory -* Start the Zettelstore software, e.g. with a double click -* A sub-directory ""zettel"" will be created in the directory where you placed the executable. +* Grab the appropriate executable and copy it to any directory +* Start the Zettelstore software, e.g. with a double click[^On Windows and macOS, the operating system tries to protect you from possible malicious software. + If you encounter a problem, please refer to the [[Troubleshooting|00001018000000]] page.] +* A sub-directory ""zettel"" will be created in the directory where you put the executable. It will contain your future zettel. -* Open the URI [[http://localhost:23123]] with your web browser. - It will present you a mostly empty Zettelstore. +* Open the URI [[http://localhost:23123/]] with your web browser. + A mostly empty Zettelstore is presented. There will be a zettel titled ""[[Home|00010000000000]]"" that contains some helpful information. -* Please read the instructions for the web-based user interface and learn about the various ways to write zettel. +* Please read the instructions for the [[web-based user interface|00001014000000]] and learn about the various ways to write zettel. * If you restart your device, please make sure to start your Zettelstore again. === The intermediate user -You already tried the Zettelstore software and now you want to use it permanently. +You have already tried the Zettelstore software and now you want to use it permanently. +Zettelstore should start automatically when you log into your computer. -* Grab the appropriate executable and copy it into the appropriate directory -* ... +Please follow [[these instructions|00001003300000]]. === The server administrator You want to provide a shared Zettelstore that can be used from your various devices. Installing Zettelstore as a Linux service is not that hard. -Grab the appropriate executable and copy it into the appropriate directory: -```sh -# sudo mv zettelstore /usr/local/bin/zettelstore -``` -Create a group named ''zettelstore'': -```sh -# sudo groupadd --system zettelstore -``` -Create a system user of that group, named ''zettelstore'', with a home folder: -```sh -# sudo useradd --system --gid zettelstore \ - --create-home --home-dir /var/lib/zettelstore \ - --shell /usr/sbin/nologin \ - --comment "Zettelstore server" \ - zettelstore -``` -Create a systemd service file and place it into ''/etc/systemd/system/zettelstore.service'': -```ini -[Unit] -Description=Zettelstore -After=network.target - -[Service] -Type=simple -User=zettelstore -Group=zettelstore -ExecStart=/usr/local/bin/zettelstore run -d /var/lib/zettelstore -WorkingDirectory=/var/lib/zettelstore - -[Install] -WantedBy=multi-user.target -``` -Double-check everything. Now you can enable and start the zettelstore as a service: -```sh -# sudo systemctl daemon-reload -# sudo systemctl enable zettelstore -# sudo systemctl start zettelstore -``` -Use the commands ``systemctl``{=sh} and ``journalctl``{=sh} to manage the service, e.g.: -```sh -# sudo systemctl status zettelstore # verify that it is running -# sudo journalctl -u zettelstore # obtain the output of the running zettelstore -``` +Please follow [[these instructions|00001003600000]]. ADDED docs/manual/00001003300000.zettel Index: docs/manual/00001003300000.zettel ================================================================== --- /dev/null +++ docs/manual/00001003300000.zettel @@ -0,0 +1,33 @@ +id: 00001003300000 +title: Zettelstore installation for the intermediate user +role: manual +tags: #installation #manual #zettelstore +syntax: zmk +created: 20211125191727 +modified: 20250627152419 + +You have already tried the Zettelstore software and now you want to use it permanently. +Zettelstore should start automatically when you log into your computer. + +* Grab the appropriate executable and copy it into the appropriate directory +* If you want to place your zettel into another directory, or if you want more than one [[Zettelstore box|00001004011200]], or if you want to [[enable authentication|00001010040100]], or if you want to tweak your Zettelstore in some other way, create an appropriate [[startup configuration file|00001004010000]]. +* If you created a startup configuration file, you need to test it: +** Start a command line prompt for your operating system. +** Navigate to the directory, where you placed the Zettelstore executable. + In most cases, this is done by the command ``cd DIR``, where ''DIR'' denotes the directory, where you placed the executable. +** Start the Zettelstore: +*** On Windows execute the command ``zettelstore.exe run -c CONFIG_FILE`` +*** On macOS execute the command ``./zettelstore run -c CONFIG_FILE`` +*** On Linux execute the command ``./zettelstore run -c CONFIG_FILE`` +** In all cases, ''CONFIG_FILE'' must be replaced with the name of the file where you wrote the startup configuration. +** If you encounter some error messages, update the startup configuration, and try again. +* Depending on your operating system, there are different ways to register Zettelstore to start automatically: +** [[Windows|00001003305000]] +** [[macOS|00001003310000]] +** [[Linux|00001003315000]] + +A word of caution: Never expose Zettelstore directly to the Internet. +As a personal service, Zettelstore is not designed to handle all aspects of the open web. +For instance, it lacks support for certificate handling, which is necessary for encrypted HTTP connections. +To ensure security, [[install Zettelstore on a server|00001003600000]] and place it behind a proxy server designed for Internet exposure. +For more details, see: [[External server to encrypt message transport|00001010090100]]. ADDED docs/manual/00001003305000.zettel Index: docs/manual/00001003305000.zettel ================================================================== --- /dev/null +++ docs/manual/00001003305000.zettel @@ -0,0 +1,120 @@ +id: 00001003305000 +title: Enable Zettelstore to start automatically on Windows +role: manual +tags: #installation #manual #zettelstore +syntax: zmk +created: 20211125191727 +modified: 20250701130205 + +Windows is a complicated beast. There are several ways to automatically start Zettelstore. + +=== Startup folder + +One way is to use the [[autostart folder|https://support.microsoft.com/en-us/windows/configure-startup-applications-in-windows-115a420a-0bff-4a6f-90e0-1934c844e473]]. +Open the folder where you have placed in the Explorer. +Create a shortcut file for the Zettelstore executable. +There are some ways to do this: +* Execute a right-click on the executable, and choose the menu entry ""Create shortcut"", +* Execute a right-click on the executable, and then click Send To > Desktop (Create shortcut). +* Drag the executable to your Desktop with pressing the ''Alt''-Key. + +If you have created the shortcut file, you must move it into the Startup folder. +Press the Windows logo key and the key ''R'', type ''shell:startup''. +Select the OK button. +This will open the Startup folder. +Move the shortcut file into this folder. + +The next time you log into your computer, Zettelstore will be started automatically. +However, it remains visible, at least in the task bar. + +You can modify the behavior by changing some properties of the shortcut file. + +=== Task scheduler + +The Windows Task scheduler allows you to start Zettelstore as a background task. + +This is both an advantage and a disadvantage. + +On the plus side, Zettelstore runs in the background, and it does not disturb you. +All you have to do is to open your web browser, enter the appropriate URL, and there you go. + +On the negative side, you will not be notified when you enter the wrong data in the Task scheduler and Zettelstore fails to start. +This can be mitigated by first using the command line prompt to start Zettelstore with the appropriate options. +Once everything works, you can register Zettelstore to be automatically started by the task scheduler. +There you should make sure that you have followed the first steps as described on the [[parent page|00001003300000]]. + +To start the Task scheduler management console, press the Windows logo key and the key ''R'', type ''taskschd.msc''. +Select the OK button. + +{{00001003305102}} + +This will start the ""Task Scheduler"". + +Now, create a new task with ""Create Task ..."" + +{{00001003305104}} + +Enter a name for the task, e.g. ""Zettelstore"" and select the options ""Run whether user is logged in or not"" and ""Do not store password."" + +{{00001003305106}} + +Create a new trigger. + +{{00001003305108}} + +Select the option ""At startup"". + +{{00001003305110}} + +Create a new action. + +{{00001003305112}} + +The next steps are the trickiest. + +If you did not create a startup configuration file, then create an action that starts a program. +Enter the file path where you placed the Zettelstore executable. +The ""Browse ..."" button helps you with that.[^I store my Zettelstore executable in the sub-directory ''bin'' of my home directory.] + +It is essential that you also enter a directory, which serves as the environment for your zettelstore. +The (sub-) directory ''zettel'', which will contain your zettel, will be placed in this directory. +If you leave the field ""Start in (optional)"" empty, the directory will be an internal Windows system directory (most likely: ''C:\\Windows\\System32''). + +If you press the OK button, the ""Create Task"" tab shows up as on the right image. + +{{00001003305114}}\ {{00001003305116}} + +If you have created a startup configuration file, you must enter something into the field ""Add arguments (optional)"". +Unfortunately, the text box is too narrow to fully see its content. + +I have entered the string ''run -c "C:\\Users\\Detlef Stern\\bin\\zsconfig.txt"'', because my startup configuration file has the name ''zsconfig.txt'' and I placed it into the same folder that also contains the Zettelstore executable. +Maybe you have to adapt to this. + +You must also enter appropriate data for the other form fields. +If you press the OK button, the ""Create Task"" tab shows up as on the right image. + +{{00001003305118}}\ {{00001003305120}} + +You should disable any additional conditions, since you typically want to use Zettelstore unconditionally. +Especially, make sure that ""Start the task only if the computer is on AC power"" is disabled. +Otherwise Zettelstore will not start if you run on battery power. + +{{00001003305122}} + +On the ""Settings"" tab, you should disable the option ""Stop the task if it runs longer than:"". + +{{00001003305124}} + +After entering the data, press the OK button. +Under some circumstances, Windows asks for permission and you have to enter your password. + +As the last step, you could run the freshly created task manually. + +Open your browser, enter the appropriate URL and use your Zettelstore. +In case of errors, the task will most likely stop immediately. +Make sure that all data you have entered is valid. +Do not forget to check the content of the startup configuration file. +Use the command prompt to debug your configuration. + +Sometimes, for example when your computer was in stand-by and it wakes up, these tasks are not started. +In this case execute the task scheduler and run the task manually. ADDED docs/manual/00001003305102.png Index: docs/manual/00001003305102.png ================================================================== --- /dev/null +++ docs/manual/00001003305102.png cannot compute difference between binary files ADDED docs/manual/00001003305104.png Index: docs/manual/00001003305104.png ================================================================== --- /dev/null +++ docs/manual/00001003305104.png cannot compute difference between binary files ADDED docs/manual/00001003305106.png Index: docs/manual/00001003305106.png ================================================================== --- /dev/null +++ docs/manual/00001003305106.png cannot compute difference between binary files ADDED docs/manual/00001003305108.png Index: docs/manual/00001003305108.png ================================================================== --- /dev/null +++ docs/manual/00001003305108.png cannot compute difference between binary files ADDED docs/manual/00001003305110.png Index: docs/manual/00001003305110.png ================================================================== --- /dev/null +++ docs/manual/00001003305110.png cannot compute difference between binary files ADDED docs/manual/00001003305112.png Index: docs/manual/00001003305112.png ================================================================== --- /dev/null +++ docs/manual/00001003305112.png cannot compute difference between binary files ADDED docs/manual/00001003305114.png Index: docs/manual/00001003305114.png ================================================================== --- /dev/null +++ docs/manual/00001003305114.png cannot compute difference between binary files ADDED docs/manual/00001003305116.png Index: docs/manual/00001003305116.png ================================================================== --- /dev/null +++ docs/manual/00001003305116.png cannot compute difference between binary files ADDED docs/manual/00001003305118.png Index: docs/manual/00001003305118.png ================================================================== --- /dev/null +++ docs/manual/00001003305118.png cannot compute difference between binary files ADDED docs/manual/00001003305120.png Index: docs/manual/00001003305120.png ================================================================== --- /dev/null +++ docs/manual/00001003305120.png cannot compute difference between binary files ADDED docs/manual/00001003305122.png Index: docs/manual/00001003305122.png ================================================================== --- /dev/null +++ docs/manual/00001003305122.png cannot compute difference between binary files ADDED docs/manual/00001003305124.png Index: docs/manual/00001003305124.png ================================================================== --- /dev/null +++ docs/manual/00001003305124.png cannot compute difference between binary files ADDED docs/manual/00001003310000.zettel Index: docs/manual/00001003310000.zettel ================================================================== --- /dev/null +++ docs/manual/00001003310000.zettel @@ -0,0 +1,95 @@ +id: 00001003310000 +title: Enable Zettelstore to start automatically on macOS +role: manual +tags: #installation #manual #zettelstore +syntax: zmk +created: 20220114181521 +modified: 20220119124635 + +There are several ways to automatically start Zettelstore. + +* [[Login Items|#login-items]] +* [[Launch Agent|#launch-agent]] + +=== Login Items + +Via macOS's system preferences, everybody is able to specify executables that are started when a user is logged in. +To do this, start system preferences and select ""Users & Groups"". + +{{00001003310104}} + +In the next screen, select the current user and then click on ""Login Items"". + +{{00001003310106}} + +Click on the plus sign at the bottom and select the Zettelstore executable. + +{{00001003310108}} + +Optionally select the ""Hide"" check box. + +{{00001003310110}} + +The next time you log into your macOS computer, Zettelstore will be started automatically. + +Unfortunately, hiding the Zettelstore windows does not always work. +Therefore, this method is just a way to automate navigating to the directory where the Zettelstore executable is placed and to click on that icon. + +If you don't want the Zettelstore window, you should try the next method. + +=== Launch Agent + +If you want to execute Zettelstore automatically and less visible, and if you know a little bit about working in the terminal application, then you could try to run Zettelstore under the control of the [[Launchd system|https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/Introduction.html]]. + +First, you have to create a description for ""Launchd"". +This is a text file named ''zettelstore.plist'' with the following content. +It assumes that you have copied the Zettelstore executable in a local folder called ''~/bin'' and have created a file for [[startup configuration|00001004010000]] called ''zettelstore.cfg'', which is placed in the same folder[^If you are not using a configuration file, just remove the lines ``-c`` and ``/Users/USERNAME/bin/zettelstore.cfg``.]: + +``` + + + + + Label + de.zettelstore + + ProgramArguments + + /Users/USERNAME/bin/zettelstore + run + -c + /Users/USERNAME/bin/zettelstore.cfg + + + WorkingDirectory + /Users/USERNAME + + EnvironmentVariables + + HOME + /Users/USERNAME + + + KeepAlive + + + LowPriorityIO + + + ProcessType + Background + + StandardOutPath + /Users/USERNAME/Library/Logs/Zettelstore.log + + StandardErrorPath + /Users/USERNAME/Library/Logs/Zettelstore-Errors.log + + +``` + +You must substitute all occurrences of ''USERNAME'' with your user name. + +Place this file into the user specific folder ''~/Library/LaunchAgents''. + +Log out and in again, or execute the command ``launchctl load ~/Library/LaunchAgents/zettelstore.plist``. ADDED docs/manual/00001003310104.png Index: docs/manual/00001003310104.png ================================================================== --- /dev/null +++ docs/manual/00001003310104.png cannot compute difference between binary files ADDED docs/manual/00001003310106.png Index: docs/manual/00001003310106.png ================================================================== --- /dev/null +++ docs/manual/00001003310106.png cannot compute difference between binary files ADDED docs/manual/00001003310108.png Index: docs/manual/00001003310108.png ================================================================== --- /dev/null +++ docs/manual/00001003310108.png cannot compute difference between binary files ADDED docs/manual/00001003310110.png Index: docs/manual/00001003310110.png ================================================================== --- /dev/null +++ docs/manual/00001003310110.png cannot compute difference between binary files ADDED docs/manual/00001003315000.zettel Index: docs/manual/00001003315000.zettel ================================================================== --- /dev/null +++ docs/manual/00001003315000.zettel @@ -0,0 +1,43 @@ +id: 00001003315000 +title: Enable Zettelstore to start automatically on Linux +role: manual +tags: #installation #manual #zettelstore +syntax: zmk +created: 20220114181521 +modified: 20250701135817 + +Since there is no such thing as the one Linux, there are too many different ways to automatically start Zettelstore. + +* One way is to interpret your Linux desktop system as a server and use the [[recipe to install Zettelstore on a server|00001003600000]]. +** See below for a lighter alternative. +* If you are using the [[Gnome Desktop|https://www.gnome.org/]], you could use the tool [[GNOME Tweaks|https://gitlab.gnome.org/GNOME/gnome-tweaks]]. + It allows you to specify applications that should run on startup / login. +* [[KDE|https://kde.org/]] provides a system setting to [[autostart|https://docs.kde.org/stable5/en/plasma-workspace/kcontrol/autostart/]] applications. +* [[Xfce|https://xfce.org/]] allows to specify [[autostart applications|https://docs.xfce.org/xfce/xfce4-session/preferences#application_autostart]]. +* [[LXDE|https://www.lxde.org/]] uses ""LXSession Edit"" to allow users to specify autostart applications. + +If you're using a different desktop environment, try searching for its name together with the word ""autostart"". + +Yet another way is to make use of the middleware that is provided. +Many Linux distributions make use of [[systemd|https://systemd.io/]], which allows to start processes on behalf of a user. +On the command line, adapt the following script to your own needs and execute it: +``` +# mkdir -p "$HOME/.config/systemd/user" +# cd "$HOME/.config/systemd/user" +# cat <<__EOF__ > zettelstore.service +[Unit] +Description=Zettelstore +After=network.target home.mount + +[Service] +ExecStart=/usr/local/bin/zettelstore run -d zettel + +[Install] +WantedBy=default.target +__EOF__ +# systemctl --user daemon-reload +# systemctl --user enable zettelstore.service +# systemctl --user start zettelstore.service +# systemctl --user status zettelstore.service +``` +The last command should output some lines to indicate success. ADDED docs/manual/00001003600000.zettel Index: docs/manual/00001003600000.zettel ================================================================== --- /dev/null +++ docs/manual/00001003600000.zettel @@ -0,0 +1,60 @@ +id: 00001003600000 +title: Installation of Zettelstore on a server +role: manual +tags: #installation #manual #zettelstore +syntax: zmk +created: 20211125191727 +modified: 20250227220033 + +You want to provide a shared Zettelstore that can be used from your various devices. +Installing Zettelstore as a Linux service is not that hard. + +Grab the appropriate executable and copy it into the appropriate directory: +```sh +# sudo mv zettelstore /usr/local/bin/zettelstore +``` +Create a group named ''zettelstore'': +```sh +# sudo groupadd --system zettelstore +``` +Create a system user of that group, named ''zettelstore'', with a home folder: +```sh +# sudo useradd --system --gid zettelstore \ + --create-home --home-dir /var/lib/zettelstore \ + --shell /usr/sbin/nologin \ + --comment "Zettelstore server" \ + zettelstore +``` +Create a systemd service file and store it into ''/etc/systemd/system/zettelstore.service'': +```ini +[Unit] +Description=Zettelstore +After=network.target + +[Service] +Type=simple +User=zettelstore +Group=zettelstore +ExecStart=/usr/local/bin/zettelstore run -d /var/lib/zettelstore +WorkingDirectory=/var/lib/zettelstore + +[Install] +WantedBy=multi-user.target +``` +Double-check everything. Now you can enable and start the zettelstore as a service: +```sh +# sudo systemctl daemon-reload +# sudo systemctl enable zettelstore +# sudo systemctl start zettelstore +``` +Use the commands ``systemctl``{=sh} and ``journalctl``{=sh} to manage the service, e.g.: +```sh +# sudo systemctl status zettelstore # verify that it is running +# sudo journalctl -u zettelstore # obtain the output of the running zettelstore +``` + +A word of caution: Never expose Zettelstore directly to the Internet. +As a personal service, Zettelstore is not designed to handle all aspects of the open web. +For instance, it lacks support for certificate handling, which is necessary for encrypted HTTP connections. +To ensure security, place Zettelstore behind a proxy server designed for Internet exposure. +For more details, see: [[External server to encrypt message transport|00001010090100]]. Index: docs/manual/00001004000000.zettel ================================================================== --- docs/manual/00001004000000.zettel +++ docs/manual/00001004000000.zettel @@ -1,13 +1,14 @@ id: 00001004000000 title: Configuration of Zettelstore role: manual tags: #configuration #manual #zettelstore syntax: zmk -modified: 20210510153233 +created: 20210126175322 +modified: 20250102181034 -There are some levels to change the behavior and/or the appearance of Zettelstore. +There are several levels to change the behavior and/or the appearance of Zettelstore. # The first level is the way to start Zettelstore services and to manage it via command line (and, in part, via a graphical user interface). #* [[Command line parameters|00001004050000]] # As an intermediate user, you usually want to have more control over how Zettelstore is started. This may include the URI under which your Zettelstore is accessible, or the directories in which your Zettel are stored. Index: docs/manual/00001004010000.zettel ================================================================== --- docs/manual/00001004010000.zettel +++ docs/manual/00001004010000.zettel @@ -1,86 +1,174 @@ id: 00001004010000 title: Zettelstore startup configuration role: manual tags: #configuration #manual #zettelstore syntax: zmk -modified: 20210525121644 +created: 20210126175322 +modified: 20250627155145 -The configuration file, as specified by the ''-c CONFIGFILE'' [[command line option|00001004051000]], allows you to specify some startup options. -These options cannot be stored in a [[configuration zettel|00001004020000]] because either they are needed before Zettelstore can start or because of security reasons. -For example, Zettelstore need to know in advance, on which network address is must listen or where zettel are placed. +The configuration file, specified by the ''-c CONFIGFILE'' [[command line option|00001004051000]], allows you to specify some startup options. +These cannot be stored in a [[configuration zettel|00001004020000]] because they are needed before Zettelstore can start or because of security reasons. +For example, Zettelstore needs to know in advance on which network address it must listen or where zettel are stored. An attacker that is able to change the owner can do anything. -Therefore only the owner of the computer on which Zettelstore runs can change this information. +Therefore, only the owner of the computer on which Zettelstore runs can change this information. The file for startup configuration must be created via a text editor in advance. The syntax of the configuration file is the same as for any zettel metadata. The following keys are supported: -; [!admin-port]''admin-port'' +; [!admin-port|''admin-port''] : Specifies the TCP port through which you can reach the [[administrator console|00001004100000]]. - A value of ''0'' (the default) disables the administrators console. - - On most operating systems, the value must be greater than ''1024'' unless you start Zettelstore with the full privileges of a system administrator (which is not recommended). - - Default: ''0'' -; [!default-dir-place-type]''default-dir-place-type'' -: Specifies the default value for the (sub-) type of [[directory places|00001004011400#type]]. - Zettel are typically stored in such places. - - Default: ''notify'' -; [!insecure-cookie]''insecure-cookie'' -: Must be set to ''true'', if authentication is enabled and Zettelstore is not accessible not via HTTPS (but via HTTP). - Otherwise web browser are free to ignore the authentication cookie. - - Default: ''false'' -; [!listen-addr]''listen-addr'' -: Configures the network address, where is zettel web service is listening for requests. - Syntax is: ''[NETWORKIP]:PORT'', where ''NETWORKIP'' is the IP-address of the networking interface (or something like ''0.0.0.0'' if you want to listen on all network interfaces, and ''PORT'' is the TCP port. - - Default value: ''"127.0.0.1:23123"'' -; [!owner]''owner'' + A value of ""0"" (the default) disables it. + The administrator console will only be enabled if Zettelstore is started with the [[''run'' sub-command|00001004051000]]. + + On most operating systems, the value must be greater than ""1024"" unless you start Zettelstore with the full privileges of a system administrator (which is not recommended). + + Default: ""0"" +; [!asset-dir|''asset-dir''] +: Allows to specify a directory whose files are allowed to be transferred directly with the help of the web server. + The URL prefix for these files is ''/assets/''. + You can use this if you want to transfer files that are too large for a zettel, such as presentation, PDF, music or video files. + + Files within the given directory will not be managed by Zettelstore.[^They will be managed by Zettelstore just in the very special case that the directory is one of the configured [[boxes|#box-uri-x]].] + + If you specify only the URL prefix in your web client, the contents of the directory are listed. + To avoid this, create an empty file in the directory named ""index.html"". + + Default: """", no asset directory is set, the URL prefix ''/assets/'' is invalid. +; [!base-url|''base-url''] +: Sets the absolute base URL for the service. + + Note: [[''url-prefix''|#url-prefix]] must be the suffix of ''base-url'', otherwise the web service will not start. + + Default: ""http://127.0.0.1:23123/"". +; [!box-uri-x|''box-uri-X''], where __X__ is a number greater or equal to one +: Specifies a [[box|00001004011200]] where zettel are stored. + During startup, __X__ is incremented, starting with one, until no key is found. + This allows you to configure than one box. + + If no ''box-uri-1'' key is given, the overall effect will be the same as if only ''box-uri-1'' was specified with the value ""dir://.zettel"". + In this case, even a key ''box-uri-2'' will be ignored. +; [!debug-mode|''debug-mode''] +: If set to [[true|00001006030500]], allows to debug the Zettelstore software (mostly used by Zettelstore developers). + Disables any timeout values of the internal web server and does not send some security-related data. + Sets [[''log-level''|#log-level]] to ""debug"". + Enables [[''runtime-profiling''|#runtime-profiling]]. + + Do not enable it for a production server. + + Default: ""false"" +; [!default-dir-box-type|''default-dir-box-type''] +: Specifies the default value for the (sub-)type of [[directory boxes|00001004011400#type]], in which Zettel are typically stored. + + Default: ""notify"" +; [!insecure-cookie|''insecure-cookie''] +: Must be set to [[true|00001006030500]] if authentication is enabled and Zettelstore is not accessible via HTTPS (but via HTTP). + Otherwise web browsers are free to ignore the authentication cookie. + + Default: ""false"" +; [!insecure-html|''insecure-html''] +: Allows to use HTML, e.g. within supported markup languages, even if this might introduce security-related problems. + However, HTML containing the ``\nokay\n", "html": "\n

okay

\n", - "example": 140, - "start_line": 2411, - "end_line": 2425, + "example": 170, + "start_line": 2756, + "end_line": 2770, + "section": "HTML blocks" + }, + { + "markdown": "\n", + "html": "\n", + "example": 171, + "start_line": 2775, + "end_line": 2791, "section": "HTML blocks" }, { "markdown": "\nh1 {color:red;}\n\np {color:blue;}\n\nokay\n", "html": "\nh1 {color:red;}\n\np {color:blue;}\n\n

okay

\n", - "example": 141, - "start_line": 2430, - "end_line": 2446, + "example": 172, + "start_line": 2795, + "end_line": 2811, "section": "HTML blocks" }, { "markdown": "\n\nfoo\n", "html": "\n\nfoo\n", - "example": 142, - "start_line": 2453, - "end_line": 2463, + "example": 173, + "start_line": 2818, + "end_line": 2828, "section": "HTML blocks" }, { "markdown": ">
\n> foo\n\nbar\n", "html": "
\n
\nfoo\n
\n

bar

\n", - "example": 143, - "start_line": 2466, - "end_line": 2477, + "example": 174, + "start_line": 2831, + "end_line": 2842, "section": "HTML blocks" }, { "markdown": "-
\n- foo\n", "html": "
    \n
  • \n
    \n
    • \n
  • foo
    • \n
\n", - "example": 144, - "start_line": 2480, - "end_line": 2490, + "example": 175, + "start_line": 2845, + "end_line": 2855, "section": "HTML blocks" }, { "markdown": "\n*foo*\n", "html": "\n

foo

\n", - "example": 145, - "start_line": 2495, - "end_line": 2501, + "example": 176, + "start_line": 2860, + "end_line": 2866, "section": "HTML blocks" }, { "markdown": "*bar*\n*baz*\n", "html": "*bar*\n

baz

\n", - "example": 146, - "start_line": 2504, - "end_line": 2510, + "example": 177, + "start_line": 2869, + "end_line": 2875, "section": "HTML blocks" }, { "markdown": "1. *bar*\n", "html": "1. *bar*\n", - "example": 147, - "start_line": 2516, - "end_line": 2524, + "example": 178, + "start_line": 2881, + "end_line": 2889, "section": "HTML blocks" }, { "markdown": "\nokay\n", "html": "\n

okay

\n", - "example": 148, - "start_line": 2529, - "end_line": 2541, + "example": 179, + "start_line": 2894, + "end_line": 2906, "section": "HTML blocks" }, { "markdown": "';\n\n?>\nokay\n", "html": "';\n\n?>\n

okay

\n", - "example": 149, - "start_line": 2547, - "end_line": 2561, + "example": 180, + "start_line": 2912, + "end_line": 2926, "section": "HTML blocks" }, { "markdown": "\n", "html": "\n", - "example": 150, - "start_line": 2566, - "end_line": 2570, + "example": 181, + "start_line": 2931, + "end_line": 2935, "section": "HTML blocks" }, { "markdown": "\nokay\n", "html": "\n

okay

\n", - "example": 151, - "start_line": 2575, - "end_line": 2603, + "example": 182, + "start_line": 2940, + "end_line": 2968, "section": "HTML blocks" }, { "markdown": " \n\n \n", "html": " \n
<!-- foo -->\n
\n", - "example": 152, - "start_line": 2608, - "end_line": 2616, + "example": 183, + "start_line": 2974, + "end_line": 2982, "section": "HTML blocks" }, { "markdown": "
\n\n
\n", "html": "
\n
<div>\n
\n", - "example": 153, - "start_line": 2619, - "end_line": 2627, + "example": 184, + "start_line": 2985, + "end_line": 2993, "section": "HTML blocks" }, { "markdown": "Foo\n
\nbar\n
\n", "html": "

Foo

\n
\nbar\n
\n", - "example": 154, - "start_line": 2633, - "end_line": 2643, + "example": 185, + "start_line": 2999, + "end_line": 3009, "section": "HTML blocks" }, { "markdown": "
\nbar\n
\n*foo*\n", "html": "
\nbar\n
\n*foo*\n", - "example": 155, - "start_line": 2650, - "end_line": 2660, + "example": 186, + "start_line": 3016, + "end_line": 3026, "section": "HTML blocks" }, { "markdown": "Foo\n\nbaz\n", "html": "

Foo\n\nbaz

\n", - "example": 156, - "start_line": 2665, - "end_line": 2673, + "example": 187, + "start_line": 3031, + "end_line": 3039, "section": "HTML blocks" }, { "markdown": "
\n\n*Emphasized* text.\n\n
\n", "html": "
\n

Emphasized text.

\n
\n", - "example": 157, - "start_line": 2706, - "end_line": 2716, + "example": 188, + "start_line": 3072, + "end_line": 3082, "section": "HTML blocks" }, { "markdown": "
\n*Emphasized* text.\n
\n", "html": "
\n*Emphasized* text.\n
\n", - "example": 158, - "start_line": 2719, - "end_line": 2727, + "example": 189, + "start_line": 3085, + "end_line": 3093, "section": "HTML blocks" }, { "markdown": "\n\n\n\n\n\n\n\n
\nHi\n
\n", "html": "\n\n\n\n
\nHi\n
\n", - "example": 159, - "start_line": 2741, - "end_line": 2761, + "example": 190, + "start_line": 3107, + "end_line": 3127, "section": "HTML blocks" }, { "markdown": "\n\n \n\n \n\n \n\n
\n Hi\n
\n", "html": "\n \n
<td>\n  Hi\n</td>\n
\n \n
\n", - "example": 160, - "start_line": 2768, - "end_line": 2789, + "example": 191, + "start_line": 3134, + "end_line": 3155, "section": "HTML blocks" }, { "markdown": "[foo]: /url \"title\"\n\n[foo]\n", "html": "

foo

\n", - "example": 161, - "start_line": 2816, - "end_line": 2822, + "example": 192, + "start_line": 3183, + "end_line": 3189, "section": "Link reference definitions" }, { "markdown": " [foo]: \n /url \n 'the title' \n\n[foo]\n", "html": "

foo

\n", - "example": 162, - "start_line": 2825, - "end_line": 2833, + "example": 193, + "start_line": 3192, + "end_line": 3200, "section": "Link reference definitions" }, { "markdown": "[Foo*bar\\]]:my_(url) 'title (with parens)'\n\n[Foo*bar\\]]\n", "html": "

Foo*bar]

\n", - "example": 163, - "start_line": 2836, - "end_line": 2842, + "example": 194, + "start_line": 3203, + "end_line": 3209, "section": "Link reference definitions" }, { "markdown": "[Foo bar]:\n\n'title'\n\n[Foo bar]\n", "html": "

Foo bar

\n", - "example": 164, - "start_line": 2845, - "end_line": 2853, + "example": 195, + "start_line": 3212, + "end_line": 3220, "section": "Link reference definitions" }, { "markdown": "[foo]: /url '\ntitle\nline1\nline2\n'\n\n[foo]\n", "html": "

foo

\n", - "example": 165, - "start_line": 2858, - "end_line": 2872, + "example": 196, + "start_line": 3225, + "end_line": 3239, "section": "Link reference definitions" }, { "markdown": "[foo]: /url 'title\n\nwith blank line'\n\n[foo]\n", "html": "

[foo]: /url 'title

\n

with blank line'

\n

[foo]

\n", - "example": 166, - "start_line": 2877, - "end_line": 2887, + "example": 197, + "start_line": 3244, + "end_line": 3254, "section": "Link reference definitions" }, { "markdown": "[foo]:\n/url\n\n[foo]\n", "html": "

foo

\n", - "example": 167, - "start_line": 2892, - "end_line": 2899, + "example": 198, + "start_line": 3259, + "end_line": 3266, "section": "Link reference definitions" }, { "markdown": "[foo]:\n\n[foo]\n", "html": "

[foo]:

\n

[foo]

\n", - "example": 168, - "start_line": 2904, - "end_line": 2911, + "example": 199, + "start_line": 3271, + "end_line": 3278, "section": "Link reference definitions" }, { "markdown": "[foo]: <>\n\n[foo]\n", "html": "

foo

\n", - "example": 169, - "start_line": 2916, - "end_line": 2922, + "example": 200, + "start_line": 3283, + "end_line": 3289, "section": "Link reference definitions" }, { "markdown": "[foo]: (baz)\n\n[foo]\n", "html": "

[foo]: (baz)

\n

[foo]

\n", - "example": 170, - "start_line": 2927, - "end_line": 2934, + "example": 201, + "start_line": 3294, + "end_line": 3301, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\\bar\\*baz \"foo\\\"bar\\baz\"\n\n[foo]\n", "html": "

foo

\n", - "example": 171, - "start_line": 2940, - "end_line": 2946, + "example": 202, + "start_line": 3307, + "end_line": 3313, "section": "Link reference definitions" }, { "markdown": "[foo]\n\n[foo]: url\n", "html": "

foo

\n", - "example": 172, - "start_line": 2951, - "end_line": 2957, + "example": 203, + "start_line": 3318, + "end_line": 3324, "section": "Link reference definitions" }, { "markdown": "[foo]\n\n[foo]: first\n[foo]: second\n", "html": "

foo

\n", - "example": 173, - "start_line": 2963, - "end_line": 2970, + "example": 204, + "start_line": 3330, + "end_line": 3337, "section": "Link reference definitions" }, { "markdown": "[FOO]: /url\n\n[Foo]\n", "html": "

Foo

\n", - "example": 174, - "start_line": 2976, - "end_line": 2982, + "example": 205, + "start_line": 3343, + "end_line": 3349, "section": "Link reference definitions" }, { "markdown": "[ΑΓΩ]: /φου\n\n[αγω]\n", "html": "

αγω

\n", - "example": 175, - "start_line": 2985, - "end_line": 2991, + "example": 206, + "start_line": 3352, + "end_line": 3358, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\n", "html": "", - "example": 176, - "start_line": 2997, - "end_line": 3000, + "example": 207, + "start_line": 3367, + "end_line": 3370, "section": "Link reference definitions" }, { "markdown": "[\nfoo\n]: /url\nbar\n", "html": "

bar

\n", - "example": 177, - "start_line": 3005, - "end_line": 3012, + "example": 208, + "start_line": 3375, + "end_line": 3382, "section": "Link reference definitions" }, { "markdown": "[foo]: /url \"title\" ok\n", "html": "

[foo]: /url "title" ok

\n", - "example": 178, - "start_line": 3018, - "end_line": 3022, + "example": 209, + "start_line": 3388, + "end_line": 3392, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\n\"title\" ok\n", "html": "

"title" ok

\n", - "example": 179, - "start_line": 3027, - "end_line": 3032, + "example": 210, + "start_line": 3397, + "end_line": 3402, "section": "Link reference definitions" }, { "markdown": " [foo]: /url \"title\"\n\n[foo]\n", "html": "
[foo]: /url "title"\n
\n

[foo]

\n", - "example": 180, - "start_line": 3038, - "end_line": 3046, + "example": 211, + "start_line": 3408, + "end_line": 3416, "section": "Link reference definitions" }, { "markdown": "```\n[foo]: /url\n```\n\n[foo]\n", "html": "
[foo]: /url\n
\n

[foo]

\n", - "example": 181, - "start_line": 3052, - "end_line": 3062, + "example": 212, + "start_line": 3422, + "end_line": 3432, "section": "Link reference definitions" }, { "markdown": "Foo\n[bar]: /baz\n\n[bar]\n", "html": "

Foo\n[bar]: /baz

\n

[bar]

\n", - "example": 182, - "start_line": 3067, - "end_line": 3076, + "example": 213, + "start_line": 3437, + "end_line": 3446, "section": "Link reference definitions" }, { "markdown": "# [Foo]\n[foo]: /url\n> bar\n", "html": "

Foo

\n
\n

bar

\n
\n", - "example": 183, - "start_line": 3082, - "end_line": 3091, + "example": 214, + "start_line": 3452, + "end_line": 3461, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\nbar\n===\n[foo]\n", "html": "

bar

\n

foo

\n", - "example": 184, - "start_line": 3093, - "end_line": 3101, + "example": 215, + "start_line": 3463, + "end_line": 3471, "section": "Link reference definitions" }, { "markdown": "[foo]: /url\n===\n[foo]\n", "html": "

===\nfoo

\n", - "example": 185, - "start_line": 3103, - "end_line": 3110, + "example": 216, + "start_line": 3473, + "end_line": 3480, "section": "Link reference definitions" }, { "markdown": "[foo]: /foo-url \"foo\"\n[bar]: /bar-url\n \"bar\"\n[baz]: /baz-url\n\n[foo],\n[bar],\n[baz]\n", "html": "

foo,\nbar,\nbaz

\n", - "example": 186, - "start_line": 3116, - "end_line": 3129, + "example": 217, + "start_line": 3486, + "end_line": 3499, "section": "Link reference definitions" }, { "markdown": "[foo]\n\n> [foo]: /url\n", "html": "

foo

\n
\n
\n", - "example": 187, - "start_line": 3137, - "end_line": 3145, - "section": "Link reference definitions" - }, - { - "markdown": "[foo]: /url\n", - "html": "", - "example": 188, - "start_line": 3154, - "end_line": 3157, + "example": 218, + "start_line": 3507, + "end_line": 3515, "section": "Link reference definitions" }, { "markdown": "aaa\n\nbbb\n", "html": "

aaa

\n

bbb

\n", - "example": 189, - "start_line": 3171, - "end_line": 3178, + "example": 219, + "start_line": 3529, + "end_line": 3536, "section": "Paragraphs" }, { "markdown": "aaa\nbbb\n\nccc\nddd\n", "html": "

aaa\nbbb

\n

ccc\nddd

\n", - "example": 190, - "start_line": 3183, - "end_line": 3194, + "example": 220, + "start_line": 3541, + "end_line": 3552, "section": "Paragraphs" }, { "markdown": "aaa\n\n\nbbb\n", "html": "

aaa

\n

bbb

\n", - "example": 191, - "start_line": 3199, - "end_line": 3207, + "example": 221, + "start_line": 3557, + "end_line": 3565, "section": "Paragraphs" }, { "markdown": " aaa\n bbb\n", "html": "

aaa\nbbb

\n", - "example": 192, - "start_line": 3212, - "end_line": 3218, + "example": 222, + "start_line": 3570, + "end_line": 3576, "section": "Paragraphs" }, { "markdown": "aaa\n bbb\n ccc\n", "html": "

aaa\nbbb\nccc

\n", - "example": 193, - "start_line": 3224, - "end_line": 3232, + "example": 223, + "start_line": 3582, + "end_line": 3590, "section": "Paragraphs" }, { "markdown": " aaa\nbbb\n", "html": "

aaa\nbbb

\n", - "example": 194, - "start_line": 3238, - "end_line": 3244, + "example": 224, + "start_line": 3596, + "end_line": 3602, "section": "Paragraphs" }, { "markdown": " aaa\nbbb\n", "html": "
aaa\n
\n

bbb

\n", - "example": 195, - "start_line": 3247, - "end_line": 3254, + "example": 225, + "start_line": 3605, + "end_line": 3612, "section": "Paragraphs" }, { "markdown": "aaa \nbbb \n", "html": "

aaa
\nbbb

\n", - "example": 196, - "start_line": 3261, - "end_line": 3267, + "example": 226, + "start_line": 3619, + "end_line": 3625, "section": "Paragraphs" }, { "markdown": " \n\naaa\n \n\n# aaa\n\n \n", "html": "

aaa

\n

aaa

\n", - "example": 197, - "start_line": 3278, - "end_line": 3290, + "example": 227, + "start_line": 3636, + "end_line": 3648, "section": "Blank lines" }, { "markdown": "> # Foo\n> bar\n> baz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", - "example": 198, - "start_line": 3344, - "end_line": 3354, + "example": 228, + "start_line": 3704, + "end_line": 3714, "section": "Block quotes" }, { "markdown": "># Foo\n>bar\n> baz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", - "example": 199, - "start_line": 3359, - "end_line": 3369, + "example": 229, + "start_line": 3719, + "end_line": 3729, "section": "Block quotes" }, { "markdown": " > # Foo\n > bar\n > baz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", - "example": 200, - "start_line": 3374, - "end_line": 3384, + "example": 230, + "start_line": 3734, + "end_line": 3744, "section": "Block quotes" }, { "markdown": " > # Foo\n > bar\n > baz\n", "html": "
> # Foo\n> bar\n> baz\n
\n", - "example": 201, - "start_line": 3389, - "end_line": 3398, + "example": 231, + "start_line": 3749, + "end_line": 3758, "section": "Block quotes" }, { "markdown": "> # Foo\n> bar\nbaz\n", "html": "
\n

Foo

\n

bar\nbaz

\n
\n", - "example": 202, - "start_line": 3404, - "end_line": 3414, + "example": 232, + "start_line": 3764, + "end_line": 3774, "section": "Block quotes" }, { "markdown": "> bar\nbaz\n> foo\n", "html": "
\n

bar\nbaz\nfoo

\n
\n", - "example": 203, - "start_line": 3420, - "end_line": 3430, + "example": 233, + "start_line": 3780, + "end_line": 3790, "section": "Block quotes" }, { "markdown": "> foo\n---\n", "html": "
\n

foo

\n
\n
\n", - "example": 204, - "start_line": 3444, - "end_line": 3452, + "example": 234, + "start_line": 3804, + "end_line": 3812, "section": "Block quotes" }, { "markdown": "> - foo\n- bar\n", "html": "
\n
    \n
  • foo
    • \n
\n
\n
    \n
  • bar
    • \n
\n", - "example": 205, - "start_line": 3464, - "end_line": 3476, + "example": 235, + "start_line": 3824, + "end_line": 3836, "section": "Block quotes" }, { "markdown": "> foo\n bar\n", "html": "
\n
foo\n
\n
\n
bar\n
\n", - "example": 206, - "start_line": 3482, - "end_line": 3492, + "example": 236, + "start_line": 3842, + "end_line": 3852, "section": "Block quotes" }, { "markdown": "> ```\nfoo\n```\n", "html": "
\n
\n
\n

foo

\n
\n", - "example": 207, - "start_line": 3495, - "end_line": 3505, + "example": 237, + "start_line": 3855, + "end_line": 3865, "section": "Block quotes" }, { "markdown": "> foo\n - bar\n", "html": "
\n

foo\n- bar

\n
\n", - "example": 208, - "start_line": 3511, - "end_line": 3519, + "example": 238, + "start_line": 3871, + "end_line": 3879, "section": "Block quotes" }, { "markdown": ">\n", "html": "
\n
\n", - "example": 209, - "start_line": 3535, - "end_line": 3540, + "example": 239, + "start_line": 3895, + "end_line": 3900, "section": "Block quotes" }, { "markdown": ">\n> \n> \n", "html": "
\n
\n", - "example": 210, - "start_line": 3543, - "end_line": 3550, + "example": 240, + "start_line": 3903, + "end_line": 3910, "section": "Block quotes" }, { "markdown": ">\n> foo\n> \n", "html": "
\n

foo

\n
\n", - "example": 211, - "start_line": 3555, - "end_line": 3563, + "example": 241, + "start_line": 3915, + "end_line": 3923, "section": "Block quotes" }, { "markdown": "> foo\n\n> bar\n", "html": "
\n

foo

\n
\n
\n

bar

\n
\n", - "example": 212, - "start_line": 3568, - "end_line": 3579, + "example": 242, + "start_line": 3928, + "end_line": 3939, "section": "Block quotes" }, { "markdown": "> foo\n> bar\n", "html": "
\n

foo\nbar

\n
\n", - "example": 213, - "start_line": 3590, - "end_line": 3598, + "example": 243, + "start_line": 3950, + "end_line": 3958, "section": "Block quotes" }, { "markdown": "> foo\n>\n> bar\n", "html": "
\n

foo

\n

bar

\n
\n", - "example": 214, - "start_line": 3603, - "end_line": 3612, + "example": 244, + "start_line": 3963, + "end_line": 3972, "section": "Block quotes" }, { "markdown": "foo\n> bar\n", "html": "

foo

\n
\n

bar

\n
\n", - "example": 215, - "start_line": 3617, - "end_line": 3625, + "example": 245, + "start_line": 3977, + "end_line": 3985, "section": "Block quotes" }, { "markdown": "> aaa\n***\n> bbb\n", "html": "
\n

aaa

\n
\n
\n
\n

bbb

\n
\n", - "example": 216, - "start_line": 3631, - "end_line": 3643, + "example": 246, + "start_line": 3991, + "end_line": 4003, "section": "Block quotes" }, { "markdown": "> bar\nbaz\n", "html": "
\n

bar\nbaz

\n
\n", - "example": 217, - "start_line": 3649, - "end_line": 3657, + "example": 247, + "start_line": 4009, + "end_line": 4017, "section": "Block quotes" }, { "markdown": "> bar\n\nbaz\n", "html": "
\n

bar

\n
\n

baz

\n", - "example": 218, - "start_line": 3660, - "end_line": 3669, + "example": 248, + "start_line": 4020, + "end_line": 4029, "section": "Block quotes" }, { "markdown": "> bar\n>\nbaz\n", "html": "
\n

bar

\n
\n

baz

\n", - "example": 219, - "start_line": 3672, - "end_line": 3681, + "example": 249, + "start_line": 4032, + "end_line": 4041, "section": "Block quotes" }, { "markdown": "> > > foo\nbar\n", "html": "
\n
\n
\n

foo\nbar

\n
\n
\n
\n", - "example": 220, - "start_line": 3688, - "end_line": 3700, + "example": 250, + "start_line": 4048, + "end_line": 4060, "section": "Block quotes" }, { "markdown": ">>> foo\n> bar\n>>baz\n", "html": "
\n
\n
\n

foo\nbar\nbaz

\n
\n
\n
\n", - "example": 221, - "start_line": 3703, - "end_line": 3717, + "example": 251, + "start_line": 4063, + "end_line": 4077, "section": "Block quotes" }, { "markdown": "> code\n\n> not code\n", "html": "
\n
code\n
\n
\n
\n

not code

\n
\n", - "example": 222, - "start_line": 3725, - "end_line": 3737, + "example": 252, + "start_line": 4085, + "end_line": 4097, "section": "Block quotes" }, { "markdown": "A paragraph\nwith two lines.\n\n indented code\n\n> A block quote.\n", "html": "

A paragraph\nwith two lines.

\n
indented code\n
\n
\n

A block quote.

\n
\n", - "example": 223, - "start_line": 3779, - "end_line": 3794, + "example": 253, + "start_line": 4139, + "end_line": 4154, "section": "List items" }, { "markdown": "1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
    2. \n
\n", - "example": 224, - "start_line": 3801, - "end_line": 3820, + "example": 254, + "start_line": 4161, + "end_line": 4180, "section": "List items" }, { "markdown": "- one\n\n two\n", "html": "
    \n
  • one
    • \n
\n

two

\n", - "example": 225, - "start_line": 3834, - "end_line": 3843, + "example": 255, + "start_line": 4194, + "end_line": 4203, "section": "List items" }, { "markdown": "- one\n\n two\n", "html": "
    \n
  • \n

    one

    \n

    two

    \n
    • \n
\n", - "example": 226, - "start_line": 3846, - "end_line": 3857, + "example": 256, + "start_line": 4206, + "end_line": 4217, "section": "List items" }, { "markdown": " - one\n\n two\n", "html": "
    \n
  • one
    • \n
\n
 two\n
\n", - "example": 227, - "start_line": 3860, - "end_line": 3870, + "example": 257, + "start_line": 4220, + "end_line": 4230, "section": "List items" }, { "markdown": " - one\n\n two\n", "html": "
    \n
  • \n

    one

    \n

    two

    \n
    • \n
\n", - "example": 228, - "start_line": 3873, - "end_line": 3884, + "example": 258, + "start_line": 4233, + "end_line": 4244, "section": "List items" }, { "markdown": " > > 1. one\n>>\n>> two\n", "html": "
\n
\n
    \n
  1. \n

    one

    \n

    two

    \n
    2. \n
\n
\n
\n", - "example": 229, - "start_line": 3895, - "end_line": 3910, + "example": 259, + "start_line": 4255, + "end_line": 4270, "section": "List items" }, { "markdown": ">>- one\n>>\n > > two\n", "html": "
\n
\n
    \n
  • one
    • \n
\n

two

\n
\n
\n", - "example": 230, - "start_line": 3922, - "end_line": 3935, + "example": 260, + "start_line": 4282, + "end_line": 4295, "section": "List items" }, { "markdown": "-one\n\n2.two\n", "html": "

-one

\n

2.two

\n", - "example": 231, - "start_line": 3941, - "end_line": 3948, + "example": 261, + "start_line": 4301, + "end_line": 4308, "section": "List items" }, { "markdown": "- foo\n\n\n bar\n", "html": "
    \n
  • \n

    foo

    \n

    bar

    \n
    • \n
\n", - "example": 232, - "start_line": 3954, - "end_line": 3966, + "example": 262, + "start_line": 4314, + "end_line": 4326, "section": "List items" }, { "markdown": "1. foo\n\n ```\n bar\n ```\n\n baz\n\n > bam\n", "html": "
    \n
  1. \n

    foo

    \n
    bar\n
    \n

    baz

    \n
    \n

    bam

    \n
    \n
    2. \n
\n", - "example": 233, - "start_line": 3971, - "end_line": 3993, + "example": 263, + "start_line": 4331, + "end_line": 4353, "section": "List items" }, { "markdown": "- Foo\n\n bar\n\n\n baz\n", "html": "
    \n
  • \n

    Foo

    \n
    bar\n\n\nbaz\n
    \n
    • \n
\n", - "example": 234, - "start_line": 3999, - "end_line": 4017, + "example": 264, + "start_line": 4359, + "end_line": 4377, "section": "List items" }, { "markdown": "123456789. ok\n", "html": "
    \n
  1. ok
    2. \n
\n", - "example": 235, - "start_line": 4021, - "end_line": 4027, + "example": 265, + "start_line": 4381, + "end_line": 4387, "section": "List items" }, { "markdown": "1234567890. not ok\n", "html": "

1234567890. not ok

\n", - "example": 236, - "start_line": 4030, - "end_line": 4034, + "example": 266, + "start_line": 4390, + "end_line": 4394, "section": "List items" }, { "markdown": "0. ok\n", "html": "
    \n
  1. ok
    2. \n
\n", - "example": 237, - "start_line": 4039, - "end_line": 4045, + "example": 267, + "start_line": 4399, + "end_line": 4405, "section": "List items" }, { "markdown": "003. ok\n", "html": "
    \n
  1. ok
    2. \n
\n", - "example": 238, - "start_line": 4048, - "end_line": 4054, + "example": 268, + "start_line": 4408, + "end_line": 4414, "section": "List items" }, { "markdown": "-1. not ok\n", "html": "

-1. not ok

\n", - "example": 239, - "start_line": 4059, - "end_line": 4063, + "example": 269, + "start_line": 4419, + "end_line": 4423, "section": "List items" }, { "markdown": "- foo\n\n bar\n", "html": "
    \n
  • \n

    foo

    \n
    bar\n
    \n
    • \n
\n", - "example": 240, - "start_line": 4082, - "end_line": 4094, + "example": 270, + "start_line": 4442, + "end_line": 4454, "section": "List items" }, { "markdown": " 10. foo\n\n bar\n", "html": "
    \n
  1. \n

    foo

    \n
    bar\n
    \n
    2. \n
\n", - "example": 241, - "start_line": 4099, - "end_line": 4111, + "example": 271, + "start_line": 4459, + "end_line": 4471, "section": "List items" }, { "markdown": " indented code\n\nparagraph\n\n more code\n", "html": "
indented code\n
\n

paragraph

\n
more code\n
\n", - "example": 242, - "start_line": 4118, - "end_line": 4130, + "example": 272, + "start_line": 4478, + "end_line": 4490, "section": "List items" }, { "markdown": "1. indented code\n\n paragraph\n\n more code\n", "html": "
    \n
  1. \n
    indented code\n
    \n

    paragraph

    \n
    more code\n
    \n
    2. \n
\n", - "example": 243, - "start_line": 4133, - "end_line": 4149, + "example": 273, + "start_line": 4493, + "end_line": 4509, "section": "List items" }, { "markdown": "1. indented code\n\n paragraph\n\n more code\n", "html": "
    \n
  1. \n
     indented code\n
    \n

    paragraph

    \n
    more code\n
    \n
    2. \n
\n", - "example": 244, - "start_line": 4155, - "end_line": 4171, + "example": 274, + "start_line": 4515, + "end_line": 4531, "section": "List items" }, { "markdown": " foo\n\nbar\n", "html": "

foo

\n

bar

\n", - "example": 245, - "start_line": 4182, - "end_line": 4189, + "example": 275, + "start_line": 4542, + "end_line": 4549, "section": "List items" }, { "markdown": "- foo\n\n bar\n", "html": "
    \n
  • foo
    • \n
\n

bar

\n", - "example": 246, - "start_line": 4192, - "end_line": 4201, + "example": 276, + "start_line": 4552, + "end_line": 4561, "section": "List items" }, { "markdown": "- foo\n\n bar\n", "html": "
    \n
  • \n

    foo

    \n

    bar

    \n
    • \n
\n", - "example": 247, - "start_line": 4209, - "end_line": 4220, + "example": 277, + "start_line": 4569, + "end_line": 4580, "section": "List items" }, { "markdown": "-\n foo\n-\n ```\n bar\n ```\n-\n baz\n", "html": "
    \n
  • foo
    • \n
  • \n
    bar\n
    \n
    • \n
  • \n
    baz\n
    \n
    • \n
\n", - "example": 248, - "start_line": 4237, - "end_line": 4258, + "example": 278, + "start_line": 4596, + "end_line": 4617, "section": "List items" }, { "markdown": "- \n foo\n", "html": "
    \n
  • foo
    • \n
\n", - "example": 249, - "start_line": 4263, - "end_line": 4270, + "example": 279, + "start_line": 4622, + "end_line": 4629, "section": "List items" }, { "markdown": "-\n\n foo\n", "html": "
    \n
  • \n
\n

foo

\n", - "example": 250, - "start_line": 4277, - "end_line": 4286, + "example": 280, + "start_line": 4636, + "end_line": 4645, "section": "List items" }, { "markdown": "- foo\n-\n- bar\n", "html": "
    \n
  • foo
    • \n
  • \n
  • bar
    • \n
\n", - "example": 251, - "start_line": 4291, - "end_line": 4301, + "example": 281, + "start_line": 4650, + "end_line": 4660, "section": "List items" }, { "markdown": "- foo\n- \n- bar\n", "html": "
    \n
  • foo
    • \n
  • \n
  • bar
    • \n
\n", - "example": 252, - "start_line": 4306, - "end_line": 4316, + "example": 282, + "start_line": 4665, + "end_line": 4675, "section": "List items" }, { "markdown": "1. foo\n2.\n3. bar\n", "html": "
    \n
  1. foo
    2. \n
  2. \n
  3. bar
    4. \n
\n", - "example": 253, - "start_line": 4321, - "end_line": 4331, + "example": 283, + "start_line": 4680, + "end_line": 4690, "section": "List items" }, { "markdown": "*\n", "html": "
    \n
  • \n
\n", - "example": 254, - "start_line": 4336, - "end_line": 4342, + "example": 284, + "start_line": 4695, + "end_line": 4701, "section": "List items" }, { "markdown": "foo\n*\n\nfoo\n1.\n", "html": "

foo\n*

\n

foo\n1.

\n", - "example": 255, - "start_line": 4346, - "end_line": 4357, + "example": 285, + "start_line": 4705, + "end_line": 4716, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
    2. \n
\n", - "example": 256, - "start_line": 4368, - "end_line": 4387, + "example": 286, + "start_line": 4727, + "end_line": 4746, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
    2. \n
\n", - "example": 257, - "start_line": 4392, - "end_line": 4411, + "example": 287, + "start_line": 4751, + "end_line": 4770, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
    2. \n
\n", - "example": 258, - "start_line": 4416, - "end_line": 4435, + "example": 288, + "start_line": 4775, + "end_line": 4794, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n\n indented code\n\n > A block quote.\n", "html": "
1.  A paragraph\n    with two lines.\n\n        indented code\n\n    > A block quote.\n
\n", - "example": 259, - "start_line": 4440, - "end_line": 4455, + "example": 289, + "start_line": 4799, + "end_line": 4814, "section": "List items" }, { "markdown": " 1. A paragraph\nwith two lines.\n\n indented code\n\n > A block quote.\n", "html": "
    \n
  1. \n

    A paragraph\nwith two lines.

    \n
    indented code\n
    \n
    \n

    A block quote.

    \n
    \n
    2. \n
\n", - "example": 260, - "start_line": 4470, - "end_line": 4489, + "example": 290, + "start_line": 4829, + "end_line": 4848, "section": "List items" }, { "markdown": " 1. A paragraph\n with two lines.\n", "html": "
    \n
  1. A paragraph\nwith two lines.
    2. \n
\n", - "example": 261, - "start_line": 4494, - "end_line": 4502, + "example": 291, + "start_line": 4853, + "end_line": 4861, "section": "List items" }, { "markdown": "> 1. > Blockquote\ncontinued here.\n", "html": "
\n
    \n
  1. \n
    \n

    Blockquote\ncontinued here.

    \n
    \n
    2. \n
\n
\n", - "example": 262, - "start_line": 4507, - "end_line": 4521, + "example": 292, + "start_line": 4866, + "end_line": 4880, "section": "List items" }, { "markdown": "> 1. > Blockquote\n> continued here.\n", "html": "
\n
    \n
  1. \n
    \n

    Blockquote\ncontinued here.

    \n
    \n
    2. \n
\n
\n", - "example": 263, - "start_line": 4524, - "end_line": 4538, + "example": 293, + "start_line": 4883, + "end_line": 4897, "section": "List items" }, { "markdown": "- foo\n - bar\n - baz\n - boo\n", "html": "
    \n
  • foo\n
      \n
    • bar\n
        \n
      • baz\n
          \n
        • boo
          • \n
        \n
        • \n
      \n
      • \n
    \n
    • \n
\n", - "example": 264, - "start_line": 4552, - "end_line": 4573, + "example": 294, + "start_line": 4911, + "end_line": 4932, "section": "List items" }, { "markdown": "- foo\n - bar\n - baz\n - boo\n", "html": "
    \n
  • foo
    • \n
  • bar
    • \n
  • baz
    • \n
  • boo
    • \n
\n", - "example": 265, - "start_line": 4578, - "end_line": 4590, + "example": 295, + "start_line": 4937, + "end_line": 4949, "section": "List items" }, { "markdown": "10) foo\n - bar\n", "html": "
    \n
  1. foo\n
      \n
    • bar
      • \n
    \n
    2. \n
\n", - "example": 266, - "start_line": 4595, - "end_line": 4606, + "example": 296, + "start_line": 4954, + "end_line": 4965, "section": "List items" }, { "markdown": "10) foo\n - bar\n", "html": "
    \n
  1. foo
    2. \n
\n
    \n
  • bar
    • \n
\n", - "example": 267, - "start_line": 4611, - "end_line": 4621, + "example": 297, + "start_line": 4970, + "end_line": 4980, "section": "List items" }, { "markdown": "- - foo\n", "html": "
    \n
  • \n
      \n
    • foo
      • \n
    \n
    • \n
\n", - "example": 268, - "start_line": 4626, - "end_line": 4636, + "example": 298, + "start_line": 4985, + "end_line": 4995, "section": "List items" }, { "markdown": "1. - 2. foo\n", "html": "
    \n
  1. \n
      \n
    • \n
        \n
      1. foo
        2. \n
      \n
      • \n
    \n
    2. \n
\n", - "example": 269, - "start_line": 4639, - "end_line": 4653, + "example": 299, + "start_line": 4998, + "end_line": 5012, "section": "List items" }, { "markdown": "- # Foo\n- Bar\n ---\n baz\n", "html": "
    \n
  • \n

    Foo

    \n
    • \n
  • \n

    Bar

    \nbaz
    • \n
\n", - "example": 270, - "start_line": 4658, - "end_line": 4672, + "example": 300, + "start_line": 5017, + "end_line": 5031, "section": "List items" }, { "markdown": "- foo\n- bar\n+ baz\n", "html": "
    \n
  • foo
    • \n
  • bar
    • \n
\n
    \n
  • baz
    • \n
\n", - "example": 271, - "start_line": 4894, - "end_line": 4906, + "example": 301, + "start_line": 5253, + "end_line": 5265, "section": "Lists" }, { "markdown": "1. foo\n2. bar\n3) baz\n", "html": "
    \n
  1. foo
    2. \n
  2. bar
    3. \n
\n
    \n
  1. baz
    2. \n
\n", - "example": 272, - "start_line": 4909, - "end_line": 4921, + "example": 302, + "start_line": 5268, + "end_line": 5280, "section": "Lists" }, { "markdown": "Foo\n- bar\n- baz\n", "html": "

Foo

\n
    \n
  • bar
    • \n
  • baz
    • \n
\n", - "example": 273, - "start_line": 4928, - "end_line": 4938, + "example": 303, + "start_line": 5287, + "end_line": 5297, "section": "Lists" }, { "markdown": "The number of windows in my house is\n14. The number of doors is 6.\n", "html": "

The number of windows in my house is\n14. The number of doors is 6.

\n", - "example": 274, - "start_line": 5005, - "end_line": 5011, + "example": 304, + "start_line": 5364, + "end_line": 5370, "section": "Lists" }, { "markdown": "The number of windows in my house is\n1. The number of doors is 6.\n", "html": "

The number of windows in my house is

\n
    \n
  1. The number of doors is 6.
    2. \n
\n", - "example": 275, - "start_line": 5015, - "end_line": 5023, + "example": 305, + "start_line": 5374, + "end_line": 5382, "section": "Lists" }, { "markdown": "- foo\n\n- bar\n\n\n- baz\n", "html": "
    \n
  • \n

    foo

    \n
    • \n
  • \n

    bar

    \n
    • \n
  • \n

    baz

    \n
    • \n
\n", - "example": 276, - "start_line": 5029, - "end_line": 5048, + "example": 306, + "start_line": 5388, + "end_line": 5407, "section": "Lists" }, { "markdown": "- foo\n - bar\n - baz\n\n\n bim\n", "html": "
    \n
  • foo\n
      \n
    • bar\n
        \n
      • \n

        baz

        \n

        bim

        \n
        • \n
      \n
      • \n
    \n
    • \n
\n", - "example": 277, - "start_line": 5050, - "end_line": 5072, + "example": 307, + "start_line": 5409, + "end_line": 5431, "section": "Lists" }, { "markdown": "- foo\n- bar\n\n\n\n- baz\n- bim\n", "html": "
    \n
  • foo
    • \n
  • bar
    • \n
\n\n
    \n
  • baz
    • \n
  • bim
    • \n
\n", - "example": 278, - "start_line": 5080, - "end_line": 5098, + "example": 308, + "start_line": 5439, + "end_line": 5457, "section": "Lists" }, { "markdown": "- foo\n\n notcode\n\n- foo\n\n\n\n code\n", "html": "
    \n
  • \n

    foo

    \n

    notcode

    \n
    • \n
  • \n

    foo

    \n
    • \n
\n\n
code\n
\n", - "example": 279, - "start_line": 5101, - "end_line": 5124, + "example": 309, + "start_line": 5460, + "end_line": 5483, "section": "Lists" }, { "markdown": "- a\n - b\n - c\n - d\n - e\n - f\n- g\n", "html": "
    \n
  • a
    • \n
  • b
    • \n
  • c
    • \n
  • d
    • \n
  • e
    • \n
  • f
    • \n
  • g
    • \n
\n", - "example": 280, - "start_line": 5132, - "end_line": 5150, + "example": 310, + "start_line": 5491, + "end_line": 5509, "section": "Lists" }, { "markdown": "1. a\n\n 2. b\n\n 3. c\n", "html": "
    \n
  1. \n

    a

    \n
    2. \n
  2. \n

    b

    \n
    3. \n
  3. \n

    c

    \n
    4. \n
\n", - "example": 281, - "start_line": 5153, - "end_line": 5171, + "example": 311, + "start_line": 5512, + "end_line": 5530, "section": "Lists" }, { "markdown": "- a\n - b\n - c\n - d\n - e\n", "html": "
    \n
  • a
    • \n
  • b
    • \n
  • c
    • \n
  • d\n- e
    • \n
\n", - "example": 282, - "start_line": 5177, - "end_line": 5191, + "example": 312, + "start_line": 5536, + "end_line": 5550, "section": "Lists" }, { "markdown": "1. a\n\n 2. b\n\n 3. c\n", "html": "
    \n
  1. \n

    a

    \n
    2. \n
  2. \n

    b

    \n
    3. \n
\n
3. c\n
\n", - "example": 283, - "start_line": 5197, - "end_line": 5214, + "example": 313, + "start_line": 5556, + "end_line": 5573, "section": "Lists" }, { "markdown": "- a\n- b\n\n- c\n", "html": "
    \n
  • \n

    a

    \n
    • \n
  • \n

    b

    \n
    • \n
  • \n

    c

    \n
    • \n
\n", - "example": 284, - "start_line": 5220, - "end_line": 5237, + "example": 314, + "start_line": 5579, + "end_line": 5596, "section": "Lists" }, { "markdown": "* a\n*\n\n* c\n", "html": "
    \n
  • \n

    a

    \n
    • \n
  • \n
  • \n

    c

    \n
    • \n
\n", - "example": 285, - "start_line": 5242, - "end_line": 5257, + "example": 315, + "start_line": 5601, + "end_line": 5616, "section": "Lists" }, { "markdown": "- a\n- b\n\n c\n- d\n", "html": "
    \n
  • \n

    a

    \n
    • \n
  • \n

    b

    \n

    c

    \n
    • \n
  • \n

    d

    \n
    • \n
\n", - "example": 286, - "start_line": 5264, - "end_line": 5283, + "example": 316, + "start_line": 5623, + "end_line": 5642, "section": "Lists" }, { "markdown": "- a\n- b\n\n [ref]: /url\n- d\n", "html": "
    \n
  • \n

    a

    \n
    • \n
  • \n

    b

    \n
    • \n
  • \n

    d

    \n
    • \n
\n", - "example": 287, - "start_line": 5286, - "end_line": 5304, + "example": 317, + "start_line": 5645, + "end_line": 5663, "section": "Lists" }, { "markdown": "- a\n- ```\n b\n\n\n ```\n- c\n", "html": "
    \n
  • a
    • \n
  • \n
    b\n\n\n
    \n
    • \n
  • c
    • \n
\n", - "example": 288, - "start_line": 5309, - "end_line": 5328, + "example": 318, + "start_line": 5668, + "end_line": 5687, "section": "Lists" }, { "markdown": "- a\n - b\n\n c\n- d\n", "html": "
    \n
  • a\n
      \n
    • \n

      b

      \n

      c

      \n
      • \n
    \n
    • \n
  • d
    • \n
\n", - "example": 289, - "start_line": 5335, - "end_line": 5353, + "example": 319, + "start_line": 5694, + "end_line": 5712, "section": "Lists" }, { "markdown": "* a\n > b\n >\n* c\n", "html": "
    \n
  • a\n
    \n

    b

    \n
    \n
    • \n
  • c
    • \n
\n", - "example": 290, - "start_line": 5359, - "end_line": 5373, + "example": 320, + "start_line": 5718, + "end_line": 5732, "section": "Lists" }, { "markdown": "- a\n > b\n ```\n c\n ```\n- d\n", "html": "
    \n
  • a\n
    \n

    b

    \n
    \n
    c\n
    \n
    • \n
  • d
    • \n
\n", - "example": 291, - "start_line": 5379, - "end_line": 5397, + "example": 321, + "start_line": 5738, + "end_line": 5756, "section": "Lists" }, { "markdown": "- a\n", "html": "
    \n
  • a
    • \n
\n", - "example": 292, - "start_line": 5402, - "end_line": 5408, + "example": 322, + "start_line": 5761, + "end_line": 5767, "section": "Lists" }, { "markdown": "- a\n - b\n", "html": "
    \n
  • a\n
      \n
    • b
      • \n
    \n
    • \n
\n", - "example": 293, - "start_line": 5411, - "end_line": 5422, + "example": 323, + "start_line": 5770, + "end_line": 5781, "section": "Lists" }, { "markdown": "1. ```\n foo\n ```\n\n bar\n", "html": "
    \n
  1. \n
    foo\n
    \n

    bar

    \n
    2. \n
\n", - "example": 294, - "start_line": 5428, - "end_line": 5442, + "example": 324, + "start_line": 5787, + "end_line": 5801, "section": "Lists" }, { "markdown": "* foo\n * bar\n\n baz\n", "html": "
    \n
  • \n

    foo

    \n
      \n
    • bar
      • \n
    \n

    baz

    \n
    • \n
\n", - "example": 295, - "start_line": 5447, - "end_line": 5462, + "example": 325, + "start_line": 5806, + "end_line": 5821, "section": "Lists" }, { "markdown": "- a\n - b\n - c\n\n- d\n - e\n - f\n", "html": "
    \n
  • \n

    a

    \n
      \n
    • b
      • \n
    • c
      • \n
    \n
    • \n
  • \n

    d

    \n
      \n
    • e
      • \n
    • f
      • \n
    \n
    • \n
\n", - "example": 296, - "start_line": 5465, - "end_line": 5490, + "example": 326, + "start_line": 5824, + "end_line": 5849, "section": "Lists" }, { "markdown": "`hi`lo`\n", "html": "

hilo`

\n", - "example": 297, - "start_line": 5499, - "end_line": 5503, - "section": "Inlines" - }, - { - "markdown": "\\!\\\"\\#\\$\\%\\&\\'\\(\\)\\*\\+\\,\\-\\.\\/\\:\\;\\<\\=\\>\\?\\@\\[\\\\\\]\\^\\_\\`\\{\\|\\}\\~\n", - "html": "

!"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~

\n", - "example": 298, - "start_line": 5513, - "end_line": 5517, - "section": "Backslash escapes" - }, - { - "markdown": "\\\t\\A\\a\\ \\3\\φ\\«\n", - "html": "

\\\t\\A\\a\\ \\3\\φ\\«

\n", - "example": 299, - "start_line": 5523, - "end_line": 5527, - "section": "Backslash escapes" - }, - { - "markdown": "\\*not emphasized*\n\\
not a tag\n\\[not a link](/foo)\n\\`not code`\n1\\. not a list\n\\* not a list\n\\# not a heading\n\\[foo]: /url \"not a reference\"\n\\ö not a character entity\n", - "html": "

*not emphasized*\n<br/> not a tag\n[not a link](/foo)\n`not code`\n1. not a list\n* not a list\n# not a heading\n[foo]: /url "not a reference"\n&ouml; not a character entity

\n", - "example": 300, - "start_line": 5533, - "end_line": 5553, - "section": "Backslash escapes" - }, - { - "markdown": "\\\\*emphasis*\n", - "html": "

\\emphasis

\n", - "example": 301, - "start_line": 5558, - "end_line": 5562, - "section": "Backslash escapes" - }, - { - "markdown": "foo\\\nbar\n", - "html": "

foo
\nbar

\n", - "example": 302, - "start_line": 5567, - "end_line": 5573, - "section": "Backslash escapes" - }, - { - "markdown": "`` \\[\\` ``\n", - "html": "

\\[\\`

\n", - "example": 303, - "start_line": 5579, - "end_line": 5583, - "section": "Backslash escapes" - }, - { - "markdown": " \\[\\]\n", - "html": "
\\[\\]\n
\n", - "example": 304, - "start_line": 5586, - "end_line": 5591, - "section": "Backslash escapes" - }, - { - "markdown": "~~~\n\\[\\]\n~~~\n", - "html": "
\\[\\]\n
\n", - "example": 305, - "start_line": 5594, - "end_line": 5601, - "section": "Backslash escapes" - }, - { - "markdown": "\n", - "html": "

http://example.com?find=\\*

\n", - "example": 306, - "start_line": 5604, - "end_line": 5608, - "section": "Backslash escapes" - }, - { - "markdown": "\n", - "html": "\n", - "example": 307, - "start_line": 5611, - "end_line": 5615, - "section": "Backslash escapes" - }, - { - "markdown": "[foo](/bar\\* \"ti\\*tle\")\n", - "html": "

foo

\n", - "example": 308, - "start_line": 5621, - "end_line": 5625, - "section": "Backslash escapes" - }, - { - "markdown": "[foo]\n\n[foo]: /bar\\* \"ti\\*tle\"\n", - "html": "

foo

\n", - "example": 309, - "start_line": 5628, - "end_line": 5634, - "section": "Backslash escapes" - }, - { - "markdown": "``` foo\\+bar\nfoo\n```\n", - "html": "
foo\n
\n", - "example": 310, - "start_line": 5637, - "end_line": 5644, - "section": "Backslash escapes" - }, - { - "markdown": "  & © Æ Ď\n¾ ℋ ⅆ\n∲ ≧̸\n", - "html": "

  & © Æ Ď\n¾ ℋ ⅆ\n∲ ≧̸

\n", - "example": 311, - "start_line": 5674, - "end_line": 5682, - "section": "Entity and numeric character references" - }, - { - "markdown": "# Ӓ Ϡ �\n", - "html": "

# Ӓ Ϡ �

\n", - "example": 312, - "start_line": 5693, - "end_line": 5697, - "section": "Entity and numeric character references" - }, - { - "markdown": "" ആ ಫ\n", - "html": "

" ആ ಫ

\n", - "example": 313, - "start_line": 5706, - "end_line": 5710, - "section": "Entity and numeric character references" - }, - { - "markdown": "  &x; &#; &#x;\n�\n&#abcdef0;\n&ThisIsNotDefined; &hi?;\n", - "html": "

&nbsp &x; &#; &#x;\n&#987654321;\n&#abcdef0;\n&ThisIsNotDefined; &hi?;

\n", - "example": 314, - "start_line": 5715, - "end_line": 5725, - "section": "Entity and numeric character references" - }, - { - "markdown": "©\n", - "html": "

&copy

\n", - "example": 315, - "start_line": 5732, - "end_line": 5736, - "section": "Entity and numeric character references" - }, - { - "markdown": "&MadeUpEntity;\n", - "html": "

&MadeUpEntity;

\n", - "example": 316, - "start_line": 5742, - "end_line": 5746, - "section": "Entity and numeric character references" - }, - { - "markdown": "\n", - "html": "\n", - "example": 317, - "start_line": 5753, - "end_line": 5757, - "section": "Entity and numeric character references" - }, - { - "markdown": "[foo](/föö \"föö\")\n", - "html": "

foo

\n", - "example": 318, - "start_line": 5760, - "end_line": 5764, - "section": "Entity and numeric character references" - }, - { - "markdown": "[foo]\n\n[foo]: /föö \"föö\"\n", - "html": "

foo

\n", - "example": 319, - "start_line": 5767, - "end_line": 5773, - "section": "Entity and numeric character references" - }, - { - "markdown": "``` föö\nfoo\n```\n", - "html": "
foo\n
\n", - "example": 320, - "start_line": 5776, - "end_line": 5783, - "section": "Entity and numeric character references" - }, - { - "markdown": "`föö`\n", - "html": "

f&ouml;&ouml;

\n", - "example": 321, - "start_line": 5789, - "end_line": 5793, - "section": "Entity and numeric character references" - }, - { - "markdown": " föfö\n", - "html": "
f&ouml;f&ouml;\n
\n", - "example": 322, - "start_line": 5796, - "end_line": 5801, - "section": "Entity and numeric character references" - }, - { - "markdown": "*foo*\n*foo*\n", - "html": "

*foo*\nfoo

\n", - "example": 323, - "start_line": 5808, - "end_line": 5814, - "section": "Entity and numeric character references" - }, - { - "markdown": "* foo\n\n* foo\n", - "html": "

* foo

\n
    \n
  • foo
    • \n
\n", - "example": 324, - "start_line": 5816, - "end_line": 5825, - "section": "Entity and numeric character references" - }, - { - "markdown": "foo bar\n", - "html": "

foo\n\nbar

\n", - "example": 325, - "start_line": 5827, - "end_line": 5833, - "section": "Entity and numeric character references" - }, - { - "markdown": " foo\n", - "html": "

\tfoo

\n", - "example": 326, - "start_line": 5835, - "end_line": 5839, - "section": "Entity and numeric character references" - }, - { - "markdown": "[a](url "tit")\n", - "html": "

[a](url "tit")

\n", "example": 327, - "start_line": 5842, - "end_line": 5846, - "section": "Entity and numeric character references" + "start_line": 5858, + "end_line": 5862, + "section": "Inlines" }, { "markdown": "`foo`\n", "html": "

foo

\n", "example": 328, - "start_line": 5870, - "end_line": 5874, + "start_line": 5890, + "end_line": 5894, "section": "Code spans" }, { "markdown": "`` foo ` bar ``\n", "html": "

foo ` bar

\n", "example": 329, - "start_line": 5881, - "end_line": 5885, + "start_line": 5901, + "end_line": 5905, "section": "Code spans" }, { "markdown": "` `` `\n", "html": "

``

\n", "example": 330, - "start_line": 5891, - "end_line": 5895, + "start_line": 5911, + "end_line": 5915, "section": "Code spans" }, { "markdown": "` `` `\n", "html": "

``

\n", "example": 331, - "start_line": 5899, - "end_line": 5903, + "start_line": 5919, + "end_line": 5923, "section": "Code spans" }, { "markdown": "` a`\n", "html": "

a

\n", "example": 332, - "start_line": 5908, - "end_line": 5912, + "start_line": 5928, + "end_line": 5932, "section": "Code spans" }, { "markdown": "` b `\n", "html": "

 b 

\n", "example": 333, - "start_line": 5917, - "end_line": 5921, + "start_line": 5937, + "end_line": 5941, "section": "Code spans" }, { "markdown": "` `\n` `\n", "html": "

 \n

\n", "example": 334, - "start_line": 5925, - "end_line": 5931, + "start_line": 5945, + "end_line": 5951, "section": "Code spans" }, { "markdown": "``\nfoo\nbar \nbaz\n``\n", "html": "

foo bar baz

\n", "example": 335, - "start_line": 5936, - "end_line": 5944, + "start_line": 5956, + "end_line": 5964, "section": "Code spans" }, { "markdown": "``\nfoo \n``\n", "html": "

foo

\n", "example": 336, - "start_line": 5946, - "end_line": 5952, + "start_line": 5966, + "end_line": 5972, "section": "Code spans" }, { "markdown": "`foo bar \nbaz`\n", "html": "

foo bar baz

\n", "example": 337, - "start_line": 5957, - "end_line": 5962, + "start_line": 5977, + "end_line": 5982, "section": "Code spans" }, { "markdown": "`foo\\`bar`\n", "html": "

foo\\bar`

\n", "example": 338, - "start_line": 5974, - "end_line": 5978, + "start_line": 5994, + "end_line": 5998, "section": "Code spans" }, { "markdown": "``foo`bar``\n", "html": "

foo`bar

\n", "example": 339, - "start_line": 5985, - "end_line": 5989, + "start_line": 6005, + "end_line": 6009, "section": "Code spans" }, { "markdown": "` foo `` bar `\n", "html": "

foo `` bar

\n", "example": 340, - "start_line": 5991, - "end_line": 5995, + "start_line": 6011, + "end_line": 6015, "section": "Code spans" }, { "markdown": "*foo`*`\n", "html": "

*foo*

\n", "example": 341, - "start_line": 6003, - "end_line": 6007, + "start_line": 6023, + "end_line": 6027, "section": "Code spans" }, { "markdown": "[not a `link](/foo`)\n", "html": "

[not a link](/foo)

\n", "example": 342, - "start_line": 6012, - "end_line": 6016, + "start_line": 6032, + "end_line": 6036, "section": "Code spans" }, { "markdown": "``\n", "html": "

<a href="">`

\n", "example": 343, - "start_line": 6022, - "end_line": 6026, + "start_line": 6042, + "end_line": 6046, "section": "Code spans" }, { "markdown": "`\n", "html": "

`

\n", "example": 344, - "start_line": 6031, - "end_line": 6035, + "start_line": 6051, + "end_line": 6055, "section": "Code spans" }, { "markdown": "``\n", "html": "

<http://foo.bar.baz>`

\n", "example": 345, - "start_line": 6040, - "end_line": 6044, + "start_line": 6060, + "end_line": 6064, "section": "Code spans" }, { "markdown": "`\n", "html": "

http://foo.bar.`baz`

\n", "example": 346, - "start_line": 6049, - "end_line": 6053, + "start_line": 6069, + "end_line": 6073, "section": "Code spans" }, { "markdown": "```foo``\n", "html": "

```foo``

\n", "example": 347, - "start_line": 6059, - "end_line": 6063, + "start_line": 6079, + "end_line": 6083, "section": "Code spans" }, { "markdown": "`foo\n", "html": "

`foo

\n", "example": 348, - "start_line": 6066, - "end_line": 6070, + "start_line": 6086, + "end_line": 6090, "section": "Code spans" }, { "markdown": "`foo``bar``\n", "html": "

`foobar

\n", "example": 349, - "start_line": 6075, - "end_line": 6079, + "start_line": 6095, + "end_line": 6099, "section": "Code spans" }, { "markdown": "*foo bar*\n", "html": "

foo bar

\n", "example": 350, - "start_line": 6292, - "end_line": 6296, + "start_line": 6312, + "end_line": 6316, "section": "Emphasis and strong emphasis" }, { "markdown": "a * foo bar*\n", "html": "

a * foo bar*

\n", "example": 351, - "start_line": 6302, - "end_line": 6306, + "start_line": 6322, + "end_line": 6326, "section": "Emphasis and strong emphasis" }, { "markdown": "a*\"foo\"*\n", "html": "

a*"foo"*

\n", "example": 352, - "start_line": 6313, - "end_line": 6317, + "start_line": 6333, + "end_line": 6337, "section": "Emphasis and strong emphasis" }, { "markdown": "* a *\n", "html": "

* a *

\n", "example": 353, - "start_line": 6322, - "end_line": 6326, + "start_line": 6342, + "end_line": 6346, "section": "Emphasis and strong emphasis" }, { "markdown": "foo*bar*\n", "html": "

foobar

\n", "example": 354, - "start_line": 6331, - "end_line": 6335, + "start_line": 6351, + "end_line": 6355, "section": "Emphasis and strong emphasis" }, { "markdown": "5*6*78\n", "html": "

5678

\n", "example": 355, - "start_line": 6338, - "end_line": 6342, + "start_line": 6358, + "end_line": 6362, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo bar_\n", "html": "

foo bar

\n", "example": 356, - "start_line": 6347, - "end_line": 6351, + "start_line": 6367, + "end_line": 6371, "section": "Emphasis and strong emphasis" }, { "markdown": "_ foo bar_\n", "html": "

_ foo bar_

\n", "example": 357, - "start_line": 6357, - "end_line": 6361, + "start_line": 6377, + "end_line": 6381, "section": "Emphasis and strong emphasis" }, { "markdown": "a_\"foo\"_\n", "html": "

a_"foo"_

\n", "example": 358, - "start_line": 6367, - "end_line": 6371, + "start_line": 6387, + "end_line": 6391, "section": "Emphasis and strong emphasis" }, { "markdown": "foo_bar_\n", "html": "

foo_bar_

\n", "example": 359, - "start_line": 6376, - "end_line": 6380, + "start_line": 6396, + "end_line": 6400, "section": "Emphasis and strong emphasis" }, { "markdown": "5_6_78\n", "html": "

5_6_78

\n", "example": 360, - "start_line": 6383, - "end_line": 6387, + "start_line": 6403, + "end_line": 6407, "section": "Emphasis and strong emphasis" }, { "markdown": "пристаням_стремятся_\n", "html": "

пристаням_стремятся_

\n", "example": 361, - "start_line": 6390, - "end_line": 6394, + "start_line": 6410, + "end_line": 6414, "section": "Emphasis and strong emphasis" }, { "markdown": "aa_\"bb\"_cc\n", "html": "

aa_"bb"_cc

\n", "example": 362, - "start_line": 6400, - "end_line": 6404, + "start_line": 6420, + "end_line": 6424, "section": "Emphasis and strong emphasis" }, { "markdown": "foo-_(bar)_\n", "html": "

foo-(bar)

\n", "example": 363, - "start_line": 6411, - "end_line": 6415, + "start_line": 6431, + "end_line": 6435, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo*\n", "html": "

_foo*

\n", "example": 364, - "start_line": 6423, - "end_line": 6427, + "start_line": 6443, + "end_line": 6447, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo bar *\n", "html": "

*foo bar *

\n", "example": 365, - "start_line": 6433, - "end_line": 6437, + "start_line": 6453, + "end_line": 6457, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo bar\n*\n", "html": "

*foo bar\n*

\n", "example": 366, - "start_line": 6442, - "end_line": 6448, + "start_line": 6462, + "end_line": 6468, "section": "Emphasis and strong emphasis" }, { "markdown": "*(*foo)\n", "html": "

*(*foo)

\n", "example": 367, - "start_line": 6455, - "end_line": 6459, + "start_line": 6475, + "end_line": 6479, "section": "Emphasis and strong emphasis" }, { "markdown": "*(*foo*)*\n", "html": "

(foo)

\n", "example": 368, - "start_line": 6465, - "end_line": 6469, + "start_line": 6485, + "end_line": 6489, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo*bar\n", "html": "

foobar

\n", "example": 369, - "start_line": 6474, - "end_line": 6478, + "start_line": 6494, + "end_line": 6498, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo bar _\n", "html": "

_foo bar _

\n", "example": 370, - "start_line": 6487, - "end_line": 6491, + "start_line": 6507, + "end_line": 6511, "section": "Emphasis and strong emphasis" }, { "markdown": "_(_foo)\n", "html": "

_(_foo)

\n", "example": 371, - "start_line": 6497, - "end_line": 6501, + "start_line": 6517, + "end_line": 6521, "section": "Emphasis and strong emphasis" }, { "markdown": "_(_foo_)_\n", "html": "

(foo)

\n", "example": 372, - "start_line": 6506, - "end_line": 6510, + "start_line": 6526, + "end_line": 6530, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo_bar\n", "html": "

_foo_bar

\n", "example": 373, - "start_line": 6515, - "end_line": 6519, + "start_line": 6535, + "end_line": 6539, "section": "Emphasis and strong emphasis" }, { "markdown": "_пристаням_стремятся\n", "html": "

_пристаням_стремятся

\n", "example": 374, - "start_line": 6522, - "end_line": 6526, + "start_line": 6542, + "end_line": 6546, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo_bar_baz_\n", "html": "

foo_bar_baz

\n", "example": 375, - "start_line": 6529, - "end_line": 6533, + "start_line": 6549, + "end_line": 6553, "section": "Emphasis and strong emphasis" }, { "markdown": "_(bar)_.\n", "html": "

(bar).

\n", "example": 376, - "start_line": 6540, - "end_line": 6544, + "start_line": 6560, + "end_line": 6564, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo bar**\n", "html": "

foo bar

\n", "example": 377, - "start_line": 6549, - "end_line": 6553, + "start_line": 6569, + "end_line": 6573, "section": "Emphasis and strong emphasis" }, { "markdown": "** foo bar**\n", "html": "

** foo bar**

\n", "example": 378, - "start_line": 6559, - "end_line": 6563, + "start_line": 6579, + "end_line": 6583, "section": "Emphasis and strong emphasis" }, { "markdown": "a**\"foo\"**\n", "html": "

a**"foo"**

\n", "example": 379, - "start_line": 6570, - "end_line": 6574, + "start_line": 6590, + "end_line": 6594, "section": "Emphasis and strong emphasis" }, { "markdown": "foo**bar**\n", "html": "

foobar

\n", "example": 380, - "start_line": 6579, - "end_line": 6583, + "start_line": 6599, + "end_line": 6603, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo bar__\n", "html": "

foo bar

\n", "example": 381, - "start_line": 6588, - "end_line": 6592, + "start_line": 6608, + "end_line": 6612, "section": "Emphasis and strong emphasis" }, { "markdown": "__ foo bar__\n", "html": "

__ foo bar__

\n", "example": 382, - "start_line": 6598, - "end_line": 6602, + "start_line": 6618, + "end_line": 6622, "section": "Emphasis and strong emphasis" }, { "markdown": "__\nfoo bar__\n", "html": "

__\nfoo bar__

\n", "example": 383, - "start_line": 6606, - "end_line": 6612, + "start_line": 6626, + "end_line": 6632, "section": "Emphasis and strong emphasis" }, { "markdown": "a__\"foo\"__\n", "html": "

a__"foo"__

\n", "example": 384, - "start_line": 6618, - "end_line": 6622, + "start_line": 6638, + "end_line": 6642, "section": "Emphasis and strong emphasis" }, { "markdown": "foo__bar__\n", "html": "

foo__bar__

\n", "example": 385, - "start_line": 6627, - "end_line": 6631, + "start_line": 6647, + "end_line": 6651, "section": "Emphasis and strong emphasis" }, { "markdown": "5__6__78\n", "html": "

5__6__78

\n", "example": 386, - "start_line": 6634, - "end_line": 6638, + "start_line": 6654, + "end_line": 6658, "section": "Emphasis and strong emphasis" }, { "markdown": "пристаням__стремятся__\n", "html": "

пристаням__стремятся__

\n", "example": 387, - "start_line": 6641, - "end_line": 6645, + "start_line": 6661, + "end_line": 6665, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo, __bar__, baz__\n", "html": "

foo, bar, baz

\n", "example": 388, - "start_line": 6648, - "end_line": 6652, + "start_line": 6668, + "end_line": 6672, "section": "Emphasis and strong emphasis" }, { "markdown": "foo-__(bar)__\n", "html": "

foo-(bar)

\n", "example": 389, - "start_line": 6659, - "end_line": 6663, + "start_line": 6679, + "end_line": 6683, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo bar **\n", "html": "

**foo bar **

\n", "example": 390, - "start_line": 6672, - "end_line": 6676, + "start_line": 6692, + "end_line": 6696, "section": "Emphasis and strong emphasis" }, { "markdown": "**(**foo)\n", "html": "

**(**foo)

\n", "example": 391, - "start_line": 6685, - "end_line": 6689, + "start_line": 6705, + "end_line": 6709, "section": "Emphasis and strong emphasis" }, { "markdown": "*(**foo**)*\n", "html": "

(foo)

\n", "example": 392, - "start_line": 6695, - "end_line": 6699, + "start_line": 6715, + "end_line": 6719, "section": "Emphasis and strong emphasis" }, { "markdown": "**Gomphocarpus (*Gomphocarpus physocarpus*, syn.\n*Asclepias physocarpa*)**\n", "html": "

Gomphocarpus (Gomphocarpus physocarpus, syn.\nAsclepias physocarpa)

\n", "example": 393, - "start_line": 6702, - "end_line": 6708, + "start_line": 6722, + "end_line": 6728, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo \"*bar*\" foo**\n", "html": "

foo "bar" foo

\n", "example": 394, - "start_line": 6711, - "end_line": 6715, + "start_line": 6731, + "end_line": 6735, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo**bar\n", "html": "

foobar

\n", "example": 395, - "start_line": 6720, - "end_line": 6724, + "start_line": 6740, + "end_line": 6744, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo bar __\n", "html": "

__foo bar __

\n", "example": 396, - "start_line": 6732, - "end_line": 6736, + "start_line": 6752, + "end_line": 6756, "section": "Emphasis and strong emphasis" }, { "markdown": "__(__foo)\n", "html": "

__(__foo)

\n", "example": 397, - "start_line": 6742, - "end_line": 6746, + "start_line": 6762, + "end_line": 6766, "section": "Emphasis and strong emphasis" }, { "markdown": "_(__foo__)_\n", "html": "

(foo)

\n", "example": 398, - "start_line": 6752, - "end_line": 6756, + "start_line": 6772, + "end_line": 6776, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo__bar\n", "html": "

__foo__bar

\n", "example": 399, - "start_line": 6761, - "end_line": 6765, + "start_line": 6781, + "end_line": 6785, "section": "Emphasis and strong emphasis" }, { "markdown": "__пристаням__стремятся\n", "html": "

__пристаням__стремятся

\n", "example": 400, - "start_line": 6768, - "end_line": 6772, + "start_line": 6788, + "end_line": 6792, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo__bar__baz__\n", "html": "

foo__bar__baz

\n", "example": 401, - "start_line": 6775, - "end_line": 6779, + "start_line": 6795, + "end_line": 6799, "section": "Emphasis and strong emphasis" }, { "markdown": "__(bar)__.\n", "html": "

(bar).

\n", "example": 402, - "start_line": 6786, - "end_line": 6790, + "start_line": 6806, + "end_line": 6810, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo [bar](/url)*\n", "html": "

foo bar

\n", "example": 403, - "start_line": 6798, - "end_line": 6802, + "start_line": 6818, + "end_line": 6822, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo\nbar*\n", "html": "

foo\nbar

\n", "example": 404, - "start_line": 6805, - "end_line": 6811, + "start_line": 6825, + "end_line": 6831, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo __bar__ baz_\n", "html": "

foo bar baz

\n", "example": 405, - "start_line": 6817, - "end_line": 6821, + "start_line": 6837, + "end_line": 6841, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo _bar_ baz_\n", "html": "

foo bar baz

\n", "example": 406, - "start_line": 6824, - "end_line": 6828, + "start_line": 6844, + "end_line": 6848, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo_ bar_\n", "html": "

foo bar

\n", "example": 407, - "start_line": 6831, - "end_line": 6835, + "start_line": 6851, + "end_line": 6855, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo *bar**\n", "html": "

foo bar

\n", "example": 408, - "start_line": 6838, - "end_line": 6842, + "start_line": 6858, + "end_line": 6862, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo **bar** baz*\n", "html": "

foo bar baz

\n", "example": 409, - "start_line": 6845, - "end_line": 6849, + "start_line": 6865, + "end_line": 6869, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo**bar**baz*\n", "html": "

foobarbaz

\n", "example": 410, - "start_line": 6851, - "end_line": 6855, + "start_line": 6871, + "end_line": 6875, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo**bar*\n", "html": "

foo**bar

\n", "example": 411, - "start_line": 6875, - "end_line": 6879, + "start_line": 6895, + "end_line": 6899, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo** bar*\n", "html": "

foo bar

\n", "example": 412, - "start_line": 6888, - "end_line": 6892, + "start_line": 6908, + "end_line": 6912, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo **bar***\n", "html": "

foo bar

\n", "example": 413, - "start_line": 6895, - "end_line": 6899, + "start_line": 6915, + "end_line": 6919, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo**bar***\n", "html": "

foobar

\n", "example": 414, - "start_line": 6902, - "end_line": 6906, + "start_line": 6922, + "end_line": 6926, "section": "Emphasis and strong emphasis" }, { "markdown": "foo***bar***baz\n", "html": "

foobarbaz

\n", "example": 415, - "start_line": 6913, - "end_line": 6917, + "start_line": 6933, + "end_line": 6937, "section": "Emphasis and strong emphasis" }, { "markdown": "foo******bar*********baz\n", "html": "

foobar***baz

\n", "example": 416, - "start_line": 6919, - "end_line": 6923, + "start_line": 6939, + "end_line": 6943, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo **bar *baz* bim** bop*\n", "html": "

foo bar baz bim bop

\n", "example": 417, - "start_line": 6928, - "end_line": 6932, + "start_line": 6948, + "end_line": 6952, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo [*bar*](/url)*\n", "html": "

foo bar

\n", "example": 418, - "start_line": 6935, - "end_line": 6939, + "start_line": 6955, + "end_line": 6959, "section": "Emphasis and strong emphasis" }, { "markdown": "** is not an empty emphasis\n", "html": "

** is not an empty emphasis

\n", "example": 419, - "start_line": 6944, - "end_line": 6948, + "start_line": 6964, + "end_line": 6968, "section": "Emphasis and strong emphasis" }, { "markdown": "**** is not an empty strong emphasis\n", "html": "

**** is not an empty strong emphasis

\n", "example": 420, - "start_line": 6951, - "end_line": 6955, + "start_line": 6971, + "end_line": 6975, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo [bar](/url)**\n", "html": "

foo bar

\n", "example": 421, - "start_line": 6964, - "end_line": 6968, + "start_line": 6984, + "end_line": 6988, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo\nbar**\n", "html": "

foo\nbar

\n", "example": 422, - "start_line": 6971, - "end_line": 6977, + "start_line": 6991, + "end_line": 6997, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo _bar_ baz__\n", "html": "

foo bar baz

\n", "example": 423, - "start_line": 6983, - "end_line": 6987, + "start_line": 7003, + "end_line": 7007, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo __bar__ baz__\n", "html": "

foo bar baz

\n", "example": 424, - "start_line": 6990, - "end_line": 6994, + "start_line": 7010, + "end_line": 7014, "section": "Emphasis and strong emphasis" }, { "markdown": "____foo__ bar__\n", "html": "

foo bar

\n", "example": 425, - "start_line": 6997, - "end_line": 7001, + "start_line": 7017, + "end_line": 7021, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo **bar****\n", "html": "

foo bar

\n", "example": 426, - "start_line": 7004, - "end_line": 7008, + "start_line": 7024, + "end_line": 7028, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo *bar* baz**\n", "html": "

foo bar baz

\n", "example": 427, - "start_line": 7011, - "end_line": 7015, + "start_line": 7031, + "end_line": 7035, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo*bar*baz**\n", "html": "

foobarbaz

\n", "example": 428, - "start_line": 7018, - "end_line": 7022, + "start_line": 7038, + "end_line": 7042, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo* bar**\n", "html": "

foo bar

\n", "example": 429, - "start_line": 7025, - "end_line": 7029, + "start_line": 7045, + "end_line": 7049, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo *bar***\n", "html": "

foo bar

\n", "example": 430, - "start_line": 7032, - "end_line": 7036, + "start_line": 7052, + "end_line": 7056, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo *bar **baz**\nbim* bop**\n", "html": "

foo bar baz\nbim bop

\n", "example": 431, - "start_line": 7041, - "end_line": 7047, + "start_line": 7061, + "end_line": 7067, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo [*bar*](/url)**\n", "html": "

foo bar

\n", "example": 432, - "start_line": 7050, - "end_line": 7054, + "start_line": 7070, + "end_line": 7074, "section": "Emphasis and strong emphasis" }, { "markdown": "__ is not an empty emphasis\n", "html": "

__ is not an empty emphasis

\n", "example": 433, - "start_line": 7059, - "end_line": 7063, + "start_line": 7079, + "end_line": 7083, "section": "Emphasis and strong emphasis" }, { "markdown": "____ is not an empty strong emphasis\n", "html": "

____ is not an empty strong emphasis

\n", "example": 434, - "start_line": 7066, - "end_line": 7070, + "start_line": 7086, + "end_line": 7090, "section": "Emphasis and strong emphasis" }, { "markdown": "foo ***\n", "html": "

foo ***

\n", "example": 435, - "start_line": 7076, - "end_line": 7080, + "start_line": 7096, + "end_line": 7100, "section": "Emphasis and strong emphasis" }, { "markdown": "foo *\\**\n", "html": "

foo *

\n", "example": 436, - "start_line": 7083, - "end_line": 7087, + "start_line": 7103, + "end_line": 7107, "section": "Emphasis and strong emphasis" }, { "markdown": "foo *_*\n", "html": "

foo _

\n", "example": 437, - "start_line": 7090, - "end_line": 7094, + "start_line": 7110, + "end_line": 7114, "section": "Emphasis and strong emphasis" }, { "markdown": "foo *****\n", "html": "

foo *****

\n", "example": 438, - "start_line": 7097, - "end_line": 7101, + "start_line": 7117, + "end_line": 7121, "section": "Emphasis and strong emphasis" }, { "markdown": "foo **\\***\n", "html": "

foo *

\n", "example": 439, - "start_line": 7104, - "end_line": 7108, + "start_line": 7124, + "end_line": 7128, "section": "Emphasis and strong emphasis" }, { "markdown": "foo **_**\n", "html": "

foo _

\n", "example": 440, - "start_line": 7111, - "end_line": 7115, + "start_line": 7131, + "end_line": 7135, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo*\n", "html": "

*foo

\n", "example": 441, - "start_line": 7122, - "end_line": 7126, + "start_line": 7142, + "end_line": 7146, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo**\n", "html": "

foo*

\n", "example": 442, - "start_line": 7129, - "end_line": 7133, + "start_line": 7149, + "end_line": 7153, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo**\n", "html": "

*foo

\n", "example": 443, - "start_line": 7136, - "end_line": 7140, + "start_line": 7156, + "end_line": 7160, "section": "Emphasis and strong emphasis" }, { "markdown": "****foo*\n", "html": "

***foo

\n", "example": 444, - "start_line": 7143, - "end_line": 7147, + "start_line": 7163, + "end_line": 7167, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo***\n", "html": "

foo*

\n", "example": 445, - "start_line": 7150, - "end_line": 7154, + "start_line": 7170, + "end_line": 7174, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo****\n", "html": "

foo***

\n", "example": 446, - "start_line": 7157, - "end_line": 7161, + "start_line": 7177, + "end_line": 7181, "section": "Emphasis and strong emphasis" }, { "markdown": "foo ___\n", "html": "

foo ___

\n", "example": 447, - "start_line": 7167, - "end_line": 7171, + "start_line": 7187, + "end_line": 7191, "section": "Emphasis and strong emphasis" }, { "markdown": "foo _\\__\n", "html": "

foo _

\n", "example": 448, - "start_line": 7174, - "end_line": 7178, + "start_line": 7194, + "end_line": 7198, "section": "Emphasis and strong emphasis" }, { "markdown": "foo _*_\n", "html": "

foo *

\n", "example": 449, - "start_line": 7181, - "end_line": 7185, + "start_line": 7201, + "end_line": 7205, "section": "Emphasis and strong emphasis" }, { "markdown": "foo _____\n", "html": "

foo _____

\n", "example": 450, - "start_line": 7188, - "end_line": 7192, + "start_line": 7208, + "end_line": 7212, "section": "Emphasis and strong emphasis" }, { "markdown": "foo __\\___\n", "html": "

foo _

\n", "example": 451, - "start_line": 7195, - "end_line": 7199, + "start_line": 7215, + "end_line": 7219, "section": "Emphasis and strong emphasis" }, { "markdown": "foo __*__\n", "html": "

foo *

\n", "example": 452, - "start_line": 7202, - "end_line": 7206, + "start_line": 7222, + "end_line": 7226, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo_\n", "html": "

_foo

\n", "example": 453, - "start_line": 7209, - "end_line": 7213, + "start_line": 7229, + "end_line": 7233, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo__\n", "html": "

foo_

\n", "example": 454, - "start_line": 7220, - "end_line": 7224, + "start_line": 7240, + "end_line": 7244, "section": "Emphasis and strong emphasis" }, { "markdown": "___foo__\n", "html": "

_foo

\n", "example": 455, - "start_line": 7227, - "end_line": 7231, + "start_line": 7247, + "end_line": 7251, "section": "Emphasis and strong emphasis" }, { "markdown": "____foo_\n", "html": "

___foo

\n", "example": 456, - "start_line": 7234, - "end_line": 7238, + "start_line": 7254, + "end_line": 7258, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo___\n", "html": "

foo_

\n", "example": 457, - "start_line": 7241, - "end_line": 7245, + "start_line": 7261, + "end_line": 7265, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo____\n", "html": "

foo___

\n", "example": 458, - "start_line": 7248, - "end_line": 7252, + "start_line": 7268, + "end_line": 7272, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo**\n", "html": "

foo

\n", "example": 459, - "start_line": 7258, - "end_line": 7262, + "start_line": 7278, + "end_line": 7282, "section": "Emphasis and strong emphasis" }, { "markdown": "*_foo_*\n", "html": "

foo

\n", "example": 460, - "start_line": 7265, - "end_line": 7269, + "start_line": 7285, + "end_line": 7289, "section": "Emphasis and strong emphasis" }, { "markdown": "__foo__\n", "html": "

foo

\n", "example": 461, - "start_line": 7272, - "end_line": 7276, + "start_line": 7292, + "end_line": 7296, "section": "Emphasis and strong emphasis" }, { "markdown": "_*foo*_\n", "html": "

foo

\n", "example": 462, - "start_line": 7279, - "end_line": 7283, + "start_line": 7299, + "end_line": 7303, "section": "Emphasis and strong emphasis" }, { "markdown": "****foo****\n", "html": "

foo

\n", "example": 463, - "start_line": 7289, - "end_line": 7293, + "start_line": 7309, + "end_line": 7313, "section": "Emphasis and strong emphasis" }, { "markdown": "____foo____\n", "html": "

foo

\n", "example": 464, - "start_line": 7296, - "end_line": 7300, + "start_line": 7316, + "end_line": 7320, "section": "Emphasis and strong emphasis" }, { "markdown": "******foo******\n", "html": "

foo

\n", "example": 465, - "start_line": 7307, - "end_line": 7311, + "start_line": 7327, + "end_line": 7331, "section": "Emphasis and strong emphasis" }, { "markdown": "***foo***\n", "html": "

foo

\n", "example": 466, - "start_line": 7316, - "end_line": 7320, + "start_line": 7336, + "end_line": 7340, "section": "Emphasis and strong emphasis" }, { "markdown": "_____foo_____\n", "html": "

foo

\n", "example": 467, - "start_line": 7323, - "end_line": 7327, + "start_line": 7343, + "end_line": 7347, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo _bar* baz_\n", "html": "

foo _bar baz_

\n", "example": 468, - "start_line": 7332, - "end_line": 7336, + "start_line": 7352, + "end_line": 7356, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo __bar *baz bim__ bam*\n", "html": "

foo bar *baz bim bam

\n", "example": 469, - "start_line": 7339, - "end_line": 7343, + "start_line": 7359, + "end_line": 7363, "section": "Emphasis and strong emphasis" }, { "markdown": "**foo **bar baz**\n", "html": "

**foo bar baz

\n", "example": 470, - "start_line": 7348, - "end_line": 7352, + "start_line": 7368, + "end_line": 7372, "section": "Emphasis and strong emphasis" }, { "markdown": "*foo *bar baz*\n", "html": "

*foo bar baz

\n", "example": 471, - "start_line": 7355, - "end_line": 7359, + "start_line": 7375, + "end_line": 7379, "section": "Emphasis and strong emphasis" }, { "markdown": "*[bar*](/url)\n", "html": "

*bar*

\n", "example": 472, - "start_line": 7364, - "end_line": 7368, + "start_line": 7384, + "end_line": 7388, "section": "Emphasis and strong emphasis" }, { "markdown": "_foo [bar_](/url)\n", "html": "

_foo bar_

\n", "example": 473, - "start_line": 7371, - "end_line": 7375, + "start_line": 7391, + "end_line": 7395, "section": "Emphasis and strong emphasis" }, { "markdown": "*\n", "html": "

*

\n", "example": 474, - "start_line": 7378, - "end_line": 7382, + "start_line": 7398, + "end_line": 7402, "section": "Emphasis and strong emphasis" }, { "markdown": "**\n", "html": "

**

\n", "example": 475, - "start_line": 7385, - "end_line": 7389, + "start_line": 7405, + "end_line": 7409, "section": "Emphasis and strong emphasis" }, { "markdown": "__\n", "html": "

__

\n", "example": 476, - "start_line": 7392, - "end_line": 7396, + "start_line": 7412, + "end_line": 7416, "section": "Emphasis and strong emphasis" }, { "markdown": "*a `*`*\n", "html": "

a *

\n", "example": 477, - "start_line": 7399, - "end_line": 7403, + "start_line": 7419, + "end_line": 7423, "section": "Emphasis and strong emphasis" }, { "markdown": "_a `_`_\n", "html": "

a _

\n", "example": 478, - "start_line": 7406, - "end_line": 7410, + "start_line": 7426, + "end_line": 7430, "section": "Emphasis and strong emphasis" }, { "markdown": "**a\n", "html": "

**ahttp://foo.bar/?q=**

\n", "example": 479, - "start_line": 7413, - "end_line": 7417, + "start_line": 7433, + "end_line": 7437, "section": "Emphasis and strong emphasis" }, { "markdown": "__a\n", "html": "

__ahttp://foo.bar/?q=__

\n", "example": 480, - "start_line": 7420, - "end_line": 7424, + "start_line": 7440, + "end_line": 7444, "section": "Emphasis and strong emphasis" }, { "markdown": "[link](/uri \"title\")\n", "html": "

link

\n", "example": 481, - "start_line": 7503, - "end_line": 7507, + "start_line": 7528, + "end_line": 7532, "section": "Links" }, { "markdown": "[link](/uri)\n", "html": "

link

\n", "example": 482, - "start_line": 7512, - "end_line": 7516, + "start_line": 7538, + "end_line": 7542, + "section": "Links" + }, + { + "markdown": "[](./target.md)\n", + "html": "

\n", + "example": 483, + "start_line": 7544, + "end_line": 7548, "section": "Links" }, { "markdown": "[link]()\n", "html": "

link

\n", - "example": 483, - "start_line": 7521, - "end_line": 7525, + "example": 484, + "start_line": 7551, + "end_line": 7555, "section": "Links" }, { "markdown": "[link](<>)\n", "html": "

link

\n", - "example": 484, - "start_line": 7528, - "end_line": 7532, + "example": 485, + "start_line": 7558, + "end_line": 7562, + "section": "Links" + }, + { + "markdown": "[]()\n", + "html": "

\n", + "example": 486, + "start_line": 7565, + "end_line": 7569, "section": "Links" }, { "markdown": "[link](/my uri)\n", "html": "

[link](/my uri)

\n", - "example": 485, - "start_line": 7537, - "end_line": 7541, + "example": 487, + "start_line": 7574, + "end_line": 7578, "section": "Links" }, { "markdown": "[link]()\n", "html": "

link

\n", - "example": 486, - "start_line": 7543, - "end_line": 7547, + "example": 488, + "start_line": 7580, + "end_line": 7584, "section": "Links" }, { "markdown": "[link](foo\nbar)\n", "html": "

[link](foo\nbar)

\n", - "example": 487, - "start_line": 7552, - "end_line": 7558, + "example": 489, + "start_line": 7589, + "end_line": 7595, "section": "Links" }, { "markdown": "[link]()\n", "html": "

[link]()

\n", - "example": 488, - "start_line": 7560, - "end_line": 7566, + "example": 490, + "start_line": 7597, + "end_line": 7603, "section": "Links" }, { "markdown": "[a]()\n", "html": "

a

\n", - "example": 489, - "start_line": 7571, - "end_line": 7575, + "example": 491, + "start_line": 7608, + "end_line": 7612, "section": "Links" }, { "markdown": "[link]()\n", "html": "

[link](<foo>)

\n", - "example": 490, - "start_line": 7579, - "end_line": 7583, + "example": 492, + "start_line": 7616, + "end_line": 7620, "section": "Links" }, { "markdown": "[a](\n[a](c)\n", "html": "

[a](<b)c\n[a](<b)c>\n[a](c)

\n", - "example": 491, - "start_line": 7588, - "end_line": 7596, + "example": 493, + "start_line": 7625, + "end_line": 7633, "section": "Links" }, { "markdown": "[link](\\(foo\\))\n", "html": "

link

\n", - "example": 492, - "start_line": 7600, - "end_line": 7604, + "example": 494, + "start_line": 7637, + "end_line": 7641, "section": "Links" }, { "markdown": "[link](foo(and(bar)))\n", "html": "

link

\n", - "example": 493, - "start_line": 7609, - "end_line": 7613, + "example": 495, + "start_line": 7646, + "end_line": 7650, + "section": "Links" + }, + { + "markdown": "[link](foo(and(bar))\n", + "html": "

[link](foo(and(bar))

\n", + "example": 496, + "start_line": 7655, + "end_line": 7659, "section": "Links" }, { "markdown": "[link](foo\\(and\\(bar\\))\n", "html": "

link

\n", - "example": 494, - "start_line": 7618, - "end_line": 7622, + "example": 497, + "start_line": 7662, + "end_line": 7666, "section": "Links" }, { "markdown": "[link]()\n", "html": "

link

\n", - "example": 495, - "start_line": 7625, - "end_line": 7629, + "example": 498, + "start_line": 7669, + "end_line": 7673, "section": "Links" }, { "markdown": "[link](foo\\)\\:)\n", "html": "

link

\n", - "example": 496, - "start_line": 7635, - "end_line": 7639, + "example": 499, + "start_line": 7679, + "end_line": 7683, "section": "Links" }, { "markdown": "[link](#fragment)\n\n[link](http://example.com#fragment)\n\n[link](http://example.com?foo=3#frag)\n", "html": "

link

\n

link

\n

link

\n", - "example": 497, - "start_line": 7644, - "end_line": 7654, + "example": 500, + "start_line": 7688, + "end_line": 7698, "section": "Links" }, { "markdown": "[link](foo\\bar)\n", "html": "

link

\n", - "example": 498, - "start_line": 7660, - "end_line": 7664, + "example": 501, + "start_line": 7704, + "end_line": 7708, "section": "Links" }, { "markdown": "[link](foo%20bä)\n", "html": "

link

\n", - "example": 499, - "start_line": 7676, - "end_line": 7680, + "example": 502, + "start_line": 7720, + "end_line": 7724, "section": "Links" }, { "markdown": "[link](\"title\")\n", "html": "

link

\n", - "example": 500, - "start_line": 7687, - "end_line": 7691, + "example": 503, + "start_line": 7731, + "end_line": 7735, "section": "Links" }, { "markdown": "[link](/url \"title\")\n[link](/url 'title')\n[link](/url (title))\n", "html": "

link\nlink\nlink

\n", - "example": 501, - "start_line": 7696, - "end_line": 7704, + "example": 504, + "start_line": 7740, + "end_line": 7748, "section": "Links" }, { "markdown": "[link](/url \"title \\\""\")\n", "html": "

link

\n", - "example": 502, - "start_line": 7710, - "end_line": 7714, + "example": 505, + "start_line": 7754, + "end_line": 7758, "section": "Links" }, { "markdown": "[link](/url \"title\")\n", "html": "

link

\n", - "example": 503, - "start_line": 7720, - "end_line": 7724, + "example": 506, + "start_line": 7765, + "end_line": 7769, "section": "Links" }, { "markdown": "[link](/url \"title \"and\" title\")\n", "html": "

[link](/url "title "and" title")

\n", - "example": 504, - "start_line": 7729, - "end_line": 7733, + "example": 507, + "start_line": 7774, + "end_line": 7778, "section": "Links" }, { "markdown": "[link](/url 'title \"and\" title')\n", "html": "

link

\n", - "example": 505, - "start_line": 7738, - "end_line": 7742, - "section": "Links" - }, - { - "markdown": "[link]( /uri\n \"title\" )\n", - "html": "

link

\n", - "example": 506, - "start_line": 7762, - "end_line": 7767, - "section": "Links" - }, - { - "markdown": "[link] (/uri)\n", - "html": "

[link] (/uri)

\n", - "example": 507, - "start_line": 7773, - "end_line": 7777, - "section": "Links" - }, - { - "markdown": "[link [foo [bar]]](/uri)\n", - "html": "

link [foo [bar]]

\n", "example": 508, "start_line": 7783, "end_line": 7787, "section": "Links" }, + { + "markdown": "[link]( /uri\n \"title\" )\n", + "html": "

link

\n", + "example": 509, + "start_line": 7808, + "end_line": 7813, + "section": "Links" + }, + { + "markdown": "[link] (/uri)\n", + "html": "

[link] (/uri)

\n", + "example": 510, + "start_line": 7819, + "end_line": 7823, + "section": "Links" + }, + { + "markdown": "[link [foo [bar]]](/uri)\n", + "html": "

link [foo [bar]]

\n", + "example": 511, + "start_line": 7829, + "end_line": 7833, + "section": "Links" + }, { "markdown": "[link] bar](/uri)\n", "html": "

[link] bar](/uri)

\n", - "example": 509, - "start_line": 7790, - "end_line": 7794, + "example": 512, + "start_line": 7836, + "end_line": 7840, "section": "Links" }, { "markdown": "[link [bar](/uri)\n", "html": "

[link bar

\n", - "example": 510, - "start_line": 7797, - "end_line": 7801, + "example": 513, + "start_line": 7843, + "end_line": 7847, "section": "Links" }, { "markdown": "[link \\[bar](/uri)\n", "html": "

link [bar

\n", - "example": 511, - "start_line": 7804, - "end_line": 7808, + "example": 514, + "start_line": 7850, + "end_line": 7854, "section": "Links" }, { "markdown": "[link *foo **bar** `#`*](/uri)\n", "html": "

link foo bar #

\n", - "example": 512, - "start_line": 7813, - "end_line": 7817, + "example": 515, + "start_line": 7859, + "end_line": 7863, "section": "Links" }, { "markdown": "[![moon](moon.jpg)](/uri)\n", "html": "

\"moon\"

\n", - "example": 513, - "start_line": 7820, - "end_line": 7824, + "example": 516, + "start_line": 7866, + "end_line": 7870, "section": "Links" }, { "markdown": "[foo [bar](/uri)](/uri)\n", "html": "

[foo bar](/uri)

\n", - "example": 514, - "start_line": 7829, - "end_line": 7833, + "example": 517, + "start_line": 7875, + "end_line": 7879, "section": "Links" }, { "markdown": "[foo *[bar [baz](/uri)](/uri)*](/uri)\n", "html": "

[foo [bar baz](/uri)](/uri)

\n", - "example": 515, - "start_line": 7836, - "end_line": 7840, + "example": 518, + "start_line": 7882, + "end_line": 7886, "section": "Links" }, { "markdown": "![[[foo](uri1)](uri2)](uri3)\n", "html": "

\"[foo](uri2)\"

\n", - "example": 516, - "start_line": 7843, - "end_line": 7847, + "example": 519, + "start_line": 7889, + "end_line": 7893, "section": "Links" }, { "markdown": "*[foo*](/uri)\n", "html": "

*foo*

\n", - "example": 517, - "start_line": 7853, - "end_line": 7857, + "example": 520, + "start_line": 7899, + "end_line": 7903, "section": "Links" }, { "markdown": "[foo *bar](baz*)\n", "html": "

foo *bar

\n", - "example": 518, - "start_line": 7860, - "end_line": 7864, + "example": 521, + "start_line": 7906, + "end_line": 7910, "section": "Links" }, { "markdown": "*foo [bar* baz]\n", "html": "

foo [bar baz]

\n", - "example": 519, - "start_line": 7870, - "end_line": 7874, + "example": 522, + "start_line": 7916, + "end_line": 7920, "section": "Links" }, { "markdown": "[foo \n", "html": "

[foo

\n", - "example": 520, - "start_line": 7880, - "end_line": 7884, + "example": 523, + "start_line": 7926, + "end_line": 7930, "section": "Links" }, { "markdown": "[foo`](/uri)`\n", "html": "

[foo](/uri)

\n", - "example": 521, - "start_line": 7887, - "end_line": 7891, + "example": 524, + "start_line": 7933, + "end_line": 7937, "section": "Links" }, { "markdown": "[foo\n", "html": "

[foohttp://example.com/?search=](uri)

\n", - "example": 522, - "start_line": 7894, - "end_line": 7898, + "example": 525, + "start_line": 7940, + "end_line": 7944, "section": "Links" }, { "markdown": "[foo][bar]\n\n[bar]: /url \"title\"\n", "html": "

foo

\n", - "example": 523, - "start_line": 7932, - "end_line": 7938, + "example": 526, + "start_line": 7978, + "end_line": 7984, "section": "Links" }, { "markdown": "[link [foo [bar]]][ref]\n\n[ref]: /uri\n", "html": "

link [foo [bar]]

\n", - "example": 524, - "start_line": 7947, - "end_line": 7953, + "example": 527, + "start_line": 7993, + "end_line": 7999, "section": "Links" }, { "markdown": "[link \\[bar][ref]\n\n[ref]: /uri\n", "html": "

link [bar

\n", - "example": 525, - "start_line": 7956, - "end_line": 7962, + "example": 528, + "start_line": 8002, + "end_line": 8008, "section": "Links" }, { "markdown": "[link *foo **bar** `#`*][ref]\n\n[ref]: /uri\n", "html": "

link foo bar #

\n", - "example": 526, - "start_line": 7967, - "end_line": 7973, + "example": 529, + "start_line": 8013, + "end_line": 8019, "section": "Links" }, { "markdown": "[![moon](moon.jpg)][ref]\n\n[ref]: /uri\n", "html": "

\"moon\"

\n", - "example": 527, - "start_line": 7976, - "end_line": 7982, + "example": 530, + "start_line": 8022, + "end_line": 8028, "section": "Links" }, { "markdown": "[foo [bar](/uri)][ref]\n\n[ref]: /uri\n", "html": "

[foo bar]ref

\n", - "example": 528, - "start_line": 7987, - "end_line": 7993, + "example": 531, + "start_line": 8033, + "end_line": 8039, "section": "Links" }, { "markdown": "[foo *bar [baz][ref]*][ref]\n\n[ref]: /uri\n", "html": "

[foo bar baz]ref

\n", - "example": 529, - "start_line": 7996, - "end_line": 8002, + "example": 532, + "start_line": 8042, + "end_line": 8048, "section": "Links" }, { "markdown": "*[foo*][ref]\n\n[ref]: /uri\n", "html": "

*foo*

\n", - "example": 530, - "start_line": 8011, - "end_line": 8017, + "example": 533, + "start_line": 8057, + "end_line": 8063, "section": "Links" }, { - "markdown": "[foo *bar][ref]\n\n[ref]: /uri\n", - "html": "

foo *bar

\n", - "example": 531, - "start_line": 8020, - "end_line": 8026, + "markdown": "[foo *bar][ref]*\n\n[ref]: /uri\n", + "html": "

foo *bar*

\n", + "example": 534, + "start_line": 8066, + "end_line": 8072, "section": "Links" }, { "markdown": "[foo \n\n[ref]: /uri\n", "html": "

[foo

\n", - "example": 532, - "start_line": 8032, - "end_line": 8038, + "example": 535, + "start_line": 8078, + "end_line": 8084, "section": "Links" }, { "markdown": "[foo`][ref]`\n\n[ref]: /uri\n", "html": "

[foo][ref]

\n", - "example": 533, - "start_line": 8041, - "end_line": 8047, + "example": 536, + "start_line": 8087, + "end_line": 8093, "section": "Links" }, { "markdown": "[foo\n\n[ref]: /uri\n", "html": "

[foohttp://example.com/?search=][ref]

\n", - "example": 534, - "start_line": 8050, - "end_line": 8056, + "example": 537, + "start_line": 8096, + "end_line": 8102, "section": "Links" }, { "markdown": "[foo][BaR]\n\n[bar]: /url \"title\"\n", "html": "

foo

\n", - "example": 535, - "start_line": 8061, - "end_line": 8067, + "example": 538, + "start_line": 8107, + "end_line": 8113, "section": "Links" }, { - "markdown": "[Толпой][Толпой] is a Russian word.\n\n[ТОЛПОЙ]: /url\n", - "html": "

Толпой is a Russian word.

\n", - "example": 536, - "start_line": 8072, - "end_line": 8078, + "markdown": "[ẞ]\n\n[SS]: /url\n", + "html": "

ẞ

\n", + "example": 539, + "start_line": 8118, + "end_line": 8124, "section": "Links" }, { "markdown": "[Foo\n bar]: /url\n\n[Baz][Foo bar]\n", "html": "

Baz

\n", - "example": 537, - "start_line": 8084, - "end_line": 8091, + "example": 540, + "start_line": 8130, + "end_line": 8137, "section": "Links" }, { "markdown": "[foo] [bar]\n\n[bar]: /url \"title\"\n", "html": "

[foo] bar

\n", - "example": 538, - "start_line": 8097, - "end_line": 8103, + "example": 541, + "start_line": 8143, + "end_line": 8149, "section": "Links" }, { "markdown": "[foo]\n[bar]\n\n[bar]: /url \"title\"\n", "html": "

[foo]\nbar

\n", - "example": 539, - "start_line": 8106, - "end_line": 8114, + "example": 542, + "start_line": 8152, + "end_line": 8160, "section": "Links" }, { "markdown": "[foo]: /url1\n\n[foo]: /url2\n\n[bar][foo]\n", "html": "

bar

\n", - "example": 540, - "start_line": 8147, - "end_line": 8155, + "example": 543, + "start_line": 8193, + "end_line": 8201, "section": "Links" }, { "markdown": "[bar][foo\\!]\n\n[foo!]: /url\n", "html": "

[bar][foo!]

\n", - "example": 541, - "start_line": 8162, - "end_line": 8168, + "example": 544, + "start_line": 8208, + "end_line": 8214, "section": "Links" }, { "markdown": "[foo][ref[]\n\n[ref[]: /uri\n", "html": "

[foo][ref[]

\n

[ref[]: /uri

\n", - "example": 542, - "start_line": 8174, - "end_line": 8181, + "example": 545, + "start_line": 8220, + "end_line": 8227, "section": "Links" }, { "markdown": "[foo][ref[bar]]\n\n[ref[bar]]: /uri\n", "html": "

[foo][ref[bar]]

\n

[ref[bar]]: /uri

\n", - "example": 543, - "start_line": 8184, - "end_line": 8191, + "example": 546, + "start_line": 8230, + "end_line": 8237, "section": "Links" }, { "markdown": "[[[foo]]]\n\n[[[foo]]]: /url\n", "html": "

[[[foo]]]

\n

[[[foo]]]: /url

\n", - "example": 544, - "start_line": 8194, - "end_line": 8201, + "example": 547, + "start_line": 8240, + "end_line": 8247, "section": "Links" }, { "markdown": "[foo][ref\\[]\n\n[ref\\[]: /uri\n", "html": "

foo

\n", - "example": 545, - "start_line": 8204, - "end_line": 8210, + "example": 548, + "start_line": 8250, + "end_line": 8256, "section": "Links" }, { "markdown": "[bar\\\\]: /uri\n\n[bar\\\\]\n", "html": "

bar\\

\n", - "example": 546, - "start_line": 8215, - "end_line": 8221, + "example": 549, + "start_line": 8261, + "end_line": 8267, "section": "Links" }, { "markdown": "[]\n\n[]: /uri\n", "html": "

[]

\n

[]: /uri

\n", - "example": 547, - "start_line": 8226, - "end_line": 8233, + "example": 550, + "start_line": 8273, + "end_line": 8280, "section": "Links" }, { "markdown": "[\n ]\n\n[\n ]: /uri\n", "html": "

[\n]

\n

[\n]: /uri

\n", - "example": 548, - "start_line": 8236, - "end_line": 8247, + "example": 551, + "start_line": 8283, + "end_line": 8294, "section": "Links" }, { "markdown": "[foo][]\n\n[foo]: /url \"title\"\n", "html": "

foo

\n", - "example": 549, - "start_line": 8259, - "end_line": 8265, + "example": 552, + "start_line": 8306, + "end_line": 8312, "section": "Links" }, { "markdown": "[*foo* bar][]\n\n[*foo* bar]: /url \"title\"\n", "html": "

foo bar

\n", - "example": 550, - "start_line": 8268, - "end_line": 8274, + "example": 553, + "start_line": 8315, + "end_line": 8321, "section": "Links" }, { "markdown": "[Foo][]\n\n[foo]: /url \"title\"\n", "html": "

Foo

\n", - "example": 551, - "start_line": 8279, - "end_line": 8285, + "example": 554, + "start_line": 8326, + "end_line": 8332, "section": "Links" }, { "markdown": "[foo] \n[]\n\n[foo]: /url \"title\"\n", "html": "

foo\n[]

\n", - "example": 552, - "start_line": 8292, - "end_line": 8300, + "example": 555, + "start_line": 8339, + "end_line": 8347, "section": "Links" }, { "markdown": "[foo]\n\n[foo]: /url \"title\"\n", "html": "

foo

\n", - "example": 553, - "start_line": 8312, - "end_line": 8318, + "example": 556, + "start_line": 8359, + "end_line": 8365, "section": "Links" }, { "markdown": "[*foo* bar]\n\n[*foo* bar]: /url \"title\"\n", "html": "

foo bar

\n", - "example": 554, - "start_line": 8321, - "end_line": 8327, + "example": 557, + "start_line": 8368, + "end_line": 8374, "section": "Links" }, { "markdown": "[[*foo* bar]]\n\n[*foo* bar]: /url \"title\"\n", "html": "

[foo bar]

\n", - "example": 555, - "start_line": 8330, - "end_line": 8336, + "example": 558, + "start_line": 8377, + "end_line": 8383, "section": "Links" }, { "markdown": "[[bar [foo]\n\n[foo]: /url\n", "html": "

[[bar foo

\n", - "example": 556, - "start_line": 8339, - "end_line": 8345, + "example": 559, + "start_line": 8386, + "end_line": 8392, "section": "Links" }, { "markdown": "[Foo]\n\n[foo]: /url \"title\"\n", "html": "

Foo

\n", - "example": 557, - "start_line": 8350, - "end_line": 8356, + "example": 560, + "start_line": 8397, + "end_line": 8403, "section": "Links" }, { "markdown": "[foo] bar\n\n[foo]: /url\n", "html": "

foo bar

\n", - "example": 558, - "start_line": 8361, - "end_line": 8367, + "example": 561, + "start_line": 8408, + "end_line": 8414, "section": "Links" }, { "markdown": "\\[foo]\n\n[foo]: /url \"title\"\n", "html": "

[foo]

\n", - "example": 559, - "start_line": 8373, - "end_line": 8379, + "example": 562, + "start_line": 8420, + "end_line": 8426, "section": "Links" }, { "markdown": "[foo*]: /url\n\n*[foo*]\n", "html": "

*foo*

\n", - "example": 560, - "start_line": 8385, - "end_line": 8391, + "example": 563, + "start_line": 8432, + "end_line": 8438, "section": "Links" }, { "markdown": "[foo][bar]\n\n[foo]: /url1\n[bar]: /url2\n", "html": "

foo

\n", - "example": 561, - "start_line": 8397, - "end_line": 8404, + "example": 564, + "start_line": 8444, + "end_line": 8451, "section": "Links" }, { "markdown": "[foo][]\n\n[foo]: /url1\n", "html": "

foo

\n", - "example": 562, - "start_line": 8406, - "end_line": 8412, + "example": 565, + "start_line": 8453, + "end_line": 8459, "section": "Links" }, { "markdown": "[foo]()\n\n[foo]: /url1\n", "html": "

foo

\n", - "example": 563, - "start_line": 8416, - "end_line": 8422, + "example": 566, + "start_line": 8463, + "end_line": 8469, "section": "Links" }, { "markdown": "[foo](not a link)\n\n[foo]: /url1\n", "html": "

foo(not a link)

\n", - "example": 564, - "start_line": 8424, - "end_line": 8430, + "example": 567, + "start_line": 8471, + "end_line": 8477, "section": "Links" }, { "markdown": "[foo][bar][baz]\n\n[baz]: /url\n", "html": "

[foo]bar

\n", - "example": 565, - "start_line": 8435, - "end_line": 8441, + "example": 568, + "start_line": 8482, + "end_line": 8488, "section": "Links" }, { "markdown": "[foo][bar][baz]\n\n[baz]: /url1\n[bar]: /url2\n", "html": "

foobaz

\n", - "example": 566, - "start_line": 8447, - "end_line": 8454, + "example": 569, + "start_line": 8494, + "end_line": 8501, "section": "Links" }, { "markdown": "[foo][bar][baz]\n\n[baz]: /url1\n[foo]: /url2\n", "html": "

[foo]bar

\n", - "example": 567, - "start_line": 8460, - "end_line": 8467, + "example": 570, + "start_line": 8507, + "end_line": 8514, "section": "Links" }, { "markdown": "![foo](/url \"title\")\n", "html": "

\"foo\"

\n", - "example": 568, - "start_line": 8483, - "end_line": 8487, + "example": 571, + "start_line": 8530, + "end_line": 8534, "section": "Images" }, { "markdown": "![foo *bar*]\n\n[foo *bar*]: train.jpg \"train & tracks\"\n", "html": "

\"foo

\n", - "example": 569, - "start_line": 8490, - "end_line": 8496, + "example": 572, + "start_line": 8537, + "end_line": 8543, "section": "Images" }, { "markdown": "![foo ![bar](/url)](/url2)\n", "html": "

\"foo

\n", - "example": 570, - "start_line": 8499, - "end_line": 8503, + "example": 573, + "start_line": 8546, + "end_line": 8550, "section": "Images" }, { "markdown": "![foo [bar](/url)](/url2)\n", "html": "

\"foo

\n", - "example": 571, - "start_line": 8506, - "end_line": 8510, + "example": 574, + "start_line": 8553, + "end_line": 8557, "section": "Images" }, { "markdown": "![foo *bar*][]\n\n[foo *bar*]: train.jpg \"train & tracks\"\n", "html": "

\"foo

\n", - "example": 572, - "start_line": 8520, - "end_line": 8526, + "example": 575, + "start_line": 8567, + "end_line": 8573, "section": "Images" }, { "markdown": "![foo *bar*][foobar]\n\n[FOOBAR]: train.jpg \"train & tracks\"\n", "html": "

\"foo

\n", - "example": 573, - "start_line": 8529, - "end_line": 8535, + "example": 576, + "start_line": 8576, + "end_line": 8582, "section": "Images" }, { "markdown": "![foo](train.jpg)\n", "html": "

\"foo\"

\n", - "example": 574, - "start_line": 8538, - "end_line": 8542, + "example": 577, + "start_line": 8585, + "end_line": 8589, "section": "Images" }, { "markdown": "My ![foo bar](/path/to/train.jpg \"title\" )\n", "html": "

My \"foo

\n", - "example": 575, - "start_line": 8545, - "end_line": 8549, + "example": 578, + "start_line": 8592, + "end_line": 8596, "section": "Images" }, { "markdown": "![foo]()\n", "html": "

\"foo\"

\n", - "example": 576, - "start_line": 8552, - "end_line": 8556, + "example": 579, + "start_line": 8599, + "end_line": 8603, "section": "Images" }, { "markdown": "![](/url)\n", "html": "

\"\"

\n", - "example": 577, - "start_line": 8559, - "end_line": 8563, + "example": 580, + "start_line": 8606, + "end_line": 8610, "section": "Images" }, { "markdown": "![foo][bar]\n\n[bar]: /url\n", "html": "

\"foo\"

\n", - "example": 578, - "start_line": 8568, - "end_line": 8574, + "example": 581, + "start_line": 8615, + "end_line": 8621, "section": "Images" }, { "markdown": "![foo][bar]\n\n[BAR]: /url\n", "html": "

\"foo\"

\n", - "example": 579, - "start_line": 8577, - "end_line": 8583, + "example": 582, + "start_line": 8624, + "end_line": 8630, "section": "Images" }, { "markdown": "![foo][]\n\n[foo]: /url \"title\"\n", "html": "

\"foo\"

\n", - "example": 580, - "start_line": 8588, - "end_line": 8594, + "example": 583, + "start_line": 8635, + "end_line": 8641, "section": "Images" }, { "markdown": "![*foo* bar][]\n\n[*foo* bar]: /url \"title\"\n", "html": "

\"foo

\n", - "example": 581, - "start_line": 8597, - "end_line": 8603, + "example": 584, + "start_line": 8644, + "end_line": 8650, "section": "Images" }, { "markdown": "![Foo][]\n\n[foo]: /url \"title\"\n", "html": "

\"Foo\"

\n", - "example": 582, - "start_line": 8608, - "end_line": 8614, + "example": 585, + "start_line": 8655, + "end_line": 8661, "section": "Images" }, { "markdown": "![foo] \n[]\n\n[foo]: /url \"title\"\n", "html": "

\"foo\"\n[]

\n", - "example": 583, - "start_line": 8620, - "end_line": 8628, + "example": 586, + "start_line": 8667, + "end_line": 8675, "section": "Images" }, { "markdown": "![foo]\n\n[foo]: /url \"title\"\n", "html": "

\"foo\"

\n", - "example": 584, - "start_line": 8633, - "end_line": 8639, + "example": 587, + "start_line": 8680, + "end_line": 8686, "section": "Images" }, { "markdown": "![*foo* bar]\n\n[*foo* bar]: /url \"title\"\n", "html": "

\"foo

\n", - "example": 585, - "start_line": 8642, - "end_line": 8648, + "example": 588, + "start_line": 8689, + "end_line": 8695, "section": "Images" }, { "markdown": "![[foo]]\n\n[[foo]]: /url \"title\"\n", "html": "

![[foo]]

\n

[[foo]]: /url "title"

\n", - "example": 586, - "start_line": 8653, - "end_line": 8660, + "example": 589, + "start_line": 8700, + "end_line": 8707, "section": "Images" }, { "markdown": "![Foo]\n\n[foo]: /url \"title\"\n", "html": "

\"Foo\"

\n", - "example": 587, - "start_line": 8665, - "end_line": 8671, + "example": 590, + "start_line": 8712, + "end_line": 8718, "section": "Images" }, { "markdown": "!\\[foo]\n\n[foo]: /url \"title\"\n", "html": "

![foo]

\n", - "example": 588, - "start_line": 8677, - "end_line": 8683, + "example": 591, + "start_line": 8724, + "end_line": 8730, "section": "Images" }, { "markdown": "\\![foo]\n\n[foo]: /url \"title\"\n", "html": "

!foo

\n", - "example": 589, - "start_line": 8689, - "end_line": 8695, + "example": 592, + "start_line": 8736, + "end_line": 8742, "section": "Images" }, { "markdown": "\n", "html": "

http://foo.bar.baz

\n", - "example": 590, - "start_line": 8722, - "end_line": 8726, + "example": 593, + "start_line": 8769, + "end_line": 8773, "section": "Autolinks" }, { "markdown": "\n", "html": "

http://foo.bar.baz/test?q=hello&id=22&boolean

\n", - "example": 591, - "start_line": 8729, - "end_line": 8733, + "example": 594, + "start_line": 8776, + "end_line": 8780, "section": "Autolinks" }, { "markdown": "\n", "html": "

irc://foo.bar:2233/baz

\n", - "example": 592, - "start_line": 8736, - "end_line": 8740, + "example": 595, + "start_line": 8783, + "end_line": 8787, "section": "Autolinks" }, { "markdown": "\n", "html": "

MAILTO:FOO@BAR.BAZ

\n", - "example": 593, - "start_line": 8745, - "end_line": 8749, + "example": 596, + "start_line": 8792, + "end_line": 8796, "section": "Autolinks" }, { "markdown": "\n", "html": "

a+b+c:d

\n", - "example": 594, - "start_line": 8757, - "end_line": 8761, + "example": 597, + "start_line": 8804, + "end_line": 8808, "section": "Autolinks" }, { "markdown": "\n", "html": "

made-up-scheme://foo,bar

\n", - "example": 595, - "start_line": 8764, - "end_line": 8768, + "example": 598, + "start_line": 8811, + "end_line": 8815, "section": "Autolinks" }, { "markdown": "\n", "html": "

http://../

\n", - "example": 596, - "start_line": 8771, - "end_line": 8775, + "example": 599, + "start_line": 8818, + "end_line": 8822, "section": "Autolinks" }, { "markdown": "\n", "html": "

localhost:5001/foo

\n", - "example": 597, - "start_line": 8778, - "end_line": 8782, + "example": 600, + "start_line": 8825, + "end_line": 8829, "section": "Autolinks" }, { "markdown": "\n", "html": "

<http://foo.bar/baz bim>

\n", - "example": 598, - "start_line": 8787, - "end_line": 8791, + "example": 601, + "start_line": 8834, + "end_line": 8838, "section": "Autolinks" }, { "markdown": "\n", "html": "

http://example.com/\\[\\

\n", - "example": 599, - "start_line": 8796, - "end_line": 8800, + "example": 602, + "start_line": 8843, + "end_line": 8847, "section": "Autolinks" }, { "markdown": "\n", "html": "

foo@bar.example.com

\n", - "example": 600, - "start_line": 8818, - "end_line": 8822, + "example": 603, + "start_line": 8865, + "end_line": 8869, "section": "Autolinks" }, { "markdown": "\n", "html": "

foo+special@Bar.baz-bar0.com

\n", - "example": 601, - "start_line": 8825, - "end_line": 8829, + "example": 604, + "start_line": 8872, + "end_line": 8876, "section": "Autolinks" }, { "markdown": "\n", "html": "

<foo+@bar.example.com>

\n", - "example": 602, - "start_line": 8834, - "end_line": 8838, + "example": 605, + "start_line": 8881, + "end_line": 8885, "section": "Autolinks" }, { "markdown": "<>\n", "html": "

<>

\n", - "example": 603, - "start_line": 8843, - "end_line": 8847, + "example": 606, + "start_line": 8890, + "end_line": 8894, "section": "Autolinks" }, { "markdown": "< http://foo.bar >\n", "html": "

< http://foo.bar >

\n", - "example": 604, - "start_line": 8850, - "end_line": 8854, + "example": 607, + "start_line": 8897, + "end_line": 8901, "section": "Autolinks" }, { "markdown": "\n", "html": "

<m:abc>

\n", - "example": 605, - "start_line": 8857, - "end_line": 8861, + "example": 608, + "start_line": 8904, + "end_line": 8908, "section": "Autolinks" }, { "markdown": "\n", "html": "

<foo.bar.baz>

\n", - "example": 606, - "start_line": 8864, - "end_line": 8868, + "example": 609, + "start_line": 8911, + "end_line": 8915, "section": "Autolinks" }, { "markdown": "http://example.com\n", "html": "

http://example.com

\n", - "example": 607, - "start_line": 8871, - "end_line": 8875, + "example": 610, + "start_line": 8918, + "end_line": 8922, "section": "Autolinks" }, { "markdown": "foo@bar.example.com\n", "html": "

foo@bar.example.com

\n", - "example": 608, - "start_line": 8878, - "end_line": 8882, + "example": 611, + "start_line": 8925, + "end_line": 8929, "section": "Autolinks" }, { "markdown": "\n", "html": "

\n", - "example": 609, - "start_line": 8960, - "end_line": 8964, + "example": 612, + "start_line": 9006, + "end_line": 9010, "section": "Raw HTML" }, { "markdown": "\n", "html": "

\n", - "example": 610, - "start_line": 8969, - "end_line": 8973, + "example": 613, + "start_line": 9015, + "end_line": 9019, "section": "Raw HTML" }, { "markdown": "\n", "html": "

\n", - "example": 611, - "start_line": 8978, - "end_line": 8984, + "example": 614, + "start_line": 9024, + "end_line": 9030, "section": "Raw HTML" }, { "markdown": "\n", "html": "

\n", - "example": 612, - "start_line": 8989, - "end_line": 8995, + "example": 615, + "start_line": 9035, + "end_line": 9041, "section": "Raw HTML" }, { "markdown": "Foo \n", "html": "

Foo

\n", - "example": 613, - "start_line": 9000, - "end_line": 9004, + "example": 616, + "start_line": 9046, + "end_line": 9050, "section": "Raw HTML" }, { "markdown": "<33> <__>\n", "html": "

<33> <__>

\n", - "example": 614, - "start_line": 9009, - "end_line": 9013, + "example": 617, + "start_line": 9055, + "end_line": 9059, "section": "Raw HTML" }, { "markdown": "\n", "html": "

<a h*#ref="hi">

\n", - "example": 615, - "start_line": 9018, - "end_line": 9022, + "example": 618, + "start_line": 9064, + "end_line": 9068, "section": "Raw HTML" }, { "markdown": " \n", "html": "

<a href="hi'> <a href=hi'>

\n", - "example": 616, - "start_line": 9027, - "end_line": 9031, + "example": 619, + "start_line": 9073, + "end_line": 9077, "section": "Raw HTML" }, { "markdown": "< a><\nfoo>\n\n", "html": "

< a><\nfoo><bar/ >\n<foo bar=baz\nbim!bop />

\n", - "example": 617, - "start_line": 9036, - "end_line": 9046, + "example": 620, + "start_line": 9082, + "end_line": 9092, "section": "Raw HTML" }, { "markdown": "\n", "html": "

<a href='bar'title=title>

\n", - "example": 618, - "start_line": 9051, - "end_line": 9055, + "example": 621, + "start_line": 9097, + "end_line": 9101, "section": "Raw HTML" }, { "markdown": "\n", "html": "

\n", - "example": 619, - "start_line": 9060, - "end_line": 9064, + "example": 622, + "start_line": 9106, + "end_line": 9110, "section": "Raw HTML" }, { "markdown": "\n", "html": "

</a href="foo">

\n", - "example": 620, - "start_line": 9069, - "end_line": 9073, + "example": 623, + "start_line": 9115, + "end_line": 9119, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", - "example": 621, - "start_line": 9078, - "end_line": 9084, + "example": 624, + "start_line": 9124, + "end_line": 9130, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo <!-- not a comment -- two hyphens -->

\n", - "example": 622, - "start_line": 9087, - "end_line": 9091, + "example": 625, + "start_line": 9133, + "end_line": 9137, "section": "Raw HTML" }, { "markdown": "foo foo -->\n\nfoo \n", "html": "

foo <!--> foo -->

\n

foo <!-- foo--->

\n", - "example": 623, - "start_line": 9096, - "end_line": 9103, + "example": 626, + "start_line": 9142, + "end_line": 9149, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", - "example": 624, - "start_line": 9108, - "end_line": 9112, + "example": 627, + "start_line": 9154, + "end_line": 9158, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", - "example": 625, - "start_line": 9117, - "end_line": 9121, + "example": 628, + "start_line": 9163, + "end_line": 9167, "section": "Raw HTML" }, { "markdown": "foo &<]]>\n", "html": "

foo &<]]>

\n", - "example": 626, - "start_line": 9126, - "end_line": 9130, + "example": 629, + "start_line": 9172, + "end_line": 9176, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", - "example": 627, - "start_line": 9136, - "end_line": 9140, + "example": 630, + "start_line": 9182, + "end_line": 9186, "section": "Raw HTML" }, { "markdown": "foo \n", "html": "

foo

\n", - "example": 628, - "start_line": 9145, - "end_line": 9149, + "example": 631, + "start_line": 9191, + "end_line": 9195, "section": "Raw HTML" }, { "markdown": "\n", "html": "

<a href=""">

\n", - "example": 629, - "start_line": 9152, - "end_line": 9156, + "example": 632, + "start_line": 9198, + "end_line": 9202, "section": "Raw HTML" }, { "markdown": "foo \nbaz\n", "html": "

foo
\nbaz

\n", - "example": 630, - "start_line": 9166, - "end_line": 9172, + "example": 633, + "start_line": 9212, + "end_line": 9218, "section": "Hard line breaks" }, { "markdown": "foo\\\nbaz\n", "html": "

foo
\nbaz

\n", - "example": 631, - "start_line": 9178, - "end_line": 9184, + "example": 634, + "start_line": 9224, + "end_line": 9230, "section": "Hard line breaks" }, { "markdown": "foo \nbaz\n", "html": "

foo
\nbaz

\n", - "example": 632, - "start_line": 9189, - "end_line": 9195, + "example": 635, + "start_line": 9235, + "end_line": 9241, "section": "Hard line breaks" }, { "markdown": "foo \n bar\n", "html": "

foo
\nbar

\n", - "example": 633, - "start_line": 9200, - "end_line": 9206, + "example": 636, + "start_line": 9246, + "end_line": 9252, "section": "Hard line breaks" }, { "markdown": "foo\\\n bar\n", "html": "

foo
\nbar

\n", - "example": 634, - "start_line": 9209, - "end_line": 9215, + "example": 637, + "start_line": 9255, + "end_line": 9261, "section": "Hard line breaks" }, { "markdown": "*foo \nbar*\n", "html": "

foo
\nbar

\n", - "example": 635, - "start_line": 9221, - "end_line": 9227, + "example": 638, + "start_line": 9267, + "end_line": 9273, "section": "Hard line breaks" }, { "markdown": "*foo\\\nbar*\n", "html": "

foo
\nbar

\n", - "example": 636, - "start_line": 9230, - "end_line": 9236, + "example": 639, + "start_line": 9276, + "end_line": 9282, "section": "Hard line breaks" }, { - "markdown": "`code \nspan`\n", - "html": "

code span

\n", - "example": 637, - "start_line": 9241, - "end_line": 9246, + "markdown": "`code \nspan`\n", + "html": "

code span

\n", + "example": 640, + "start_line": 9287, + "end_line": 9292, "section": "Hard line breaks" }, { "markdown": "`code\\\nspan`\n", "html": "

code\\ span

\n", - "example": 638, - "start_line": 9249, - "end_line": 9254, + "example": 641, + "start_line": 9295, + "end_line": 9300, "section": "Hard line breaks" }, { "markdown": "\n", "html": "

\n", - "example": 639, - "start_line": 9259, - "end_line": 9265, + "example": 642, + "start_line": 9305, + "end_line": 9311, "section": "Hard line breaks" }, { "markdown": "\n", "html": "

\n", - "example": 640, - "start_line": 9268, - "end_line": 9274, + "example": 643, + "start_line": 9314, + "end_line": 9320, "section": "Hard line breaks" }, { "markdown": "foo\\\n", "html": "

foo\\

\n", - "example": 641, - "start_line": 9281, - "end_line": 9285, + "example": 644, + "start_line": 9327, + "end_line": 9331, "section": "Hard line breaks" }, { "markdown": "foo \n", "html": "

foo

\n", - "example": 642, - "start_line": 9288, - "end_line": 9292, + "example": 645, + "start_line": 9334, + "end_line": 9338, "section": "Hard line breaks" }, { "markdown": "### foo\\\n", "html": "

foo\\

\n", - "example": 643, - "start_line": 9295, - "end_line": 9299, + "example": 646, + "start_line": 9341, + "end_line": 9345, "section": "Hard line breaks" }, { "markdown": "### foo \n", "html": "

foo

\n", - "example": 644, - "start_line": 9302, - "end_line": 9306, + "example": 647, + "start_line": 9348, + "end_line": 9352, "section": "Hard line breaks" }, { "markdown": "foo\nbaz\n", "html": "

foo\nbaz

\n", - "example": 645, - "start_line": 9317, - "end_line": 9323, + "example": 648, + "start_line": 9363, + "end_line": 9369, "section": "Soft line breaks" }, { "markdown": "foo \n baz\n", "html": "

foo\nbaz

\n", - "example": 646, - "start_line": 9329, - "end_line": 9335, + "example": 649, + "start_line": 9375, + "end_line": 9381, "section": "Soft line breaks" }, { "markdown": "hello $.;'there\n", "html": "

hello $.;'there

\n", - "example": 647, - "start_line": 9349, - "end_line": 9353, + "example": 650, + "start_line": 9395, + "end_line": 9399, "section": "Textual content" }, { "markdown": "Foo χρῆν\n", "html": "

Foo χρῆν

\n", - "example": 648, - "start_line": 9356, - "end_line": 9360, + "example": 651, + "start_line": 9402, + "end_line": 9406, "section": "Textual content" }, { "markdown": "Multiple spaces\n", "html": "

Multiple spaces

\n", - "example": 649, - "start_line": 9365, - "end_line": 9369, + "example": 652, + "start_line": 9411, + "end_line": 9415, "section": "Textual content" } ] Index: testdata/meta/title/20200310110300.zettel ================================================================== --- testdata/meta/title/20200310110300.zettel +++ testdata/meta/title/20200310110300.zettel @@ -1,1 +1,1 @@ -title: A ""Title"" with //Markup//, ``Zettelmarkup``{=zmk} +title: A ""Title"" with __Markup__, ``Zettelmarkup``{=zmk} DELETED testdata/mustache/comments.json Index: testdata/mustache/comments.json ================================================================== --- testdata/mustache/comments.json +++ /dev/null @@ -1,1 +0,0 @@ -{"__ATTN__":"Do not edit this file; changes belong in the appropriate YAML file.","overview":"Comment tags represent content that should never appear in the resulting\noutput.\n\nThe tag's content may contain any substring (including newlines) EXCEPT the\nclosing delimiter.\n\nComment tags SHOULD be treated as standalone when appropriate.\n","tests":[{"name":"Inline","data":{},"expected":"1234567890","template":"12345{{! Comment Block! }}67890","desc":"Comment blocks should be removed from the template."},{"name":"Multiline","data":{},"expected":"1234567890\n","template":"12345{{!\n This is a\n multi-line comment...\n}}67890\n","desc":"Multiline comments should be permitted."},{"name":"Standalone","data":{},"expected":"Begin.\nEnd.\n","template":"Begin.\n{{! Comment Block! }}\nEnd.\n","desc":"All standalone comment lines should be removed."},{"name":"Indented Standalone","data":{},"expected":"Begin.\nEnd.\n","template":"Begin.\n {{! Indented Comment Block! }}\nEnd.\n","desc":"All standalone comment lines should be removed."},{"name":"Standalone Line Endings","data":{},"expected":"|\r\n|","template":"|\r\n{{! Standalone Comment }}\r\n|","desc":"\"\\r\\n\" should be considered a newline for standalone tags."},{"name":"Standalone Without Previous Line","data":{},"expected":"!","template":" {{! I'm Still Standalone }}\n!","desc":"Standalone tags should not require a newline to precede them."},{"name":"Standalone Without Newline","data":{},"expected":"!\n","template":"!\n {{! I'm Still Standalone }}","desc":"Standalone tags should not require a newline to follow them."},{"name":"Multiline Standalone","data":{},"expected":"Begin.\nEnd.\n","template":"Begin.\n{{!\nSomething's going on here...\n}}\nEnd.\n","desc":"All standalone comment lines should be removed."},{"name":"Indented Multiline Standalone","data":{},"expected":"Begin.\nEnd.\n","template":"Begin.\n {{!\n Something's going on here...\n }}\nEnd.\n","desc":"All standalone comment lines should be removed."},{"name":"Indented Inline","data":{},"expected":" 12 \n","template":" 12 {{! 34 }}\n","desc":"Inline comments should not strip whitespace"},{"name":"Surrounding Whitespace","data":{},"expected":"12345 67890","template":"12345 {{! Comment Block! }} 67890","desc":"Comment removal should preserve surrounding whitespace."}]} DELETED testdata/mustache/delimiters.json Index: testdata/mustache/delimiters.json ================================================================== --- testdata/mustache/delimiters.json +++ /dev/null @@ -1,1 +0,0 @@ -{"__ATTN__":"Do not edit this file; changes belong in the appropriate YAML file.","overview":"Set Delimiter tags are used to change the tag delimiters for all content\nfollowing the tag in the current compilation unit.\n\nThe tag's content MUST be any two non-whitespace sequences (separated by\nwhitespace) EXCEPT an equals sign ('=') followed by the current closing\ndelimiter.\n\nSet Delimiter tags SHOULD be treated as standalone when appropriate.\n","tests":[{"name":"Pair Behavior","data":{"text":"Hey!"},"expected":"(Hey!)","template":"{{=<% %>=}}(<%text%>)","desc":"The equals sign (used on both sides) should permit delimiter changes."},{"name":"Special Characters","data":{"text":"It worked!"},"expected":"(It worked!)","template":"({{=[ ]=}}[text])","desc":"Characters with special meaning regexen should be valid delimiters."},{"name":"Sections","data":{"section":true,"data":"I got interpolated."},"expected":"[\n I got interpolated.\n |data|\n\n {{data}}\n I got interpolated.\n]\n","template":"[\n{{#section}}\n {{data}}\n |data|\n{{/section}}\n\n{{= | | =}}\n|#section|\n {{data}}\n |data|\n|/section|\n]\n","desc":"Delimiters set outside sections should persist."},{"name":"Inverted Sections","data":{"section":false,"data":"I got interpolated."},"expected":"[\n I got interpolated.\n |data|\n\n {{data}}\n I got interpolated.\n]\n","template":"[\n{{^section}}\n {{data}}\n |data|\n{{/section}}\n\n{{= | | =}}\n|^section|\n {{data}}\n |data|\n|/section|\n]\n","desc":"Delimiters set outside inverted sections should persist."},{"name":"Partial Inheritence","data":{"value":"yes"},"expected":"[ .yes. ]\n[ .yes. ]\n","template":"[ {{>include}} ]\n{{= | | =}}\n[ |>include| ]\n","desc":"Delimiters set in a parent template should not affect a partial.","partials":{"include":".{{value}}."}},{"name":"Post-Partial Behavior","data":{"value":"yes"},"expected":"[ .yes. .yes. ]\n[ .yes. .|value|. ]\n","template":"[ {{>include}} ]\n[ .{{value}}. .|value|. ]\n","desc":"Delimiters set in a partial should not affect the parent template.","partials":{"include":".{{value}}. {{= | | =}} .|value|."}},{"name":"Surrounding Whitespace","data":{},"expected":"| |","template":"| {{=@ @=}} |","desc":"Surrounding whitespace should be left untouched."},{"name":"Outlying Whitespace (Inline)","data":{},"expected":" | \n","template":" | {{=@ @=}}\n","desc":"Whitespace should be left untouched."},{"name":"Standalone Tag","data":{},"expected":"Begin.\nEnd.\n","template":"Begin.\n{{=@ @=}}\nEnd.\n","desc":"Standalone lines should be removed from the template."},{"name":"Indented Standalone Tag","data":{},"expected":"Begin.\nEnd.\n","template":"Begin.\n {{=@ @=}}\nEnd.\n","desc":"Indented standalone lines should be removed from the template."},{"name":"Standalone Line Endings","data":{},"expected":"|\r\n|","template":"|\r\n{{= @ @ =}}\r\n|","desc":"\"\\r\\n\" should be considered a newline for standalone tags."},{"name":"Standalone Without Previous Line","data":{},"expected":"=","template":" {{=@ @=}}\n=","desc":"Standalone tags should not require a newline to precede them."},{"name":"Standalone Without Newline","data":{},"expected":"=\n","template":"=\n {{=@ @=}}","desc":"Standalone tags should not require a newline to follow them."},{"name":"Pair with Padding","data":{},"expected":"||","template":"|{{= @ @ =}}|","desc":"Superfluous in-tag whitespace should be ignored."}]} DELETED testdata/mustache/interpolation.json Index: testdata/mustache/interpolation.json ================================================================== --- testdata/mustache/interpolation.json +++ /dev/null @@ -1,1 +0,0 @@ -{"__ATTN__":"Do not edit this file; changes belong in the appropriate YAML file.","overview":"Interpolation tags are used to integrate dynamic content into the template.\n\nThe tag's content MUST be a non-whitespace character sequence NOT containing\nthe current closing delimiter.\n\nThis tag's content names the data to replace the tag. A single period (`.`)\nindicates that the item currently sitting atop the context stack should be\nused; otherwise, name resolution is as follows:\n 1) Split the name on periods; the first part is the name to resolve, any\n remaining parts should be retained.\n 2) Walk the context stack from top to bottom, finding the first context\n that is a) a hash containing the name as a key OR b) an object responding\n to a method with the given name.\n 3) If the context is a hash, the data is the value associated with the\n name.\n 4) If the context is an object, the data is the value returned by the\n method with the given name.\n 5) If any name parts were retained in step 1, each should be resolved\n against a context stack containing only the result from the former\n resolution. If any part fails resolution, the result should be considered\n falsey, and should interpolate as the empty string.\nData should be coerced into a string (and escaped, if appropriate) before\ninterpolation.\n\nThe Interpolation tags MUST NOT be treated as standalone.\n","tests":[{"name":"No Interpolation","data":{},"expected":"Hello from {Mustache}!\n","template":"Hello from {Mustache}!\n","desc":"Mustache-free templates should render as-is."},{"name":"Basic Interpolation","data":{"subject":"world"},"expected":"Hello, world!\n","template":"Hello, {{subject}}!\n","desc":"Unadorned tags should interpolate content into the template."},{"name":"HTML Escaping","data":{"forbidden":"& \" < >"},"expected":"These characters should be HTML escaped: & " < >\n","template":"These characters should be HTML escaped: {{forbidden}}\n","desc":"Basic interpolation should be HTML escaped."},{"name":"Triple Mustache","data":{"forbidden":"& \" < >"},"expected":"These characters should not be HTML escaped: & \" < >\n","template":"These characters should not be HTML escaped: {{{forbidden}}}\n","desc":"Triple mustaches should interpolate without HTML escaping."},{"name":"Ampersand","data":{"forbidden":"& \" < >"},"expected":"These characters should not be HTML escaped: & \" < >\n","template":"These characters should not be HTML escaped: {{&forbidden}}\n","desc":"Ampersand should interpolate without HTML escaping."},{"name":"Basic Integer Interpolation","data":{"mph":85},"expected":"\"85 miles an hour!\"","template":"\"{{mph}} miles an hour!\"","desc":"Integers should interpolate seamlessly."},{"name":"Triple Mustache Integer Interpolation","data":{"mph":85},"expected":"\"85 miles an hour!\"","template":"\"{{{mph}}} miles an hour!\"","desc":"Integers should interpolate seamlessly."},{"name":"Ampersand Integer Interpolation","data":{"mph":85},"expected":"\"85 miles an hour!\"","template":"\"{{&mph}} miles an hour!\"","desc":"Integers should interpolate seamlessly."},{"name":"Basic Decimal Interpolation","data":{"power":1.21},"expected":"\"1.21 jiggawatts!\"","template":"\"{{power}} jiggawatts!\"","desc":"Decimals should interpolate seamlessly with proper significance."},{"name":"Triple Mustache Decimal Interpolation","data":{"power":1.21},"expected":"\"1.21 jiggawatts!\"","template":"\"{{{power}}} jiggawatts!\"","desc":"Decimals should interpolate seamlessly with proper significance."},{"name":"Ampersand Decimal Interpolation","data":{"power":1.21},"expected":"\"1.21 jiggawatts!\"","template":"\"{{&power}} jiggawatts!\"","desc":"Decimals should interpolate seamlessly with proper significance."},{"name":"Basic Context Miss Interpolation","data":{},"expected":"I () be seen!","template":"I ({{cannot}}) be seen!","desc":"Failed context lookups should default to empty strings."},{"name":"Triple Mustache Context Miss Interpolation","data":{},"expected":"I () be seen!","template":"I ({{{cannot}}}) be seen!","desc":"Failed context lookups should default to empty strings."},{"name":"Ampersand Context Miss Interpolation","data":{},"expected":"I () be seen!","template":"I ({{&cannot}}) be seen!","desc":"Failed context lookups should default to empty strings."},{"name":"Dotted Names - Basic Interpolation","data":{"person":{"name":"Joe"}},"expected":"\"Joe\" == \"Joe\"","template":"\"{{person.name}}\" == \"{{#person}}{{name}}{{/person}}\"","desc":"Dotted names should be considered a form of shorthand for sections."},{"name":"Dotted Names - Triple Mustache Interpolation","data":{"person":{"name":"Joe"}},"expected":"\"Joe\" == \"Joe\"","template":"\"{{{person.name}}}\" == \"{{#person}}{{{name}}}{{/person}}\"","desc":"Dotted names should be considered a form of shorthand for sections."},{"name":"Dotted Names - Ampersand Interpolation","data":{"person":{"name":"Joe"}},"expected":"\"Joe\" == \"Joe\"","template":"\"{{&person.name}}\" == \"{{#person}}{{&name}}{{/person}}\"","desc":"Dotted names should be considered a form of shorthand for sections."},{"name":"Dotted Names - Arbitrary Depth","data":{"a":{"b":{"c":{"d":{"e":{"name":"Phil"}}}}}},"expected":"\"Phil\" == \"Phil\"","template":"\"{{a.b.c.d.e.name}}\" == \"Phil\"","desc":"Dotted names should be functional to any level of nesting."},{"name":"Dotted Names - Broken Chains","data":{"a":{}},"expected":"\"\" == \"\"","template":"\"{{a.b.c}}\" == \"\"","desc":"Any falsey value prior to the last part of the name should yield ''."},{"name":"Dotted Names - Broken Chain Resolution","data":{"a":{"b":{}},"c":{"name":"Jim"}},"expected":"\"\" == \"\"","template":"\"{{a.b.c.name}}\" == \"\"","desc":"Each part of a dotted name should resolve only against its parent."},{"name":"Dotted Names - Initial Resolution","data":{"a":{"b":{"c":{"d":{"e":{"name":"Phil"}}}}},"b":{"c":{"d":{"e":{"name":"Wrong"}}}}},"expected":"\"Phil\" == \"Phil\"","template":"\"{{#a}}{{b.c.d.e.name}}{{/a}}\" == \"Phil\"","desc":"The first part of a dotted name should resolve as any other name."},{"name":"Interpolation - Surrounding Whitespace","data":{"string":"---"},"expected":"| --- |","template":"| {{string}} |","desc":"Interpolation should not alter surrounding whitespace."},{"name":"Triple Mustache - Surrounding Whitespace","data":{"string":"---"},"expected":"| --- |","template":"| {{{string}}} |","desc":"Interpolation should not alter surrounding whitespace."},{"name":"Ampersand - Surrounding Whitespace","data":{"string":"---"},"expected":"| --- |","template":"| {{&string}} |","desc":"Interpolation should not alter surrounding whitespace."},{"name":"Interpolation - Standalone","data":{"string":"---"},"expected":" ---\n","template":" {{string}}\n","desc":"Standalone interpolation should not alter surrounding whitespace."},{"name":"Triple Mustache - Standalone","data":{"string":"---"},"expected":" ---\n","template":" {{{string}}}\n","desc":"Standalone interpolation should not alter surrounding whitespace."},{"name":"Ampersand - Standalone","data":{"string":"---"},"expected":" ---\n","template":" {{&string}}\n","desc":"Standalone interpolation should not alter surrounding whitespace."},{"name":"Interpolation With Padding","data":{"string":"---"},"expected":"|---|","template":"|{{ string }}|","desc":"Superfluous in-tag whitespace should be ignored."},{"name":"Triple Mustache With Padding","data":{"string":"---"},"expected":"|---|","template":"|{{{ string }}}|","desc":"Superfluous in-tag whitespace should be ignored."},{"name":"Ampersand With Padding","data":{"string":"---"},"expected":"|---|","template":"|{{& string }}|","desc":"Superfluous in-tag whitespace should be ignored."}]} DELETED testdata/mustache/inverted.json Index: testdata/mustache/inverted.json ================================================================== --- testdata/mustache/inverted.json +++ /dev/null @@ -1,1 +0,0 @@ -{"__ATTN__":"Do not edit this file; changes belong in the appropriate YAML file.","overview":"Inverted Section tags and End Section tags are used in combination to wrap a\nsection of the template.\n\nThese tags' content MUST be a non-whitespace character sequence NOT\ncontaining the current closing delimiter; each Inverted Section tag MUST be\nfollowed by an End Section tag with the same content within the same\nsection.\n\nThis tag's content names the data to replace the tag. Name resolution is as\nfollows:\n 1) Split the name on periods; the first part is the name to resolve, any\n remaining parts should be retained.\n 2) Walk the context stack from top to bottom, finding the first context\n that is a) a hash containing the name as a key OR b) an object responding\n to a method with the given name.\n 3) If the context is a hash, the data is the value associated with the\n name.\n 4) If the context is an object and the method with the given name has an\n arity of 1, the method SHOULD be called with a String containing the\n unprocessed contents of the sections; the data is the value returned.\n 5) Otherwise, the data is the value returned by calling the method with\n the given name.\n 6) If any name parts were retained in step 1, each should be resolved\n against a context stack containing only the result from the former\n resolution. If any part fails resolution, the result should be considered\n falsey, and should interpolate as the empty string.\nIf the data is not of a list type, it is coerced into a list as follows: if\nthe data is truthy (e.g. `!!data == true`), use a single-element list\ncontaining the data, otherwise use an empty list.\n\nThis section MUST NOT be rendered unless the data list is empty.\n\nInverted Section and End Section tags SHOULD be treated as standalone when\nappropriate.\n","tests":[{"name":"Falsey","data":{"boolean":false},"expected":"\"This should be rendered.\"","template":"\"{{^boolean}}This should be rendered.{{/boolean}}\"","desc":"Falsey sections should have their contents rendered."},{"name":"Truthy","data":{"boolean":true},"expected":"\"\"","template":"\"{{^boolean}}This should not be rendered.{{/boolean}}\"","desc":"Truthy sections should have their contents omitted."},{"name":"Context","data":{"context":{"name":"Joe"}},"expected":"\"\"","template":"\"{{^context}}Hi {{name}}.{{/context}}\"","desc":"Objects and hashes should behave like truthy values."},{"name":"List","data":{"list":[{"n":1},{"n":2},{"n":3}]},"expected":"\"\"","template":"\"{{^list}}{{n}}{{/list}}\"","desc":"Lists should behave like truthy values."},{"name":"Empty List","data":{"list":[]},"expected":"\"Yay lists!\"","template":"\"{{^list}}Yay lists!{{/list}}\"","desc":"Empty lists should behave like falsey values."},{"name":"Doubled","data":{"two":"second","bool":false},"expected":"* first\n* second\n* third\n","template":"{{^bool}}\n* first\n{{/bool}}\n* {{two}}\n{{^bool}}\n* third\n{{/bool}}\n","desc":"Multiple inverted sections per template should be permitted."},{"name":"Nested (Falsey)","data":{"bool":false},"expected":"| A B C D E |","template":"| A {{^bool}}B {{^bool}}C{{/bool}} D{{/bool}} E |","desc":"Nested falsey sections should have their contents rendered."},{"name":"Nested (Truthy)","data":{"bool":true},"expected":"| A E |","template":"| A {{^bool}}B {{^bool}}C{{/bool}} D{{/bool}} E |","desc":"Nested truthy sections should be omitted."},{"name":"Context Misses","data":{},"expected":"[Cannot find key 'missing'!]","template":"[{{^missing}}Cannot find key 'missing'!{{/missing}}]","desc":"Failed context lookups should be considered falsey."},{"name":"Dotted Names - Truthy","data":{"a":{"b":{"c":true}}},"expected":"\"\" == \"\"","template":"\"{{^a.b.c}}Not Here{{/a.b.c}}\" == \"\"","desc":"Dotted names should be valid for Inverted Section tags."},{"name":"Dotted Names - Falsey","data":{"a":{"b":{"c":false}}},"expected":"\"Not Here\" == \"Not Here\"","template":"\"{{^a.b.c}}Not Here{{/a.b.c}}\" == \"Not Here\"","desc":"Dotted names should be valid for Inverted Section tags."},{"name":"Dotted Names - Broken Chains","data":{"a":{}},"expected":"\"Not Here\" == \"Not Here\"","template":"\"{{^a.b.c}}Not Here{{/a.b.c}}\" == \"Not Here\"","desc":"Dotted names that cannot be resolved should be considered falsey."},{"name":"Surrounding Whitespace","data":{"boolean":false},"expected":" | \t|\t | \n","template":" | {{^boolean}}\t|\t{{/boolean}} | \n","desc":"Inverted sections should not alter surrounding whitespace."},{"name":"Internal Whitespace","data":{"boolean":false},"expected":" | \n | \n","template":" | {{^boolean}} {{! Important Whitespace }}\n {{/boolean}} | \n","desc":"Inverted should not alter internal whitespace."},{"name":"Indented Inline Sections","data":{"boolean":false},"expected":" NO\n WAY\n","template":" {{^boolean}}NO{{/boolean}}\n {{^boolean}}WAY{{/boolean}}\n","desc":"Single-line sections should not alter surrounding whitespace."},{"name":"Standalone Lines","data":{"boolean":false},"expected":"| This Is\n|\n| A Line\n","template":"| This Is\n{{^boolean}}\n|\n{{/boolean}}\n| A Line\n","desc":"Standalone lines should be removed from the template."},{"name":"Standalone Indented Lines","data":{"boolean":false},"expected":"| This Is\n|\n| A Line\n","template":"| This Is\n {{^boolean}}\n|\n {{/boolean}}\n| A Line\n","desc":"Standalone indented lines should be removed from the template."},{"name":"Standalone Line Endings","data":{"boolean":false},"expected":"|\r\n|","template":"|\r\n{{^boolean}}\r\n{{/boolean}}\r\n|","desc":"\"\\r\\n\" should be considered a newline for standalone tags."},{"name":"Standalone Without Previous Line","data":{"boolean":false},"expected":"^\n/","template":" {{^boolean}}\n^{{/boolean}}\n/","desc":"Standalone tags should not require a newline to precede them."},{"name":"Standalone Without Newline","data":{"boolean":false},"expected":"^\n/\n","template":"^{{^boolean}}\n/\n {{/boolean}}","desc":"Standalone tags should not require a newline to follow them."},{"name":"Padding","data":{"boolean":false},"expected":"|=|","template":"|{{^ boolean }}={{/ boolean }}|","desc":"Superfluous in-tag whitespace should be ignored."}]} DELETED testdata/mustache/partials.json Index: testdata/mustache/partials.json ================================================================== --- testdata/mustache/partials.json +++ /dev/null @@ -1,1 +0,0 @@ -{"__ATTN__":"Do not edit this file; changes belong in the appropriate YAML file.","overview":"Partial tags are used to expand an external template into the current\ntemplate.\n\nThe tag's content MUST be a non-whitespace character sequence NOT containing\nthe current closing delimiter.\n\nThis tag's content names the partial to inject. Set Delimiter tags MUST NOT\naffect the parsing of a partial. The partial MUST be rendered against the\ncontext stack local to the tag. If the named partial cannot be found, the\nempty string SHOULD be used instead, as in interpolations.\n\nPartial tags SHOULD be treated as standalone when appropriate. If this tag\nis used standalone, any whitespace preceding the tag should treated as\nindentation, and prepended to each line of the partial before rendering.\n","tests":[{"name":"Basic Behavior","data":{},"expected":"\"from partial\"","template":"\"{{>text}}\"","desc":"The greater-than operator should expand to the named partial.","partials":{"text":"from partial"}},{"name":"Failed Lookup","data":{},"expected":"\"\"","template":"\"{{>text}}\"","desc":"The empty string should be used when the named partial is not found.","partials":{}},{"name":"Context","data":{"text":"content"},"expected":"\"*content*\"","template":"\"{{>partial}}\"","desc":"The greater-than operator should operate within the current context.","partials":{"partial":"*{{text}}*"}},{"name":"Recursion","data":{"content":"X","nodes":[{"content":"Y","nodes":[]}]},"expected":"X>","template":"{{>node}}","desc":"The greater-than operator should properly recurse.","partials":{"node":"{{content}}<{{#nodes}}{{>node}}{{/nodes}}>"}},{"name":"Surrounding Whitespace","data":{},"expected":"| \t|\t |","template":"| {{>partial}} |","desc":"The greater-than operator should not alter surrounding whitespace.","partials":{"partial":"\t|\t"}},{"name":"Inline Indentation","data":{"data":"|"},"expected":" | >\n>\n","template":" {{data}} {{> partial}}\n","desc":"Whitespace should be left untouched.","partials":{"partial":">\n>"}},{"name":"Standalone Line Endings","data":{},"expected":"|\r\n>|","template":"|\r\n{{>partial}}\r\n|","desc":"\"\\r\\n\" should be considered a newline for standalone tags.","partials":{"partial":">"}},{"name":"Standalone Without Previous Line","data":{},"expected":" >\n >>","template":" {{>partial}}\n>","desc":"Standalone tags should not require a newline to precede them.","partials":{"partial":">\n>"}},{"name":"Standalone Without Newline","data":{},"expected":">\n >\n >","template":">\n {{>partial}}","desc":"Standalone tags should not require a newline to follow them.","partials":{"partial":">\n>"}},{"name":"Standalone Indentation","data":{"content":"<\n->"},"expected":"\\\n |\n <\n->\n |\n/\n","template":"\\\n {{>partial}}\n/\n","desc":"Each line of the partial should be indented before rendering.","partials":{"partial":"|\n{{{content}}}\n|\n"}},{"name":"Padding Whitespace","data":{"boolean":true},"expected":"|[]|","template":"|{{> partial }}|","desc":"Superfluous in-tag whitespace should be ignored.","partials":{"partial":"[]"}}]} DELETED testdata/mustache/sections.json Index: testdata/mustache/sections.json ================================================================== --- testdata/mustache/sections.json +++ /dev/null @@ -1,1 +0,0 @@ -{"__ATTN__":"Do not edit this file; changes belong in the appropriate YAML file.","overview":"Section tags and End Section tags are used in combination to wrap a section\nof the template for iteration\n\nThese tags' content MUST be a non-whitespace character sequence NOT\ncontaining the current closing delimiter; each Section tag MUST be followed\nby an End Section tag with the same content within the same section.\n\nThis tag's content names the data to replace the tag. Name resolution is as\nfollows:\n 1) Split the name on periods; the first part is the name to resolve, any\n remaining parts should be retained.\n 2) Walk the context stack from top to bottom, finding the first context\n that is a) a hash containing the name as a key OR b) an object responding\n to a method with the given name.\n 3) If the context is a hash, the data is the value associated with the\n name.\n 4) If the context is an object and the method with the given name has an\n arity of 1, the method SHOULD be called with a String containing the\n unprocessed contents of the sections; the data is the value returned.\n 5) Otherwise, the data is the value returned by calling the method with\n the given name.\n 6) If any name parts were retained in step 1, each should be resolved\n against a context stack containing only the result from the former\n resolution. If any part fails resolution, the result should be considered\n falsey, and should interpolate as the empty string.\nIf the data is not of a list type, it is coerced into a list as follows: if\nthe data is truthy (e.g. `!!data == true`), use a single-element list\ncontaining the data, otherwise use an empty list.\n\nFor each element in the data list, the element MUST be pushed onto the\ncontext stack, the section MUST be rendered, and the element MUST be popped\noff the context stack.\n\nSection and End Section tags SHOULD be treated as standalone when\nappropriate.\n","tests":[{"name":"Truthy","data":{"boolean":true},"expected":"\"This should be rendered.\"","template":"\"{{#boolean}}This should be rendered.{{/boolean}}\"","desc":"Truthy sections should have their contents rendered."},{"name":"Falsey","data":{"boolean":false},"expected":"\"\"","template":"\"{{#boolean}}This should not be rendered.{{/boolean}}\"","desc":"Falsey sections should have their contents omitted."},{"name":"Context","data":{"context":{"name":"Joe"}},"expected":"\"Hi Joe.\"","template":"\"{{#context}}Hi {{name}}.{{/context}}\"","desc":"Objects and hashes should be pushed onto the context stack."},{"name":"Deeply Nested Contexts","data":{"a":{"one":1},"b":{"two":2},"c":{"three":3},"d":{"four":4},"e":{"five":5}},"expected":"1\n121\n12321\n1234321\n123454321\n1234321\n12321\n121\n1\n","template":"{{#a}}\n{{one}}\n{{#b}}\n{{one}}{{two}}{{one}}\n{{#c}}\n{{one}}{{two}}{{three}}{{two}}{{one}}\n{{#d}}\n{{one}}{{two}}{{three}}{{four}}{{three}}{{two}}{{one}}\n{{#e}}\n{{one}}{{two}}{{three}}{{four}}{{five}}{{four}}{{three}}{{two}}{{one}}\n{{/e}}\n{{one}}{{two}}{{three}}{{four}}{{three}}{{two}}{{one}}\n{{/d}}\n{{one}}{{two}}{{three}}{{two}}{{one}}\n{{/c}}\n{{one}}{{two}}{{one}}\n{{/b}}\n{{one}}\n{{/a}}\n","desc":"All elements on the context stack should be accessible."},{"name":"List","data":{"list":[{"item":1},{"item":2},{"item":3}]},"expected":"\"123\"","template":"\"{{#list}}{{item}}{{/list}}\"","desc":"Lists should be iterated; list items should visit the context stack."},{"name":"Empty List","data":{"list":[]},"expected":"\"\"","template":"\"{{#list}}Yay lists!{{/list}}\"","desc":"Empty lists should behave like falsey values."},{"name":"Doubled","data":{"two":"second","bool":true},"expected":"* first\n* second\n* third\n","template":"{{#bool}}\n* first\n{{/bool}}\n* {{two}}\n{{#bool}}\n* third\n{{/bool}}\n","desc":"Multiple sections per template should be permitted."},{"name":"Nested (Truthy)","data":{"bool":true},"expected":"| A B C D E |","template":"| A {{#bool}}B {{#bool}}C{{/bool}} D{{/bool}} E |","desc":"Nested truthy sections should have their contents rendered."},{"name":"Nested (Falsey)","data":{"bool":false},"expected":"| A E |","template":"| A {{#bool}}B {{#bool}}C{{/bool}} D{{/bool}} E |","desc":"Nested falsey sections should be omitted."},{"name":"Context Misses","data":{},"expected":"[]","template":"[{{#missing}}Found key 'missing'!{{/missing}}]","desc":"Failed context lookups should be considered falsey."},{"name":"Implicit Iterator - String","data":{"list":["a","b","c","d","e"]},"expected":"\"(a)(b)(c)(d)(e)\"","template":"\"{{#list}}({{.}}){{/list}}\"","desc":"Implicit iterators should directly interpolate strings."},{"name":"Implicit Iterator - Integer","data":{"list":[1,2,3,4,5]},"expected":"\"(1)(2)(3)(4)(5)\"","template":"\"{{#list}}({{.}}){{/list}}\"","desc":"Implicit iterators should cast integers to strings and interpolate."},{"name":"Implicit Iterator - Decimal","data":{"list":[1.1,2.2,3.3,4.4,5.5]},"expected":"\"(1.1)(2.2)(3.3)(4.4)(5.5)\"","template":"\"{{#list}}({{.}}){{/list}}\"","desc":"Implicit iterators should cast decimals to strings and interpolate."},{"name":"Implicit Iterator - Array","desc":"Implicit iterators should allow iterating over nested arrays.","data":{"list":[[1,2,3],["a","b","c"]]},"template":"\"{{#list}}({{#.}}{{.}}{{/.}}){{/list}}\"","expected":"\"(123)(abc)\""},{"name":"Dotted Names - Truthy","data":{"a":{"b":{"c":true}}},"expected":"\"Here\" == \"Here\"","template":"\"{{#a.b.c}}Here{{/a.b.c}}\" == \"Here\"","desc":"Dotted names should be valid for Section tags."},{"name":"Dotted Names - Falsey","data":{"a":{"b":{"c":false}}},"expected":"\"\" == \"\"","template":"\"{{#a.b.c}}Here{{/a.b.c}}\" == \"\"","desc":"Dotted names should be valid for Section tags."},{"name":"Dotted Names - Broken Chains","data":{"a":{}},"expected":"\"\" == \"\"","template":"\"{{#a.b.c}}Here{{/a.b.c}}\" == \"\"","desc":"Dotted names that cannot be resolved should be considered falsey."},{"name":"Surrounding Whitespace","data":{"boolean":true},"expected":" | \t|\t | \n","template":" | {{#boolean}}\t|\t{{/boolean}} | \n","desc":"Sections should not alter surrounding whitespace."},{"name":"Internal Whitespace","data":{"boolean":true},"expected":" | \n | \n","template":" | {{#boolean}} {{! Important Whitespace }}\n {{/boolean}} | \n","desc":"Sections should not alter internal whitespace."},{"name":"Indented Inline Sections","data":{"boolean":true},"expected":" YES\n GOOD\n","template":" {{#boolean}}YES{{/boolean}}\n {{#boolean}}GOOD{{/boolean}}\n","desc":"Single-line sections should not alter surrounding whitespace."},{"name":"Standalone Lines","data":{"boolean":true},"expected":"| This Is\n|\n| A Line\n","template":"| This Is\n{{#boolean}}\n|\n{{/boolean}}\n| A Line\n","desc":"Standalone lines should be removed from the template."},{"name":"Indented Standalone Lines","data":{"boolean":true},"expected":"| This Is\n|\n| A Line\n","template":"| This Is\n {{#boolean}}\n|\n {{/boolean}}\n| A Line\n","desc":"Indented standalone lines should be removed from the template."},{"name":"Standalone Line Endings","data":{"boolean":true},"expected":"|\r\n|","template":"|\r\n{{#boolean}}\r\n{{/boolean}}\r\n|","desc":"\"\\r\\n\" should be considered a newline for standalone tags."},{"name":"Standalone Without Previous Line","data":{"boolean":true},"expected":"#\n/","template":" {{#boolean}}\n#{{/boolean}}\n/","desc":"Standalone tags should not require a newline to precede them."},{"name":"Standalone Without Newline","data":{"boolean":true},"expected":"#\n/\n","template":"#{{#boolean}}\n/\n {{/boolean}}","desc":"Standalone tags should not require a newline to follow them."},{"name":"Padding","data":{"boolean":true},"expected":"|=|","template":"|{{# boolean }}={{/ boolean }}|","desc":"Superfluous in-tag whitespace should be ignored."}]} DELETED testdata/mustache/~lambdas.json Index: testdata/mustache/~lambdas.json ================================================================== --- testdata/mustache/~lambdas.json +++ /dev/null @@ -1,1 +0,0 @@ -{"__ATTN__":"Do not edit this file; changes belong in the appropriate YAML file.","overview":"Lambdas are a special-cased data type for use in interpolations and\nsections.\n\nWhen used as the data value for an Interpolation tag, the lambda MUST be\ntreatable as an arity 0 function, and invoked as such. The returned value\nMUST be rendered against the default delimiters, then interpolated in place\nof the lambda.\n\nWhen used as the data value for a Section tag, the lambda MUST be treatable\nas an arity 1 function, and invoked as such (passing a String containing the\nunprocessed section contents). The returned value MUST be rendered against\nthe current delimiters, then interpolated in place of the section.\n","tests":[{"name":"Interpolation","data":{"lambda":{"php":"return \"world\";","clojure":"(fn [] \"world\")","__tag__":"code","perl":"sub { \"world\" }","python":"lambda: \"world\"","ruby":"proc { \"world\" }","js":"function() { return \"world\" }"}},"expected":"Hello, world!","template":"Hello, {{lambda}}!","desc":"A lambda's return value should be interpolated."},{"name":"Interpolation - Expansion","data":{"planet":"world","lambda":{"php":"return \"{{planet}}\";","clojure":"(fn [] \"{{planet}}\")","__tag__":"code","perl":"sub { \"{{planet}}\" }","python":"lambda: \"{{planet}}\"","ruby":"proc { \"{{planet}}\" }","js":"function() { return \"{{planet}}\" }"}},"expected":"Hello, world!","template":"Hello, {{lambda}}!","desc":"A lambda's return value should be parsed."},{"name":"Interpolation - Alternate Delimiters","data":{"planet":"world","lambda":{"php":"return \"|planet| => {{planet}}\";","clojure":"(fn [] \"|planet| => {{planet}}\")","__tag__":"code","perl":"sub { \"|planet| => {{planet}}\" }","python":"lambda: \"|planet| => {{planet}}\"","ruby":"proc { \"|planet| => {{planet}}\" }","js":"function() { return \"|planet| => {{planet}}\" }"}},"expected":"Hello, (|planet| => world)!","template":"{{= | | =}}\nHello, (|&lambda|)!","desc":"A lambda's return value should parse with the default delimiters."},{"name":"Interpolation - Multiple Calls","data":{"lambda":{"php":"global $calls; return ++$calls;","clojure":"(def g (atom 0)) (fn [] (swap! g inc))","__tag__":"code","perl":"sub { no strict; $calls += 1 }","python":"lambda: globals().update(calls=globals().get(\"calls\",0)+1) or calls","ruby":"proc { $calls ||= 0; $calls += 1 }","js":"function() { return (g=(function(){return this})()).calls=(g.calls||0)+1 }"}},"expected":"1 == 2 == 3","template":"{{lambda}} == {{{lambda}}} == {{lambda}}","desc":"Interpolated lambdas should not be cached."},{"name":"Escaping","data":{"lambda":{"php":"return \">\";","clojure":"(fn [] \">\")","__tag__":"code","perl":"sub { \">\" }","python":"lambda: \">\"","ruby":"proc { \">\" }","js":"function() { return \">\" }"}},"expected":"<>>","template":"<{{lambda}}{{{lambda}}}","desc":"Lambda results should be appropriately escaped."},{"name":"Section","data":{"x":"Error!","lambda":{"php":"return ($text == \"{{x}}\") ? \"yes\" : \"no\";","clojure":"(fn [text] (if (= text \"{{x}}\") \"yes\" \"no\"))","__tag__":"code","perl":"sub { $_[0] eq \"{{x}}\" ? \"yes\" : \"no\" }","python":"lambda text: text == \"{{x}}\" and \"yes\" or \"no\"","ruby":"proc { |text| text == \"{{x}}\" ? \"yes\" : \"no\" }","js":"function(txt) { return (txt == \"{{x}}\" ? \"yes\" : \"no\") }"}},"expected":"","template":"<{{#lambda}}{{x}}{{/lambda}}>","desc":"Lambdas used for sections should receive the raw section string."},{"name":"Section - Expansion","data":{"planet":"Earth","lambda":{"php":"return $text . \"{{planet}}\" . $text;","clojure":"(fn [text] (str text \"{{planet}}\" text))","__tag__":"code","perl":"sub { $_[0] . \"{{planet}}\" . $_[0] }","python":"lambda text: \"%s{{planet}}%s\" % (text, text)","ruby":"proc { |text| \"#{text}{{planet}}#{text}\" }","js":"function(txt) { return txt + \"{{planet}}\" + txt }"}},"expected":"<-Earth->","template":"<{{#lambda}}-{{/lambda}}>","desc":"Lambdas used for sections should have their results parsed."},{"name":"Section - Alternate Delimiters","data":{"planet":"Earth","lambda":{"php":"return $text . \"{{planet}} => |planet|\" . $text;","clojure":"(fn [text] (str text \"{{planet}} => |planet|\" text))","__tag__":"code","perl":"sub { $_[0] . \"{{planet}} => |planet|\" . $_[0] }","python":"lambda text: \"%s{{planet}} => |planet|%s\" % (text, text)","ruby":"proc { |text| \"#{text}{{planet}} => |planet|#{text}\" }","js":"function(txt) { return txt + \"{{planet}} => |planet|\" + txt }"}},"expected":"<-{{planet}} => Earth->","template":"{{= | | =}}<|#lambda|-|/lambda|>","desc":"Lambdas used for sections should parse with the current delimiters."},{"name":"Section - Multiple Calls","data":{"lambda":{"php":"return \"__\" . $text . \"__\";","clojure":"(fn [text] (str \"__\" text \"__\"))","__tag__":"code","perl":"sub { \"__\" . $_[0] . \"__\" }","python":"lambda text: \"__%s__\" % (text)","ruby":"proc { |text| \"__#{text}__\" }","js":"function(txt) { return \"__\" + txt + \"__\" }"}},"expected":"__FILE__ != __LINE__","template":"{{#lambda}}FILE{{/lambda}} != {{#lambda}}LINE{{/lambda}}","desc":"Lambdas used for sections should not be cached."},{"name":"Inverted Section","data":{"static":"static","lambda":{"php":"return false;","clojure":"(fn [text] false)","__tag__":"code","perl":"sub { 0 }","python":"lambda text: 0","ruby":"proc { |text| false }","js":"function(txt) { return false }"}},"expected":"<>","template":"<{{^lambda}}{{static}}{{/lambda}}>","desc":"Lambdas used for inverted sections should be considered truthy."}]} ADDED testdata/naughty/LICENSE Index: testdata/naughty/LICENSE ================================================================== --- /dev/null +++ testdata/naughty/LICENSE @@ -0,0 +1,22 @@ +The MIT License (MIT) + +Copyright (c) 2015-2020 Max Woolf + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + ADDED testdata/naughty/README.md Index: testdata/naughty/README.md ================================================================== --- /dev/null +++ testdata/naughty/README.md @@ -0,0 +1,6 @@ +# Big List of Naughty Strings + +A list of strings which have a high probability of causing issues when used as user-input data. + +* Source: https://github.com/minimaxir/big-list-of-naughty-strings +* License: MIT, (c) 2015-2020 Max Woolf (see file LICENSE) ADDED testdata/naughty/blns.txt Index: testdata/naughty/blns.txt ================================================================== --- /dev/null +++ testdata/naughty/blns.txt @@ -0,0 +1,742 @@ +# Reserved Strings +# +# Strings which may be used elsewhere in code + +undefined +undef +null +NULL +(null) +nil +NIL +true +false +True +False +TRUE +FALSE +None +hasOwnProperty +then +constructor +\ +\\ + +# Numeric Strings +# +# Strings which can be interpreted as numeric + +0 +1 +1.00 +$1.00 +1/2 +1E2 +1E02 +1E+02 +-1 +-1.00 +-$1.00 +-1/2 +-1E2 +-1E02 +-1E+02 +1/0 +0/0 +-2147483648/-1 +-9223372036854775808/-1 +-0 +-0.0 ++0 ++0.0 +0.00 +0..0 +. +0.0.0 +0,00 +0,,0 +, +0,0,0 +0.0/0 +1.0/0.0 +0.0/0.0 +1,0/0,0 +0,0/0,0 +--1 +- +-. +-, +999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 +NaN +Infinity +-Infinity +INF +1#INF +-1#IND +1#QNAN +1#SNAN +1#IND +0x0 +0xffffffff +0xffffffffffffffff +0xabad1dea +123456789012345678901234567890123456789 +1,000.00 +1 000.00 +1'000.00 +1,000,000.00 +1 000 000.00 +1'000'000.00 +1.000,00 +1 000,00 +1'000,00 +1.000.000,00 +1 000 000,00 +1'000'000,00 +01000 +08 +09 +2.2250738585072011e-308 + +# Special Characters +# +# ASCII punctuation. All of these characters may need to be escaped in some +# contexts. Divided into three groups based on (US-layout) keyboard position. + +,./;'[]\-= +<>?:"{}|_+ +!@#$%^&*()`~ + +# Non-whitespace C0 controls: U+0001 through U+0008, U+000E through U+001F, +# and U+007F (DEL) +# Often forbidden to appear in various text-based file formats (e.g. XML), +# or reused for internal delimiters on the theory that they should never +# appear in input. +# The next line may appear to be blank or mojibake in some viewers. + + +# Non-whitespace C1 controls: U+0080 through U+0084 and U+0086 through U+009F. +# Commonly misinterpreted as additional graphic characters. +# The next line may appear to be blank, mojibake, or dingbats in some viewers. +€‚ƒ„†‡ˆ‰Š‹ŒŽ‘’“”•–—˜™š›œžŸ + +# Whitespace: all of the characters with category Zs, Zl, or Zp (in Unicode +# version 8.0.0), plus U+0009 (HT), U+000B (VT), U+000C (FF), U+0085 (NEL), +# and U+200B (ZERO WIDTH SPACE), which are in the C categories but are often +# treated as whitespace in some contexts. +# This file unfortunately cannot express strings containing +# U+0000, U+000A, or U+000D (NUL, LF, CR). +# The next line may appear to be blank or mojibake in some viewers. +# The next line may be flagged for "trailing whitespace" in some viewers. + …             ​

  　 + +# Unicode additional control characters: all of the characters with +# general category Cf (in Unicode 8.0.0). +# The next line may appear to be blank or mojibake in some viewers. +­؀؁؂؃؄؅؜۝܏᠎​‌‍‎‏‪‫‬‭‮⁠⁡⁢⁣⁤⁦⁧⁨⁩𑂽𛲠𛲡𛲢𛲣𝅳𝅴𝅵𝅶𝅷𝅸𝅹𝅺󠀁󠀠󠀡󠀢󠀣󠀤󠀥󠀦󠀧󠀨󠀩󠀪󠀫󠀬󠀭󠀮󠀯󠀰󠀱󠀲󠀳󠀴󠀵󠀶󠀷󠀸󠀹󠀺󠀻󠀼󠀽󠀾󠀿󠁀󠁁󠁂󠁃󠁄󠁅󠁆󠁇󠁈󠁉󠁊󠁋󠁌󠁍󠁎󠁏󠁐󠁑󠁒󠁓󠁔󠁕󠁖󠁗󠁘󠁙󠁚󠁛󠁜󠁝󠁞󠁟󠁠󠁡󠁢󠁣󠁤󠁥󠁦󠁧󠁨󠁩󠁪󠁫󠁬󠁭󠁮󠁯󠁰󠁱󠁲󠁳󠁴󠁵󠁶󠁷󠁸󠁹󠁺󠁻󠁼󠁽󠁾󠁿 + +# "Byte order marks", U+FEFF and U+FFFE, each on its own line. +# The next two lines may appear to be blank or mojibake in some viewers. + +￾ + +# Unicode Symbols +# +# Strings which contain common unicode symbols (e.g. smart quotes) + +Ω≈ç√∫˜µ≤≥÷ +åß∂ƒ©˙∆˚¬…æ +œ∑´®†¥¨ˆøπ“‘ +¡™£¢∞§¶•ªº–≠ +¸˛Ç◊ı˜Â¯˘¿ +ÅÍÎÏ˝ÓÔÒÚÆ☃ +Œ„´‰ˇÁ¨ˆØ∏”’ +`⁄€‹›fifl‡°·‚—± +⅛⅜⅝⅞ +ЁЂЃЄЅІЇЈЉЊЋЌЍЎЏАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯабвгдежзийклмнопрстуфхцчшщъыьэюя +٠١٢٣٤٥٦٧٨٩ + +# Unicode Subscript/Superscript/Accents +# +# Strings which contain unicode subscripts/superscripts; can cause rendering issues + +⁰⁴⁵ +₀₁₂ +⁰⁴⁵₀₁₂ +ด้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็ ด้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็ ด้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็็้้้้้้้้็็็็็้้้้้็็็็ + +# Quotation Marks +# +# Strings which contain misplaced quotation marks; can cause encoding errors + +' +" +'' +"" +'"' +"''''"'" +"'"'"''''" + + + + + +# Two-Byte Characters +# +# Strings which contain two-byte characters: can cause rendering issues or character-length issues + +田中さんにあげて下さい +パーティーへ行かないか +和製漢語 +部落格 +사회과학원 어학연구소 +찦차를 타고 온 펲시맨과 쑛다리 똠방각하 +社會科學院語學研究所 +울란바토르 +𠜎𠜱𠝹𠱓𠱸𠲖𠳏 + +# Strings which contain two-byte letters: can cause issues with naïve UTF-16 capitalizers which think that 16 bits == 1 character + +𐐜 𐐔𐐇𐐝𐐀𐐡𐐇𐐓 𐐙𐐊𐐡𐐝𐐓/𐐝𐐇𐐗𐐊𐐤𐐔 𐐒𐐋𐐗 𐐒𐐌 𐐜 𐐡𐐀𐐖𐐇𐐤𐐓𐐝 𐐱𐑂 𐑄 𐐔𐐇𐐝𐐀𐐡𐐇𐐓 𐐏𐐆𐐅𐐤𐐆𐐚𐐊𐐡𐐝𐐆𐐓𐐆 + +# Special Unicode Characters Union +# +# A super string recommended by VMware Inc. Globalization Team: can effectively cause rendering issues or character-length issues to validate product globalization readiness. +# +# 表 CJK_UNIFIED_IDEOGRAPHS (U+8868) +# ポ KATAKANA LETTER PO (U+30DD) +# あ HIRAGANA LETTER A (U+3042) +# A LATIN CAPITAL LETTER A (U+0041) +# 鷗 CJK_UNIFIED_IDEOGRAPHS (U+9DD7) +# Œ LATIN SMALL LIGATURE OE (U+0153) +# é LATIN SMALL LETTER E WITH ACUTE (U+00E9) +# Ｂ FULLWIDTH LATIN CAPITAL LETTER B (U+FF22) +# 逍 CJK_UNIFIED_IDEOGRAPHS (U+900D) +# Ü LATIN SMALL LETTER U WITH DIAERESIS (U+00FC) +# ß LATIN SMALL LETTER SHARP S (U+00DF) +# ª FEMININE ORDINAL INDICATOR (U+00AA) +# ą LATIN SMALL LETTER A WITH OGONEK (U+0105) +# ñ LATIN SMALL LETTER N WITH TILDE (U+00F1) +# 丂 CJK_UNIFIED_IDEOGRAPHS (U+4E02) +# 㐀 CJK Ideograph Extension A, First (U+3400) +# 𠀀 CJK Ideograph Extension B, First (U+20000) + +表ポあA鷗ŒéＢ逍Üߪąñ丂㐀𠀀 + +# Changing length when lowercased +# +# Characters which increase in length (2 to 3 bytes) when lowercased +# Credit: https://twitter.com/jifa/status/625776454479970304 + +Ⱥ +Ⱦ + +# Japanese Emoticons +# +# Strings which consists of Japanese-style emoticons which are popular on the web + +ヽ༼ຈل͜ຈ༽ﾉ ヽ༼ຈل͜ຈ༽ﾉ +(｡◕ ∀ ◕｡) +｀ｨ(´∀｀∩ +__ﾛ(,_,*) +・(￣∀￣)・:*: +ﾟ･✿ヾ╲(｡◕‿◕｡)╱✿･ﾟ +,。・:*:・゜’( ☻ ω ☻ )。・:*:・゜’ +(╯°□°）╯︵ ┻━┻) +(ﾉಥ益ಥ）ﾉ ┻━┻ +┬─┬ノ( º _ ºノ) +( ͡° ͜ʖ ͡°) +¯\_(ツ)_/¯ + +# Emoji +# +# Strings which contain Emoji; should be the same behavior as two-byte characters, but not always + +😍 +👩🏽 +👨‍🦰 👨🏿‍🦰 👨‍🦱 👨🏿‍🦱 🦹🏿‍♂️ +👾 🙇 💁 🙅 🙆 🙋 🙎 🙍 +🐵 🙈 🙉 🙊 +❤️ 💔 💌 💕 💞 💓 💗 💖 💘 💝 💟 💜 💛 💚 💙 +✋🏿 💪🏿 👐🏿 🙌🏿 👏🏿 🙏🏿 +👨‍👩‍👦 👨‍👩‍👧‍👦 👨‍👨‍👦 👩‍👩‍👧 👨‍👦 👨‍👧‍👦 👩‍👦 👩‍👧‍👦 +🚾 🆒 🆓 🆕 🆖 🆗 🆙 🏧 +0️⃣ 1️⃣ 2️⃣ 3️⃣ 4️⃣ 5️⃣ 6️⃣ 7️⃣ 8️⃣ 9️⃣ 🔟 + +# Regional Indicator Symbols +# +# Regional Indicator Symbols can be displayed differently across +# fonts, and have a number of special behaviors + +🇺🇸🇷🇺🇸 🇦🇫🇦🇲🇸 +🇺🇸🇷🇺🇸🇦🇫🇦🇲 +🇺🇸🇷🇺🇸🇦 + +# Unicode Numbers +# +# Strings which contain unicode numbers; if the code is localized, it should see the input as numeric + +１２３ +١٢٣ + +# Right-To-Left Strings +# +# Strings which contain text that should be rendered RTL if possible (e.g. Arabic, Hebrew) + +ثم نفس سقطت وبالتحديد،, جزيرتي باستخدام أن دنو. إذ هنا؟ الستار وتنصيب كان. أهّل ايطاليا، بريطانيا-فرنسا قد أخذ. سليمان، إتفاقية بين ما, يذكر الحدود أي بعد, معاملة بولندا، الإطلاق عل إيو. +בְּרֵאשִׁית, בָּרָא אֱלֹהִים, אֵת הַשָּׁמַיִם, וְאֵת הָאָרֶץ +הָיְתָהtestالصفحات التّحول +﷽ +ﷺ +مُنَاقَشَةُ سُبُلِ اِسْتِخْدَامِ اللُّغَةِ فِي النُّظُمِ الْقَائِمَةِ وَفِيم يَخُصَّ التَّطْبِيقَاتُ الْحاسُوبِيَّةُ، +الكل في المجمو عة (5) + +# Ogham Text +# +# The only unicode alphabet to use a space which isn't empty but should still act like a space. + +᚛ᚄᚓᚐᚋᚒᚄ ᚑᚄᚂᚑᚏᚅ᚜ +᚛                 ᚜ + +# Trick Unicode +# +# Strings which contain unicode with unusual properties (e.g. Right-to-left override) (c.f. http://www.unicode.org/charts/PDF/U2000.pdf) + +‪‪test‪ +‫test‫ +
test
 +test⁠test‫ +⁦test⁧ + +# Zalgo Text +# +# Strings which contain "corrupted" text. The corruption will not appear in non-HTML text, however. (via http://www.eeemo.net) + +Ṱ̺̺̕o͞ ̷i̲̬͇̪͙n̝̗͕v̟̜̘̦͟o̶̙̰̠kè͚̮̺̪̹̱̤ ̖t̝͕̳̣̻̪͞h̼͓̲̦̳̘̲e͇̣̰̦̬͎ ̢̼̻̱̘h͚͎͙̜̣̲ͅi̦̲̣̰̤v̻͍e̺̭̳̪̰-m̢iͅn̖̺̞̲̯̰d̵̼̟͙̩̼̘̳ ̞̥̱̳̭r̛̗̘e͙p͠r̼̞̻̭̗e̺̠̣͟s̘͇̳͍̝͉e͉̥̯̞̲͚̬͜ǹ̬͎͎̟̖͇̤t͍̬̤͓̼̭͘ͅi̪̱n͠g̴͉ ͏͉ͅc̬̟h͡a̫̻̯͘o̫̟̖͍̙̝͉s̗̦̲.̨̹͈̣ +̡͓̞ͅI̗̘̦͝n͇͇͙v̮̫ok̲̫̙͈i̖͙̭̹̠̞n̡̻̮̣̺g̲͈͙̭͙̬͎ ̰t͔̦h̞̲e̢̤ ͍̬̲͖f̴̘͕̣è͖ẹ̥̩l͖͔͚i͓͚̦͠n͖͍̗͓̳̮g͍ ̨o͚̪͡f̘̣̬ ̖̘͖̟͙̮c҉͔̫͖͓͇͖ͅh̵̤̣͚͔á̗̼͕ͅo̼̣̥s̱͈̺̖̦̻͢.̛̖̞̠̫̰ +̗̺͖̹̯͓Ṯ̤͍̥͇͈h̲́e͏͓̼̗̙̼̣͔ ͇̜̱̠͓͍ͅN͕͠e̗̱z̘̝̜̺͙p̤̺̹͍̯͚e̠̻̠͜r̨̤͍̺̖͔̖̖d̠̟̭̬̝͟i̦͖̩͓͔̤a̠̗̬͉̙n͚͜ ̻̞̰͚ͅh̵͉i̳̞v̢͇ḙ͎͟-҉̭̩̼͔m̤̭̫i͕͇̝̦n̗͙ḍ̟ ̯̲͕͞ǫ̟̯̰̲͙̻̝f ̪̰̰̗̖̭̘͘c̦͍̲̞͍̩̙ḥ͚a̮͎̟̙͜ơ̩̹͎s̤.̝̝ ҉Z̡̖̜͖̰̣͉̜a͖̰͙̬͡l̲̫̳͍̩g̡̟̼̱͚̞̬ͅo̗͜.̟ +̦H̬̤̗̤͝e͜ ̜̥̝̻͍̟́w̕h̖̯͓o̝͙̖͎̱̮ ҉̺̙̞̟͈W̷̼̭a̺̪͍į͈͕̭͙̯̜t̶̼̮s̘͙͖̕ ̠̫̠B̻͍͙͉̳ͅe̵h̵̬͇̫͙i̹͓̳̳̮͎̫̕n͟d̴̪̜̖ ̰͉̩͇͙̲͞ͅT͖̼͓̪͢h͏͓̮̻e̬̝̟ͅ ̤̹̝W͙̞̝͔͇͝ͅa͏͓͔̹̼̣l̴͔̰̤̟͔ḽ̫.͕ +Z̮̞̠͙͔ͅḀ̗̞͈̻̗Ḷ͙͎̯̹̞͓G̻O̭̗̮ + +# Unicode Upsidedown +# +# Strings which contain unicode with an "upsidedown" effect (via http://www.upsidedowntext.com) + +˙ɐnbᴉlɐ ɐuƃɐɯ ǝɹolop ʇǝ ǝɹoqɐl ʇn ʇunpᴉpᴉɔuᴉ ɹodɯǝʇ poɯsnᴉǝ op pǝs 'ʇᴉlǝ ƃuᴉɔsᴉdᴉpɐ ɹnʇǝʇɔǝsuoɔ 'ʇǝɯɐ ʇᴉs ɹolop ɯnsdᴉ ɯǝɹo˥ +00˙Ɩ$- + +# Unicode font +# +# Strings which contain bold/italic/etc. versions of normal characters + +Ｔｈｅ ｑｕｉｃｋ ｂｒｏｗｎ ｆｏｘ ｊｕｍｐｓ ｏｖｅｒ ｔｈｅ ｌａｚｙ ｄｏｇ +𝐓𝐡𝐞 𝐪𝐮𝐢𝐜𝐤 𝐛𝐫𝐨𝐰𝐧 𝐟𝐨𝐱 𝐣𝐮𝐦𝐩𝐬 𝐨𝐯𝐞𝐫 𝐭𝐡𝐞 𝐥𝐚𝐳𝐲 𝐝𝐨𝐠 +𝕿𝖍𝖊 𝖖𝖚𝖎𝖈𝖐 𝖇𝖗𝖔𝖜𝖓 𝖋𝖔𝖝 𝖏𝖚𝖒𝖕𝖘 𝖔𝖛𝖊𝖗 𝖙𝖍𝖊 𝖑𝖆𝖟𝖞 𝖉𝖔𝖌 +𝑻𝒉𝒆 𝒒𝒖𝒊𝒄𝒌 𝒃𝒓𝒐𝒘𝒏 𝒇𝒐𝒙 𝒋𝒖𝒎𝒑𝒔 𝒐𝒗𝒆𝒓 𝒕𝒉𝒆 𝒍𝒂𝒛𝒚 𝒅𝒐𝒈 +𝓣𝓱𝓮 𝓺𝓾𝓲𝓬𝓴 𝓫𝓻𝓸𝔀𝓷 𝓯𝓸𝔁 𝓳𝓾𝓶𝓹𝓼 𝓸𝓿𝓮𝓻 𝓽𝓱𝓮 𝓵𝓪𝔃𝔂 𝓭𝓸𝓰 +𝕋𝕙𝕖 𝕢𝕦𝕚𝕔𝕜 𝕓𝕣𝕠𝕨𝕟 𝕗𝕠𝕩 𝕛𝕦𝕞𝕡𝕤 𝕠𝕧𝕖𝕣 𝕥𝕙𝕖 𝕝𝕒𝕫𝕪 𝕕𝕠𝕘 +𝚃𝚑𝚎 𝚚𝚞𝚒𝚌𝚔 𝚋𝚛𝚘𝚠𝚗 𝚏𝚘𝚡 𝚓𝚞𝚖𝚙𝚜 𝚘𝚟𝚎𝚛 𝚝𝚑𝚎 𝚕𝚊𝚣𝚢 𝚍𝚘𝚐 +⒯⒣⒠ ⒬⒰⒤⒞⒦ ⒝⒭⒪⒲⒩ ⒡⒪⒳ ⒥⒰⒨⒫⒮ ⒪⒱⒠⒭ ⒯⒣⒠ ⒧⒜⒵⒴ ⒟⒪⒢ + +# Script Injection +# +# Strings which attempt to invoke a benign script injection; shows vulnerability to XSS + + +<script>alert('1');</script> + + +"> +'> +> + +< / script >< script >alert(8)< / script > + onfocus=JaVaSCript:alert(9) autofocus +" onfocus=JaVaSCript:alert(10) autofocus +' onfocus=JaVaSCript:alert(11) autofocus +＜script＞alert(12)＜/script＞ +ript>alert(13)ript> +--> +";alert(15);t=" +';alert(16);t=' +JavaSCript:alert(17) +;alert(18); +src=JaVaSCript:prompt(19) +">javascript:alert(25); +javascript:alert(26); +javascript:alert(27); +javascript:alert(28); +javascript:alert(29); +javascript:alert(30); +javascript:alert(31); +'`"><\x3Cscript>javascript:alert(32) +'`"><\x00script>javascript:alert(33) +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +ABC
DEF +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +test +`"'> +`"'> +`"'> +`"'> +`"'> +`"'> +`"'> +`"'> +`"'> +`"'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> +"`'> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +XXX + + + +<a href=http://foo.bar/#x=`y></a><img alt="`><img src=x:x onerror=javascript:alert(203)></a>"> +<!--[if]><script>javascript:alert(204)</script --> +<!--[if<img src=x onerror=javascript:alert(205)//]> --> +<script src="/\%(jscript)s"></script> +<script src="\\%(jscript)s"></script> +<IMG """><SCRIPT>alert("206")</SCRIPT>"> +<IMG SRC=javascript:alert(String.fromCharCode(50,48,55))> +<IMG SRC=# onmouseover="alert('208')"> +<IMG SRC= onmouseover="alert('209')"> +<IMG onmouseover="alert('210')"> +<IMG SRC=javascript:alert('211')> +<IMG SRC=javascript:alert('212')> +<IMG SRC=javascript:alert('213')> +<IMG SRC="jav   ascript:alert('214');"> +<IMG SRC="jav ascript:alert('215');"> +<IMG SRC="jav ascript:alert('216');"> +<IMG SRC="jav ascript:alert('217');"> +perl -e 'print "<IMG SRC=java\0script:alert(\"218\")>";' > out +<IMG SRC="   javascript:alert('219');"> +<SCRIPT/XSS SRC="http://ha.ckers.org/xss.js"></SCRIPT> +<BODY onload!#$%&()*~+-_.,:;?@[/|\]^`=alert("220")> +<SCRIPT/SRC="http://ha.ckers.org/xss.js"></SCRIPT> +<<SCRIPT>alert("221");//<</SCRIPT> +<SCRIPT SRC=http://ha.ckers.org/xss.js?< B > +<SCRIPT SRC=//ha.ckers.org/.j> +<IMG SRC="javascript:alert('222')" +<iframe src=http://ha.ckers.org/scriptlet.html < +\";alert('223');// +<u oncopy=alert()> Copy me</u> +<i onwheel=alert(224)> Scroll over me </i> +<plaintext> +http://a/%%30%30 +</textarea><script>alert(225)</script> + +# SQL Injection +# +# Strings which can cause a SQL injection if inputs are not sanitized + +1;DROP TABLE users +1'; DROP TABLE users-- 1 +' OR 1=1 -- 1 +' OR '1'='1 +'; EXEC sp_MSForEachTable 'DROP TABLE ?'; -- + +% +_ + +# Server Code Injection +# +# Strings which can cause user to run code on server as a privileged user (c.f. https://news.ycombinator.com/item?id=7665153) + +- +-- +--version +--help +$USER +/dev/null; touch /tmp/blns.fail ; echo +`touch /tmp/blns.fail` +$(touch /tmp/blns.fail) +@{[system "touch /tmp/blns.fail"]} + +# Command Injection (Ruby) +# +# Strings which can call system commands within Ruby/Rails applications + +eval("puts 'hello world'") +System("ls -al /") +`ls -al /` +Kernel.exec("ls -al /") +Kernel.exit(1) +%x('ls -al /') + +# XXE Injection (XML) +# +# String which can reveal system files when parsed by a badly configured XML parser + +<?xml version="1.0" encoding="ISO-8859-1"?><!DOCTYPE foo [ <!ELEMENT foo ANY ><!ENTITY xxe SYSTEM "file:///etc/passwd" >]><foo>&xxe;</foo> + +# Unwanted Interpolation +# +# Strings which can be accidentally expanded into different strings if evaluated in the wrong context, e.g. used as a printf format string or via Perl or shell eval. Might expose sensitive data from the program doing the interpolation, or might just represent the wrong string. + +$HOME +$ENV{'HOME'} +%d +%s%s%s%s%s +{0} +%*.*s +%@ +%n +File:/// + +# File Inclusion +# +# Strings which can cause user to pull in files that should not be a part of a web server + +../../../../../../../../../../../etc/passwd%00 +../../../../../../../../../../../etc/hosts + +# Known CVEs and Vulnerabilities +# +# Strings that test for known vulnerabilities + +() { 0; }; touch /tmp/blns.shellshock1.fail; +() { _; } >_[$($())] { touch /tmp/blns.shellshock2.fail; } +<<< %s(un='%s') = %u ++++ATH0 + +# MSDOS/Windows Special Filenames +# +# Strings which are reserved characters in MSDOS/Windows + +CON +PRN +AUX +CLOCK$ +NUL +A: +ZZ: +COM1 +LPT1 +LPT2 +LPT3 +COM2 +COM3 +COM4 + +# IRC specific strings +# +# Strings that may occur on IRC clients that make security products freak out + +DCC SEND STARTKEYLOGGER 0 0 0 + +# Scunthorpe Problem +# +# Innocuous strings which may be blocked by profanity filters (https://en.wikipedia.org/wiki/Scunthorpe_problem) + +Scunthorpe General Hospital +Penistone Community Church +Lightwater Country Park +Jimmy Clitheroe +Horniman Museum +shitake mushrooms +RomansInSussex.co.uk +http://www.cum.qc.ca/ +Craig Cockburn, Software Specialist +Linda Callahan +Dr. Herman I. Libshitz +magna cum laude +Super Bowl XXX +medieval erection of parapets +evaluate +mocha +expression +Arsenal canal +classic +Tyson Gay +Dick Van Dyke +basement + +# Human injection +# +# Strings which may cause human to reinterpret worldview + +If you're reading this, you've been in a coma for almost 20 years now. We're trying a new technique. We don't know where this message will end up in your dream, but we hope it works. Please wake up, we miss you. + +# Terminal escape codes +# +# Strings which punish the fools who use cat/type on this file + +Roses are red, violets are blue. Hope you enjoy terminal hue +But now...for my greatest trick... +The quick brown fox... [Beeeep] + +# iOS Vulnerabilities +# +# Strings which crashed iMessage in various versions of iOS + +Powerلُلُصّبُلُلصّبُررً ॣ ॣh ॣ ॣ冗 +🏳0🌈️ +జ్ఞ‌ా + +# Persian special characters +# +# This is a four characters string which includes Persian special characters (گچپژ) + +گچپژ + +# jinja2 injection +# +# first one is supposed to raise "MemoryError" exception +# second, obviously, prints contents of /etc/passwd + +{% print 'x' * 64 * 1024**3 %} +{{ "".__class__.__mro__[2].__subclasses__()[40]("/etc/passwd").read() }} ADDED testdata/testbox/00000000000100.zettel Index: testdata/testbox/00000000000100.zettel ================================================================== --- /dev/null +++ testdata/testbox/00000000000100.zettel @@ -0,0 +1,8 @@ +id: 00000000000100 +title: Zettelstore Runtime Configuration +role: configuration +syntax: none +expert-mode: true +modified: 20220215171142 +visibility: owner + ADDED testdata/testbox/00000999999999.zettel Index: testdata/testbox/00000999999999.zettel ================================================================== --- /dev/null +++ testdata/testbox/00000999999999.zettel @@ -0,0 +1,11 @@ +id: 00000999999999 +title: Zettelstore Application Directory +role: configuration +syntax: none +app-zid: 00000999999999 +created: 20240703235900 +lang: en +modified: 20240708125724 +nozid-zid: 9999999998 +noappzid: 00000999999999 +visibility: login ADDED testdata/testbox/19700101000000.zettel Index: testdata/testbox/19700101000000.zettel ================================================================== --- /dev/null +++ testdata/testbox/19700101000000.zettel @@ -0,0 +1,12 @@ +id: 19700101000000 +title: Startup Configuration +role: configuration +tags: #invisible +syntax: none +box-uri-1: mem: +box-uri-2: dir:testdata/testbox?readonly +modified: 20210629174022 +owner: 20210629163300 +secret: 1234567890123456 +token-lifetime-api: 1 +visibility: owner ADDED testdata/testbox/20210629163300.zettel Index: testdata/testbox/20210629163300.zettel ================================================================== --- /dev/null +++ testdata/testbox/20210629163300.zettel @@ -0,0 +1,9 @@ +id: 20210629163300 +title: Owena +role: user +tags: #test #user +syntax: none +credential: $2a$10$gcKyVmQ50fwgpOjyiiCm4eba/ILrNXoxTUCopgTEnYTa4yuceHMC6 +modified: 20211220131749 +user-id: owner + ADDED testdata/testbox/20210629165000.zettel Index: testdata/testbox/20210629165000.zettel ================================================================== --- /dev/null +++ testdata/testbox/20210629165000.zettel @@ -0,0 +1,10 @@ +id: 20210629165000 +title: Woody +role: user +tags: #test #user +syntax: none +credential: $2a$10$VmHPyXa0Bm8DE4MJ.pQnbuuQmweWtyGya0L/bFA4nIuCn1EvPQflK +modified: 20211220132007 +user-id: writer +user-role: writer + ADDED testdata/testbox/20210629165024.zettel Index: testdata/testbox/20210629165024.zettel ================================================================== --- /dev/null +++ testdata/testbox/20210629165024.zettel @@ -0,0 +1,10 @@ +id: 20210629165024 +title: Reanna +role: user +tags: #test #user +syntax: none +credential: $2a$10$uC7LV2JdFhasw2HqSWZbSOihvFpwtaEXjXp98yzGfE3FHudq.vg.u +modified: 20211220131906 +user-id: reader +user-role: reader + ADDED testdata/testbox/20210629165050.zettel Index: testdata/testbox/20210629165050.zettel ================================================================== --- /dev/null +++ testdata/testbox/20210629165050.zettel @@ -0,0 +1,10 @@ +id: 20210629165050 +title: Creighton +role: user +tags: #test #user +syntax: none +credential: $2a$10$z85253tqhbHlXPZpt0hJpughLR4WXY8iYJbm1LlBhrKsL1YfkRy2q +modified: 20211220131520 +user-id: creator +user-role: creator + ADDED testdata/testbox/20211019200500.zettel Index: testdata/testbox/20211019200500.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211019200500.zettel @@ -0,0 +1,10 @@ +id: 20211019200500 +title: Collection of all users +role: zettel +syntax: zmk +modified: 20211220132017 + +* [[Owena|20210629163300]] +* [[Woody|20210629165000]] +* [[Reanna|20210629165024]] +* [[Creighton|20210629165050]] ADDED testdata/testbox/20211020121000.zettel Index: testdata/testbox/20211020121000.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020121000.zettel @@ -0,0 +1,8 @@ +id: 20211020121000 +title: ABC +role: zettel +syntax: zmk +modified: 20211020191331 +visibility: owner + +Ab C ADDED testdata/testbox/20211020121100.zettel Index: testdata/testbox/20211020121100.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020121100.zettel @@ -0,0 +1,6 @@ +id: 20211020121100 +title: 10*ABC +role: zettel +syntax: zmk + +{{20211020121000}}{{20211020121000}}{{20211020121000}}{{20211020121000}}{{20211020121000}}{{20211020121000}}{{20211020121000}}{{20211020121000}}{{20211020121000}}{{20211020121000}} ADDED testdata/testbox/20211020121145.zettel Index: testdata/testbox/20211020121145.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020121145.zettel @@ -0,0 +1,6 @@ +id: 20211020121145 +title: 10*10*ABC +role: zettel +syntax: zmk + +{{20211020121100}}{{20211020121100}}{{20211020121100}}{{20211020121100}}{{20211020121100}}{{20211020121100}}{{20211020121100}}{{20211020121100}}{{20211020121100}}{{20211020121100}} ADDED testdata/testbox/20211020121300.zettel Index: testdata/testbox/20211020121300.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020121300.zettel @@ -0,0 +1,6 @@ +id: 20211020121300 +title: 10*10*10*ABC +role: zettel +syntax: zmk + +{{20211020121145}}{{20211020121145}}{{20211020121145}}{{20211020121145}}{{20211020121145}}{{20211020121145}}{{20211020121145}}{{20211020121145}}{{20211020121145}}{{20211020121145}} ADDED testdata/testbox/20211020121400.zettel Index: testdata/testbox/20211020121400.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020121400.zettel @@ -0,0 +1,6 @@ +id: 20211020121400 +title: 10*10*10*10*ABC +role: zettel +syntax: zmk + +{{20211020121300}}{{20211020121300}}{{20211020121300}}{{20211020121300}}{{20211020121300}}{{20211020121300}}{{20211020121300}}{{20211020121300}}{{20211020121300}}{{20211020121300}} ADDED testdata/testbox/20211020182600.zettel Index: testdata/testbox/20211020182600.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020182600.zettel @@ -0,0 +1,7 @@ +id: 20211020182600 +title: Self-recursive transclusion +role: zettel +syntax: zmk +modified: 20211020182712 + +{{20211020182600}} ADDED testdata/testbox/20211020183700.zettel Index: testdata/testbox/20211020183700.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020183700.zettel @@ -0,0 +1,7 @@ +id: 20211020183700 +title: Indirect Recursive 1 +role: zettel +syntax: zmk +modified: 20211020183932 + +{{20211020183800}} ADDED testdata/testbox/20211020183800.zettel Index: testdata/testbox/20211020183800.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020183800.zettel @@ -0,0 +1,6 @@ +id: 20211020183800 +title: Indirect Recursive 2 +role: zettel +syntax: zmk + +{{20211020183700}} ADDED testdata/testbox/20211020184300.zettel Index: testdata/testbox/20211020184300.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020184300.zettel @@ -0,0 +1,5 @@ +id: 20211020184300 +title: Empty Zettel +role: zettel +syntax: zmk + ADDED testdata/testbox/20211020184342.zettel Index: testdata/testbox/20211020184342.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020184342.zettel @@ -0,0 +1,6 @@ +id: 20211020184342 +title: Transclude empty Zettel +role: zettel +syntax: zmk + +{{20211020184300}} ADDED testdata/testbox/20211020185400.zettel Index: testdata/testbox/20211020185400.zettel ================================================================== --- /dev/null +++ testdata/testbox/20211020185400.zettel @@ -0,0 +1,8 @@ +id: 20211020185400 +title: Self embed zettel +role: zettel +syntax: zmk +modified: 20211119150218 + +{{#mark}} +[!mark] Home ADDED testdata/testbox/20230929102100.zettel Index: testdata/testbox/20230929102100.zettel ================================================================== --- /dev/null +++ testdata/testbox/20230929102100.zettel @@ -0,0 +1,7 @@ +id: 20230929102100 +title: #test +role: tag +syntax: zmk +created: 20230929102125 + +Zettel with this tag are testing the Zettelstore. ADDED tests/client/client_test.go Index: tests/client/client_test.go ================================================================== --- /dev/null +++ tests/client/client_test.go @@ -0,0 +1,522 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2021-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore client is licensed under the latest version of the EUPL +// (European Union Public License). Please see file LICENSE.txt for your rights +// and obligations under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2021-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package client provides a client for accessing the Zettelstore via its API. +package client_test + +import ( + "context" + "flag" + "fmt" + "io" + "net/http" + "net/url" + "slices" + "strconv" + "testing" + + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/client" + "t73f.de/r/zsc/domain/id" + "t73f.de/r/zsc/domain/meta" + "zettelstore.de/z/internal/kernel" +) + +func nextZid(zid id.Zid) id.Zid { return zid + 1 } + +func TestNextZid(t *testing.T) { + testCases := []struct { + zid, exp id.Zid + }{ + {1, 2}, + } + for i, tc := range testCases { + if got := nextZid(tc.zid); got != tc.exp { + t.Errorf("%d: zid=%q, exp=%q, got=%q", i, tc.zid, tc.exp, got) + } + + } +} + +func TestListZettel(t *testing.T) { + const ( + ownerZettel = 58 + configRoleZettel = 36 + writerZettel = ownerZettel - 24 + readerZettel = ownerZettel - 24 + creatorZettel = 11 + publicZettel = 6 + ) + + testdata := []struct { + user string + exp int + }{ + {"", publicZettel}, + {"creator", creatorZettel}, + {"reader", readerZettel}, + {"writer", writerZettel}, + {"owner", ownerZettel}, + } + + t.Parallel() + c := getClient() + for i, tc := range testdata { + t.Run(fmt.Sprintf("User %d/%q", i, tc.user), func(tt *testing.T) { + c.SetAuth(tc.user, tc.user) + q, h, l, err := c.QueryZettelData(context.Background(), "") + if err != nil { + tt.Error(err) + return + } + if q != "" { + tt.Errorf("Query should be empty, but is %q", q) + } + if h != "" { + tt.Errorf("Human should be empty, but is %q", q) + } + got := len(l) + if got != tc.exp { + tt.Errorf("List of length %d expected, but got %d\n%v", tc.exp, got, l) + } + }) + } + search := meta.KeyRole + api.SearchOperatorHas + meta.ValueRoleConfiguration + " ORDER id" + q, h, l, err := c.QueryZettelData(context.Background(), search) + if err != nil { + t.Error(err) + return + } + expQ := "role:configuration ORDER id" + if q != expQ { + t.Errorf("Query should be %q, but is %q", expQ, q) + } + expH := "role HAS configuration ORDER id" + if h != expH { + t.Errorf("Human should be %q, but is %q", expH, h) + } + got := len(l) + if got != configRoleZettel { + t.Errorf("List of length %d expected, but got %d\n%v", configRoleZettel, got, l) + } + + pl, err := c.QueryZettel(context.Background(), search) + if err != nil { + t.Error(err) + return + } + compareZettelList(t, pl, l) +} + +func compareZettelList(t *testing.T, pl [][]byte, l []api.ZidMetaRights) { + t.Helper() + if len(pl) != len(l) { + t.Errorf("Different list lenght: Plain=%d, Data=%d", len(pl), len(l)) + } else { + for i, line := range pl { + got, err := id.Parse(string(line[:14])) + if err == nil && got != l[i].ID { + t.Errorf("%d: Data=%q, got=%q", i, l[i].ID, got) + } + } + } +} + +func TestGetZettelData(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + z, err := c.GetZettelData(context.Background(), id.ZidDefaultHome) + if err != nil { + t.Error(err) + return + } + if m := z.Meta; len(m) == 0 { + t.Errorf("Exptected non-empty meta, but got %v", z.Meta) + } + if z.Content == "" || z.Encoding != "" { + t.Errorf("Expect non-empty content, but empty encoding (got %q)", z.Encoding) + } + + mr, err := c.GetMetaData(context.Background(), id.ZidDefaultHome) + if err != nil { + t.Error(err) + return + } + if mr.Rights == api.ZettelCanNone { + t.Error("rights must be greater zero") + } + if len(mr.Meta) != len(z.Meta) { + t.Errorf("Pure meta differs from zettel meta: %s vs %s", mr.Meta, z.Meta) + return + } + for k, v := range z.Meta { + got, ok := mr.Meta[k] + if !ok { + t.Errorf("Pure meta has no key %q", k) + continue + } + if got != v { + t.Errorf("Pure meta has different value for key %q: %q vs %q", k, got, v) + } + } +} + +func TestGetParsedEvaluatedZettel(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + encodings := []api.EncodingEnum{ + api.EncoderHTML, + api.EncoderSz, + api.EncoderText, + } + for _, enc := range encodings { + content, err := c.GetParsedZettel(context.Background(), id.ZidDefaultHome, enc) + if err != nil { + t.Error(err) + continue + } + if len(content) == 0 { + t.Errorf("Empty content for parsed encoding %v", enc) + } + content, err = c.GetEvaluatedZettel(context.Background(), id.ZidDefaultHome, enc) + if err != nil { + t.Error(err) + continue + } + if len(content) == 0 { + t.Errorf("Empty content for evaluated encoding %v", enc) + } + } +} + +func checkListZid(t *testing.T, l []api.ZidMetaRights, pos int, expected id.Zid) { + t.Helper() + if got := l[pos].ID; got != expected { + t.Errorf("Expected result[%d]=%v, but got %v", pos, expected, got) + } +} + +func TestGetZettelOrder(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + _, _, metaSeq, err := c.QueryZettelData(context.Background(), id.ZidTOCNewTemplate.String()+" "+api.ItemsDirective) + if err != nil { + t.Error(err) + return + } + if got := len(metaSeq); got != 4 { + t.Errorf("Expected list of length 4, got %d", got) + return + } + checkListZid(t, metaSeq, 0, id.ZidTemplateNewZettel) + checkListZid(t, metaSeq, 1, id.ZidTemplateNewRole) + checkListZid(t, metaSeq, 2, id.ZidTemplateNewTag) + checkListZid(t, metaSeq, 3, id.ZidTemplateNewUser) +} + +func TestGetZettelContext(t *testing.T) { + const ( + allUserZid = id.Zid(20211019200500) + ownerZid = id.Zid(20210629163300) + writerZid = id.Zid(20210629165000) + readerZid = id.Zid(20210629165024) + creatorZid = id.Zid(20210629165050) + limitAll = 3 + ) + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + rl, err := c.QueryZettel(context.Background(), ownerZid.String()+" CONTEXT LIMIT "+strconv.Itoa(limitAll)) + if err != nil { + t.Error(err) + return + } + checkZidList(t, []id.Zid{ownerZid, allUserZid, writerZid}, rl) + + rl, err = c.QueryZettel(context.Background(), ownerZid.String()+" CONTEXT BACKWARD") + if err != nil { + t.Error(err) + return + } + checkZidList(t, []id.Zid{ownerZid, allUserZid}, rl) +} +func checkZidList(t *testing.T, exp []id.Zid, got [][]byte) { + t.Helper() + if len(exp) != len(got) { + t.Errorf("expected a list fo length %d, but got %d", len(exp), len(got)) + return + } + for i, expZid := range exp { + gotZid, err := id.Parse(string(got[i][:14])) + if err != nil || expZid != gotZid { + t.Errorf("lists differ at pos %d: expected id %v, but got %v", i, expZid, gotZid) + } + } +} + +func TestGetUnlinkedReferences(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + _, _, metaSeq, err := c.QueryZettelData(context.Background(), id.ZidDefaultHome.String()+" "+api.UnlinkedDirective) + if err != nil { + t.Error(err) + return + } + if got := len(metaSeq); got != 1 { + t.Errorf("Expected list of length 1, got %d:\n%v", got, metaSeq) + return + } +} + +func TestGetReferences(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + urls, err := c.GetReferences(context.Background(), id.ZidDefaultHome, "zettel") + if err != nil { + t.Error(err) + return + } + exp := []string{ + "https://zettelstore.de/", + "https://zettelstore.de/home/doc/trunk/www/download.wiki", + "https://zettelstore.de/home/doc/trunk/www/changes.wiki", + "mailto:ds@zettelstore.de", + } + if !slices.Equal(urls, exp) { + t.Errorf("wrong references of home zettel: expected\n%v, but got\n%v", exp, urls) + } +} + +func TestExecuteCommand(t *testing.T) { + c := getClient() + err := c.ExecuteCommand(context.Background(), api.Command("xyz")) + failNoErrorOrNoCode(t, err, http.StatusBadRequest) + err = c.ExecuteCommand(context.Background(), api.CommandAuthenticated) + failNoErrorOrNoCode(t, err, http.StatusUnauthorized) + err = c.ExecuteCommand(context.Background(), api.CommandRefresh) + failNoErrorOrNoCode(t, err, http.StatusForbidden) + + c.SetAuth("owner", "owner") + err = c.ExecuteCommand(context.Background(), api.CommandAuthenticated) + if err != nil { + t.Error(err) + } + err = c.ExecuteCommand(context.Background(), api.CommandRefresh) + if err != nil { + t.Error(err) + } +} +func failNoErrorOrNoCode(t *testing.T, err error, goodCode int) { + if err != nil { + if cErr, ok := err.(*client.Error); ok { + if cErr.StatusCode == goodCode { + return + } + t.Errorf("Expect status code %d, but got client error %v", goodCode, cErr) + } else { + t.Errorf("Expect status code %d, but got non-client error %v", goodCode, err) + } + } else { + t.Errorf("No error returned, but status code %d expected", goodCode) + } +} + +func TestListTags(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + agg, err := c.QueryAggregate(context.Background(), api.ActionSeparator+meta.KeyTags) + if err != nil { + t.Error(err) + return + } + tags := []struct { + key string + size int + }{ + {"#invisible", 1}, + {"#user", 4}, + {"#test", 4}, + } + if len(agg) != len(tags) { + t.Errorf("Expected %d different tags, but got %d (%v)", len(tags), len(agg), agg) + } + for _, tag := range tags { + if zl, ok := agg[tag.key]; !ok { + t.Errorf("No tag %v: %v", tag.key, agg) + } else if len(zl) != tag.size { + t.Errorf("Expected %d zettel with tag %v, but got %v", tag.size, tag.key, zl) + } + } + for i, id := range agg["#user"] { + if id != agg["#test"][i] { + t.Errorf("Tags #user and #test have different content: %v vs %v", agg["#user"], agg["#test"]) + } + } +} + +func TestTagZettel(t *testing.T) { + t.Parallel() + c := getClient() + c.AllowRedirect(true) + c.SetAuth("owner", "owner") + ctx := context.Background() + zid, err := c.TagZettel(ctx, "nosuchtag") + if err != nil { + t.Error(err) + } else if zid != id.Invalid { + t.Errorf("no zid expected, but got %q", zid) + } + zid, err = c.TagZettel(ctx, "#test") + exp := id.Zid(20230929102100) + if err != nil { + t.Error(err) + } else if zid != exp { + t.Errorf("tag zettel for #test should be %q, but got %q", exp, zid) + } +} + +func TestListRoles(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + agg, err := c.QueryAggregate(context.Background(), api.ActionSeparator+meta.KeyRole) + if err != nil { + t.Error(err) + return + } + exp := []string{"configuration", "role", "user", "tag", "zettel"} + if len(agg) != len(exp) { + t.Errorf("Expected %d different roles, but got %d (%v)", len(exp), len(agg), agg) + } + for _, id := range exp { + if _, found := agg[id]; !found { + t.Errorf("Role map expected key %q", id) + } + } +} + +func TestRoleZettel(t *testing.T) { + t.Parallel() + c := getClient() + c.AllowRedirect(true) + c.SetAuth("owner", "owner") + ctx := context.Background() + zid, err := c.RoleZettel(ctx, "nosuchrole") + if err != nil { + t.Error("AAA", err) + } else if zid != id.Invalid { + t.Errorf("no zid expected, but got %q", zid) + } + zid, err = c.RoleZettel(ctx, "zettel") + exp := id.Zid(60010) + if err != nil { + t.Error(err) + } else if zid != exp { + t.Errorf("role zettel for zettel should be %q, but got %q", exp, zid) + } +} + +func TestRedirect(t *testing.T) { + t.Parallel() + c := getClient() + search := "emoji" + api.ActionSeparator + api.RedirectAction + ub := c.NewURLBuilder('z').AppendQuery(search) + respRedirect, err := http.Get(ub.String()) + if err != nil { + t.Error(err) + return + } + defer func() { _ = respRedirect.Body.Close() }() + bodyRedirect, err := io.ReadAll(respRedirect.Body) + if err != nil { + t.Error(err) + return + } + ub.ClearQuery().SetZid(id.ZidEmoji) + respEmoji, err := http.Get(ub.String()) + if err != nil { + t.Error(err) + return + } + defer func() { _ = respEmoji.Body.Close() }() + bodyEmoji, err := io.ReadAll(respEmoji.Body) + if err != nil { + t.Error(err) + return + } + if !slices.Equal(bodyRedirect, bodyEmoji) { + t.Error("Wrong redirect") + t.Error("REDIRECT", respRedirect) + t.Error("EXPECTED", respEmoji) + } +} + +func TestVersion(t *testing.T) { + t.Parallel() + c := getClient() + ver, err := c.GetVersionInfo(context.Background()) + if err != nil { + t.Error(err) + return + } + if ver.Major != -1 || ver.Minor != -1 || ver.Patch != -1 || ver.Info != kernel.CoreDefaultVersion || ver.Hash != "" { + t.Error(ver) + } +} + +func TestApplicationZid(t *testing.T) { + c := getClient() + c.SetAuth("reader", "reader") + zid, err := c.GetApplicationZid(context.Background(), "app") + if err != nil { + t.Error(err) + return + } + if zid != id.ZidAppDirectory { + t.Errorf("c.GetApplicationZid(\"app\") should result in %q, but got: %q", id.ZidAppDirectory, zid) + } + if zid, err = c.GetApplicationZid(context.Background(), "noappzid"); err == nil { + t.Errorf(`c.GetApplicationZid("nozid") should result in error, but got: %v`, zid) + } + if zid, err = c.GetApplicationZid(context.Background(), "nozid"); err == nil { + t.Errorf(`c.GetApplicationZid("nozid") should result in error, but got: %v`, zid) + } +} + +var baseURL string + +func init() { + flag.StringVar(&baseURL, "base-url", "", "Base URL") +} + +func getClient() *client.Client { + u, err := url.Parse(baseURL) + if err != nil { + panic(err) + } + return client.NewClient(u) +} + +// TestMain controls whether client API tests should run or not. +func TestMain(m *testing.M) { + flag.Parse() + if baseURL != "" { + m.Run() + } +} ADDED tests/client/crud_test.go Index: tests/client/crud_test.go ================================================================== --- /dev/null +++ tests/client/crud_test.go @@ -0,0 +1,191 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2021-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore client is licensed under the latest version of the EUPL +// (European Union Public License). Please see file LICENSE.txt for your rights +// and obligations under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2021-present Detlef Stern +//----------------------------------------------------------------------------- + +package client_test + +import ( + "context" + "strings" + "testing" + + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/client" + "t73f.de/r/zsc/domain/id" + "t73f.de/r/zsc/domain/meta" +) + +// --------------------------------------------------------------------------- +// Tests that change the Zettelstore must nor run parallel to other tests. + +func TestCreateGetDeleteZettel(t *testing.T) { + // Is not to be allowed to run in parallel with other tests. + zettel := `title: A Test + +Example content.` + c := getClient() + c.SetAuth("owner", "owner") + zid, err := c.CreateZettel(context.Background(), []byte(zettel)) + if err != nil { + t.Error("Cannot create zettel:", err) + return + } + if !zid.IsValid() { + t.Error("Invalid zettel ID", zid) + return + } + data, err := c.GetZettel(context.Background(), zid, api.PartZettel) + if err != nil { + t.Error("Cannot read zettel", zid, err) + return + } + exp := `title: A Test + +Example content.` + if string(data) != exp { + t.Errorf("Expected zettel data: %q, but got %q", exp, data) + } + + doDelete(t, c, zid) +} + +func TestCreateGetDeleteZettelDataCreator(t *testing.T) { + // Is not to be allowed to run in parallel with other tests. + c := getClient() + c.SetAuth("creator", "creator") + zid, err := c.CreateZettelData(context.Background(), api.ZettelData{ + Meta: nil, + Encoding: "", + Content: "Example", + }) + if err != nil { + t.Error("Cannot create zettel:", err) + return + } + if !zid.IsValid() { + t.Error("Invalid zettel ID", zid) + return + } + + c.SetAuth("owner", "owner") + doDelete(t, c, zid) +} + +func TestCreateGetDeleteZettelData(t *testing.T) { + // Is not to be allowed to run in parallel with other tests. + c := getClient() + c.SetAuth("owner", "owner") + wrongModified := "19691231115959" + zid, err := c.CreateZettelData(context.Background(), api.ZettelData{ + Meta: api.ZettelMeta{ + meta.KeyTitle: "A\nTitle", // \n must be converted into a space + meta.KeyModified: wrongModified, + }, + }) + if err != nil { + t.Error("Cannot create zettel:", err) + return + } + z, err := c.GetZettelData(context.Background(), zid) + if err != nil { + t.Error("Cannot get zettel:", zid, err) + } else { + exp := "A Title" + if got := z.Meta[meta.KeyTitle]; got != exp { + t.Errorf("Expected title %q, but got %q", exp, got) + } + if got := z.Meta[meta.KeyModified]; got != "" { + t.Errorf("Create allowed to set the modified key: %q", got) + } + } + doDelete(t, c, zid) +} + +func TestUpdateZettel(t *testing.T) { + c := getClient() + c.SetAuth("owner", "owner") + z, err := c.GetZettel(context.Background(), id.ZidDefaultHome, api.PartZettel) + if err != nil { + t.Error(err) + return + } + if !strings.HasPrefix(string(z), "title: Home\n") { + t.Error("Got unexpected zettel", z) + return + } + newZettel := `title: Empty Home +role: zettel +syntax: zmk + +Empty` + err = c.UpdateZettel(context.Background(), id.ZidDefaultHome, []byte(newZettel)) + if err != nil { + t.Error(err) + return + } + zt, err := c.GetZettel(context.Background(), id.ZidDefaultHome, api.PartZettel) + if err != nil { + t.Error(err) + return + } + if string(zt) != newZettel { + t.Errorf("Expected zettel %q, got %q", newZettel, zt) + } + // Must delete to clean up for next tests + doDelete(t, c, id.ZidDefaultHome) +} + +func TestUpdateZettelData(t *testing.T) { + c := getClient() + c.SetAuth("writer", "writer") + z, err := c.GetZettelData(context.Background(), id.ZidDefaultHome) + if err != nil { + t.Error(err) + return + } + if got := z.Meta[meta.KeyTitle]; got != "Home" { + t.Errorf("Title of zettel is not \"Home\", but %q", got) + return + } + newTitle := "New Home" + z.Meta[meta.KeyTitle] = newTitle + wrongModified := "19691231235959" + z.Meta[meta.KeyModified] = wrongModified + err = c.UpdateZettelData(context.Background(), id.ZidDefaultHome, z) + if err != nil { + t.Error(err) + return + } + zt, err := c.GetZettelData(context.Background(), id.ZidDefaultHome) + if err != nil { + t.Error(err) + return + } + if got := zt.Meta[meta.KeyTitle]; got != newTitle { + t.Errorf("Title of zettel is not %q, but %q", newTitle, got) + } + if got := zt.Meta[meta.KeyModified]; got == wrongModified { + t.Errorf("Update did not change the modified key: %q", got) + } + + // Must delete to clean up for next tests + c.SetAuth("owner", "owner") + doDelete(t, c, id.ZidDefaultHome) +} + +func doDelete(t *testing.T, c *client.Client, zid id.Zid) { + err := c.DeleteZettel(context.Background(), zid) + if err != nil { + t.Helper() + t.Error("Cannot delete", zid, ":", err) + } +} ADDED tests/client/embed_test.go Index: tests/client/embed_test.go ================================================================== --- /dev/null +++ tests/client/embed_test.go @@ -0,0 +1,184 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2021-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2021-present Detlef Stern +//----------------------------------------------------------------------------- + +package client_test + +import ( + "context" + "strings" + "testing" + + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/domain/id" +) + +const ( + abcZid = id.Zid(20211020121000) + abc10Zid = id.Zid(20211020121100) +) + +func TestZettelTransclusion(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + + const abc10000Zid = id.Zid(20211020121400) + contentMap := map[id.Zid]int{ + abcZid: 1, + abc10Zid: 10, + id.Zid(20211020121145): 100, + id.Zid(20211020121300): 1000, + } + content, err := c.GetZettel(context.Background(), abcZid, api.PartContent) + if err != nil { + t.Error(err) + return + } + baseContent := string(content) + for zid, siz := range contentMap { + content, err = c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML) + if err != nil { + t.Error(err) + continue + } + sContent := string(content) + prefix := "<p>" + if !strings.HasPrefix(sContent, prefix) { + t.Errorf("Content of zettel %q does not start with %q: %q", zid, prefix, stringHead(sContent)) + continue + } + suffix := "</p>" + if !strings.HasSuffix(sContent, suffix) { + t.Errorf("Content of zettel %q does not end with %q: %q", zid, suffix, stringTail(sContent)) + continue + } + got := sContent[len(prefix) : len(content)-len(suffix)] + if expect := strings.Repeat(baseContent, siz); expect != got { + t.Errorf("Unexpected content for zettel %q\nExpect: %q\nGot: %q", zid, expect, got) + } + } + + content, err = c.GetEvaluatedZettel(context.Background(), abc10000Zid, api.EncoderHTML) + if err != nil { + t.Error(err) + return + } + checkContentContains(t, abc10000Zid, string(content), "Too many transclusions") +} + +func TestZettelTransclusionNoPrivilegeEscalation(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("reader", "reader") + + zettelData, err := c.GetZettelData(context.Background(), id.ZidEmoji) + if err != nil { + t.Error(err) + return + } + expectedEnc := "base64" + if got := zettelData.Encoding; expectedEnc != got { + t.Errorf("Zettel %q: encoding %q expected, but got %q", abcZid, expectedEnc, got) + } + + content, err := c.GetEvaluatedZettel(context.Background(), abc10Zid, api.EncoderHTML) + if err != nil { + t.Error(err) + return + } + if exp, got := "", string(content); exp != got { + t.Errorf("Zettel %q must contain %q, but got %q", abc10Zid, exp, got) + } +} + +func stringHead(s string) string { + const maxLen = 40 + if len(s) <= maxLen { + return s + } + return s[:maxLen-3] + "..." +} + +func stringTail(s string) string { + const maxLen = 40 + if len(s) <= maxLen { + return s + } + return "..." + s[len(s)-maxLen-3:] +} + +func TestRecursiveTransclusion(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + + const ( + selfRecursiveZid = id.Zid(20211020182600) + indirectRecursive1Zid = id.Zid(20211020183700) + indirectRecursive2Zid = id.Zid(20211020183800) + ) + recursiveZettel := map[id.Zid]id.Zid{ + selfRecursiveZid: selfRecursiveZid, + indirectRecursive1Zid: indirectRecursive2Zid, + indirectRecursive2Zid: indirectRecursive1Zid, + } + for zid, errZid := range recursiveZettel { + content, err := c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML) + if err != nil { + t.Error(err) + continue + } + sContent := string(content) + checkContentContains(t, zid, sContent, "Recursive transclusion") + checkContentContains(t, zid, sContent, errZid.String()) + } +} +func TestNothingToTransclude(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + + const ( + transZid = id.Zid(20211020184342) + emptyZid = id.Zid(20211020184300) + ) + content, err := c.GetEvaluatedZettel(context.Background(), transZid, api.EncoderHTML) + if err != nil { + t.Error(err) + return + } + sContent := string(content) + checkContentContains(t, transZid, sContent, "<!-- Nothing to transclude") + checkContentContains(t, transZid, sContent, emptyZid.String()) +} + +func TestSelfEmbedRef(t *testing.T) { + t.Parallel() + c := getClient() + c.SetAuth("owner", "owner") + + const selfEmbedZid = id.Zid(20211020185400) + content, err := c.GetEvaluatedZettel(context.Background(), selfEmbedZid, api.EncoderHTML) + if err != nil { + t.Error(err) + return + } + checkContentContains(t, selfEmbedZid, string(content), "Self embed reference") +} + +func checkContentContains(t *testing.T, zid id.Zid, content, expected string) { + if !strings.Contains(content, expected) { + t.Helper() + t.Errorf("Zettel %q should contain %q, but does not: %q", zid, expected, content) + } +} Index: tests/markdown_test.go ================================================================== --- tests/markdown_test.go +++ tests/markdown_test.go @@ -1,37 +1,36 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- -// Package tests provides some higher-level tests. package tests import ( + "bytes" "encoding/json" "fmt" "os" - "regexp" "strings" "testing" - "zettelstore.de/z/ast" - "zettelstore.de/z/encoder" - _ "zettelstore.de/z/encoder/htmlenc" - _ "zettelstore.de/z/encoder/jsonenc" - _ "zettelstore.de/z/encoder/nativeenc" - _ "zettelstore.de/z/encoder/textenc" - _ "zettelstore.de/z/encoder/zmkenc" - "zettelstore.de/z/input" - "zettelstore.de/z/parser" - _ "zettelstore.de/z/parser/markdown" - _ "zettelstore.de/z/parser/zettelmark" + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/domain/meta" + "t73f.de/r/zsx/input" + + "zettelstore.de/z/internal/ast" + "zettelstore.de/z/internal/config" + "zettelstore.de/z/internal/encoder" + "zettelstore.de/z/internal/parser" ) type markdownTestCase struct { Markdown string `json:"markdown"` HTML string `json:"html"` @@ -39,137 +38,103 @@ StartLine int `json:"start_line"` EndLine int `json:"end_line"` Section string `json:"section"` } -// exceptions lists all CommonMark tests that should not be tested for identical HTML output -var exceptions = []string{ - " - foo\n - bar\n\t - baz\n", // 9 - "<script type=\"text/javascript\">\n// JavaScript example\n\ndocument.getElementById(\"demo\").innerHTML = \"Hello JavaScript!\";\n</script>\nokay\n", // 140 - "<script>\nfoo\n</script>1. *bar*\n", // 147 - "- foo\n - bar\n - baz\n - boo\n", // 264 - "10) foo\n - bar\n", // 266 - "- # Foo\n- Bar\n ---\n baz\n", // 270 - "- foo\n\n- bar\n\n\n- baz\n", // 276 - "- foo\n - bar\n - baz\n\n\n bim\n", // 277 - "1. a\n\n 2. b\n\n 3. c\n", // 281 - "1. a\n\n 2. b\n\n 3. c\n", // 283 - "- a\n- b\n\n- c\n", // 284 - "* a\n*\n\n* c\n", // 285 - "- a\n- b\n\n [ref]: /url\n- d\n", // 287 - "- a\n - b\n\n c\n- d\n", // 289 - "* a\n > b\n >\n* c\n", // 290 - "- a\n > b\n ```\n c\n ```\n- d\n", // 291 - "- a\n - b\n", // 293 - "<http://example.com?find=\\*>\n", // 306 - "<http://foo.bar.`baz>`\n", // 346 - "[foo<http://example.com/?search=](uri)>\n", // 522 - "[foo<http://example.com/?search=][ref]>\n\n[ref]: /uri\n", // 534 - "<http://foo.bar.baz/test?q=hello&id=22&boolean>\n", // 591 -} - -var reHeadingID = regexp.MustCompile(` id="[^"]*"`) - func TestEncoderAvailability(t *testing.T) { + t.Parallel() encoderMissing := false - for _, format := range formats { - enc := encoder.Create(format, nil) + for _, enc := range encodings { + enc := encoder.Create(enc, &encoder.CreateParameter{Lang: meta.ValueLangEN}) if enc == nil { - t.Errorf("No encoder for %q found", format) + t.Errorf("No encoder for %q found", enc) encoderMissing = true } } if encoderMissing { panic("At least one encoder is missing. See test log") } } func TestMarkdownSpec(t *testing.T) { + t.Parallel() content, err := os.ReadFile("../testdata/markdown/spec.json") if err != nil { panic(err) } var testcases []markdownTestCase if err = json.Unmarshal(content, &testcases); err != nil { panic(err) } - excMap := make(map[string]bool, len(exceptions)) - for _, exc := range exceptions { - excMap[exc] = true - } - for _, tc := range testcases { - ast := parser.ParseBlocks(input.NewInput(tc.Markdown), nil, "markdown") - testAllEncodings(t, tc, ast) - if _, found := excMap[tc.Markdown]; !found { - testHTMLEncoding(t, tc, ast) - } - testZmkEncoding(t, tc, ast) - } -} - -func testAllEncodings(t *testing.T, tc markdownTestCase, ast ast.BlockSlice) { - var sb strings.Builder - testID := tc.Example*100 + 1 - for _, format := range formats { - t.Run(fmt.Sprintf("Encode %v %v", format, testID), func(st *testing.T) { - encoder.Create(format, nil).WriteBlocks(&sb, ast) - sb.Reset() - }) - } -} - -func testHTMLEncoding(t *testing.T, tc markdownTestCase, ast ast.BlockSlice) { - htmlEncoder := encoder.Create("html", &encoder.Environment{Xhtml: true}) - var sb strings.Builder - testID := tc.Example*100 + 1 - t.Run(fmt.Sprintf("Encode md html %v", testID), func(st *testing.T) { - htmlEncoder.WriteBlocks(&sb, ast) - gotHTML := sb.String() - sb.Reset() - - mdHTML := tc.HTML - mdHTML = strings.ReplaceAll(mdHTML, "\"MAILTO:", "\"mailto:") - gotHTML = strings.ReplaceAll(gotHTML, " class=\"zs-external\"", "") - gotHTML = strings.ReplaceAll(gotHTML, "%2A", "*") // url.QueryEscape - if strings.Count(gotHTML, "<h") > 0 { - gotHTML = reHeadingID.ReplaceAllString(gotHTML, "") - } - if gotHTML != mdHTML { - mdHTML = strings.ReplaceAll(mdHTML, "<li>\n", "<li>") - if gotHTML != mdHTML { - st.Errorf("\nCMD: %q\nExp: %q\nGot: %q", tc.Markdown, mdHTML, gotHTML) - } - } - }) -} - -func testZmkEncoding(t *testing.T, tc markdownTestCase, ast ast.BlockSlice) { - zmkEncoder := encoder.Create("zmk", nil) - var sb strings.Builder - testID := tc.Example*100 + 1 - t.Run(fmt.Sprintf("Encode zmk %14d", testID), func(st *testing.T) { - zmkEncoder.WriteBlocks(&sb, ast) - gotFirst := sb.String() - sb.Reset() - - testID = tc.Example*100 + 2 - secondAst := parser.ParseBlocks(input.NewInput(gotFirst), nil, "zmk") - zmkEncoder.WriteBlocks(&sb, secondAst) - gotSecond := sb.String() - sb.Reset() + + for _, tc := range testcases { + ast := createMDBlockSlice(tc.Markdown, config.NoHTML) + testAllEncodings(t, tc, &ast) + testZmkEncoding(t, tc, &ast) + } +} + +func createMDBlockSlice(markdown string, hi config.HTMLInsecurity) ast.BlockSlice { + return parser.Parse(input.NewInput([]byte(markdown)), nil, meta.ValueSyntaxMarkdown, hi) +} + +func testAllEncodings(t *testing.T, tc markdownTestCase, ast *ast.BlockSlice) { + var sb strings.Builder + testID := tc.Example*100 + 1 + for _, enc := range encodings { + t.Run(fmt.Sprintf("Encode %v %v", enc, testID), func(*testing.T) { + _, _ = encoder.Create(enc, &encoder.CreateParameter{Lang: meta.ValueLangEN}).WriteBlocks(&sb, ast) + sb.Reset() + }) + } +} + +func testZmkEncoding(t *testing.T, tc markdownTestCase, ast *ast.BlockSlice) { + zmkEncoder := encoder.Create(api.EncoderZmk, nil) + var buf bytes.Buffer + testID := tc.Example*100 + 1 + t.Run(fmt.Sprintf("Encode zmk %14d", testID), func(st *testing.T) { + buf.Reset() + _, _ = zmkEncoder.WriteBlocks(&buf, ast) + // gotFirst := buf.String() + + testID = tc.Example*100 + 2 + secondAst := parser.Parse(input.NewInput(buf.Bytes()), nil, meta.ValueSyntaxZmk, config.NoHTML) + buf.Reset() + _, _ = zmkEncoder.WriteBlocks(&buf, &secondAst) + gotSecond := buf.String() // if gotFirst != gotSecond { // st.Errorf("\nCMD: %q\n1st: %q\n2nd: %q", tc.Markdown, gotFirst, gotSecond) // } testID = tc.Example*100 + 3 - thirdAst := parser.ParseBlocks(input.NewInput(gotFirst), nil, "zmk") - zmkEncoder.WriteBlocks(&sb, thirdAst) - gotThird := sb.String() - sb.Reset() + thirdAst := parser.Parse(input.NewInput(buf.Bytes()), nil, meta.ValueSyntaxZmk, config.NoHTML) + buf.Reset() + _, _ = zmkEncoder.WriteBlocks(&buf, &thirdAst) + gotThird := buf.String() if gotSecond != gotThird { st.Errorf("\n1st: %q\n2nd: %q", gotSecond, gotThird) } }) +} +func TestAdditionalMarkdown(t *testing.T) { + testcases := []struct { + md string + exp string + }{ + {`abc<br>def`, "abc``<br>``{=\"html\"}def"}, + } + zmkEncoder := encoder.Create(api.EncoderZmk, nil) + var sb strings.Builder + for i, tc := range testcases { + ast := createMDBlockSlice(tc.md, config.MarkdownHTML) + sb.Reset() + _, _ = zmkEncoder.WriteBlocks(&sb, &ast) + got := sb.String() + if got != tc.exp { + t.Errorf("%d: %q -> %q, but got %q", i, tc.md, tc.exp, got) + } + } } ADDED tests/naughtystrings_test.go Index: tests/naughtystrings_test.go ================================================================== --- /dev/null +++ tests/naughtystrings_test.go @@ -0,0 +1,97 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2020-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern +//----------------------------------------------------------------------------- + +package tests + +import ( + "bufio" + "io" + "os" + "path/filepath" + "testing" + + "t73f.de/r/zsc/domain/meta" + "t73f.de/r/zsx/input" + + "zettelstore.de/z/internal/config" + "zettelstore.de/z/internal/encoder" + "zettelstore.de/z/internal/parser" + + _ "zettelstore.de/z/cmd" +) + +// Test all parser / encoder with a list of "naughty strings", i.e. unusual strings +// that often crash software. + +func getNaughtyStrings() (result []string, err error) { + fpath := filepath.Join("..", "testdata", "naughty", "blns.txt") + file, err := os.Open(fpath) + if err != nil { + return nil, err + } + defer func() { _ = file.Close() }() + scanner := bufio.NewScanner(file) + for scanner.Scan() { + if text := scanner.Text(); text != "" && text[0] != '#' { + result = append(result, text) + } + } + return result, scanner.Err() +} + +func getAllParser() (result []*parser.Info) { + for _, pname := range parser.GetSyntaxes() { + pinfo := parser.Get(pname) + if pname == pinfo.Name { + result = append(result, pinfo) + } + } + return result +} + +func getAllEncoder() (result []encoder.Encoder) { + for _, enc := range encoder.GetEncodings() { + e := encoder.Create(enc, &encoder.CreateParameter{Lang: meta.ValueLangEN}) + result = append(result, e) + } + return result +} + +func TestNaughtyStringParser(t *testing.T) { + blns, err := getNaughtyStrings() + if err != nil { + t.Fatal(err) + } + if len(blns) == 0 { + t.Fatal("no naughty strings found") + } + pinfos := getAllParser() + if len(pinfos) == 0 { + t.Fatal("no parser found") + } + encs := getAllEncoder() + if len(encs) == 0 { + t.Fatal("no encoder found") + } + for _, s := range blns { + for _, pinfo := range pinfos { + bs := parser.Parse(input.NewInput([]byte(s)), &meta.Meta{}, pinfo.Name, config.NoHTML) + for _, enc := range encs { + _, err = enc.WriteBlocks(io.Discard, &bs) + if err != nil { + t.Error(err) + } + } + } + } +} Index: tests/regression_test.go ================================================================== --- tests/regression_test.go +++ tests/regression_test.go @@ -1,13 +1,16 @@ //----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern +// Copyright (c) 2020-present Detlef Stern // -// This file is part of zettelstore. +// This file is part of Zettelstore. // // Zettelstore is licensed under the latest version of the EUPL (European Union // Public License). Please see file LICENSE.txt for your rights and obligations // under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2020-present Detlef Stern //----------------------------------------------------------------------------- // Package tests provides some higher-level tests. package tests @@ -19,64 +22,68 @@ "os" "path/filepath" "strings" "testing" - "zettelstore.de/z/ast" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/kernel" - "zettelstore.de/z/parser" - "zettelstore.de/z/place" - "zettelstore.de/z/place/manager" - - _ "zettelstore.de/z/encoder/htmlenc" - _ "zettelstore.de/z/encoder/jsonenc" - _ "zettelstore.de/z/encoder/nativeenc" - _ "zettelstore.de/z/encoder/textenc" - _ "zettelstore.de/z/encoder/zmkenc" - _ "zettelstore.de/z/parser/blob" - _ "zettelstore.de/z/parser/zettelmark" - _ "zettelstore.de/z/place/dirplace" -) - -var formats = []string{"html", "djson", "native", "text"} - -func getFilePlaces(wd string, kind string) (root string, places []place.ManagedPlace) { + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/domain/meta" + + "zettelstore.de/z/internal/ast" + "zettelstore.de/z/internal/box" + "zettelstore.de/z/internal/box/manager" + "zettelstore.de/z/internal/config" + "zettelstore.de/z/internal/encoder" + "zettelstore.de/z/internal/kernel" + "zettelstore.de/z/internal/parser" + "zettelstore.de/z/internal/query" + + _ "zettelstore.de/z/internal/box/dirbox" +) + +var encodings = []api.EncodingEnum{ + api.EncoderHTML, + api.EncoderSz, + api.EncoderText, +} + +func getFileBoxes(wd, kind string) (root string, boxes []box.ManagedBox) { root = filepath.Clean(filepath.Join(wd, "..", "testdata", kind)) entries, err := os.ReadDir(root) if err != nil { panic(err) } - cdata := manager.ConnectData{Config: testConfig, Enricher: &noEnrich{}, Notify: nil} + cdata := manager.ConnectData{ + Number: 0, + Config: testConfig, + Enricher: &noEnrich{}, + Notify: nil, + } for _, entry := range entries { if entry.IsDir() { - u, err := url.Parse("dir://" + filepath.Join(root, entry.Name()) + "?type=" + kernel.PlaceDirTypeSimple) - if err != nil { - panic(err) - } - place, err := manager.Connect(u, &noAuth{}, &cdata) - if err != nil { - panic(err) - } - places = append(places, place) + u, err2 := url.Parse("dir://" + filepath.Join(root, entry.Name()) + "?type=" + kernel.BoxDirTypeSimple) + if err2 != nil { + panic(err2) + } + box, err2 := manager.Connect(u, &noAuth{}, &cdata) + if err2 != nil { + panic(err2) + } + boxes = append(boxes, box) } } - return root, places + return root, boxes } type noEnrich struct{} -func (nf *noEnrich) Enrich(ctx context.Context, m *meta.Meta) {} -func (nf *noEnrich) Remove(ctx context.Context, m *meta.Meta) {} +func (*noEnrich) Enrich(context.Context, *meta.Meta, int) {} +func (*noEnrich) Remove(context.Context, *meta.Meta) {} type noAuth struct{} -func (na *noAuth) IsReadonly() bool { return false } +func (*noAuth) IsReadonly() bool { return false } func trimLastEOL(s string) string { if lastPos := len(s) - 1; lastPos >= 0 && s[lastPos] == '\n' { return s[:lastPos] } @@ -86,16 +93,19 @@ func resultFile(file string) (data string, err error) { f, err := os.Open(file) if err != nil { return "", err } - defer f.Close() src, err := io.ReadAll(f) + err2 := f.Close() + if err == nil { + err = err2 + } return string(src), err } -func checkFileContent(t *testing.T, filename string, gotContent string) { +func checkFileContent(t *testing.T, filename, gotContent string) { t.Helper() wantContent, err := resultFile(filename) if err != nil { t.Error(err) return @@ -105,156 +115,82 @@ if gotContent != wantContent { t.Errorf("\nWant: %q\nGot: %q", wantContent, gotContent) } } -func checkBlocksFile(t *testing.T, resultName string, zn *ast.ZettelNode, format string) { - t.Helper() - var env encoder.Environment - if enc := encoder.Create(format, &env); enc != nil { - var sb strings.Builder - enc.WriteBlocks(&sb, zn.Ast) - checkFileContent(t, resultName, sb.String()) - return - } - panic(fmt.Sprintf("Unknown writer format %q", format)) -} - -func checkZmkEncoder(t *testing.T, zn *ast.ZettelNode) { - zmkEncoder := encoder.Create("zmk", nil) - var sb strings.Builder - zmkEncoder.WriteBlocks(&sb, zn.Ast) - gotFirst := sb.String() - sb.Reset() - - newZettel := parser.ParseZettel(domain.Zettel{ - Meta: zn.Meta, Content: domain.NewContent("\n" + gotFirst)}, "", testConfig) - zmkEncoder.WriteBlocks(&sb, newZettel.Ast) - gotSecond := sb.String() - sb.Reset() - - if gotFirst != gotSecond { - t.Errorf("\n1st: %q\n2nd: %q", gotFirst, gotSecond) - } -} - -func getPlaceName(p place.ManagedPlace, root string) string { +func getBoxName(p box.ManagedBox, root string) string { u, err := url.Parse(p.Location()) if err != nil { panic("Unable to parse URL '" + p.Location() + "': " + err.Error()) } return u.Path[len(root):] } -func match(*meta.Meta) bool { return true } +func checkMetaFile(t *testing.T, resultName string, zn *ast.ZettelNode, enc api.EncodingEnum) { + t.Helper() + + if enc := encoder.Create(enc, &encoder.CreateParameter{Lang: meta.ValueLangEN}); enc != nil { + var sf strings.Builder + _, _ = enc.WriteMeta(&sf, zn.Meta) + checkFileContent(t, resultName, sf.String()) + return + } + panic(fmt.Sprintf("Unknown writer encoding %q", enc)) +} -func checkContentPlace(t *testing.T, p place.ManagedPlace, wd, placeName string) { - ss := p.(place.StartStopper) +func checkMetaBox(t *testing.T, p box.ManagedBox, wd, boxName string) { + ss := p.(box.StartStopper) if err := ss.Start(context.Background()); err != nil { panic(err) } - metaList, err := p.SelectMeta(context.Background(), match) - if err != nil { + metaList := []*meta.Meta{} + if err := p.ApplyMeta(context.Background(), + func(m *meta.Meta) { metaList = append(metaList, m) }, + query.AlwaysIncluded); err != nil { panic(err) } for _, meta := range metaList { zettel, err := p.GetZettel(context.Background(), meta.Zid) if err != nil { panic(err) } - z := parser.ParseZettel(zettel, "", testConfig) - for _, format := range formats { - t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, format), func(st *testing.T) { - resultName := filepath.Join(wd, "result", "content", placeName, z.Zid.String()+"."+format) - checkBlocksFile(st, resultName, z, format) - }) - } - t.Run(fmt.Sprintf("%s::%d", p.Location(), meta.Zid), func(st *testing.T) { - checkZmkEncoder(st, z) - }) - } - if err := ss.Stop(context.Background()); err != nil { - panic(err) - } -} - -func TestContentRegression(t *testing.T) { - wd, err := os.Getwd() - if err != nil { - panic(err) - } - root, places := getFilePlaces(wd, "content") - for _, p := range places { - checkContentPlace(t, p, wd, getPlaceName(p, root)) - } -} - -func checkMetaFile(t *testing.T, resultName string, zn *ast.ZettelNode, format string) { - t.Helper() - - if enc := encoder.Create(format, nil); enc != nil { - var sb strings.Builder - enc.WriteMeta(&sb, zn.Meta) - checkFileContent(t, resultName, sb.String()) - return - } - panic(fmt.Sprintf("Unknown writer format %q", format)) -} - -func checkMetaPlace(t *testing.T, p place.ManagedPlace, wd, placeName string) { - ss := p.(place.StartStopper) - if err := ss.Start(context.Background()); err != nil { - panic(err) - } - metaList, err := p.SelectMeta(context.Background(), match) - if err != nil { - panic(err) - } - for _, meta := range metaList { - zettel, err := p.GetZettel(context.Background(), meta.Zid) - if err != nil { - panic(err) - } - z := parser.ParseZettel(zettel, "", testConfig) - for _, format := range formats { - t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, format), func(st *testing.T) { - resultName := filepath.Join(wd, "result", "meta", placeName, z.Zid.String()+"."+format) - checkMetaFile(st, resultName, z, format) - }) - } - } - if err := ss.Stop(context.Background()); err != nil { - panic(err) - } -} - -type myConfig struct{} - -func (cfg *myConfig) AddDefaultValues(m *meta.Meta) *meta.Meta { return m } -func (cfg *myConfig) GetDefaultTitle() string { return "" } -func (cfg *myConfig) GetDefaultRole() string { return meta.ValueRoleZettel } -func (cfg *myConfig) GetDefaultSyntax() string { return meta.ValueSyntaxZmk } -func (cfg *myConfig) GetDefaultLang() string { return "" } -func (cfg *myConfig) GetDefaultVisibility() meta.Visibility { return meta.VisibilityPublic } -func (cfg *myConfig) GetFooterHTML() string { return "" } -func (cfg *myConfig) GetHomeZettel() id.Zid { return id.Invalid } -func (cfg *myConfig) GetListPageSize() int { return 0 } -func (cfg *myConfig) GetMarkerExternal() string { return "" } -func (cfg *myConfig) GetSiteName() string { return "" } -func (cfg *myConfig) GetYAMLHeader() bool { return false } -func (cfg *myConfig) GetZettelFileSyntax() []string { return nil } - -func (cfg *myConfig) GetExpertMode() bool { return false } -func (cfg *myConfig) GetVisibility(*meta.Meta) meta.Visibility { return cfg.GetDefaultVisibility() } - -var testConfig = &myConfig{} - -func TestMetaRegression(t *testing.T) { - wd, err := os.Getwd() - if err != nil { - panic(err) - } - root, places := getFilePlaces(wd, "meta") - for _, p := range places { - checkMetaPlace(t, p, wd, getPlaceName(p, root)) + z := parser.ParseZettel(context.Background(), zettel, "", testConfig) + for _, enc := range encodings { + t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, enc), func(st *testing.T) { + resultName := filepath.Join(wd, "result", "meta", boxName, z.Zid.String()+"."+enc.String()) + checkMetaFile(st, resultName, z, enc) + }) + } + } + ss.Stop(context.Background()) +} + +type myConfig struct{} + +func (*myConfig) Get(context.Context, *meta.Meta, string) string { return "" } +func (*myConfig) AddDefaultValues(_ context.Context, m *meta.Meta) *meta.Meta { + return m +} +func (*myConfig) GetHTMLInsecurity() config.HTMLInsecurity { return config.NoHTML } +func (*myConfig) GetListPageSize() int { return 0 } +func (*myConfig) GetSiteName() string { return "" } +func (*myConfig) GetYAMLHeader() bool { return false } +func (*myConfig) GetZettelFileSyntax() []meta.Value { return nil } + +func (*myConfig) GetSimpleMode() bool { return false } +func (*myConfig) GetExpertMode() bool { return false } +func (*myConfig) GetVisibility(*meta.Meta) meta.Visibility { return meta.VisibilityPublic } +func (*myConfig) GetMaxTransclusions() int { return 1024 } + +var testConfig = &myConfig{} + +func TestMetaRegression(t *testing.T) { + t.Parallel() + wd, err := os.Getwd() + if err != nil { + panic(err) + } + root, boxes := getFileBoxes(wd, "meta") + for _, p := range boxes { + checkMetaBox(t, p, wd, getBoxName(p, root)) } } DELETED tests/result/content/blockcomment/20200215204700.djson Index: tests/result/content/blockcomment/20200215204700.djson ================================================================== --- tests/result/content/blockcomment/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"CommentBlock","l":["No render"]},{"t":"CommentBlock","a":{"-":""},"l":["Render"]}] DELETED tests/result/content/blockcomment/20200215204700.html Index: tests/result/content/blockcomment/20200215204700.html ================================================================== --- tests/result/content/blockcomment/20200215204700.html +++ /dev/null @@ -1,3 +0,0 @@ -<!-- -Render ---> DELETED tests/result/content/blockcomment/20200215204700.native Index: tests/result/content/blockcomment/20200215204700.native ================================================================== --- tests/result/content/blockcomment/20200215204700.native +++ /dev/null @@ -1,2 +0,0 @@ -[CommentBlock "No render"], -[CommentBlock ("",[-]) "Render"] DELETED tests/result/content/blockcomment/20200215204700.text Index: tests/result/content/blockcomment/20200215204700.text ================================================================== --- tests/result/content/blockcomment/20200215204700.text +++ /dev/null DELETED tests/result/content/cite/20200215204700.djson Index: tests/result/content/cite/20200215204700.djson ================================================================== --- tests/result/content/cite/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Cite","a":{"-":""},"s":"Stern18"}]}] DELETED tests/result/content/cite/20200215204700.html Index: tests/result/content/cite/20200215204700.html ================================================================== --- tests/result/content/cite/20200215204700.html +++ /dev/null @@ -1,1 +0,0 @@ -<p>Stern18</p> DELETED tests/result/content/cite/20200215204700.native Index: tests/result/content/cite/20200215204700.native ================================================================== --- tests/result/content/cite/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Cite ("",[-]) "Stern18"] DELETED tests/result/content/cite/20200215204700.text Index: tests/result/content/cite/20200215204700.text ================================================================== --- tests/result/content/cite/20200215204700.text +++ /dev/null DELETED tests/result/content/comment/20200215204700.djson Index: tests/result/content/comment/20200215204700.djson ================================================================== --- tests/result/content/comment/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Text","s":"%"},{"t":"Space"},{"t":"Text","s":"No"},{"t":"Space"},{"t":"Text","s":"comment"},{"t":"Soft"},{"t":"Comment","s":"Comment"}]}] DELETED tests/result/content/comment/20200215204700.html Index: tests/result/content/comment/20200215204700.html ================================================================== --- tests/result/content/comment/20200215204700.html +++ /dev/null @@ -1,2 +0,0 @@ -<p>% No comment -<!-- Comment --></p> DELETED tests/result/content/comment/20200215204700.native Index: tests/result/content/comment/20200215204700.native ================================================================== --- tests/result/content/comment/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Text "%",Space,Text "No",Space,Text "comment",Space,Comment "Comment"] DELETED tests/result/content/comment/20200215204700.text Index: tests/result/content/comment/20200215204700.text ================================================================== --- tests/result/content/comment/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -% No comment DELETED tests/result/content/descrlist/20200226122100.djson Index: tests/result/content/descrlist/20200226122100.djson ================================================================== --- tests/result/content/descrlist/20200226122100.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"DescriptionList","g":[[[{"t":"Text","s":"Zettel"}],[{"t":"Para","i":[{"t":"Text","s":"Paper"}]}],[{"t":"Para","i":[{"t":"Text","s":"Note"}]}]],[[{"t":"Text","s":"Zettelkasten"}],[{"t":"Para","i":[{"t":"Text","s":"Slip"},{"t":"Space"},{"t":"Text","s":"box"}]}]]]}] DELETED tests/result/content/descrlist/20200226122100.html Index: tests/result/content/descrlist/20200226122100.html ================================================================== --- tests/result/content/descrlist/20200226122100.html +++ /dev/null @@ -1,7 +0,0 @@ -<dl> -<dt>Zettel</dt> -<dd>Paper</dd> -<dd>Note</dd> -<dt>Zettelkasten</dt> -<dd>Slip box</dd> -</dl> DELETED tests/result/content/descrlist/20200226122100.native Index: tests/result/content/descrlist/20200226122100.native ================================================================== --- tests/result/content/descrlist/20200226122100.native +++ /dev/null @@ -1,9 +0,0 @@ -[DescriptionList - [Term [Text "Zettel"], - [Description - [Para Text "Paper"]], - [Description - [Para Text "Note"]]], - [Term [Text "Zettelkasten"], - [Description - [Para Text "Slip",Space,Text "box"]]]] DELETED tests/result/content/descrlist/20200226122100.text Index: tests/result/content/descrlist/20200226122100.text ================================================================== --- tests/result/content/descrlist/20200226122100.text +++ /dev/null @@ -1,5 +0,0 @@ -Zettel -Paper -Note -Zettelkasten -Slip box DELETED tests/result/content/edit/20200215204700.djson Index: tests/result/content/edit/20200215204700.djson ================================================================== --- tests/result/content/edit/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Delete","i":[{"t":"Text","s":"delete"}]},{"t":"Soft"},{"t":"Insert","i":[{"t":"Text","s":"insert"}]},{"t":"Soft"},{"t":"Delete","i":[{"t":"Text","s":"kill"}]},{"t":"Insert","i":[{"t":"Text","s":"create"}]}]}] DELETED tests/result/content/edit/20200215204700.html Index: tests/result/content/edit/20200215204700.html ================================================================== --- tests/result/content/edit/20200215204700.html +++ /dev/null @@ -1,3 +0,0 @@ -<p><del>delete</del> -<ins>insert</ins> -<del>kill</del><ins>create</ins></p> DELETED tests/result/content/edit/20200215204700.native Index: tests/result/content/edit/20200215204700.native ================================================================== --- tests/result/content/edit/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Delete [Text "delete"],Space,Insert [Text "insert"],Space,Delete [Text "kill"],Insert [Text "create"]] DELETED tests/result/content/edit/20200215204700.text Index: tests/result/content/edit/20200215204700.text ================================================================== --- tests/result/content/edit/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -delete insert killcreate DELETED tests/result/content/footnote/20200215204700.djson Index: tests/result/content/footnote/20200215204700.djson ================================================================== --- tests/result/content/footnote/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Text","s":"Text"},{"t":"Footnote","a":{"":"sidebar"},"i":[{"t":"Text","s":"foot"}]}]}] DELETED tests/result/content/footnote/20200215204700.html Index: tests/result/content/footnote/20200215204700.html ================================================================== --- tests/result/content/footnote/20200215204700.html +++ /dev/null @@ -1,4 +0,0 @@ -<p>Text<sup id="fnref:1"><a href="#fn:1" class="zs-footnote-ref" role="doc-noteref">1</a></sup></p> -<ol class="zs-endnotes"> -<li id="fn:1" role="doc-endnote">foot <a href="#fnref:1" class="zs-footnote-backref" role="doc-backlink">↩︎</a></li> -</ol> DELETED tests/result/content/footnote/20200215204700.native Index: tests/result/content/footnote/20200215204700.native ================================================================== --- tests/result/content/footnote/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Text "Text",Footnote ("sidebar",[]) [Text "foot"]] DELETED tests/result/content/footnote/20200215204700.text Index: tests/result/content/footnote/20200215204700.text ================================================================== --- tests/result/content/footnote/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -Text foot DELETED tests/result/content/format/20200215204700.djson Index: tests/result/content/format/20200215204700.djson ================================================================== --- tests/result/content/format/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Italic","i":[{"t":"Text","s":"italic"}]},{"t":"Soft"},{"t":"Emph","i":[{"t":"Text","s":"emph"}]},{"t":"Soft"},{"t":"Bold","i":[{"t":"Text","s":"bold"}]},{"t":"Soft"},{"t":"Strong","i":[{"t":"Text","s":"strong"}]},{"t":"Soft"},{"t":"Underline","i":[{"t":"Text","s":"unterline"}]},{"t":"Soft"},{"t":"Strikethrough","i":[{"t":"Text","s":"strike"}]},{"t":"Soft"},{"t":"Mono","i":[{"t":"Text","s":"monospace"}]},{"t":"Soft"},{"t":"Super","i":[{"t":"Text","s":"superscript"}]},{"t":"Soft"},{"t":"Sub","i":[{"t":"Text","s":"subscript"}]},{"t":"Soft"},{"t":"Quote","i":[{"t":"Text","s":"Quotes"}]},{"t":"Soft"},{"t":"Quotation","i":[{"t":"Text","s":"Quotation"}]},{"t":"Soft"},{"t":"Small","i":[{"t":"Text","s":"small"}]},{"t":"Soft"},{"t":"Span","i":[{"t":"Text","s":"span"}]},{"t":"Soft"},{"t":"Code","s":"code"},{"t":"Soft"},{"t":"Input","s":"input"},{"t":"Soft"},{"t":"Output","s":"output"}]}] DELETED tests/result/content/format/20200215204700.html Index: tests/result/content/format/20200215204700.html ================================================================== --- tests/result/content/format/20200215204700.html +++ /dev/null @@ -1,16 +0,0 @@ -<p><i>italic</i> -<em>emph</em> -<b>bold</b> -<strong>strong</strong> -<u>unterline</u> -<s>strike</s> -<span style="font-family:monospace">monospace</span> -<sup>superscript</sup> -<sub>subscript</sub> -"Quotes" -<q>Quotation</q> -<small>small</small> -<span>span</span> -<code>code</code> -<kbd>input</kbd> -<samp>output</samp></p> DELETED tests/result/content/format/20200215204700.native Index: tests/result/content/format/20200215204700.native ================================================================== --- tests/result/content/format/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Italic [Text "italic"],Space,Emph [Text "emph"],Space,Bold [Text "bold"],Space,Strong [Text "strong"],Space,Underline [Text "unterline"],Space,Strikethrough [Text "strike"],Space,Mono [Text "monospace"],Space,Super [Text "superscript"],Space,Sub [Text "subscript"],Space,Quote [Text "Quotes"],Space,Quotation [Text "Quotation"],Space,Small [Text "small"],Space,Span [Text "span"],Space,Code "code",Space,Input "input",Space,Output "output"] DELETED tests/result/content/format/20200215204700.text Index: tests/result/content/format/20200215204700.text ================================================================== --- tests/result/content/format/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -italic emph bold strong unterline strike monospace superscript subscript Quotes Quotation small span code input output DELETED tests/result/content/format/20201107164400.djson Index: tests/result/content/format/20201107164400.djson ================================================================== --- tests/result/content/format/20201107164400.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Span","a":{"lang":"fr"},"i":[{"t":"Quote","i":[{"t":"Text","s":"abc"}]}]}]}] DELETED tests/result/content/format/20201107164400.html Index: tests/result/content/format/20201107164400.html ================================================================== --- tests/result/content/format/20201107164400.html +++ /dev/null @@ -1,1 +0,0 @@ -<p><span lang="fr">« abc »</span></p> DELETED tests/result/content/format/20201107164400.native Index: tests/result/content/format/20201107164400.native ================================================================== --- tests/result/content/format/20201107164400.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Span ("",[lang="fr"]) [Quote [Text "abc"]]] DELETED tests/result/content/format/20201107164400.text Index: tests/result/content/format/20201107164400.text ================================================================== --- tests/result/content/format/20201107164400.text +++ /dev/null @@ -1,1 +0,0 @@ -abc DELETED tests/result/content/heading/20200215204700.djson Index: tests/result/content/heading/20200215204700.djson ================================================================== --- tests/result/content/heading/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Heading","n":2,"s":"first","i":[{"t":"Text","s":"First"}]}] DELETED tests/result/content/heading/20200215204700.html Index: tests/result/content/heading/20200215204700.html ================================================================== --- tests/result/content/heading/20200215204700.html +++ /dev/null @@ -1,1 +0,0 @@ -<h2 id="first">First</h2> DELETED tests/result/content/heading/20200215204700.native Index: tests/result/content/heading/20200215204700.native ================================================================== --- tests/result/content/heading/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Heading 2 "first" Text "First"] DELETED tests/result/content/heading/20200215204700.text Index: tests/result/content/heading/20200215204700.text ================================================================== --- tests/result/content/heading/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -First DELETED tests/result/content/hrule/20200215204700.djson Index: tests/result/content/hrule/20200215204700.djson ================================================================== --- tests/result/content/hrule/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Hrule"}] DELETED tests/result/content/hrule/20200215204700.html Index: tests/result/content/hrule/20200215204700.html ================================================================== --- tests/result/content/hrule/20200215204700.html +++ /dev/null @@ -1,1 +0,0 @@ -<hr> DELETED tests/result/content/hrule/20200215204700.native Index: tests/result/content/hrule/20200215204700.native ================================================================== --- tests/result/content/hrule/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Hrule] DELETED tests/result/content/hrule/20200215204700.text Index: tests/result/content/hrule/20200215204700.text ================================================================== --- tests/result/content/hrule/20200215204700.text +++ /dev/null DELETED tests/result/content/image/20200215204700.djson Index: tests/result/content/image/20200215204700.djson ================================================================== --- tests/result/content/image/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Image","s":"abc"}]}] DELETED tests/result/content/image/20200215204700.html Index: tests/result/content/image/20200215204700.html ================================================================== --- tests/result/content/image/20200215204700.html +++ /dev/null @@ -1,1 +0,0 @@ -<p><img src="abc" alt=""></p> DELETED tests/result/content/image/20200215204700.native Index: tests/result/content/image/20200215204700.native ================================================================== --- tests/result/content/image/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Image "abc"] DELETED tests/result/content/image/20200215204700.text Index: tests/result/content/image/20200215204700.text ================================================================== --- tests/result/content/image/20200215204700.text +++ /dev/null DELETED tests/result/content/link/20200215204700.djson Index: tests/result/content/link/20200215204700.djson ================================================================== --- tests/result/content/link/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Link","q":"external","s":"https://zettelstore.de/z","i":[{"t":"Text","s":"Home"}]},{"t":"Soft"},{"t":"Link","q":"external","s":"https://zettelstore.de","i":[{"t":"Text","s":"https://zettelstore.de"}]},{"t":"Soft"},{"t":"Link","q":"zettel","s":"00000000000100","i":[{"t":"Text","s":"Config"}]},{"t":"Soft"},{"t":"Link","q":"zettel","s":"00000000000100","i":[{"t":"Text","s":"00000000000100"}]},{"t":"Soft"},{"t":"Link","q":"self","s":"#frag","i":[{"t":"Text","s":"Frag"}]},{"t":"Soft"},{"t":"Link","q":"self","s":"#frag","i":[{"t":"Text","s":"#frag"}]},{"t":"Soft"},{"t":"Link","q":"local","s":"/hosted","i":[{"t":"Text","s":"H"}]},{"t":"Soft"},{"t":"Link","q":"based","s":"/based","i":[{"t":"Text","s":"B"}]},{"t":"Soft"},{"t":"Link","q":"local","s":"../rel","i":[{"t":"Text","s":"R"}]}]}] DELETED tests/result/content/link/20200215204700.html Index: tests/result/content/link/20200215204700.html ================================================================== --- tests/result/content/link/20200215204700.html +++ /dev/null @@ -1,9 +0,0 @@ -<p><a href="https://zettelstore.de/z" class="zs-external">Home</a> -<a href="https://zettelstore.de" class="zs-external">https://zettelstore.de</a> -<a href="00000000000100">Config</a> -<a href="00000000000100">00000000000100</a> -<a href="#frag">Frag</a> -<a href="#frag">#frag</a> -<a href="/hosted">H</a> -<a href="/based">B</a> -<a href="../rel">R</a></p> DELETED tests/result/content/link/20200215204700.native Index: tests/result/content/link/20200215204700.native ================================================================== --- tests/result/content/link/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Link EXTERNAL "https://zettelstore.de/z" [Text "Home"],Space,Link EXTERNAL "https://zettelstore.de" [],Space,Link ZETTEL "00000000000100" [Text "Config"],Space,Link ZETTEL "00000000000100" [],Space,Link SELF "#frag" [Text "Frag"],Space,Link SELF "#frag" [],Space,Link LOCAL "/hosted" [Text "H"],Space,Link BASED "/based" [Text "B"],Space,Link LOCAL "../rel" [Text "R"]] DELETED tests/result/content/link/20200215204700.text Index: tests/result/content/link/20200215204700.text ================================================================== --- tests/result/content/link/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -Home Config Frag H B R DELETED tests/result/content/list/20200215204700.djson Index: tests/result/content/list/20200215204700.djson ================================================================== --- tests/result/content/list/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"BulletList","c":[[{"t":"Para","i":[{"t":"Text","s":"Item"},{"t":"Space"},{"t":"Text","s":"1"}]}],[{"t":"Para","i":[{"t":"Text","s":"Item"},{"t":"Space"},{"t":"Text","s":"2"}]}],[{"t":"Para","i":[{"t":"Text","s":"Item"},{"t":"Space"},{"t":"Text","s":"3"}]}]]}] DELETED tests/result/content/list/20200215204700.html Index: tests/result/content/list/20200215204700.html ================================================================== --- tests/result/content/list/20200215204700.html +++ /dev/null @@ -1,5 +0,0 @@ -<ul> -<li>Item 1</li> -<li>Item 2</li> -<li>Item 3</li> -</ul> DELETED tests/result/content/list/20200215204700.native Index: tests/result/content/list/20200215204700.native ================================================================== --- tests/result/content/list/20200215204700.native +++ /dev/null @@ -1,4 +0,0 @@ -[BulletList - [[Para Text "Item",Space,Text "1"]], - [[Para Text "Item",Space,Text "2"]], - [[Para Text "Item",Space,Text "3"]]] DELETED tests/result/content/list/20200215204700.text Index: tests/result/content/list/20200215204700.text ================================================================== --- tests/result/content/list/20200215204700.text +++ /dev/null @@ -1,3 +0,0 @@ -Item 1 -Item 2 -Item 3 DELETED tests/result/content/list/20200217194800.djson Index: tests/result/content/list/20200217194800.djson ================================================================== --- tests/result/content/list/20200217194800.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"BulletList","c":[[{"t":"Para","i":[{"t":"Text","s":"Item1.1"}]}],[{"t":"Para","i":[{"t":"Text","s":"Item1.2"}]}],[{"t":"Para","i":[{"t":"Text","s":"Item1.3"}]}],[{"t":"Para","i":[{"t":"Text","s":"Item2.1"}]}],[{"t":"Para","i":[{"t":"Text","s":"Item2.2"}]}]]}] DELETED tests/result/content/list/20200217194800.html Index: tests/result/content/list/20200217194800.html ================================================================== --- tests/result/content/list/20200217194800.html +++ /dev/null @@ -1,7 +0,0 @@ -<ul> -<li>Item1.1</li> -<li>Item1.2</li> -<li>Item1.3</li> -<li>Item2.1</li> -<li>Item2.2</li> -</ul> DELETED tests/result/content/list/20200217194800.native Index: tests/result/content/list/20200217194800.native ================================================================== --- tests/result/content/list/20200217194800.native +++ /dev/null @@ -1,6 +0,0 @@ -[BulletList - [[Para Text "Item1.1"]], - [[Para Text "Item1.2"]], - [[Para Text "Item1.3"]], - [[Para Text "Item2.1"]], - [[Para Text "Item2.2"]]] DELETED tests/result/content/list/20200217194800.text Index: tests/result/content/list/20200217194800.text ================================================================== --- tests/result/content/list/20200217194800.text +++ /dev/null @@ -1,5 +0,0 @@ -Item1.1 -Item1.2 -Item1.3 -Item2.1 -Item2.2 DELETED tests/result/content/list/20200516105700.djson Index: tests/result/content/list/20200516105700.djson ================================================================== --- tests/result/content/list/20200516105700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"BulletList","c":[[{"t":"Para","i":[{"t":"Text","s":"T1"}]},{"t":"BulletList","c":[[{"t":"Para","i":[{"t":"Text","s":"T2"}]}]]}],[{"t":"Para","i":[{"t":"Text","s":"T3"}]},{"t":"BulletList","c":[[{"t":"Para","i":[{"t":"Text","s":"T4"}]}]]}],[{"t":"Para","i":[{"t":"Text","s":"T5"}]}]]}] DELETED tests/result/content/list/20200516105700.html Index: tests/result/content/list/20200516105700.html ================================================================== --- tests/result/content/list/20200516105700.html +++ /dev/null @@ -1,14 +0,0 @@ -<ul> -<li><p>T1</p> -<ul> -<li>T2</li> -</ul> -</li> -<li><p>T3</p> -<ul> -<li>T4</li> -</ul> -</li> -<li><p>T5</p> -</li> -</ul> DELETED tests/result/content/list/20200516105700.native Index: tests/result/content/list/20200516105700.native ================================================================== --- tests/result/content/list/20200516105700.native +++ /dev/null @@ -1,8 +0,0 @@ -[BulletList - [[Para Text "T1"], - [BulletList - [[Para Text "T2"]]]], - [[Para Text "T3"], - [BulletList - [[Para Text "T4"]]]], - [[Para Text "T5"]]] DELETED tests/result/content/list/20200516105700.text Index: tests/result/content/list/20200516105700.text ================================================================== --- tests/result/content/list/20200516105700.text +++ /dev/null @@ -1,5 +0,0 @@ -T1 -T2 -T3 -T4 -T5 DELETED tests/result/content/literal/20200215204700.djson Index: tests/result/content/literal/20200215204700.djson ================================================================== --- tests/result/content/literal/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Input","s":"input"},{"t":"Soft"},{"t":"Code","s":"program"},{"t":"Soft"},{"t":"Output","s":"output"}]}] DELETED tests/result/content/literal/20200215204700.html Index: tests/result/content/literal/20200215204700.html ================================================================== --- tests/result/content/literal/20200215204700.html +++ /dev/null @@ -1,3 +0,0 @@ -<p><kbd>input</kbd> -<code>program</code> -<samp>output</samp></p> DELETED tests/result/content/literal/20200215204700.native Index: tests/result/content/literal/20200215204700.native ================================================================== --- tests/result/content/literal/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Input "input",Space,Code "program",Space,Output "output"] DELETED tests/result/content/literal/20200215204700.text Index: tests/result/content/literal/20200215204700.text ================================================================== --- tests/result/content/literal/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -input program output DELETED tests/result/content/mark/20200215204700.djson Index: tests/result/content/mark/20200215204700.djson ================================================================== --- tests/result/content/mark/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Mark","s":"mark"}]}] DELETED tests/result/content/mark/20200215204700.html Index: tests/result/content/mark/20200215204700.html ================================================================== --- tests/result/content/mark/20200215204700.html +++ /dev/null @@ -1,1 +0,0 @@ -<p><a id="mark"></a></p> DELETED tests/result/content/mark/20200215204700.native Index: tests/result/content/mark/20200215204700.native ================================================================== --- tests/result/content/mark/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Mark "mark"] DELETED tests/result/content/mark/20200215204700.text Index: tests/result/content/mark/20200215204700.text ================================================================== --- tests/result/content/mark/20200215204700.text +++ /dev/null DELETED tests/result/content/paragraph/20200215185900.djson Index: tests/result/content/paragraph/20200215185900.djson ================================================================== --- tests/result/content/paragraph/20200215185900.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Text","s":"This"},{"t":"Space"},{"t":"Text","s":"is"},{"t":"Space"},{"t":"Text","s":"a"},{"t":"Space"},{"t":"Text","s":"zettel"},{"t":"Space"},{"t":"Text","s":"for"},{"t":"Space"},{"t":"Text","s":"testing."}]}] DELETED tests/result/content/paragraph/20200215185900.html Index: tests/result/content/paragraph/20200215185900.html ================================================================== --- tests/result/content/paragraph/20200215185900.html +++ /dev/null @@ -1,1 +0,0 @@ -<p>This is a zettel for testing.</p> DELETED tests/result/content/paragraph/20200215185900.native Index: tests/result/content/paragraph/20200215185900.native ================================================================== --- tests/result/content/paragraph/20200215185900.native +++ /dev/null @@ -1,1 +0,0 @@ -[Para Text "This",Space,Text "is",Space,Text "a",Space,Text "zettel",Space,Text "for",Space,Text "testing."] DELETED tests/result/content/paragraph/20200215185900.text Index: tests/result/content/paragraph/20200215185900.text ================================================================== --- tests/result/content/paragraph/20200215185900.text +++ /dev/null @@ -1,1 +0,0 @@ -This is a zettel for testing. DELETED tests/result/content/paragraph/20200217151800.djson Index: tests/result/content/paragraph/20200217151800.djson ================================================================== --- tests/result/content/paragraph/20200217151800.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Para","i":[{"t":"Text","s":"Text"},{"t":"Space"},{"t":"Text","s":"Text"},{"t":"Soft"},{"t":"Text","s":"*abc"}]},{"t":"Para","i":[{"t":"Text","s":"Text"},{"t":"Space"},{"t":"Text","s":"Text"}]},{"t":"BulletList","c":[[{"t":"Para","i":[{"t":"Text","s":"abc"}]}]]}] DELETED tests/result/content/paragraph/20200217151800.html Index: tests/result/content/paragraph/20200217151800.html ================================================================== --- tests/result/content/paragraph/20200217151800.html +++ /dev/null @@ -1,6 +0,0 @@ -<p>Text Text -*abc</p> -<p>Text Text</p> -<ul> -<li>abc</li> -</ul> DELETED tests/result/content/paragraph/20200217151800.native Index: tests/result/content/paragraph/20200217151800.native ================================================================== --- tests/result/content/paragraph/20200217151800.native +++ /dev/null @@ -1,4 +0,0 @@ -[Para Text "Text",Space,Text "Text",Space,Text "*abc"], -[Para Text "Text",Space,Text "Text"], -[BulletList - [[Para Text "abc"]]] DELETED tests/result/content/paragraph/20200217151800.text Index: tests/result/content/paragraph/20200217151800.text ================================================================== --- tests/result/content/paragraph/20200217151800.text +++ /dev/null @@ -1,3 +0,0 @@ -Text Text *abc -Text Text -abc DELETED tests/result/content/png/20200512180900.djson Index: tests/result/content/png/20200512180900.djson ================================================================== --- tests/result/content/png/20200512180900.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Blob","q":"20200512180900","s":"png","o":"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="}] DELETED tests/result/content/png/20200512180900.html Index: tests/result/content/png/20200512180900.html ================================================================== --- tests/result/content/png/20200512180900.html +++ /dev/null @@ -1,1 +0,0 @@ -<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg==" title="20200512180900"> DELETED tests/result/content/png/20200512180900.native Index: tests/result/content/png/20200512180900.native ================================================================== --- tests/result/content/png/20200512180900.native +++ /dev/null @@ -1,1 +0,0 @@ -[BLOB "20200512180900" "png" "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="] DELETED tests/result/content/png/20200512180900.text Index: tests/result/content/png/20200512180900.text ================================================================== --- tests/result/content/png/20200512180900.text +++ /dev/null DELETED tests/result/content/quoteblock/20200215204700.djson Index: tests/result/content/quoteblock/20200215204700.djson ================================================================== --- tests/result/content/quoteblock/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"QuoteBlock","b":[{"t":"Para","i":[{"t":"Text","s":"To"},{"t":"Space"},{"t":"Text","s":"be"},{"t":"Space"},{"t":"Text","s":"or"},{"t":"Space"},{"t":"Text","s":"not"},{"t":"Space"},{"t":"Text","s":"to"},{"t":"Space"},{"t":"Text","s":"be."}]}],"i":[{"t":"Text","s":"Romeo"}]}] DELETED tests/result/content/quoteblock/20200215204700.html Index: tests/result/content/quoteblock/20200215204700.html ================================================================== --- tests/result/content/quoteblock/20200215204700.html +++ /dev/null @@ -1,4 +0,0 @@ -<blockquote> -<p>To be or not to be.</p> -<cite>Romeo</cite> -</blockquote> DELETED tests/result/content/quoteblock/20200215204700.native Index: tests/result/content/quoteblock/20200215204700.native ================================================================== --- tests/result/content/quoteblock/20200215204700.native +++ /dev/null @@ -1,3 +0,0 @@ -[QuoteBlock - [[Para Text "To",Space,Text "be",Space,Text "or",Space,Text "not",Space,Text "to",Space,Text "be."]], - [Cite Text "Romeo"]] DELETED tests/result/content/quoteblock/20200215204700.text Index: tests/result/content/quoteblock/20200215204700.text ================================================================== --- tests/result/content/quoteblock/20200215204700.text +++ /dev/null @@ -1,2 +0,0 @@ -To be or not to be. -Romeo DELETED tests/result/content/spanblock/20200215204700.djson Index: tests/result/content/spanblock/20200215204700.djson ================================================================== --- tests/result/content/spanblock/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"SpanBlock","b":[{"t":"Para","i":[{"t":"Text","s":"A"},{"t":"Space"},{"t":"Text","s":"simple"},{"t":"Soft"},{"t":"Space","n":3},{"t":"Text","s":"span"},{"t":"Soft"},{"t":"Text","s":"and"},{"t":"Space"},{"t":"Text","s":"much"},{"t":"Space"},{"t":"Text","s":"more"}]}]}] DELETED tests/result/content/spanblock/20200215204700.html Index: tests/result/content/spanblock/20200215204700.html ================================================================== --- tests/result/content/spanblock/20200215204700.html +++ /dev/null @@ -1,5 +0,0 @@ -<div> -<p>A simple - span -and much more</p> -</div> DELETED tests/result/content/spanblock/20200215204700.native Index: tests/result/content/spanblock/20200215204700.native ================================================================== --- tests/result/content/spanblock/20200215204700.native +++ /dev/null @@ -1,2 +0,0 @@ -[SpanBlock - [[Para Text "A",Space,Text "simple",Space,Space 3,Text "span",Space,Text "and",Space,Text "much",Space,Text "more"]]] DELETED tests/result/content/spanblock/20200215204700.text Index: tests/result/content/spanblock/20200215204700.text ================================================================== --- tests/result/content/spanblock/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -A simple span and much more DELETED tests/result/content/table/20200215204700.djson Index: tests/result/content/table/20200215204700.djson ================================================================== --- tests/result/content/table/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Table","p":[[],[[["",[{"t":"Text","s":"c1"}]],["",[{"t":"Text","s":"c2"}]],["",[{"t":"Text","s":"c3"}]]]]]}] DELETED tests/result/content/table/20200215204700.html Index: tests/result/content/table/20200215204700.html ================================================================== --- tests/result/content/table/20200215204700.html +++ /dev/null @@ -1,5 +0,0 @@ -<table> -<tbody> -<tr><td>c1</td><td>c2</td><td>c3</td></tr> -</tbody> -</table> DELETED tests/result/content/table/20200215204700.native Index: tests/result/content/table/20200215204700.native ================================================================== --- tests/result/content/table/20200215204700.native +++ /dev/null @@ -1,2 +0,0 @@ -[Table - [Row [Cell Default Text "c1"],[Cell Default Text "c2"],[Cell Default Text "c3"]]] DELETED tests/result/content/table/20200215204700.text Index: tests/result/content/table/20200215204700.text ================================================================== --- tests/result/content/table/20200215204700.text +++ /dev/null @@ -1,1 +0,0 @@ -c1 c2 c3 DELETED tests/result/content/table/20200618140700.djson Index: tests/result/content/table/20200618140700.djson ================================================================== --- tests/result/content/table/20200618140700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"Table","p":[[[">",[{"t":"Text","s":"h1"}]],["",[{"t":"Text","s":"h2"}]],[":",[{"t":"Text","s":"h3"}]]],[[["<",[{"t":"Text","s":"c1"}]],["",[{"t":"Text","s":"c2"}]],[":",[{"t":"Text","s":"c3"}]]],[[">",[{"t":"Text","s":"f1"}]],["",[{"t":"Text","s":"f2"}]],[":",[{"t":"Text","s":"=f3"}]]]]]}] DELETED tests/result/content/table/20200618140700.html Index: tests/result/content/table/20200618140700.html ================================================================== --- tests/result/content/table/20200618140700.html +++ /dev/null @@ -1,9 +0,0 @@ -<table> -<thead> -<tr><th style="text-align:right">h1</th><th>h2</th><th style="text-align:center">h3</th></tr> -</thead> -<tbody> -<tr><td style="text-align:left">c1</td><td>c2</td><td style="text-align:center">c3</td></tr> -<tr><td style="text-align:right">f1</td><td>f2</td><td style="text-align:center">=f3</td></tr> -</tbody> -</table> DELETED tests/result/content/table/20200618140700.native Index: tests/result/content/table/20200618140700.native ================================================================== --- tests/result/content/table/20200618140700.native +++ /dev/null @@ -1,4 +0,0 @@ -[Table - [Header [Cell Right Text "h1"],[Cell Default Text "h2"],[Cell Center Text "h3"]], - [Row [Cell Left Text "c1"],[Cell Default Text "c2"],[Cell Center Text "c3"]], - [Row [Cell Right Text "f1"],[Cell Default Text "f2"],[Cell Center Text "=f3"]]] DELETED tests/result/content/table/20200618140700.text Index: tests/result/content/table/20200618140700.text ================================================================== --- tests/result/content/table/20200618140700.text +++ /dev/null @@ -1,3 +0,0 @@ -h1 h2 h3 -c1 c2 c3 -f1 f2 =f3 DELETED tests/result/content/verbatim/20200215204700.djson Index: tests/result/content/verbatim/20200215204700.djson ================================================================== --- tests/result/content/verbatim/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"CodeBlock","l":["if __name__ == \"main\":"," print(\"Hello, World\")","exit(0)"]}] DELETED tests/result/content/verbatim/20200215204700.html Index: tests/result/content/verbatim/20200215204700.html ================================================================== --- tests/result/content/verbatim/20200215204700.html +++ /dev/null @@ -1,4 +0,0 @@ -<pre><code>if __name__ == "main": - print("Hello, World") -exit(0) -</code></pre> DELETED tests/result/content/verbatim/20200215204700.native Index: tests/result/content/verbatim/20200215204700.native ================================================================== --- tests/result/content/verbatim/20200215204700.native +++ /dev/null @@ -1,1 +0,0 @@ -[CodeBlock "if __name__ == \"main\":\n print(\"Hello, World\")\nexit(0)"] DELETED tests/result/content/verbatim/20200215204700.text Index: tests/result/content/verbatim/20200215204700.text ================================================================== --- tests/result/content/verbatim/20200215204700.text +++ /dev/null @@ -1,3 +0,0 @@ -if __name__ == "main": - print("Hello, World") -exit(0) DELETED tests/result/content/verseblock/20200215204700.djson Index: tests/result/content/verseblock/20200215204700.djson ================================================================== --- tests/result/content/verseblock/20200215204700.djson +++ /dev/null @@ -1,1 +0,0 @@ -[{"t":"VerseBlock","b":[{"t":"Para","i":[{"t":"Text","s":"A line"},{"t":"Hard"},{"t":"Text","s":"  another line"},{"t":"Hard"},{"t":"Text","s":"Back"}]},{"t":"Para","i":[{"t":"Text","s":"Paragraph"}]},{"t":"Para","i":[{"t":"Text","s":"    Spacy  Para"}]}],"i":[{"t":"Text","s":"Author"}]}] DELETED tests/result/content/verseblock/20200215204700.html Index: tests/result/content/verseblock/20200215204700.html ================================================================== --- tests/result/content/verseblock/20200215204700.html +++ /dev/null @@ -1,8 +0,0 @@ -<div> -<p>A line<br> -  another line<br> -Back</p> -<p>Paragraph</p> -<p>    Spacy  Para</p> -<cite>Author</cite> -</div> DELETED tests/result/content/verseblock/20200215204700.native Index: tests/result/content/verseblock/20200215204700.native ================================================================== --- tests/result/content/verseblock/20200215204700.native +++ /dev/null @@ -1,5 +0,0 @@ -[VerseBlock - [[Para Text "A line",Break,Text "  another line",Break,Text "Back"], - [Para Text "Paragraph"], - [Para Text "    Spacy  Para"]], - [Cite Text "Author"]] DELETED tests/result/content/verseblock/20200215204700.text Index: tests/result/content/verseblock/20200215204700.text ================================================================== --- tests/result/content/verseblock/20200215204700.text +++ /dev/null @@ -1,6 +0,0 @@ -A line -  another line -Back -Paragraph -    Spacy  Para -Author DELETED tests/result/meta/copyright/20200310125800.djson Index: tests/result/meta/copyright/20200310125800.djson ================================================================== --- tests/result/meta/copyright/20200310125800.djson +++ /dev/null @@ -1,1 +0,0 @@ -{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","copyright":"(c) 2020 Detlef Stern","license":"CC BY-SA 4.0"} ADDED tests/result/meta/copyright/20200310125800.zjson Index: tests/result/meta/copyright/20200310125800.zjson ================================================================== --- /dev/null +++ tests/result/meta/copyright/20200310125800.zjson @@ -0,0 +1,1 @@ +{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","copyright":"(c) 2020 Detlef Stern","license":"CC BY-SA 4.0"} DELETED tests/result/meta/header/20200310125800.djson Index: tests/result/meta/header/20200310125800.djson ================================================================== --- tests/result/meta/header/20200310125800.djson +++ /dev/null @@ -1,1 +0,0 @@ -{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","x-no":"00000000000000"} ADDED tests/result/meta/header/20200310125800.zjson Index: tests/result/meta/header/20200310125800.zjson ================================================================== --- /dev/null +++ tests/result/meta/header/20200310125800.zjson @@ -0,0 +1,1 @@ +{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","x-no":"00000000000000"} DELETED tests/result/meta/title/20200310110300.djson Index: tests/result/meta/title/20200310110300.djson ================================================================== --- tests/result/meta/title/20200310110300.djson +++ /dev/null @@ -1,1 +0,0 @@ -{"title":[{"t":"Text","s":"A"},{"t":"Space"},{"t":"Quote","i":[{"t":"Text","s":"Title"}]},{"t":"Space"},{"t":"Text","s":"with"},{"t":"Space"},{"t":"Italic","i":[{"t":"Text","s":"Markup"}]},{"t":"Text","s":","},{"t":"Space"},{"t":"Code","a":{"":"zmk"},"s":"Zettelmarkup"}],"role":"zettel","syntax":"zmk"} Index: tests/result/meta/title/20200310110300.native ================================================================== --- tests/result/meta/title/20200310110300.native +++ tests/result/meta/title/20200310110300.native @@ -1,3 +1,3 @@ -[Title Text "A",Space,Quote [Text "Title"],Space,Text "with",Space,Italic [Text "Markup"],Text ",",Space,Code ("zmk",[]) "Zettelmarkup"] +[Title Text "A",Space,Quote [Text "Title"],Space,Text "with",Space,Emph [Text "Markup"],Text ",",Space,Code ("zmk",[]) "Zettelmarkup"] [Role "zettel"] [Syntax "zmk"] ADDED tests/result/meta/title/20200310110300.zjson Index: tests/result/meta/title/20200310110300.zjson ================================================================== --- /dev/null +++ tests/result/meta/title/20200310110300.zjson @@ -0,0 +1,1 @@ +{"title":[{"t":"Text","s":"A"},{"t":"Space"},{"t":"Quote","i":[{"t":"Text","s":"Title"}]},{"t":"Space"},{"t":"Text","s":"with"},{"t":"Space"},{"t":"Emph","i":[{"t":"Text","s":"Markup"}]},{"t":"Text","s":","},{"t":"Space"},{"t":"Code","a":{"":"zmk"},"s":"Zettelmarkup"}],"role":"zettel","syntax":"zmk"} DELETED tools/build.go Index: tools/build.go ================================================================== --- tools/build.go +++ /dev/null @@ -1,454 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package main provides a command to build and run the software. -package main - -import ( - "archive/zip" - "bytes" - "flag" - "fmt" - "io" - "io/fs" - "os" - "os/exec" - "path/filepath" - "regexp" - "strings" - "time" - - "zettelstore.de/z/strfun" -) - -func executeCommand(env []string, name string, arg ...string) (string, error) { - if verbose { - if len(env) > 0 { - for i, e := range env { - fmt.Fprintf(os.Stderr, "ENV%d %v\n", i+1, e) - } - } - fmt.Fprintln(os.Stderr, "EXEC", name, arg) - } - if len(env) > 0 { - env = append(env, os.Environ()...) - } - var out bytes.Buffer - cmd := exec.Command(name, arg...) - cmd.Env = env - cmd.Stdin = nil - cmd.Stdout = &out - cmd.Stderr = os.Stderr - err := cmd.Run() - return out.String(), err -} - -func readVersionFile() (string, error) { - content, err := os.ReadFile("VERSION") - if err != nil { - return "", err - } - return strings.TrimFunc(string(content), func(r rune) bool { - return r <= ' ' - }), nil -} - -var fossilCheckout = regexp.MustCompile(`^checkout:\s+([0-9a-f]+)\s`) -var dirtyPrefixes = []string{ - "DELETED ", "ADDED ", "UPDATED ", "CONFLICT ", "EDITED ", "RENAMED ", "EXTRA "} - -const dirtySuffix = "-dirty" - -func readFossilVersion() (string, error) { - s, err := executeCommand(nil, "fossil", "status", "--differ") - if err != nil { - return "", err - } - var hash, suffix string - for _, line := range strfun.SplitLines(s) { - if hash == "" { - if m := fossilCheckout.FindStringSubmatch(line); len(m) > 0 { - hash = m[1][:10] - if suffix != "" { - return hash + suffix, nil - } - continue - } - } - if suffix == "" { - for _, prefix := range dirtyPrefixes { - if strings.HasPrefix(line, prefix) { - suffix = dirtySuffix - if hash != "" { - return hash + suffix, nil - } - break - } - } - } - } - return hash, nil -} - -func getVersionData() (string, string) { - base, err := readVersionFile() - if err != nil { - base = "dev" - } - fossil, err := readFossilVersion() - if err != nil { - return base, "" - } - return base, fossil -} - -func calcVersion(base, vcs string) string { return base + "+" + vcs } - -func getVersion() string { - base, vcs := getVersionData() - return calcVersion(base, vcs) -} - -func findExec(cmd string) string { - if path, err := executeCommand(nil, "which", "shadow"); err == nil && path != "" { - return path - } - return "" -} - -func cmdCheck() error { - if err := checkGoTest(); err != nil { - return err - } - if err := checkGoVet(); err != nil { - return err - } - if err := checkGoLint(); err != nil { - return err - } - if err := checkGoVetShadow(); err != nil { - return err - } - if err := checkStaticcheck(); err != nil { - return err - } - return checkFossilExtra() -} - -func checkGoTest() error { - out, err := executeCommand(nil, "go", "test", "./...") - if err != nil { - for _, line := range strfun.SplitLines(out) { - if strings.HasPrefix(line, "ok") || strings.HasPrefix(line, "?") { - continue - } - fmt.Fprintln(os.Stderr, line) - } - } - return err -} - -func checkGoVet() error { - out, err := executeCommand(nil, "go", "vet", "./...") - if err != nil { - fmt.Fprintln(os.Stderr, "Some checks failed") - if len(out) > 0 { - fmt.Fprintln(os.Stderr, out) - } - } - return err -} - -func checkGoLint() error { - out, err := executeCommand(nil, "golint", "./...") - if out != "" { - fmt.Fprintln(os.Stderr, "Some lints failed") - fmt.Fprint(os.Stderr, out) - } - return err -} - -func checkGoVetShadow() error { - path := findExec("shadow") - if path == "" { - return nil - } - out, err := executeCommand(nil, "go", "vet", "-vettool", strings.TrimSpace(path), "./...") - if err != nil { - fmt.Fprintln(os.Stderr, "Some shadowed variables found") - if len(out) > 0 { - fmt.Fprintln(os.Stderr, out) - } - } - return err -} -func checkStaticcheck() error { - out, err := executeCommand(nil, "staticcheck", "./...") - if err != nil { - fmt.Fprintln(os.Stderr, "Some staticcheck problems found") - if len(out) > 0 { - fmt.Fprintln(os.Stderr, out) - } - } - return err -} - -func checkFossilExtra() error { - out, err := executeCommand(nil, "fossil", "extra") - if err != nil { - fmt.Fprintln(os.Stderr, "Unable to execute 'fossil extra'") - return err - } - if len(out) > 0 { - fmt.Fprint(os.Stderr, "Warning: unversioned file(s):") - for i, extra := range strfun.SplitLines(out) { - if i > 0 { - fmt.Fprint(os.Stderr, ",") - } - fmt.Fprintf(os.Stderr, " %q", extra) - } - fmt.Fprintln(os.Stderr) - } - return nil -} - -func cmdBuild() error { - return doBuild(nil, getVersion(), "bin/zettelstore") -} - -func doBuild(env []string, version, target string) error { - out, err := executeCommand( - env, - "go", "build", - "-tags", "osusergo,netgo", - "-trimpath", - "-ldflags", fmt.Sprintf("-X main.version=%v -w", version), - "-o", target, - "zettelstore.de/z/cmd/zettelstore", - ) - if err != nil { - return err - } - if len(out) > 0 { - fmt.Println(out) - } - return nil -} - -func cmdManual() error { - base, _ := getReleaseVersionData() - return createManualZip(".", base) -} - -func createManualZip(path, base string) error { - manualPath := filepath.Join("docs", "manual") - entries, err := os.ReadDir(manualPath) - if err != nil { - return err - } - zipName := filepath.Join(path, "manual-"+base+".zip") - zipFile, err := os.OpenFile(zipName, os.O_RDWR|os.O_CREATE, 0600) - if err != nil { - return err - } - defer zipFile.Close() - zipWriter := zip.NewWriter(zipFile) - defer zipWriter.Close() - - for _, entry := range entries { - if err = createManualZipEntry(manualPath, entry, zipWriter); err != nil { - return err - } - } - return nil -} - -func createManualZipEntry(path string, entry fs.DirEntry, zipWriter *zip.Writer) error { - info, err := entry.Info() - if err != nil { - return err - } - fh, err := zip.FileInfoHeader(info) - if err != nil { - return err - } - fh.Name = entry.Name() - fh.Method = zip.Deflate - w, err := zipWriter.CreateHeader(fh) - if err != nil { - return err - } - manualFile, err := os.Open(filepath.Join(path, entry.Name())) - if err != nil { - return err - } - defer manualFile.Close() - _, err = io.Copy(w, manualFile) - return err -} - -func getReleaseVersionData() (string, string) { - base, fossil := getVersionData() - if strings.HasSuffix(base, "dev") { - base = base[:len(base)-3] + "preview-" + time.Now().Format("20060102") - } - if strings.HasSuffix(fossil, dirtySuffix) { - fmt.Fprintf(os.Stderr, "Warning: releasing a dirty version %v\n", fossil) - base = base + dirtySuffix - } - return base, fossil -} - -func cmdRelease() error { - if err := cmdCheck(); err != nil { - return err - } - base, fossil := getReleaseVersionData() - releases := []struct { - arch string - os string - env []string - name string - }{ - {"amd64", "linux", nil, "zettelstore"}, - {"arm", "linux", []string{"GOARM=6"}, "zettelstore"}, - {"amd64", "darwin", nil, "iZettelstore"}, - {"arm64", "darwin", nil, "iZettelstore"}, - {"amd64", "windows", nil, "zettelstore.exe"}, - } - for _, rel := range releases { - env := append(rel.env, "GOARCH="+rel.arch, "GOOS="+rel.os) - zsName := filepath.Join("releases", rel.name) - if err := doBuild(env, calcVersion(base, fossil), zsName); err != nil { - return err - } - zipName := fmt.Sprintf("zettelstore-%v-%v-%v.zip", base, rel.os, rel.arch) - if err := createReleaseZip(zsName, zipName, rel.name); err != nil { - return err - } - if err := os.Remove(zsName); err != nil { - return err - } - } - return createManualZip("releases", base) -} - -func createReleaseZip(zsName, zipName, fileName string) error { - zipFile, err := os.OpenFile(filepath.Join("releases", zipName), os.O_RDWR|os.O_CREATE, 0600) - if err != nil { - return err - } - defer zipFile.Close() - zw := zip.NewWriter(zipFile) - defer zw.Close() - err = addFileToZip(zw, zsName, fileName) - if err != nil { - return err - } - err = addFileToZip(zw, "LICENSE.txt", "LICENSE.txt") - if err != nil { - return err - } - err = addFileToZip(zw, "docs/readmezip.txt", "README.txt") - return err -} - -func addFileToZip(zipFile *zip.Writer, filepath, filename string) error { - zsFile, err := os.Open(filepath) - if err != nil { - return err - } - defer zsFile.Close() - stat, err := zsFile.Stat() - if err != nil { - return err - } - fh, err := zip.FileInfoHeader(stat) - if err != nil { - return err - } - fh.Name = filename - fh.Method = zip.Deflate - w, err := zipFile.CreateHeader(fh) - if err != nil { - return err - } - _, err = io.Copy(w, zsFile) - return err -} - -func cmdClean() error { - for _, dir := range []string{"bin", "releases"} { - err := os.RemoveAll(dir) - if err != nil { - return err - } - } - return nil -} - -func cmdHelp() { - fmt.Println(`Usage: go run tools/build.go [-v] COMMAND - -Options: - -v Verbose output. - -Commands: - build Build the software for local computer. - check Check current working state: execute tests, static analysis tools, - extra files, ... - Is automatically done when releasing the software. - clean Remove all build and release directories. - help Outputs this text. - manual Create a ZIP file with all manual zettel - release Create the software for various platforms and put them in - appropriate named ZIP files. - version Print the current version of the software. - -All commands can be abbreviated as long as they remain unique.`) -} - -var ( - verbose bool -) - -func main() { - flag.BoolVar(&verbose, "v", false, "Verbose output") - flag.Parse() - var err error - args := flag.Args() - if len(args) < 1 { - cmdHelp() - } else { - switch args[0] { - case "b", "bu", "bui", "buil", "build": - err = cmdBuild() - case "m", "ma", "man", "manu", "manua", "manual": - err = cmdManual() - case "r", "re", "rel", "rele", "relea", "releas", "release": - err = cmdRelease() - case "cl", "cle", "clea", "clean": - err = cmdClean() - case "v", "ve", "ver", "vers", "versi", "versio", "version": - fmt.Print(getVersion()) - case "ch", "che", "chec", "check": - err = cmdCheck() - case "h", "he", "hel", "help": - cmdHelp() - default: - fmt.Fprintf(os.Stderr, "Unknown command %q\n", args[0]) - cmdHelp() - os.Exit(1) - } - } - if err != nil { - fmt.Fprintln(os.Stderr, err) - } -} ADDED tools/build/build.go Index: tools/build/build.go ================================================================== --- /dev/null +++ tools/build/build.go @@ -0,0 +1,327 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2021-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2021-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package main provides a command to build and run the software. +package main + +import ( + "archive/zip" + "bytes" + "flag" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "slices" + "strings" + "time" + + zerostrings "t73f.de/r/zero/strings" + "t73f.de/r/zsc/domain/id" + "t73f.de/r/zsc/domain/meta" + "t73f.de/r/zsx/input" + + "zettelstore.de/z/tools" +) + +func readVersionFile() (string, error) { + content, err := os.ReadFile("VERSION") + if err != nil { + return "", err + } + return strings.TrimFunc(string(content), func(r rune) bool { + return r <= ' ' + }), nil +} + +func getVersion() string { + base, err := readVersionFile() + if err != nil { + base = "dev" + } + return base +} + +var dirtyPrefixes = []string{ + "DELETED ", "ADDED ", "UPDATED ", "CONFLICT ", "EDITED ", "RENAMED ", "EXTRA "} + +const dirtySuffix = "-dirty" + +func readFossilDirty() (string, error) { + s, err := tools.ExecuteCommand(nil, "fossil", "status", "--differ") + if err != nil { + return "", err + } + for _, line := range zerostrings.SplitLines(s) { + for _, prefix := range dirtyPrefixes { + if strings.HasPrefix(line, prefix) { + return dirtySuffix, nil + } + } + } + return "", nil +} + +func getFossilDirty() string { + fossil, err := readFossilDirty() + if err != nil { + return "" + } + return fossil +} + +func cmdBuild() error { + return doBuild(tools.EnvDirectProxy, getVersion(), "bin/zettelstore") +} + +func doBuild(env []string, version, target string) error { + env = append(env, "CGO_ENABLED=0") + env = append(env, tools.EnvGoVCS...) + out, err := tools.ExecuteCommand( + env, + "go", "build", + "-tags", "osusergo,netgo", + "-trimpath", + "-ldflags", fmt.Sprintf("-X main.version=%v -w", version), + "-o", target, + "zettelstore.de/z/cmd/zettelstore", + ) + if err != nil { + return err + } + if len(out) > 0 { + fmt.Println(out) + } + return nil +} + +func cmdHelp() { + fmt.Println(`Usage: go run tools/build/build.go [-v] COMMAND + +Options: + -v Verbose output. + +Commands: + build Build the software for local computer. + help Output this text. + manual Create a ZIP file with all manual zettel + release Create the software for various platforms and put them in + appropriate named ZIP files. + version Print the current version of the software. + +All commands can be abbreviated as long as they remain unique.`) +} + +func main() { + flag.BoolVar(&tools.Verbose, "v", false, "Verbose output") + flag.Parse() + var err error + args := flag.Args() + if len(args) < 1 { + cmdHelp() + } else { + switch args[0] { + case "b", "bu", "bui", "buil", "build": + err = cmdBuild() + case "m", "ma", "man", "manu", "manua", "manual": + err = cmdManual() + case "r", "re", "rel", "rele", "relea", "releas", "release": + err = cmdRelease() + case "v", "ve", "ver", "vers", "versi", "versio", "version": + fmt.Print(getVersion()) + case "h", "he", "hel", "help": + cmdHelp() + default: + fmt.Fprintf(os.Stderr, "Unknown command %q\n", args[0]) + cmdHelp() + os.Exit(1) + } + } + if err != nil { + fmt.Fprintln(os.Stderr, err) + } +} + +// --- manual + +func cmdManual() error { + base := getReleaseVersionData() + return createManualZip(".", base) +} + +func createManualZip(path, base string) error { + manualPath := filepath.Join("docs", "manual") + entries, err := os.ReadDir(manualPath) + if err != nil { + return err + } + zipName := filepath.Join(path, "manual-"+base+".zip") + zipFile, err := os.OpenFile(zipName, os.O_RDWR|os.O_CREATE, 0600) + if err != nil { + return err + } + defer func() { _ = zipFile.Close() }() + zipWriter := zip.NewWriter(zipFile) + defer func() { _ = zipWriter.Close() }() + + for _, entry := range entries { + if err = createManualZipEntry(manualPath, entry, zipWriter); err != nil { + return err + } + } + return nil +} + +const versionZid = "00001000000001" + +func createManualZipEntry(path string, entry fs.DirEntry, zipWriter *zip.Writer) error { + info, err := entry.Info() + if err != nil { + return err + } + fh, err := zip.FileInfoHeader(info) + if err != nil { + return err + } + name := entry.Name() + fh.Name = name + fh.Method = zip.Deflate + w, err := zipWriter.CreateHeader(fh) + if err != nil { + return err + } + manualFile, err := os.Open(filepath.Join(path, name)) + if err != nil { + return err + } + defer func() { _ = manualFile.Close() }() + + if name != versionZid+".zettel" { + _, err = io.Copy(w, manualFile) + return err + } + + data, err := io.ReadAll(manualFile) + if err != nil { + return err + } + inp := input.NewInput(data) + m := meta.NewFromInput(id.MustParse(versionZid), inp) + m.SetNow(meta.KeyModified) + + var buf bytes.Buffer + if _, err = fmt.Fprintf(&buf, "id: %s\n", versionZid); err != nil { + return err + } + if _, err = m.WriteComputed(&buf); err != nil { + return err + } + if _, err = fmt.Fprintf(&buf, "\n%s", getVersion()); err != nil { + return err + } + _, err = io.Copy(w, &buf) + return err +} + +//--- release + +func cmdRelease() error { + if err := tools.Check(true); err != nil { + return err + } + base := getReleaseVersionData() + releases := []struct { + arch string + os string + env []string + name string + }{ + {"amd64", "linux", nil, "zettelstore"}, + {"arm", "linux", []string{"GOARM=6"}, "zettelstore"}, + {"arm64", "darwin", nil, "zettelstore"}, + {"amd64", "darwin", nil, "zettelstore"}, + {"amd64", "windows", nil, "zettelstore.exe"}, + {"arm64", "android", nil, "zettelstore"}, + } + for _, rel := range releases { + env := slices.Clone(rel.env) + env = append(env, "GOARCH="+rel.arch, "GOOS="+rel.os) + env = append(env, tools.EnvDirectProxy...) + env = append(env, tools.EnvGoVCS...) + zsName := filepath.Join("releases", rel.name) + if err := doBuild(env, base, zsName); err != nil { + return err + } + zipName := fmt.Sprintf("zettelstore-%v-%v-%v.zip", base, rel.os, rel.arch) + if err := createReleaseZip(zsName, zipName, rel.name); err != nil { + return err + } + if err := os.Remove(zsName); err != nil { + return err + } + } + return createManualZip("releases", base) +} + +func getReleaseVersionData() string { + if fossil := getFossilDirty(); fossil != "" { + fmt.Fprintln(os.Stderr, "Warning: releasing a dirty version") + } + base := getVersion() + if strings.HasSuffix(base, "dev") { + return base[:len(base)-3] + "preview-" + time.Now().Local().Format("20060102") + } + return base +} + +func createReleaseZip(zsName, zipName, fileName string) error { + zipFile, err := os.OpenFile(filepath.Join("releases", zipName), os.O_RDWR|os.O_CREATE, 0600) + if err != nil { + return err + } + defer func() { _ = zipFile.Close() }() + zw := zip.NewWriter(zipFile) + defer func() { _ = zw.Close() }() + if err = addFileToZip(zw, zsName, fileName); err != nil { + return err + } + if err = addFileToZip(zw, "LICENSE.txt", "LICENSE.txt"); err != nil { + return err + } + return addFileToZip(zw, "docs/readmezip.txt", "README.txt") +} + +func addFileToZip(zipFile *zip.Writer, filepath, filename string) error { + zsFile, err := os.Open(filepath) + if err != nil { + return err + } + defer func() { _ = zsFile.Close() }() + stat, err := zsFile.Stat() + if err != nil { + return err + } + fh, err := zip.FileInfoHeader(stat) + if err != nil { + return err + } + fh.Name = filename + fh.Method = zip.Deflate + w, err := zipFile.CreateHeader(fh) + if err != nil { + return err + } + _, err = io.Copy(w, zsFile) + return err +} ADDED tools/check/check.go Index: tools/check/check.go ================================================================== --- /dev/null +++ tools/check/check.go @@ -0,0 +1,35 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2023-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2023-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package main provides a command to execute unit tests. +package main + +import ( + "flag" + "fmt" + "os" + + "zettelstore.de/z/tools" +) + +var release bool + +func main() { + flag.BoolVar(&release, "r", false, "Release check") + flag.BoolVar(&tools.Verbose, "v", false, "Verbose output") + flag.Parse() + + if err := tools.Check(release); err != nil { + fmt.Fprintln(os.Stderr, err) + } +} ADDED tools/clean/clean.go Index: tools/clean/clean.go ================================================================== --- /dev/null +++ tools/clean/clean.go @@ -0,0 +1,56 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2023-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2023-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package main provides a command to clean / remove development artifacts. +package main + +import ( + "flag" + "fmt" + "os" + + "zettelstore.de/z/tools" +) + +func main() { + flag.BoolVar(&tools.Verbose, "v", false, "Verbose output") + flag.Parse() + + if err := cmdClean(); err != nil { + fmt.Fprintln(os.Stderr, err) + } +} + +func cmdClean() error { + for _, dir := range []string{"bin", "releases"} { + err := os.RemoveAll(dir) + if err != nil { + return err + } + } + out, err := tools.ExecuteCommand(nil, "go", "clean", "./...") + if err != nil { + return err + } + if len(out) > 0 { + fmt.Println(out) + } + out, err = tools.ExecuteCommand(nil, "go", "clean", "-cache", "-modcache", "-testcache") + if err != nil { + return err + } + if len(out) > 0 { + fmt.Println(out) + } + return nil +} ADDED tools/devtools/devtools.go Index: tools/devtools/devtools.go ================================================================== --- /dev/null +++ tools/devtools/devtools.go @@ -0,0 +1,61 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2023-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2023-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package main provides a command to install development tools. +package main + +import ( + "flag" + "fmt" + "os" + + "zettelstore.de/z/tools" +) + +func main() { + flag.BoolVar(&tools.Verbose, "v", false, "Verbose output") + flag.Parse() + + if err := cmdTools(); err != nil { + fmt.Fprintln(os.Stderr, err) + } +} + +func cmdTools() error { + tools := []struct{ name, pack string }{ + {"shadow", "golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest"}, + {"unparam", "mvdan.cc/unparam@latest"}, + {"staticcheck", "honnef.co/go/tools/cmd/staticcheck@latest"}, + {"govulncheck", "golang.org/x/vuln/cmd/govulncheck@latest"}, + {"deadcode", "golang.org/x/tools/cmd/deadcode@latest"}, + {"errcheck", "github.com/kisielk/errcheck@latest"}, + {"revive", "github.com/mgechev/revive@latest"}, + } + for _, tool := range tools { + err := doGoInstall(tool.pack) + if err != nil { + return err + } + } + return nil +} +func doGoInstall(pack string) error { + out, err := tools.ExecuteCommand(nil, "go", "install", pack) + if err != nil { + fmt.Fprintln(os.Stderr, "Unable to install package", pack) + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + return err +} ADDED tools/htmllint/htmllint.go Index: tools/htmllint/htmllint.go ================================================================== --- /dev/null +++ tools/htmllint/htmllint.go @@ -0,0 +1,206 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2023-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2023-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package main provides a tool to check the validity of HTML zettel documents. +package main + +import ( + "context" + "flag" + "fmt" + "log" + "math/rand/v2" + "net/url" + "os" + "regexp" + "slices" + "strings" + + "t73f.de/r/zsc/api" + "t73f.de/r/zsc/client" + "t73f.de/r/zsc/domain/id" + "zettelstore.de/z/tools" +) + +func main() { + flag.BoolVar(&tools.Verbose, "v", false, "Verbose output") + flag.Parse() + + if err := cmdValidateHTML(flag.Args()); err != nil { + fmt.Fprintln(os.Stderr, err) + } +} +func cmdValidateHTML(args []string) error { + rawURL := "http://localhost:23123" + if len(args) > 0 { + rawURL = args[0] + } + u, err := url.Parse(rawURL) + if err != nil { + return err + } + client := client.NewClient(u) + _, _, metaList, err := client.QueryZettelData(context.Background(), "") + if err != nil { + return err + } + zids, perm := calculateZids(metaList) + for _, kd := range keyDescr { + msgCount := 0 + fmt.Fprintf(os.Stderr, "Now checking: %s\n", kd.text) + for _, zid := range zidsToUse(zids, perm, kd.sampleSize) { + var nmsgs int + nmsgs, err = validateHTML(client, kd.uc, zid) + if err != nil { + fmt.Fprintf(os.Stderr, "* error while validating zettel %v with: %v\n", zid, err) + msgCount++ + } else { + msgCount += nmsgs + } + } + if msgCount == 1 { + fmt.Fprintln(os.Stderr, "==> found 1 possible issue") + } else if msgCount > 1 { + fmt.Fprintf(os.Stderr, "==> found %v possible issues\n", msgCount) + } + } + return nil +} + +func calculateZids(metaList []api.ZidMetaRights) ([]id.Zid, []int) { + zids := make([]id.Zid, len(metaList)) + for i, m := range metaList { + zids[i] = m.ID + } + slices.Sort(zids) + return zids, rand.Perm(len(metaList)) +} + +func zidsToUse(zids []id.Zid, perm []int, sampleSize int) []id.Zid { + if sampleSize < 0 || len(perm) <= sampleSize { + return zids + } + if sampleSize == 0 { + return nil + } + result := make([]id.Zid, sampleSize) + for i := range sampleSize { + result[i] = zids[perm[i]] + } + slices.Sort(result) + return result +} + +var keyDescr = []struct { + uc urlCreator + text string + sampleSize int +}{ + {getHTMLZettel, "zettel HTML encoding", -1}, + {createJustKey('h'), "zettel web view", -1}, + {createJustKey('i'), "zettel info view", -1}, + {createJustKey('e'), "zettel edit form", 100}, + {createJustKey('c'), "zettel create form", 10}, + {createJustKey('d'), "zettel delete dialog", 200}, +} + +type urlCreator func(*client.Client, id.Zid) *api.URLBuilder + +func createJustKey(key byte) urlCreator { + return func(c *client.Client, zid id.Zid) *api.URLBuilder { + return c.NewURLBuilder(key).SetZid(zid) + } +} + +func getHTMLZettel(client *client.Client, zid id.Zid) *api.URLBuilder { + return client.NewURLBuilder('z').SetZid(zid). + AppendKVQuery(api.QueryKeyEncoding, api.EncodingHTML). + AppendKVQuery(api.QueryKeyPart, api.PartZettel) +} + +func validateHTML(client *client.Client, uc urlCreator, zid id.Zid) (int, error) { + ub := uc(client, zid) + if tools.Verbose { + fmt.Fprintf(os.Stderr, "GET %v\n", ub) + } + data, err := client.Get(context.Background(), ub) + if err != nil { + return 0, err + } + if len(data) == 0 { + return 0, nil + } + _, stderr, err := tools.ExecuteFilter(data, nil, "tidy", "-e", "-q", "-lang", "en") + if err != nil { + switch err.Error() { + case "exit status 1": + case "exit status 2": + default: + log.Println("SERR", stderr) + return 0, err + } + } + if stderr == "" { + return 0, nil + } + if msgs := filterTidyMessages(strings.Split(stderr, "\n")); len(msgs) > 0 { + fmt.Fprintln(os.Stderr, zid) + for _, msg := range msgs { + fmt.Fprintln(os.Stderr, "-", msg) + } + return len(msgs), nil + } + return 0, nil +} + +var reLine = regexp.MustCompile(`line \d+ column \d+ - (.+): (.+)`) + +func filterTidyMessages(lines []string) []string { + result := make([]string, 0, len(lines)) + for _, line := range lines { + line = strings.TrimSpace(line) + if line == "" { + continue + } + matches := reLine.FindStringSubmatch(line) + if len(matches) <= 1 { + if line == "This document has errors that must be fixed before" || + line == "using HTML Tidy to generate a tidied up version." { + continue + } + result = append(result, "!!!"+line) + continue + } + if matches[1] == "Error" { + if len(matches) > 2 { + if matches[2] == "<search> is not recognized!" { + continue + } + } + } + if matches[1] != "Warning" { + result = append(result, "???"+line) + continue + } + if len(matches) > 2 { + switch matches[2] { + case "discarding unexpected <search>", + "discarding unexpected </search>", + `<input> proprietary attribute "inputmode"`: + continue + } + } + result = append(result, line) + } + return result +} ADDED tools/testapi/testapi.go Index: tools/testapi/testapi.go ================================================================== --- /dev/null +++ tools/testapi/testapi.go @@ -0,0 +1,110 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2023-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2023-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package main provides a command to test the API +package main + +import ( + "errors" + "flag" + "fmt" + "io" + "net" + "os" + "os/exec" + "strings" + "time" + + "zettelstore.de/z/tools" +) + +func main() { + flag.BoolVar(&tools.Verbose, "v", false, "Verbose output") + flag.Parse() + + if err := cmdTestAPI(); err != nil { + fmt.Fprintln(os.Stderr, err) + } +} + +type zsInfo struct { + cmd *exec.Cmd + out strings.Builder + adminAddress string +} + +func cmdTestAPI() error { + var err error + var info zsInfo + needServer := !addressInUse(":23123") + if needServer { + err = startZettelstore(&info) + } + if err != nil { + return err + } + err = tools.CheckGoTest("zettelstore.de/z/tests/client", "-base-url", "http://127.0.0.1:23123") + if needServer { + err1 := stopZettelstore(&info) + if err == nil { + err = err1 + } + } + return err +} + +func startZettelstore(info *zsInfo) error { + info.adminAddress = ":2323" + name, arg := "go", []string{ + "run", "cmd/zettelstore/main.go", "run", + "-c", "./testdata/testbox/19700101000000.zettel", "-a", info.adminAddress[1:]} + tools.LogCommand("FORK", nil, name, arg) + cmd := tools.PrepareCommand(tools.EnvGoVCS, name, arg, nil, &info.out, os.Stderr) + if !tools.Verbose { + cmd.Stderr = nil + } + err := cmd.Start() + time.Sleep(2 * time.Second) + for range 100 { + time.Sleep(time.Millisecond * 100) + if addressInUse(info.adminAddress) { + info.cmd = cmd + return err + } + } + time.Sleep(4 * time.Second) // Wait for all zettel to be indexed. + return errors.New("zettelstore did not start") +} + +func stopZettelstore(i *zsInfo) error { + conn, err := net.Dial("tcp", i.adminAddress) + if err != nil { + fmt.Println("Unable to stop Zettelstore") + return err + } + _, err = io.WriteString(conn, "shutdown\n") + _ = conn.Close() + if err == nil { + err = i.cmd.Wait() + } + return err +} + +func addressInUse(address string) bool { + conn, err := net.Dial("tcp", address) + if err != nil { + return false + } + _ = conn.Close() + return true +} ADDED tools/tools.go Index: tools/tools.go ================================================================== --- /dev/null +++ tools/tools.go @@ -0,0 +1,254 @@ +//----------------------------------------------------------------------------- +// Copyright (c) 2023-present Detlef Stern +// +// This file is part of Zettelstore. +// +// Zettelstore is licensed under the latest version of the EUPL (European Union +// Public License). Please see file LICENSE.txt for your rights and obligations +// under this license. +// +// SPDX-License-Identifier: EUPL-1.2 +// SPDX-FileCopyrightText: 2023-present Detlef Stern +//----------------------------------------------------------------------------- + +// Package tools provides a collection of functions to build needed tools. +package tools + +import ( + "bytes" + "errors" + "fmt" + "io" + "os" + "os/exec" + "strings" + + zerostrings "t73f.de/r/zero/strings" +) + +// Some constants to make Go work with fossil. +var ( + EnvDirectProxy = []string{"GOPROXY=direct"} + EnvGoVCS = []string{"GOVCS=zettelstore.de:fossil,t73f.de:fossil"} +) + +// Verbose signals a verbose tool execution. +var Verbose bool + +// ExecuteCommand executes a specific command. +func ExecuteCommand(env []string, name string, arg ...string) (string, error) { + LogCommand("EXEC", env, name, arg) + var out strings.Builder + cmd := PrepareCommand(env, name, arg, nil, &out, os.Stderr) + err := cmd.Run() + return out.String(), err +} + +// ExecuteFilter executes an external program to be used as a filter. +func ExecuteFilter(data []byte, env []string, name string, arg ...string) (string, string, error) { + LogCommand("EXEC", env, name, arg) + var stdout, stderr strings.Builder + cmd := PrepareCommand(env, name, arg, bytes.NewReader(data), &stdout, &stderr) + err := cmd.Run() + return stdout.String(), stderr.String(), err +} + +// PrepareCommand creates a commands to be executed. +func PrepareCommand(env []string, name string, arg []string, in io.Reader, stdout, stderr io.Writer) *exec.Cmd { + if len(env) > 0 { + env = append(env, os.Environ()...) + } + cmd := exec.Command(name, arg...) + cmd.Env = env + cmd.Stdin = in + cmd.Stdout = stdout + cmd.Stderr = stderr + return cmd +} + +// LogCommand logs the execution of a command. +func LogCommand(exec string, env []string, name string, arg []string) { + if Verbose { + if len(env) > 0 { + for i, e := range env { + fmt.Fprintf(os.Stderr, "ENV%d %v\n", i+1, e) + } + } + fmt.Fprintln(os.Stderr, exec, name, arg) + } +} + +// Check the source with some linters. +func Check(forRelease bool) error { + if err := CheckGoTest("./..."); err != nil { + return err + } + if err := checkGoVet(); err != nil { + return err + } + if err := checkShadow(forRelease); err != nil { + return err + } + if err := checkStaticcheck(); err != nil { + return err + } + if err := checkUnparam(forRelease); err != nil { + return err + } + if err := checkRevive(); err != nil { + return err + } + if err := checkErrCheck(); err != nil { + return err + } + if forRelease { + if err := checkGoVulncheck(); err != nil { + return err + } + } + return checkFossilExtra() +} + +// CheckGoTest runs all internal unti tests. +func CheckGoTest(pkg string, testParams ...string) error { + var env []string + env = append(env, EnvDirectProxy...) + env = append(env, EnvGoVCS...) + args := []string{"test", pkg} + args = append(args, testParams...) + out, err := ExecuteCommand(env, "go", args...) + if err != nil { + for _, line := range zerostrings.SplitLines(out) { + if strings.HasPrefix(line, "ok") || strings.HasPrefix(line, "?") { + continue + } + fmt.Fprintln(os.Stderr, line) + } + } + return err +} +func checkGoVet() error { + out, err := ExecuteCommand(EnvGoVCS, "go", "vet", "./...") + if err != nil { + fmt.Fprintln(os.Stderr, "Some checks failed") + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + return err +} + +func checkShadow(forRelease bool) error { + path, err := findExecStrict("shadow", forRelease) + if path == "" { + return err + } + out, err := ExecuteCommand(EnvGoVCS, path, "-strict", "./...") + if err != nil { + fmt.Fprintln(os.Stderr, "Some shadowed variables found") + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + return err +} + +func checkStaticcheck() error { + out, err := ExecuteCommand(EnvGoVCS, "staticcheck", "./...") + if err != nil { + fmt.Fprintln(os.Stderr, "Some staticcheck problems found") + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + return err +} + +func checkRevive() error { + out, err := ExecuteCommand(EnvGoVCS, "revive", "./...") + if err != nil || out != "" { + fmt.Fprintln(os.Stderr, "Some revive problems found") + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + return err +} + +func checkErrCheck() error { + out, err := ExecuteCommand(EnvGoVCS, "errcheck", "./...") + if err != nil || out != "" { + fmt.Fprintln(os.Stderr, "Some errcheck problems found") + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + return err +} + +func checkUnparam(forRelease bool) error { + path, err := findExecStrict("unparam", forRelease) + if path == "" { + return err + } + out, err := ExecuteCommand(EnvGoVCS, path, "./...") + if err != nil { + fmt.Fprintln(os.Stderr, "Some unparam problems found") + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + if forRelease { + if out2, err2 := ExecuteCommand(nil, path, "-exported", "-tests", "./..."); err2 != nil { + fmt.Fprintln(os.Stderr, "Some optional unparam problems found") + if len(out2) > 0 { + fmt.Fprintln(os.Stderr, out2) + } + } + } + return err +} + +func checkGoVulncheck() error { + out, err := ExecuteCommand(EnvGoVCS, "govulncheck", "./...") + if err != nil { + fmt.Fprintln(os.Stderr, "Some checks failed") + if len(out) > 0 { + fmt.Fprintln(os.Stderr, out) + } + } + return err +} +func findExec(cmd string) string { + if path, err := ExecuteCommand(nil, "which", cmd); err == nil && path != "" { + return strings.TrimSpace(path) + } + return "" +} + +func findExecStrict(cmd string, forRelease bool) (string, error) { + path := findExec(cmd) + if path != "" || !forRelease { + return path, nil + } + return "", errors.New("Command '" + cmd + "' not installed, but required for release") +} + +func checkFossilExtra() error { + out, err := ExecuteCommand(nil, "fossil", "extra") + if err != nil { + fmt.Fprintln(os.Stderr, "Unable to execute 'fossil extra'") + return err + } + if len(out) > 0 { + fmt.Fprint(os.Stderr, "Warning: unversioned file(s):") + for i, extra := range zerostrings.SplitLines(out) { + if i > 0 { + fmt.Fprint(os.Stderr, ",") + } + fmt.Fprintf(os.Stderr, " %q", extra) + } + fmt.Fprintln(os.Stderr) + } + return nil +} DELETED usecase/authenticate.go Index: usecase/authenticate.go ================================================================== --- usecase/authenticate.go +++ /dev/null @@ -1,92 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - "math/rand" - "time" - - "zettelstore.de/z/auth" - "zettelstore.de/z/auth/cred" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/search" -) - -// AuthenticatePort is the interface used by this use case. -type AuthenticatePort interface { - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) - SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) -} - -// Authenticate is the data for this use case. -type Authenticate struct { - token auth.TokenManager - port AuthenticatePort - ucGetUser GetUser -} - -// NewAuthenticate creates a new use case. -func NewAuthenticate(token auth.TokenManager, authz auth.AuthzManager, port AuthenticatePort) Authenticate { - return Authenticate{ - token: token, - port: port, - ucGetUser: NewGetUser(authz, port), - } -} - -// Run executes the use case. -func (uc Authenticate) Run(ctx context.Context, ident, credential string, d time.Duration, k auth.TokenKind) ([]byte, error) { - identMeta, err := uc.ucGetUser.Run(ctx, ident) - defer addDelay(time.Now(), 500*time.Millisecond, 100*time.Millisecond) - - if identMeta == nil || err != nil { - compensateCompare() - return nil, err - } - - if hashCred, ok := identMeta.Get(meta.KeyCredential); ok { - ok, err := cred.CompareHashAndCredential(hashCred, identMeta.Zid, ident, credential) - if err != nil { - return nil, err - } - if ok { - token, err := uc.token.GetToken(identMeta, d, k) - if err != nil { - return nil, err - } - return token, nil - } - return nil, nil - } - compensateCompare() - return nil, nil -} - -// compensateCompare if normal comapare is not possible, to avoid timing hints. -func compensateCompare() { - cred.CompareHashAndCredential( - "$2a$10$WHcSO3G9afJ3zlOYQR1suuf83bCXED2jmzjti/MH4YH4l2mivDuze", id.Invalid, "", "") -} - -// addDelay after credential checking to allow some CPU time for other tasks. -// durDelay is the normal delay, if time spend for checking is smaller than -// the minimum delay minDelay. In addition some jitter (+/- 50 ms) is added. -func addDelay(start time.Time, durDelay, minDelay time.Duration) { - jitter := time.Duration(rand.Intn(100)-50) * time.Millisecond - if elapsed := time.Since(start); elapsed+minDelay < durDelay { - time.Sleep(durDelay - elapsed + jitter) - } else { - time.Sleep(minDelay + jitter) - } -} DELETED usecase/context.go Index: usecase/context.go ================================================================== --- usecase/context.go +++ /dev/null @@ -1,168 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the Zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -// ZettelContextPort is the interface used by this use case. -type ZettelContextPort interface { - // GetMeta retrieves just the meta data of a specific zettel. - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) -} - -// ZettelContext is the data for this use case. -type ZettelContext struct { - port ZettelContextPort -} - -// NewZettelContext creates a new use case. -func NewZettelContext(port ZettelContextPort) ZettelContext { - return ZettelContext{port: port} -} - -// ZettelContextDirection determines the way, the context is calculated. -type ZettelContextDirection int - -// Constant values for ZettelContextDirection -const ( - _ ZettelContextDirection = iota - ZettelContextForward // Traverse all forwarding links - ZettelContextBackward // Traverse all backwaring links - ZettelContextBoth // Traverse both directions -) - -// ParseZCDirection returns a direction value for a given string. -func ParseZCDirection(s string) ZettelContextDirection { - switch s { - case "backward": - return ZettelContextBackward - case "forward": - return ZettelContextForward - } - return ZettelContextBoth -} - -// Run executes the use case. -func (uc ZettelContext) Run(ctx context.Context, zid id.Zid, dir ZettelContextDirection, depth, limit int) (result []*meta.Meta, err error) { - start, err := uc.port.GetMeta(ctx, zid) - if err != nil { - return nil, err - } - tasks := ztlCtx{depth: depth} - uc.addInitialTasks(ctx, &tasks, start) - visited := id.NewSet() - isBackward := dir == ZettelContextBoth || dir == ZettelContextBackward - isForward := dir == ZettelContextBoth || dir == ZettelContextForward - for !tasks.empty() { - m, curDepth := tasks.pop() - if _, ok := visited[m.Zid]; ok { - continue - } - visited[m.Zid] = true - result = append(result, m) - if limit > 0 && len(result) > limit { // start is the first element of result - break - } - curDepth++ - for _, p := range m.PairsRest(true) { - if p.Key == meta.KeyBackward { - if isBackward { - uc.addIDSet(ctx, &tasks, curDepth, p.Value) - } - continue - } - if p.Key == meta.KeyForward { - if isForward { - uc.addIDSet(ctx, &tasks, curDepth, p.Value) - } - continue - } - if p.Key != meta.KeyBack { - hasInverse := meta.Inverse(p.Key) != "" - if (!hasInverse || !isBackward) && (hasInverse || !isForward) { - continue - } - if t := meta.Type(p.Key); t == meta.TypeID { - uc.addID(ctx, &tasks, curDepth, p.Value) - } else if t == meta.TypeIDSet { - uc.addIDSet(ctx, &tasks, curDepth, p.Value) - } - } - } - } - return result, nil -} - -func (uc ZettelContext) addInitialTasks(ctx context.Context, tasks *ztlCtx, start *meta.Meta) { - tasks.add(start, 0) -} - -func (uc ZettelContext) addID(ctx context.Context, tasks *ztlCtx, depth int, value string) { - if zid, err := id.Parse(value); err == nil { - if m, err := uc.port.GetMeta(ctx, zid); err == nil { - tasks.add(m, depth) - } - } -} - -func (uc ZettelContext) addIDSet(ctx context.Context, tasks *ztlCtx, depth int, value string) { - for _, val := range meta.ListFromValue(value) { - uc.addID(ctx, tasks, depth, val) - } -} - -type ztlCtxTask struct { - next *ztlCtxTask - meta *meta.Meta - depth int -} - -type ztlCtx struct { - first *ztlCtxTask - last *ztlCtxTask - depth int -} - -func (zc *ztlCtx) add(m *meta.Meta, depth int) { - if zc.depth > 0 && depth > zc.depth { - return - } - task := &ztlCtxTask{next: nil, meta: m, depth: depth} - if zc.first == nil { - zc.first = task - zc.last = task - } else { - zc.last.next = task - zc.last = task - } -} - -func (zc *ztlCtx) empty() bool { - return zc.first == nil -} - -func (zc *ztlCtx) pop() (*meta.Meta, int) { - task := zc.first - if task == nil { - return nil, -1 - } - zc.first = task.next - if zc.first == nil { - zc.last = nil - } - return task.meta, task.depth -} DELETED usecase/copy_zettel.go Index: usecase/copy_zettel.go ================================================================== --- usecase/copy_zettel.go +++ /dev/null @@ -1,41 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/strfun" -) - -// CopyZettel is the data for this use case. -type CopyZettel struct{} - -// NewCopyZettel creates a new use case. -func NewCopyZettel() CopyZettel { - return CopyZettel{} -} - -// Run executes the use case. -func (uc CopyZettel) Run(origZettel domain.Zettel) domain.Zettel { - m := origZettel.Meta.Clone() - if title, ok := m.Get(meta.KeyTitle); ok { - if len(title) > 0 { - title = "Copy of " + title - } else { - title = "Copy" - } - m.Set(meta.KeyTitle, title) - } - content := strfun.TrimSpaceRight(origZettel.Content.AsString()) - return domain.Zettel{Meta: m, Content: domain.Content(content)} -} DELETED usecase/create_zettel.go Index: usecase/create_zettel.go ================================================================== --- usecase/create_zettel.go +++ /dev/null @@ -1,64 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/config" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/strfun" -) - -// CreateZettelPort is the interface used by this use case. -type CreateZettelPort interface { - // CreateZettel creates a new zettel. - CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) -} - -// CreateZettel is the data for this use case. -type CreateZettel struct { - rtConfig config.Config - port CreateZettelPort -} - -// NewCreateZettel creates a new use case. -func NewCreateZettel(rtConfig config.Config, port CreateZettelPort) CreateZettel { - return CreateZettel{ - rtConfig: rtConfig, - port: port, - } -} - -// Run executes the use case. -func (uc CreateZettel) Run(ctx context.Context, zettel domain.Zettel) (id.Zid, error) { - m := zettel.Meta - if m.Zid.IsValid() { - return m.Zid, nil // TODO: new error: already exists - } - - if title, ok := m.Get(meta.KeyTitle); !ok || title == "" { - m.Set(meta.KeyTitle, uc.rtConfig.GetDefaultTitle()) - } - if role, ok := m.Get(meta.KeyRole); !ok || role == "" { - m.Set(meta.KeyRole, uc.rtConfig.GetDefaultRole()) - } - if syntax, ok := m.Get(meta.KeySyntax); !ok || syntax == "" { - m.Set(meta.KeySyntax, uc.rtConfig.GetDefaultSyntax()) - } - m.YamlSep = uc.rtConfig.GetYAMLHeader() - - zettel.Content = domain.Content(strfun.TrimSpaceRight(zettel.Content.AsString())) - return uc.port.CreateZettel(ctx, zettel) -} DELETED usecase/delete_zettel.go Index: usecase/delete_zettel.go ================================================================== --- usecase/delete_zettel.go +++ /dev/null @@ -1,39 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain/id" -) - -// DeleteZettelPort is the interface used by this use case. -type DeleteZettelPort interface { - // DeleteZettel removes the zettel from the place. - DeleteZettel(ctx context.Context, zid id.Zid) error -} - -// DeleteZettel is the data for this use case. -type DeleteZettel struct { - port DeleteZettelPort -} - -// NewDeleteZettel creates a new use case. -func NewDeleteZettel(port DeleteZettelPort) DeleteZettel { - return DeleteZettel{port: port} -} - -// Run executes the use case. -func (uc DeleteZettel) Run(ctx context.Context, zid id.Zid) error { - return uc.port.DeleteZettel(ctx, zid) -} DELETED usecase/folge_zettel.go Index: usecase/folge_zettel.go ================================================================== --- usecase/folge_zettel.go +++ /dev/null @@ -1,48 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "zettelstore.de/z/config" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -// FolgeZettel is the data for this use case. -type FolgeZettel struct { - rtConfig config.Config -} - -// NewFolgeZettel creates a new use case. -func NewFolgeZettel(rtConfig config.Config) FolgeZettel { - return FolgeZettel{rtConfig} -} - -// Run executes the use case. -func (uc FolgeZettel) Run(origZettel domain.Zettel) domain.Zettel { - origMeta := origZettel.Meta - m := meta.New(id.Invalid) - if title, ok := origMeta.Get(meta.KeyTitle); ok { - if len(title) > 0 { - title = "Folge of " + title - } else { - title = "Folge" - } - m.Set(meta.KeyTitle, title) - } - m.Set(meta.KeyRole, config.GetRole(origMeta, uc.rtConfig)) - m.Set(meta.KeyTags, origMeta.GetDefault(meta.KeyTags, "")) - m.Set(meta.KeySyntax, uc.rtConfig.GetDefaultSyntax()) - m.Set(meta.KeyPrecursor, origMeta.Zid.String()) - return domain.Zettel{Meta: m, Content: ""} -} DELETED usecase/get_meta.go Index: usecase/get_meta.go ================================================================== --- usecase/get_meta.go +++ /dev/null @@ -1,40 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -// GetMetaPort is the interface used by this use case. -type GetMetaPort interface { - // GetMeta retrieves just the meta data of a specific zettel. - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) -} - -// GetMeta is the data for this use case. -type GetMeta struct { - port GetMetaPort -} - -// NewGetMeta creates a new use case. -func NewGetMeta(port GetMetaPort) GetMeta { - return GetMeta{port: port} -} - -// Run executes the use case. -func (uc GetMeta) Run(ctx context.Context, zid id.Zid) (*meta.Meta, error) { - return uc.port.GetMeta(ctx, zid) -} DELETED usecase/get_user.go Index: usecase/get_user.go ================================================================== --- usecase/get_user.go +++ /dev/null @@ -1,102 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/auth" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/search" -) - -// Use case: return user identified by meta key ident. -// --------------------------------------------------- - -// GetUserPort is the interface used by this use case. -type GetUserPort interface { - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) - SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) -} - -// GetUser is the data for this use case. -type GetUser struct { - authz auth.AuthzManager - port GetUserPort -} - -// NewGetUser creates a new use case. -func NewGetUser(authz auth.AuthzManager, port GetUserPort) GetUser { - return GetUser{authz: authz, port: port} -} - -// Run executes the use case. -func (uc GetUser) Run(ctx context.Context, ident string) (*meta.Meta, error) { - ctx = place.NoEnrichContext(ctx) - - // It is important to try first with the owner. First, because another user - // could give herself the same ''ident''. Second, in most cases the owner - // will authenticate. - identMeta, err := uc.port.GetMeta(ctx, uc.authz.Owner()) - if err == nil && identMeta.GetDefault(meta.KeyUserID, "") == ident { - if role, ok := identMeta.Get(meta.KeyRole); !ok || - role != meta.ValueRoleUser { - return nil, nil - } - return identMeta, nil - } - // Owner was not found or has another ident. Try via list search. - var s *search.Search - s = s.AddExpr(meta.KeyRole, meta.ValueRoleUser) - s = s.AddExpr(meta.KeyUserID, ident) - metaList, err := uc.port.SelectMeta(ctx, s) - if err != nil { - return nil, err - } - if len(metaList) < 1 { - return nil, nil - } - return metaList[len(metaList)-1], nil -} - -// Use case: return an user identified by zettel id and assert given ident value. -// ------------------------------------------------------------------------------ - -// GetUserByZidPort is the interface used by this use case. -type GetUserByZidPort interface { - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) -} - -// GetUserByZid is the data for this use case. -type GetUserByZid struct { - port GetUserByZidPort -} - -// NewGetUserByZid creates a new use case. -func NewGetUserByZid(port GetUserByZidPort) GetUserByZid { - return GetUserByZid{port: port} -} - -// GetUser executes the use case. -func (uc GetUserByZid) GetUser(ctx context.Context, zid id.Zid, ident string) (*meta.Meta, error) { - userMeta, err := uc.port.GetMeta(place.NoEnrichContext(ctx), zid) - if err != nil { - return nil, err - } - - if val, ok := userMeta.Get(meta.KeyUserID); !ok || val != ident { - return nil, nil - } - return userMeta, nil -} DELETED usecase/get_zettel.go Index: usecase/get_zettel.go ================================================================== --- usecase/get_zettel.go +++ /dev/null @@ -1,40 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" -) - -// GetZettelPort is the interface used by this use case. -type GetZettelPort interface { - // GetZettel retrieves a specific zettel. - GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) -} - -// GetZettel is the data for this use case. -type GetZettel struct { - port GetZettelPort -} - -// NewGetZettel creates a new use case. -func NewGetZettel(port GetZettelPort) GetZettel { - return GetZettel{port: port} -} - -// Run executes the use case. -func (uc GetZettel) Run(ctx context.Context, zid id.Zid) (domain.Zettel, error) { - return uc.port.GetZettel(ctx, zid) -} DELETED usecase/list_meta.go Index: usecase/list_meta.go ================================================================== --- usecase/list_meta.go +++ /dev/null @@ -1,40 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/search" -) - -// ListMetaPort is the interface used by this use case. -type ListMetaPort interface { - // SelectMeta returns all zettel meta data that match the selection criteria. - SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) -} - -// ListMeta is the data for this use case. -type ListMeta struct { - port ListMetaPort -} - -// NewListMeta creates a new use case. -func NewListMeta(port ListMetaPort) ListMeta { - return ListMeta{port: port} -} - -// Run executes the use case. -func (uc ListMeta) Run(ctx context.Context, s *search.Search) ([]*meta.Meta, error) { - return uc.port.SelectMeta(ctx, s) -} DELETED usecase/list_role.go Index: usecase/list_role.go ================================================================== --- usecase/list_role.go +++ /dev/null @@ -1,57 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - "sort" - - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/search" -) - -// ListRolePort is the interface used by this use case. -type ListRolePort interface { - // SelectMeta returns all zettel meta data that match the selection criteria. - SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) -} - -// ListRole is the data for this use case. -type ListRole struct { - port ListRolePort -} - -// NewListRole creates a new use case. -func NewListRole(port ListRolePort) ListRole { - return ListRole{port: port} -} - -// Run executes the use case. -func (uc ListRole) Run(ctx context.Context) ([]string, error) { - metas, err := uc.port.SelectMeta(place.NoEnrichContext(ctx), nil) - if err != nil { - return nil, err - } - roles := make(map[string]bool, 8) - for _, m := range metas { - if role, ok := m.Get(meta.KeyRole); ok && role != "" { - roles[role] = true - } - } - result := make([]string, 0, len(roles)) - for role := range roles { - result = append(result, role) - } - sort.Strings(result) - return result, nil -} DELETED usecase/list_tags.go Index: usecase/list_tags.go ================================================================== --- usecase/list_tags.go +++ /dev/null @@ -1,63 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/search" -) - -// ListTagsPort is the interface used by this use case. -type ListTagsPort interface { - // SelectMeta returns all zettel meta data that match the selection criteria. - SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) -} - -// ListTags is the data for this use case. -type ListTags struct { - port ListTagsPort -} - -// NewListTags creates a new use case. -func NewListTags(port ListTagsPort) ListTags { - return ListTags{port: port} -} - -// TagData associates tags with a list of all zettel meta that use this tag -type TagData map[string][]*meta.Meta - -// Run executes the use case. -func (uc ListTags) Run(ctx context.Context, minCount int) (TagData, error) { - metas, err := uc.port.SelectMeta(place.NoEnrichContext(ctx), nil) - if err != nil { - return nil, err - } - result := make(TagData) - for _, m := range metas { - if tl, ok := m.GetList(meta.KeyTags); ok && len(tl) > 0 { - for _, t := range tl { - result[t] = append(result[t], m) - } - } - } - if minCount > 1 { - for t, ms := range result { - if len(ms) < minCount { - delete(result, t) - } - } - } - return result, nil -} DELETED usecase/new_zettel.go Index: usecase/new_zettel.go ================================================================== --- usecase/new_zettel.go +++ /dev/null @@ -1,39 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "zettelstore.de/z/domain" - "zettelstore.de/z/strfun" -) - -// NewZettel is the data for this use case. -type NewZettel struct{} - -// NewNewZettel creates a new use case. -func NewNewZettel() NewZettel { - return NewZettel{} -} - -// Run executes the use case. -func (uc NewZettel) Run(origZettel domain.Zettel) domain.Zettel { - m := origZettel.Meta.Clone() - const prefix = "new-" - for _, pair := range m.PairsRest(false) { - if key := pair.Key; len(key) > len(prefix) && key[0:len(prefix)] == prefix { - m.Set(key[len(prefix):], pair.Value) - m.Delete(key) - } - } - content := strfun.TrimSpaceRight(origZettel.Content.AsString()) - return domain.Zettel{Meta: m, Content: domain.Content(content)} -} DELETED usecase/order.go Index: usecase/order.go ================================================================== --- usecase/order.go +++ /dev/null @@ -1,55 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the Zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/collect" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -// ZettelOrderPort is the interface used by this use case. -type ZettelOrderPort interface { - // GetMeta retrieves just the meta data of a specific zettel. - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) -} - -// ZettelOrder is the data for this use case. -type ZettelOrder struct { - port ZettelOrderPort - parseZettel ParseZettel -} - -// NewZettelOrder creates a new use case. -func NewZettelOrder(port ZettelOrderPort, parseZettel ParseZettel) ZettelOrder { - return ZettelOrder{port: port, parseZettel: parseZettel} -} - -// Run executes the use case. -func (uc ZettelOrder) Run( - ctx context.Context, zid id.Zid, syntax string, -) (start *meta.Meta, result []*meta.Meta, err error) { - zn, err := uc.parseZettel.Run(ctx, zid, syntax) - if err != nil { - return nil, nil, err - } - for _, ref := range collect.Order(zn) { - if zid, err := id.Parse(ref.URL.Path); err == nil { - if m, err := uc.port.GetMeta(ctx, zid); err == nil { - result = append(result, m) - } - } - } - return zn.Meta, result, nil -} DELETED usecase/parse_zettel.go Index: usecase/parse_zettel.go ================================================================== --- usecase/parse_zettel.go +++ /dev/null @@ -1,43 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/ast" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/parser" -) - -// ParseZettel is the data for this use case. -type ParseZettel struct { - rtConfig config.Config - getZettel GetZettel -} - -// NewParseZettel creates a new use case. -func NewParseZettel(rtConfig config.Config, getZettel GetZettel) ParseZettel { - return ParseZettel{rtConfig: rtConfig, getZettel: getZettel} -} - -// Run executes the use case. -func (uc ParseZettel) Run( - ctx context.Context, zid id.Zid, syntax string) (*ast.ZettelNode, error) { - zettel, err := uc.getZettel.Run(ctx, zid) - if err != nil { - return nil, err - } - - return parser.ParseZettel(zettel, syntax, uc.rtConfig), nil -} DELETED usecase/rename_zettel.go Index: usecase/rename_zettel.go ================================================================== --- usecase/rename_zettel.go +++ /dev/null @@ -1,62 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" -) - -// RenameZettelPort is the interface used by this use case. -type RenameZettelPort interface { - // GetMeta retrieves just the meta data of a specific zettel. - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) - - // Rename changes the current id to a new id. - RenameZettel(ctx context.Context, curZid, newZid id.Zid) error -} - -// RenameZettel is the data for this use case. -type RenameZettel struct { - port RenameZettelPort -} - -// ErrZidInUse is returned if the zettel id is not appropriate for the place operation. -type ErrZidInUse struct{ Zid id.Zid } - -func (err *ErrZidInUse) Error() string { - return "Zettel id already in use: " + err.Zid.String() -} - -// NewRenameZettel creates a new use case. -func NewRenameZettel(port RenameZettelPort) RenameZettel { - return RenameZettel{port: port} -} - -// Run executes the use case. -func (uc RenameZettel) Run(ctx context.Context, curZid, newZid id.Zid) error { - noEnrichCtx := place.NoEnrichContext(ctx) - if _, err := uc.port.GetMeta(noEnrichCtx, curZid); err != nil { - return err - } - if newZid == curZid { - // Nothing to do - return nil - } - if _, err := uc.port.GetMeta(noEnrichCtx, newZid); err == nil { - return &ErrZidInUse{Zid: newZid} - } - return uc.port.RenameZettel(ctx, curZid, newZid) -} DELETED usecase/search.go Index: usecase/search.go ================================================================== --- usecase/search.go +++ /dev/null @@ -1,44 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/search" -) - -// SearchPort is the interface used by this use case. -type SearchPort interface { - // SelectMeta returns all zettel meta data that match the selection criteria. - SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) -} - -// Search is the data for this use case. -type Search struct { - port SearchPort -} - -// NewSearch creates a new use case. -func NewSearch(port SearchPort) Search { - return Search{port: port} -} - -// Run executes the use case. -func (uc Search) Run(ctx context.Context, s *search.Search) ([]*meta.Meta, error) { - if !s.HasComputedMetaKey() { - ctx = place.NoEnrichContext(ctx) - } - return uc.port.SelectMeta(ctx, s) -} DELETED usecase/update_zettel.go Index: usecase/update_zettel.go ================================================================== --- usecase/update_zettel.go +++ /dev/null @@ -1,62 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package usecase provides (business) use cases for the zettelstore. -package usecase - -import ( - "context" - - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/strfun" -) - -// UpdateZettelPort is the interface used by this use case. -type UpdateZettelPort interface { - // GetZettel retrieves a specific zettel. - GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) - - // UpdateZettel updates an existing zettel. - UpdateZettel(ctx context.Context, zettel domain.Zettel) error -} - -// UpdateZettel is the data for this use case. -type UpdateZettel struct { - port UpdateZettelPort -} - -// NewUpdateZettel creates a new use case. -func NewUpdateZettel(port UpdateZettelPort) UpdateZettel { - return UpdateZettel{port: port} -} - -// Run executes the use case. -func (uc UpdateZettel) Run(ctx context.Context, zettel domain.Zettel, hasContent bool) error { - m := zettel.Meta - oldZettel, err := uc.port.GetZettel(place.NoEnrichContext(ctx), m.Zid) - if err != nil { - return err - } - if zettel.Equal(oldZettel, false) { - return nil - } - m.SetNow(meta.KeyModified) - m.YamlSep = oldZettel.Meta.YamlSep - if m.Zid == id.ConfigurationZid { - m.Set(meta.KeySyntax, meta.ValueSyntaxNone) - } - if !hasContent { - zettel.Content = domain.Content(strfun.TrimSpaceRight(oldZettel.Content.AsString())) - } - return uc.port.UpdateZettel(ctx, zettel) -} DELETED web/adapter/api/api.go Index: web/adapter/api/api.go ================================================================== --- web/adapter/api/api.go +++ /dev/null @@ -1,62 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "context" - "time" - - "zettelstore.de/z/auth" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/kernel" - "zettelstore.de/z/web/server" -) - -// API holds all data and methods for delivering API call results. -type API struct { - b server.Builder - rtConfig config.Config - authz auth.AuthzManager - token auth.TokenManager - auth server.Auth - - tokenLifetime time.Duration -} - -// New creates a new API object. -func New(b server.Builder, authz auth.AuthzManager, token auth.TokenManager, auth server.Auth, rtConfig config.Config) *API { - api := &API{ - b: b, - authz: authz, - token: token, - auth: auth, - rtConfig: rtConfig, - - tokenLifetime: kernel.Main.GetConfig(kernel.WebService, kernel.WebTokenLifetimeAPI).(time.Duration), - } - return api -} - -// GetURLPrefix returns the configured URL prefix of the web server. -func (api *API) GetURLPrefix() string { return api.b.GetURLPrefix() } - -// NewURLBuilder creates a new URL builder object with the given key. -func (api *API) NewURLBuilder(key byte) server.URLBuilder { return api.b.NewURLBuilder(key) } - -func (api *API) getAuthData(ctx context.Context) *server.AuthData { - return api.auth.GetAuthData(ctx) -} -func (api *API) withAuth() bool { return api.authz.WithAuth() } -func (api *API) getToken(ident *meta.Meta) ([]byte, error) { - return api.token.GetToken(ident, api.tokenLifetime, auth.KindJSON) -} DELETED web/adapter/api/content_type.go Index: web/adapter/api/content_type.go ================================================================== --- web/adapter/api/content_type.go +++ /dev/null @@ -1,57 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -const plainText = "text/plain; charset=utf-8" - -var mapFormat2CT = map[string]string{ - "html": "text/html; charset=utf-8", - "native": plainText, - "json": "application/json", - "djson": "application/json", - "text": plainText, - "zmk": plainText, - "raw": plainText, // In some cases... -} - -func format2ContentType(format string) string { - ct, ok := mapFormat2CT[format] - if !ok { - return "application/octet-stream" - } - return ct -} - -var mapSyntax2CT = map[string]string{ - "css": "text/css; charset=utf-8", - "gif": "image/gif", - "html": "text/html; charset=utf-8", - "jpeg": "image/jpeg", - "jpg": "image/jpeg", - "js": "text/javascript; charset=utf-8", - "pdf": "application/pdf", - "png": "image/png", - "svg": "image/svg+xml", - "xml": "text/xml; charset=utf-8", - "zmk": "text/x-zmk; charset=utf-8", - "plain": plainText, - "text": plainText, - "markdown": "text/markdown; charset=utf-8", - "md": "text/markdown; charset=utf-8", - "mustache": plainText, - //"graphviz": "text/vnd.graphviz; charset=utf-8", -} - -func syntax2contentType(syntax string) (string, bool) { - contentType, ok := mapSyntax2CT[syntax] - return contentType, ok -} DELETED web/adapter/api/get_links.go Index: web/adapter/api/get_links.go ================================================================== --- web/adapter/api/get_links.go +++ /dev/null @@ -1,226 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "encoding/json" - "net/http" - "strconv" - - "zettelstore.de/z/ast" - "zettelstore.de/z/collect" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -type jsonGetLinks struct { - ID string `json:"id"` - URL string `json:"url"` - Links struct { - Incoming []jsonIDURL `json:"incoming"` - Outgoing []jsonIDURL `json:"outgoing"` - Local []string `json:"local"` - External []string `json:"external"` - } `json:"links"` - Images struct { - Outgoing []jsonIDURL `json:"outgoing"` - Local []string `json:"local"` - External []string `json:"external"` - } `json:"images"` - Cites []string `json:"cites"` -} - -// MakeGetLinksHandler creates a new API handler to return links to other material. -func (api *API) MakeGetLinksHandler(parseZettel usecase.ParseZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - http.NotFound(w, r) - return - } - ctx := r.Context() - q := r.URL.Query() - zn, err := parseZettel.Run(ctx, zid, q.Get("syntax")) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - summary := collect.References(zn) - - kind := getKindFromValue(q.Get("kind")) - matter := getMatterFromValue(q.Get("matter")) - if !validKindMatter(kind, matter) { - adapter.BadRequest(w, "Invalid kind/matter") - return - } - - outData := jsonGetLinks{ - ID: zid.String(), - URL: api.NewURLBuilder('z').SetZid(zid).String(), - } - if kind&kindLink != 0 { - api.setupLinkJSONRefs(summary, matter, &outData) - } - if kind&kindImage != 0 { - api.setupImageJSONRefs(summary, matter, &outData) - } - if kind&kindCite != 0 { - outData.Cites = stringCites(summary.Cites) - } - - w.Header().Set(adapter.ContentType, format2ContentType("json")) - enc := json.NewEncoder(w) - enc.SetEscapeHTML(false) - enc.Encode(&outData) - } -} - -func (api *API) setupLinkJSONRefs(summary collect.Summary, matter matterType, outData *jsonGetLinks) { - if matter&matterIncoming != 0 { - outData.Links.Incoming = []jsonIDURL{} - } - zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Links) - if matter&matterOutgoing != 0 { - outData.Links.Outgoing = api.idURLRefs(zetRefs) - } - if matter&matterLocal != 0 { - outData.Links.Local = stringRefs(locRefs) - } - if matter&matterExternal != 0 { - outData.Links.External = stringRefs(extRefs) - } -} - -func (api *API) setupImageJSONRefs(summary collect.Summary, matter matterType, outData *jsonGetLinks) { - zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Images) - if matter&matterOutgoing != 0 { - outData.Images.Outgoing = api.idURLRefs(zetRefs) - } - if matter&matterLocal != 0 { - outData.Images.Local = stringRefs(locRefs) - } - if matter&matterExternal != 0 { - outData.Images.External = stringRefs(extRefs) - } -} - -func (api *API) idURLRefs(refs []*ast.Reference) []jsonIDURL { - result := make([]jsonIDURL, 0, len(refs)) - for _, ref := range refs { - path := ref.URL.Path - ub := api.NewURLBuilder('z').AppendPath(path) - if fragment := ref.URL.Fragment; len(fragment) > 0 { - ub.SetFragment(fragment) - } - result = append(result, jsonIDURL{ID: path, URL: ub.String()}) - } - return result -} - -func stringRefs(refs []*ast.Reference) []string { - result := make([]string, 0, len(refs)) - for _, ref := range refs { - result = append(result, ref.String()) - } - return result -} - -func stringCites(cites []*ast.CiteNode) []string { - mapKey := make(map[string]bool) - result := make([]string, 0, len(cites)) - for _, cn := range cites { - if _, ok := mapKey[cn.Key]; !ok { - mapKey[cn.Key] = true - result = append(result, cn.Key) - } - } - return result -} - -type kindType int - -const ( - _ kindType = 1 << iota - kindLink - kindImage - kindCite -) - -var mapKind = map[string]kindType{ - "": kindLink | kindImage | kindCite, - "link": kindLink, - "image": kindImage, - "cite": kindCite, - "both": kindLink | kindImage, - "all": kindLink | kindImage | kindCite, -} - -func getKindFromValue(value string) kindType { - if k, ok := mapKind[value]; ok { - return k - } - if n, err := strconv.Atoi(value); err == nil && n > 0 { - return kindType(n) - } - return 0 -} - -type matterType int - -const ( - _ matterType = 1 << iota - matterIncoming - matterOutgoing - matterLocal - matterExternal -) - -var mapMatter = map[string]matterType{ - "": matterIncoming | matterOutgoing | matterLocal | matterExternal, - "incoming": matterIncoming, - "outgoing": matterOutgoing, - "local": matterLocal, - "external": matterExternal, - "zettel": matterIncoming | matterOutgoing, - "material": matterLocal | matterExternal, - "all": matterIncoming | matterOutgoing | matterLocal | matterExternal, -} - -func getMatterFromValue(value string) matterType { - if m, ok := mapMatter[value]; ok { - return m - } - if n, err := strconv.Atoi(value); err == nil && n > 0 { - return matterType(n) - } - return 0 -} - -func validKindMatter(kind kindType, matter matterType) bool { - if kind == 0 { - return false - } - if kind&kindLink != 0 { - return matter != 0 - } - if kind&kindImage != 0 { - if matter == 0 || matter == matterIncoming { - return false - } - return true - } - if kind&kindCite != 0 { - return matter == matterOutgoing - } - return false -} DELETED web/adapter/api/get_order.go Index: web/adapter/api/get_order.go ================================================================== --- web/adapter/api/get_order.go +++ /dev/null @@ -1,40 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "net/http" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeGetOrderHandler creates a new API handler to return zettel references -// of a given zettel. -func (api *API) MakeGetOrderHandler(zettelOrder usecase.ZettelOrder) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - http.NotFound(w, r) - return - } - ctx := r.Context() - q := r.URL.Query() - start, metas, err := zettelOrder.Run(ctx, zid, q.Get("syntax")) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - api.writeMetaList(w, start, metas) - } -} DELETED web/adapter/api/get_role_list.go Index: web/adapter/api/get_role_list.go ================================================================== --- web/adapter/api/get_role_list.go +++ /dev/null @@ -1,64 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "fmt" - "net/http" - - "zettelstore.de/z/encoder" - "zettelstore.de/z/encoder/jsonenc" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeListRoleHandler creates a new HTTP handler for the use case "list some zettel". -func (api *API) MakeListRoleHandler(listRole usecase.ListRole) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - roleList, err := listRole.Run(r.Context()) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - - format := adapter.GetFormat(r, r.URL.Query(), encoder.GetDefaultFormat()) - switch format { - case "json": - w.Header().Set(adapter.ContentType, format2ContentType(format)) - renderListRoleJSON(w, roleList) - default: - adapter.BadRequest(w, fmt.Sprintf("Role list not available in format %q", format)) - } - - } -} - -func renderListRoleJSON(w http.ResponseWriter, roleList []string) { - buf := encoder.NewBufWriter(w) - - buf.WriteString("{\"role-list\":[") - first := true - for _, role := range roleList { - if first { - buf.WriteByte('"') - first = false - } else { - buf.WriteString("\",\"") - } - buf.Write(jsonenc.Escape(role)) - } - if !first { - buf.WriteByte('"') - } - buf.WriteString("]}") - buf.Flush() -} DELETED web/adapter/api/get_tags_list.go Index: web/adapter/api/get_tags_list.go ================================================================== --- web/adapter/api/get_tags_list.go +++ /dev/null @@ -1,80 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "fmt" - "net/http" - "sort" - "strconv" - - "zettelstore.de/z/encoder" - "zettelstore.de/z/encoder/jsonenc" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeListTagsHandler creates a new HTTP handler for the use case "list some zettel". -func (api *API) MakeListTagsHandler(listTags usecase.ListTags) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - iMinCount, _ := strconv.Atoi(r.URL.Query().Get("min")) - tagData, err := listTags.Run(r.Context(), iMinCount) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - - format := adapter.GetFormat(r, r.URL.Query(), encoder.GetDefaultFormat()) - switch format { - case "json": - w.Header().Set(adapter.ContentType, format2ContentType(format)) - renderListTagsJSON(w, tagData) - default: - adapter.BadRequest(w, fmt.Sprintf("Tags list not available in format %q", format)) - } - } -} - -func renderListTagsJSON(w http.ResponseWriter, tagData usecase.TagData) { - buf := encoder.NewBufWriter(w) - - tagList := make([]string, 0, len(tagData)) - for tag := range tagData { - tagList = append(tagList, tag) - } - sort.Strings(tagList) - - buf.WriteString("{\"tags\":{") - first := true - for _, tag := range tagList { - if first { - buf.WriteByte('"') - first = false - } else { - buf.WriteString(",\"") - } - buf.Write(jsonenc.Escape(tag)) - buf.WriteString("\":[") - for i, meta := range tagData[tag] { - if i > 0 { - buf.WriteByte(',') - } - buf.WriteByte('"') - buf.WriteString(meta.Zid.String()) - buf.WriteByte('"') - } - buf.WriteString("]") - - } - buf.WriteString("}}") - buf.Flush() -} DELETED web/adapter/api/get_zettel.go Index: web/adapter/api/get_zettel.go ================================================================== --- web/adapter/api/get_zettel.go +++ /dev/null @@ -1,129 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "errors" - "fmt" - "net/http" - - "zettelstore.de/z/ast" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeGetZettelHandler creates a new HTTP handler to return a rendered zettel. -func (api *API) MakeGetZettelHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - http.NotFound(w, r) - return - } - - ctx := r.Context() - q := r.URL.Query() - format := adapter.GetFormat(r, q, encoder.GetDefaultFormat()) - if format == "raw" { - ctx = place.NoEnrichContext(ctx) - } - zn, err := parseZettel.Run(ctx, zid, q.Get("syntax")) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - - part := getPart(q, partZettel) - if part == partUnknown { - adapter.BadRequest(w, "Unknown _part parameter") - return - } - switch format { - case "json", "djson": - w.Header().Set(adapter.ContentType, format2ContentType(format)) - err = api.getWriteMetaZettelFunc(ctx, format, part, partZettel, getMeta)(w, zn) - if err != nil { - adapter.InternalServerError(w, "Write D/JSON", err) - } - return - } - - env := encoder.Environment{ - LinkAdapter: adapter.MakeLinkAdapter(ctx, api, 'z', getMeta, part.DefString(partZettel), format), - ImageAdapter: adapter.MakeImageAdapter(ctx, api, getMeta), - CiteAdapter: nil, - Lang: config.GetLang(zn.InhMeta, api.rtConfig), - Xhtml: false, - MarkerExternal: "", - NewWindow: false, - IgnoreMeta: map[string]bool{meta.KeyLang: true}, - } - switch part { - case partZettel: - err = writeZettelPartZettel(w, zn, format, env) - case partMeta: - err = writeZettelPartMeta(w, zn, format) - case partContent: - err = api.writeZettelPartContent(w, zn, format, env) - } - if err != nil { - if errors.Is(err, adapter.ErrNoSuchFormat) { - adapter.BadRequest(w, fmt.Sprintf("Zettel %q not available in format %q", zid.String(), format)) - return - } - adapter.InternalServerError(w, "Get zettel", err) - } - } -} - -func writeZettelPartZettel(w http.ResponseWriter, zn *ast.ZettelNode, format string, env encoder.Environment) error { - enc := encoder.Create(format, &env) - if enc == nil { - return adapter.ErrNoSuchFormat - } - inhMeta := false - if format != "raw" { - w.Header().Set(adapter.ContentType, format2ContentType(format)) - inhMeta = true - } - _, err := enc.WriteZettel(w, zn, inhMeta) - return err -} - -func writeZettelPartMeta(w http.ResponseWriter, zn *ast.ZettelNode, format string) error { - w.Header().Set(adapter.ContentType, format2ContentType(format)) - if enc := encoder.Create(format, nil); enc != nil { - if format == "raw" { - _, err := enc.WriteMeta(w, zn.Meta) - return err - } - _, err := enc.WriteMeta(w, zn.InhMeta) - return err - } - return adapter.ErrNoSuchFormat -} - -func (api *API) writeZettelPartContent(w http.ResponseWriter, zn *ast.ZettelNode, format string, env encoder.Environment) error { - if format == "raw" { - if ct, ok := syntax2contentType(config.GetSyntax(zn.Meta, api.rtConfig)); ok { - w.Header().Add(adapter.ContentType, ct) - } - } else { - w.Header().Set(adapter.ContentType, format2ContentType(format)) - } - return writeContent(w, zn, format, &env) -} DELETED web/adapter/api/get_zettel_context.go Index: web/adapter/api/get_zettel_context.go ================================================================== --- web/adapter/api/get_zettel_context.go +++ /dev/null @@ -1,48 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "net/http" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeZettelContextHandler creates a new HTTP handler for the use case "zettel context". -func (api *API) MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - http.NotFound(w, r) - return - } - q := r.URL.Query() - dir := usecase.ParseZCDirection(q.Get("dir")) - depth, ok := adapter.GetInteger(q, "depth") - if !ok || depth < 0 { - depth = 5 - } - limit, ok := adapter.GetInteger(q, "limit") - if !ok || limit < 0 { - limit = 200 - } - ctx := r.Context() - metaList, err := getContext.Run(ctx, zid, dir, depth, limit) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - api.writeMetaList(w, metaList[0], metaList[1:]) - } -} DELETED web/adapter/api/get_zettel_list.go Index: web/adapter/api/get_zettel_list.go ================================================================== --- web/adapter/api/get_zettel_list.go +++ /dev/null @@ -1,86 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "fmt" - "net/http" - - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/parser" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeListMetaHandler creates a new HTTP handler for the use case "list some zettel". -func (api *API) MakeListMetaHandler( - listMeta usecase.ListMeta, - getMeta usecase.GetMeta, - parseZettel usecase.ParseZettel, -) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - q := r.URL.Query() - s := adapter.GetSearch(q, false) - format := adapter.GetFormat(r, q, encoder.GetDefaultFormat()) - part := getPart(q, partMeta) - if part == partUnknown { - adapter.BadRequest(w, "Unknown _part parameter") - return - } - ctx1 := ctx - if format == "html" || (!s.HasComputedMetaKey() && (part == partID || part == partContent)) { - ctx1 = place.NoEnrichContext(ctx1) - } - metaList, err := listMeta.Run(ctx1, s) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - - w.Header().Set(adapter.ContentType, format2ContentType(format)) - switch format { - case "html": - api.renderListMetaHTML(w, metaList) - case "json", "djson": - api.renderListMetaXJSON(ctx, w, metaList, format, part, partMeta, getMeta, parseZettel) - case "native", "raw", "text", "zmk": - adapter.NotImplemented(w, fmt.Sprintf("Zettel list in format %q not yet implemented", format)) - default: - adapter.BadRequest(w, fmt.Sprintf("Zettel list not available in format %q", format)) - } - } -} - -func (api *API) renderListMetaHTML(w http.ResponseWriter, metaList []*meta.Meta) { - env := encoder.Environment{Interactive: true} - buf := encoder.NewBufWriter(w) - buf.WriteStrings("<html lang=\"", api.rtConfig.GetDefaultLang(), "\">\n<body>\n<ul>\n") - for _, m := range metaList { - title := m.GetDefault(meta.KeyTitle, "") - htmlTitle, err := adapter.FormatInlines(parser.ParseMetadata(title), "html", &env) - if err != nil { - adapter.InternalServerError(w, "Format HTML inlines", err) - return - } - buf.WriteStrings( - "<li><a href=\"", - api.NewURLBuilder('z').SetZid(m.Zid).AppendQuery("_format", "html").String(), - "\">", - htmlTitle, - "</a></li>\n") - } - buf.WriteString("</ul>\n</body>\n</html>") - buf.Flush() -} DELETED web/adapter/api/json.go Index: web/adapter/api/json.go ================================================================== --- web/adapter/api/json.go +++ /dev/null @@ -1,342 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "context" - "encoding/json" - "io" - "net/http" - - "zettelstore.de/z/ast" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -type jsonIDURL struct { - ID string `json:"id"` - URL string `json:"url"` -} -type jsonZettel struct { - ID string `json:"id"` - URL string `json:"url"` - Meta map[string]string `json:"meta"` - Encoding string `json:"encoding"` - Content interface{} `json:"content"` -} -type jsonMeta struct { - ID string `json:"id"` - URL string `json:"url"` - Meta map[string]string `json:"meta"` -} -type jsonMetaList struct { - ID string `json:"id"` - URL string `json:"url"` - Meta map[string]string `json:"meta"` - List []jsonMeta `json:"list"` -} -type jsonContent struct { - ID string `json:"id"` - URL string `json:"url"` - Encoding string `json:"encoding"` - Content interface{} `json:"content"` -} - -func encodedContent(content domain.Content) (string, interface{}) { - if content.IsBinary() { - return "base64", content.AsBytes() - } - return "", content.AsString() -} - -var ( - djsonMetaHeader = []byte(",\"meta\":") - djsonContentHeader = []byte(",\"content\":") - djsonHeader1 = []byte("{\"id\":\"") - djsonHeader2 = []byte("\",\"url\":\"") - djsonHeader3 = []byte("?_format=") - djsonHeader4 = []byte("\"") - djsonFooter = []byte("}") -) - -func (api *API) writeDJSONHeader(w io.Writer, zid id.Zid) error { - _, err := w.Write(djsonHeader1) - if err == nil { - _, err = w.Write(zid.Bytes()) - } - if err == nil { - _, err = w.Write(djsonHeader2) - } - if err == nil { - _, err = io.WriteString(w, api.NewURLBuilder('z').SetZid(zid).String()) - } - if err == nil { - _, err = w.Write(djsonHeader3) - if err == nil { - _, err = io.WriteString(w, "djson") - } - } - if err == nil { - _, err = w.Write(djsonHeader4) - } - return err -} - -func (api *API) renderListMetaXJSON( - ctx context.Context, - w http.ResponseWriter, - metaList []*meta.Meta, - format string, - part, defPart partType, - getMeta usecase.GetMeta, - parseZettel usecase.ParseZettel, -) { - prepareZettel := api.getPrepareZettelFunc(ctx, parseZettel, part) - writeZettel := api.getWriteMetaZettelFunc(ctx, format, part, defPart, getMeta) - err := writeListXJSON(w, metaList, prepareZettel, writeZettel) - if err != nil { - adapter.InternalServerError(w, "Get list", err) - } -} - -type prepareZettelFunc func(m *meta.Meta) (*ast.ZettelNode, error) - -func (api *API) getPrepareZettelFunc(ctx context.Context, parseZettel usecase.ParseZettel, part partType) prepareZettelFunc { - switch part { - case partZettel, partContent: - return func(m *meta.Meta) (*ast.ZettelNode, error) { - return parseZettel.Run(ctx, m.Zid, "") - } - case partMeta, partID: - return func(m *meta.Meta) (*ast.ZettelNode, error) { - return &ast.ZettelNode{ - Meta: m, - Content: "", - Zid: m.Zid, - InhMeta: api.rtConfig.AddDefaultValues(m), - Ast: nil, - }, nil - } - } - return nil -} - -type writeZettelFunc func(io.Writer, *ast.ZettelNode) error - -func (api *API) getWriteMetaZettelFunc(ctx context.Context, format string, - part, defPart partType, getMeta usecase.GetMeta) writeZettelFunc { - switch part { - case partZettel: - return api.getWriteZettelFunc(ctx, format, defPart, getMeta) - case partMeta: - return api.getWriteMetaFunc(ctx, format) - case partContent: - return api.getWriteContentFunc(ctx, format, defPart, getMeta) - case partID: - return api.getWriteIDFunc(ctx, format) - default: - panic(part) - } -} - -func (api *API) getWriteZettelFunc(ctx context.Context, format string, - defPart partType, getMeta usecase.GetMeta) writeZettelFunc { - if format == "json" { - return func(w io.Writer, zn *ast.ZettelNode) error { - encoding, content := encodedContent(zn.Content) - return encodeJSONData(w, jsonZettel{ - ID: zn.Zid.String(), - URL: api.NewURLBuilder('z').SetZid(zn.Zid).String(), - Meta: zn.InhMeta.Map(), - Encoding: encoding, - Content: content, - }) - } - } - enc := encoder.Create("djson", nil) - if enc == nil { - panic("no DJSON encoder found") - } - return func(w io.Writer, zn *ast.ZettelNode) error { - err := api.writeDJSONHeader(w, zn.Zid) - if err != nil { - return err - } - _, err = w.Write(djsonMetaHeader) - if err != nil { - return err - } - _, err = enc.WriteMeta(w, zn.InhMeta) - if err != nil { - return err - } - _, err = w.Write(djsonContentHeader) - if err != nil { - return err - } - err = writeContent(w, zn, "djson", &encoder.Environment{ - LinkAdapter: adapter.MakeLinkAdapter(ctx, api, 'z', getMeta, partZettel.DefString(defPart), "djson"), - ImageAdapter: adapter.MakeImageAdapter(ctx, api, getMeta)}) - if err != nil { - return err - } - _, err = w.Write(djsonFooter) - return err - } -} -func (api *API) getWriteMetaFunc(ctx context.Context, format string) writeZettelFunc { - if format == "json" { - return func(w io.Writer, zn *ast.ZettelNode) error { - return encodeJSONData(w, jsonMeta{ - ID: zn.Zid.String(), - URL: api.NewURLBuilder('z').SetZid(zn.Zid).String(), - Meta: zn.InhMeta.Map(), - }) - } - } - enc := encoder.Create("djson", nil) - if enc == nil { - panic("no DJSON encoder found") - } - return func(w io.Writer, zn *ast.ZettelNode) error { - err := api.writeDJSONHeader(w, zn.Zid) - if err != nil { - return err - } - _, err = w.Write(djsonMetaHeader) - if err != nil { - return err - } - _, err = enc.WriteMeta(w, zn.InhMeta) - if err != nil { - return err - } - _, err = w.Write(djsonFooter) - return err - } -} -func (api *API) getWriteContentFunc(ctx context.Context, format string, - defPart partType, getMeta usecase.GetMeta) writeZettelFunc { - if format == "json" { - return func(w io.Writer, zn *ast.ZettelNode) error { - encoding, content := encodedContent(zn.Content) - return encodeJSONData(w, jsonContent{ - ID: zn.Zid.String(), - URL: api.NewURLBuilder('z').SetZid(zn.Zid).String(), - Encoding: encoding, - Content: content, - }) - } - } - return func(w io.Writer, zn *ast.ZettelNode) error { - err := api.writeDJSONHeader(w, zn.Zid) - if err != nil { - return err - } - _, err = w.Write(djsonContentHeader) - if err != nil { - return err - } - err = writeContent(w, zn, "djson", &encoder.Environment{ - LinkAdapter: adapter.MakeLinkAdapter(ctx, api, 'z', getMeta, partContent.DefString(defPart), "djson"), - ImageAdapter: adapter.MakeImageAdapter(ctx, api, getMeta)}) - if err != nil { - return err - } - _, err = w.Write(djsonFooter) - return err - } -} -func (api *API) getWriteIDFunc(ctx context.Context, format string) writeZettelFunc { - if format == "json" { - return func(w io.Writer, zn *ast.ZettelNode) error { - return encodeJSONData(w, jsonIDURL{ - ID: zn.Zid.String(), - URL: api.NewURLBuilder('z').SetZid(zn.Zid).String(), - }) - } - } - return func(w io.Writer, zn *ast.ZettelNode) error { - err := api.writeDJSONHeader(w, zn.Zid) - if err != nil { - return err - } - _, err = w.Write(djsonFooter) - return err - } -} - -var ( - jsonListHeader = []byte("{\"list\":[") - jsonListSep = []byte{','} - jsonListFooter = []byte("]}") -) - -func writeListXJSON(w http.ResponseWriter, metaList []*meta.Meta, prepareZettel prepareZettelFunc, writeZettel writeZettelFunc) error { - _, err := w.Write(jsonListHeader) - for i, m := range metaList { - if err != nil { - return err - } - if i > 0 { - _, err = w.Write(jsonListSep) - if err != nil { - return err - } - } - zn, err1 := prepareZettel(m) - if err1 != nil { - return err1 - } - err = writeZettel(w, zn) - } - if err == nil { - _, err = w.Write(jsonListFooter) - } - return err -} - -func writeContent(w io.Writer, zn *ast.ZettelNode, format string, env *encoder.Environment) error { - enc := encoder.Create(format, env) - if enc == nil { - return adapter.ErrNoSuchFormat - } - - _, err := enc.WriteContent(w, zn) - return err -} - -func encodeJSONData(w io.Writer, data interface{}) error { - enc := json.NewEncoder(w) - enc.SetEscapeHTML(false) - return enc.Encode(data) -} - -func (api *API) writeMetaList(w http.ResponseWriter, m *meta.Meta, metaList []*meta.Meta) error { - outData := jsonMetaList{ - ID: m.Zid.String(), - URL: api.NewURLBuilder('z').SetZid(m.Zid).String(), - Meta: m.Map(), - List: make([]jsonMeta, len(metaList)), - } - for i, m := range metaList { - outData.List[i].ID = m.Zid.String() - outData.List[i].URL = api.NewURLBuilder('z').SetZid(m.Zid).String() - outData.List[i].Meta = m.Map() - } - w.Header().Set(adapter.ContentType, format2ContentType("json")) - return encodeJSONData(w, outData) -} DELETED web/adapter/api/login.go Index: web/adapter/api/login.go ================================================================== --- web/adapter/api/login.go +++ /dev/null @@ -1,102 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import ( - "encoding/json" - "net/http" - "time" - - "zettelstore.de/z/auth" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakePostLoginHandlerAPI creates a new HTTP handler to authenticate the given user via API. -func (api *API) MakePostLoginHandlerAPI(ucAuth usecase.Authenticate) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - if !api.withAuth() { - w.Header().Set(adapter.ContentType, format2ContentType("json")) - writeJSONToken(w, "freeaccess", 24*366*10*time.Hour) - return - } - var token []byte - if ident, cred := retrieveIdentCred(r); ident != "" { - var err error - token, err = ucAuth.Run(r.Context(), ident, cred, api.tokenLifetime, auth.KindJSON) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - } - if len(token) == 0 { - w.Header().Set("WWW-Authenticate", `Bearer realm="Default"`) - http.Error(w, "Authentication failed", http.StatusUnauthorized) - return - } - - w.Header().Set(adapter.ContentType, format2ContentType("json")) - writeJSONToken(w, string(token), api.tokenLifetime) - } -} - -func retrieveIdentCred(r *http.Request) (string, string) { - if ident, cred, ok := adapter.GetCredentialsViaForm(r); ok { - return ident, cred - } - if ident, cred, ok := r.BasicAuth(); ok { - return ident, cred - } - return "", "" -} - -func writeJSONToken(w http.ResponseWriter, token string, lifetime time.Duration) { - je := json.NewEncoder(w) - je.Encode(struct { - Token string `json:"access_token"` - Type string `json:"token_type"` - Expires int `json:"expires_in"` - }{ - Token: token, - Type: "Bearer", - Expires: int(lifetime / time.Second), - }) -} - -// MakeRenewAuthHandler creates a new HTTP handler to renew the authenticate of a user. -func (api *API) MakeRenewAuthHandler() http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - authData := api.getAuthData(ctx) - if authData == nil || len(authData.Token) == 0 || authData.User == nil { - adapter.BadRequest(w, "Not authenticated") - return - } - totalLifetime := authData.Expires.Sub(authData.Issued) - currentLifetime := authData.Now.Sub(authData.Issued) - // If we are in the first quarter of the tokens lifetime, return the token - if currentLifetime*4 < totalLifetime { - w.Header().Set(adapter.ContentType, format2ContentType("json")) - writeJSONToken(w, string(authData.Token), totalLifetime-currentLifetime) - return - } - - // Token is a little bit aged. Create a new one - token, err := api.getToken(authData.User) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - w.Header().Set(adapter.ContentType, format2ContentType("json")) - writeJSONToken(w, string(token), api.tokenLifetime) - } -} DELETED web/adapter/api/request.go Index: web/adapter/api/request.go ================================================================== --- web/adapter/api/request.go +++ /dev/null @@ -1,65 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package api provides api handlers for web requests. -package api - -import "net/url" - -type partType int - -const ( - partUnknown partType = iota - partID - partMeta - partContent - partZettel -) - -var partMap = map[string]partType{ - "id": partID, - "meta": partMeta, - "content": partContent, - "zettel": partZettel, -} - -func getPart(q url.Values, defPart partType) partType { - p := q.Get("_part") - if p == "" { - return defPart - } - if part, ok := partMap[p]; ok { - return part - } - return partUnknown -} - -func (p partType) String() string { - switch p { - case partID: - return "id" - case partMeta: - return "meta" - case partContent: - return "content" - case partZettel: - return "zettel" - case partUnknown: - return "unknown" - } - return "" -} - -func (p partType) DefString(defPart partType) string { - if p == defPart { - return "" - } - return p.String() -} DELETED web/adapter/encoding.go Index: web/adapter/encoding.go ================================================================== --- web/adapter/encoding.go +++ /dev/null @@ -1,136 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package adapter provides handlers for web requests. -package adapter - -import ( - "context" - "errors" - "strings" - - "zettelstore.de/z/ast" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/encoder" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/server" -) - -// ErrNoSuchFormat signals an unsupported encoding format -var ErrNoSuchFormat = errors.New("no such format") - -// FormatInlines returns a string representation of the inline slice. -func FormatInlines(is ast.InlineSlice, format string, env *encoder.Environment) (string, error) { - enc := encoder.Create(format, env) - if enc == nil { - return "", ErrNoSuchFormat - } - - var content strings.Builder - _, err := enc.WriteInlines(&content, is) - if err != nil { - return "", err - } - return content.String(), nil -} - -// MakeLinkAdapter creates an adapter to change a link node during encoding. -func MakeLinkAdapter( - ctx context.Context, - b server.Builder, - key byte, - getMeta usecase.GetMeta, - part, format string, -) func(*ast.LinkNode) ast.InlineNode { - return func(origLink *ast.LinkNode) ast.InlineNode { - origRef := origLink.Ref - if origRef == nil { - return origLink - } - if origRef.State == ast.RefStateBased { - newLink := *origLink - urlPrefix := b.GetURLPrefix() - newRef := ast.ParseReference(urlPrefix + origRef.Value[1:]) - newRef.State = ast.RefStateHosted - newLink.Ref = newRef - return &newLink - } - if origRef.State != ast.RefStateZettel { - return origLink - } - zid, err := id.Parse(origRef.URL.Path) - if err != nil { - panic(err) - } - _, err = getMeta.Run(place.NoEnrichContext(ctx), zid) - if errors.Is(err, &place.ErrNotAllowed{}) { - return &ast.FormatNode{ - Code: ast.FormatSpan, - Attrs: origLink.Attrs, - Inlines: origLink.Inlines, - } - } - var newRef *ast.Reference - if err == nil { - ub := b.NewURLBuilder(key).SetZid(zid) - if part != "" { - ub.AppendQuery("_part", part) - } - if format != "" { - ub.AppendQuery("_format", format) - } - if fragment := origRef.URL.EscapedFragment(); fragment != "" { - ub.SetFragment(fragment) - } - - newRef = ast.ParseReference(ub.String()) - newRef.State = ast.RefStateFound - } else { - newRef = ast.ParseReference(origRef.Value) - newRef.State = ast.RefStateBroken - } - newLink := *origLink - newLink.Ref = newRef - return &newLink - } -} - -// MakeImageAdapter creates an adapter to change an image node during encoding. -func MakeImageAdapter(ctx context.Context, b server.Builder, getMeta usecase.GetMeta) func(*ast.ImageNode) ast.InlineNode { - return func(origImage *ast.ImageNode) ast.InlineNode { - if origImage.Ref == nil { - return origImage - } - switch origImage.Ref.State { - case ast.RefStateInvalid: - return createZettelImage(b, origImage, id.EmojiZid, ast.RefStateInvalid) - case ast.RefStateZettel: - zid, err := id.Parse(origImage.Ref.Value) - if err != nil { - panic(err) - } - _, err = getMeta.Run(place.NoEnrichContext(ctx), zid) - if err != nil { - return createZettelImage(b, origImage, id.EmojiZid, ast.RefStateBroken) - } - return createZettelImage(b, origImage, zid, ast.RefStateFound) - } - return origImage - } -} - -func createZettelImage(b server.Builder, origImage *ast.ImageNode, zid id.Zid, state ast.RefState) *ast.ImageNode { - newImage := *origImage - newImage.Ref = ast.ParseReference( - b.NewURLBuilder('z').SetZid(zid).AppendQuery("_part", "content").AppendQuery("_format", "raw").String()) - newImage.Ref.State = state - return &newImage -} DELETED web/adapter/errors.go Index: web/adapter/errors.go ================================================================== --- web/adapter/errors.go +++ /dev/null @@ -1,48 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package adapter provides handlers for web requests. -package adapter - -import ( - "log" - "net/http" -) - -// BadRequest signals HTTP status code 400. -func BadRequest(w http.ResponseWriter, text string) { - http.Error(w, text, http.StatusBadRequest) -} - -// Forbidden signals HTTP status code 403. -func Forbidden(w http.ResponseWriter, text string) { - http.Error(w, text, http.StatusForbidden) -} - -// NotFound signals HTTP status code 404. -func NotFound(w http.ResponseWriter, text string) { - http.Error(w, text, http.StatusNotFound) -} - -// InternalServerError signals HTTP status code 500. -func InternalServerError(w http.ResponseWriter, text string, err error) { - http.Error(w, "Internal Server Error", http.StatusInternalServerError) - if text == "" { - log.Println(err) - } else { - log.Printf("%v: %v", text, err) - } -} - -// NotImplemented signals HTTP status code 501 -func NotImplemented(w http.ResponseWriter, text string) { - http.Error(w, text, http.StatusNotImplemented) - log.Println(text) -} DELETED web/adapter/login.go Index: web/adapter/login.go ================================================================== --- web/adapter/login.go +++ /dev/null @@ -1,51 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package adapter provides handlers for web requests. -package adapter - -import ( - "fmt" - "log" - "net/http" - "strings" - - "zettelstore.de/z/encoder" -) - -// MakePostLoginHandler creates a new HTTP handler to authenticate the given user. -func MakePostLoginHandler(apiHandler, htmlHandler http.HandlerFunc) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - switch format := GetFormat(r, r.URL.Query(), encoder.GetDefaultFormat()); format { - case "json": - apiHandler(w, r) - case "html": - htmlHandler(w, r) - default: - BadRequest(w, fmt.Sprintf("Authentication not available in format %q", format)) - } - } -} - -// GetCredentialsViaForm retrieves the authentication credentions from a form. -func GetCredentialsViaForm(r *http.Request) (ident, cred string, ok bool) { - err := r.ParseForm() - if err != nil { - log.Println(err) - return "", "", false - } - - ident = strings.TrimSpace(r.PostFormValue("username")) - cred = r.PostFormValue("password") - if ident == "" { - return "", "", false - } - return ident, cred, true -} DELETED web/adapter/request.go Index: web/adapter/request.go ================================================================== --- web/adapter/request.go +++ /dev/null @@ -1,144 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package adapter provides handlers for web requests. -package adapter - -import ( - "net/http" - "net/url" - "strconv" - "strings" - - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/search" -) - -// GetInteger returns the integer value of the named query key. -func GetInteger(q url.Values, key string) (int, bool) { - s := q.Get(key) - if s != "" { - if val, err := strconv.Atoi(s); err == nil { - return val, true - } - } - return 0, false -} - -// ContentType defines the HTTP header value "Content-Type". -const ContentType = "Content-Type" - -// GetFormat returns the data format selected by the caller. -func GetFormat(r *http.Request, q url.Values, defFormat string) string { - format := q.Get("_format") - if len(format) > 0 { - return format - } - if format, ok := getOneFormat(r, "Accept"); ok { - return format - } - if format, ok := getOneFormat(r, ContentType); ok { - return format - } - return defFormat -} - -func getOneFormat(r *http.Request, key string) (string, bool) { - if values, ok := r.Header[key]; ok { - for _, value := range values { - if format, ok := contentType2format(value); ok { - return format, true - } - } - } - return "", false -} - -var mapCT2format = map[string]string{ - "application/json": "json", - "text/html": "html", -} - -func contentType2format(contentType string) (string, bool) { - // TODO: only check before first ';' - format, ok := mapCT2format[contentType] - return format, ok -} - -// GetSearch retrieves the specified filter and sorting options from a query. -func GetSearch(q url.Values, forSearch bool) (s *search.Search) { - sortQKey, orderQKey, offsetQKey, limitQKey, negateQKey, sQKey := getQueryKeys(forSearch) - for key, values := range q { - switch key { - case sortQKey, orderQKey: - s = extractOrderFromQuery(values, s) - case offsetQKey: - s = extractOffsetFromQuery(values, s) - case limitQKey: - s = extractLimitFromQuery(values, s) - case negateQKey: - s = s.SetNegate() - case sQKey: - s = setCleanedQueryValues(s, "", values) - default: - if !forSearch && meta.KeyIsValid(key) { - s = setCleanedQueryValues(s, key, values) - } - } - } - return s -} - -func extractOrderFromQuery(values []string, s *search.Search) *search.Search { - if len(values) > 0 { - descending := false - sortkey := values[0] - if strings.HasPrefix(sortkey, "-") { - descending = true - sortkey = sortkey[1:] - } - if meta.KeyIsValid(sortkey) || sortkey == search.RandomOrder { - s = s.AddOrder(sortkey, descending) - } - } - return s -} - -func extractOffsetFromQuery(values []string, s *search.Search) *search.Search { - if len(values) > 0 { - if offset, err := strconv.Atoi(values[0]); err == nil { - s = s.SetOffset(offset) - } - } - return s -} - -func extractLimitFromQuery(values []string, s *search.Search) *search.Search { - if len(values) > 0 { - if limit, err := strconv.Atoi(values[0]); err == nil { - s = s.SetLimit(limit) - } - } - return s -} - -func getQueryKeys(forSearch bool) (string, string, string, string, string, string) { - if forSearch { - return "sort", "order", "offset", "limit", "negate", "s" - } - return "_sort", "_order", "_offset", "_limit", "_negate", "_s" -} - -func setCleanedQueryValues(s *search.Search, key string, values []string) *search.Search { - for _, val := range values { - s = s.AddExpr(key, val) - } - return s -} DELETED web/adapter/response.go Index: web/adapter/response.go ================================================================== --- web/adapter/response.go +++ /dev/null @@ -1,67 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package adapter provides handlers for web requests. -package adapter - -import ( - "errors" - "fmt" - "log" - "net/http" - - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" -) - -// ReportUsecaseError returns an appropriate HTTP status code for errors in use cases. -func ReportUsecaseError(w http.ResponseWriter, err error) { - code, text := CodeMessageFromError(err) - if code == http.StatusInternalServerError { - log.Printf("%v: %v", text, err) - } - http.Error(w, text, code) -} - -// ErrBadRequest is returned if the caller made an invalid HTTP request. -type ErrBadRequest struct { - Text string -} - -// NewErrBadRequest creates an new bad request error. -func NewErrBadRequest(text string) error { return &ErrBadRequest{Text: text} } - -func (err *ErrBadRequest) Error() string { return err.Text } - -// CodeMessageFromError returns an appropriate HTTP status code and text from a given error. -func CodeMessageFromError(err error) (int, string) { - if err == place.ErrNotFound { - return http.StatusNotFound, http.StatusText(http.StatusNotFound) - } - if err1, ok := err.(*place.ErrNotAllowed); ok { - return http.StatusForbidden, err1.Error() - } - if err1, ok := err.(*place.ErrInvalidID); ok { - return http.StatusBadRequest, fmt.Sprintf("Zettel-ID %q not appropriate in this context", err1.Zid) - } - if err1, ok := err.(*usecase.ErrZidInUse); ok { - return http.StatusBadRequest, fmt.Sprintf("Zettel-ID %q already in use", err1.Zid) - } - if err1, ok := err.(*ErrBadRequest); ok { - return http.StatusBadRequest, err1.Text - } - if errors.Is(err, place.ErrStopped) { - return http.StatusInternalServerError, fmt.Sprintf("Zettelstore not operational: %v", err) - } - if errors.Is(err, place.ErrConflict) { - return http.StatusConflict, "Zettelstore operations conflicted" - } - return http.StatusInternalServerError, err.Error() -} DELETED web/adapter/webui/create_zettel.go Index: web/adapter/webui/create_zettel.go ================================================================== --- web/adapter/webui/create_zettel.go +++ /dev/null @@ -1,153 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "context" - "fmt" - "net/http" - - "zettelstore.de/z/config" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/input" - "zettelstore.de/z/parser" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeGetCopyZettelHandler creates a new HTTP handler to display the -// HTML edit view of a copied zettel. -func (wui *WebUI) MakeGetCopyZettelHandler(getZettel usecase.GetZettel, copyZettel usecase.CopyZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - origZettel, err := getOrigZettel(ctx, w, r, getZettel, "Copy") - if err != nil { - wui.reportError(ctx, w, err) - return - } - wui.renderZettelForm(w, r, copyZettel.Run(origZettel), "Copy Zettel", "Copy Zettel") - } -} - -// MakeGetFolgeZettelHandler creates a new HTTP handler to display the -// HTML edit view of a follow-up zettel. -func (wui *WebUI) MakeGetFolgeZettelHandler(getZettel usecase.GetZettel, folgeZettel usecase.FolgeZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - origZettel, err := getOrigZettel(ctx, w, r, getZettel, "Folge") - if err != nil { - wui.reportError(ctx, w, err) - return - } - wui.renderZettelForm(w, r, folgeZettel.Run(origZettel), "Folge Zettel", "Folgezettel") - } -} - -// MakeGetNewZettelHandler creates a new HTTP handler to display the -// HTML edit view of a zettel. -func (wui *WebUI) MakeGetNewZettelHandler(getZettel usecase.GetZettel, newZettel usecase.NewZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - origZettel, err := getOrigZettel(ctx, w, r, getZettel, "New") - if err != nil { - wui.reportError(ctx, w, err) - return - } - m := origZettel.Meta - title := parser.ParseInlines(input.NewInput(config.GetTitle(m, wui.rtConfig)), meta.ValueSyntaxZmk) - textTitle, err := adapter.FormatInlines(title, "text", nil) - if err != nil { - wui.reportError(ctx, w, err) - return - } - env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)} - htmlTitle, err := adapter.FormatInlines(title, "html", &env) - if err != nil { - wui.reportError(ctx, w, err) - return - } - wui.renderZettelForm(w, r, newZettel.Run(origZettel), textTitle, htmlTitle) - } -} - -func getOrigZettel( - ctx context.Context, - w http.ResponseWriter, - r *http.Request, - getZettel usecase.GetZettel, - op string, -) (domain.Zettel, error) { - if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" { - return domain.Zettel{}, adapter.NewErrBadRequest( - fmt.Sprintf("%v zettel not possible in format %q", op, format)) - } - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - return domain.Zettel{}, place.ErrNotFound - } - origZettel, err := getZettel.Run(place.NoEnrichContext(ctx), zid) - if err != nil { - return domain.Zettel{}, place.ErrNotFound - } - return origZettel, nil -} - -func (wui *WebUI) renderZettelForm( - w http.ResponseWriter, - r *http.Request, - zettel domain.Zettel, - title, heading string, -) { - ctx := r.Context() - user := wui.getUser(ctx) - m := zettel.Meta - var base baseData - wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), title, user, &base) - wui.renderTemplate(ctx, w, id.FormTemplateZid, &base, formZettelData{ - Heading: heading, - MetaTitle: m.GetDefault(meta.KeyTitle, ""), - MetaTags: m.GetDefault(meta.KeyTags, ""), - MetaRole: config.GetRole(m, wui.rtConfig), - MetaSyntax: config.GetSyntax(m, wui.rtConfig), - MetaPairsRest: m.PairsRest(false), - IsTextContent: !zettel.Content.IsBinary(), - Content: zettel.Content.AsString(), - }) -} - -// MakePostCreateZettelHandler creates a new HTTP handler to store content of -// an existing zettel. -func (wui *WebUI) MakePostCreateZettelHandler(createZettel usecase.CreateZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - zettel, hasContent, err := parseZettelForm(r, id.Invalid) - if err != nil { - wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read form data")) - return - } - if !hasContent { - wui.reportError(ctx, w, adapter.NewErrBadRequest("Content is missing")) - return - } - - newZid, err := createZettel.Run(ctx, zettel) - if err != nil { - wui.reportError(ctx, w, err) - return - } - redirectFound(w, r, wui.NewURLBuilder('h').SetZid(newZid)) - } -} DELETED web/adapter/webui/delete_zettel.go Index: web/adapter/webui/delete_zettel.go ================================================================== --- web/adapter/webui/delete_zettel.go +++ /dev/null @@ -1,79 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "fmt" - "net/http" - - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeGetDeleteZettelHandler creates a new HTTP handler to display the -// HTML delete view of a zettel. -func (wui *WebUI) MakeGetDeleteZettelHandler(getZettel usecase.GetZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" { - wui.reportError(ctx, w, adapter.NewErrBadRequest( - fmt.Sprintf("Delete zettel not possible in format %q", format))) - return - } - - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - zettel, err := getZettel.Run(ctx, zid) - if err != nil { - wui.reportError(ctx, w, err) - return - } - - user := wui.getUser(ctx) - m := zettel.Meta - var base baseData - wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Delete Zettel "+m.Zid.String(), user, &base) - wui.renderTemplate(ctx, w, id.DeleteTemplateZid, &base, struct { - Zid string - MetaPairs []meta.Pair - }{ - Zid: zid.String(), - MetaPairs: m.Pairs(true), - }) - } -} - -// MakePostDeleteZettelHandler creates a new HTTP handler to delete a zettel. -func (wui *WebUI) MakePostDeleteZettelHandler(deleteZettel usecase.DeleteZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - if err := deleteZettel.Run(r.Context(), zid); err != nil { - wui.reportError(ctx, w, err) - return - } - redirectFound(w, r, wui.NewURLBuilder('/')) - } -} DELETED web/adapter/webui/edit_zettel.go Index: web/adapter/webui/edit_zettel.go ================================================================== --- web/adapter/webui/edit_zettel.go +++ /dev/null @@ -1,89 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "fmt" - "net/http" - - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeEditGetZettelHandler creates a new HTTP handler to display the -// HTML edit view of a zettel. -func (wui *WebUI) MakeEditGetZettelHandler(getZettel usecase.GetZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - zettel, err := getZettel.Run(place.NoEnrichContext(ctx), zid) - if err != nil { - wui.reportError(ctx, w, err) - return - } - - if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" { - wui.reportError(ctx, w, adapter.NewErrBadRequest( - fmt.Sprintf("Edit zettel %q not possible in format %q", zid, format))) - return - } - - user := wui.getUser(ctx) - m := zettel.Meta - var base baseData - wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Edit Zettel", user, &base) - wui.renderTemplate(ctx, w, id.FormTemplateZid, &base, formZettelData{ - Heading: base.Title, - MetaTitle: m.GetDefault(meta.KeyTitle, ""), - MetaRole: m.GetDefault(meta.KeyRole, ""), - MetaTags: m.GetDefault(meta.KeyTags, ""), - MetaSyntax: m.GetDefault(meta.KeySyntax, ""), - MetaPairsRest: m.PairsRest(false), - IsTextContent: !zettel.Content.IsBinary(), - Content: zettel.Content.AsString(), - }) - } -} - -// MakeEditSetZettelHandler creates a new HTTP handler to store content of -// an existing zettel. -func (wui *WebUI) MakeEditSetZettelHandler(updateZettel usecase.UpdateZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - zettel, hasContent, err := parseZettelForm(r, zid) - if err != nil { - wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read zettel form")) - return - } - - if err := updateZettel.Run(r.Context(), zettel, hasContent); err != nil { - wui.reportError(ctx, w, err) - return - } - redirectFound(w, r, wui.NewURLBuilder('h').SetZid(zid)) - } -} DELETED web/adapter/webui/forms.go Index: web/adapter/webui/forms.go ================================================================== --- web/adapter/webui/forms.go +++ /dev/null @@ -1,82 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "net/http" - "strings" - - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/input" -) - -type formZettelData struct { - Heading string - MetaTitle string - MetaRole string - MetaTags string - MetaSyntax string - MetaPairsRest []meta.Pair - IsTextContent bool - Content string -} - -func parseZettelForm(r *http.Request, zid id.Zid) (domain.Zettel, bool, error) { - err := r.ParseForm() - if err != nil { - return domain.Zettel{}, false, err - } - - var m *meta.Meta - if postMeta, ok := trimmedFormValue(r, "meta"); ok { - m = meta.NewFromInput(zid, input.NewInput(postMeta)) - } else { - m = meta.New(zid) - } - if postTitle, ok := trimmedFormValue(r, "title"); ok { - m.Set(meta.KeyTitle, postTitle) - } - if postTags, ok := trimmedFormValue(r, "tags"); ok { - if tags := strings.Fields(postTags); len(tags) > 0 { - m.SetList(meta.KeyTags, tags) - } - } - if postRole, ok := trimmedFormValue(r, "role"); ok { - m.Set(meta.KeyRole, postRole) - } - if postSyntax, ok := trimmedFormValue(r, "syntax"); ok { - m.Set(meta.KeySyntax, postSyntax) - } - if values, ok := r.PostForm["content"]; ok && len(values) > 0 { - return domain.Zettel{ - Meta: m, - Content: domain.NewContent( - strings.ReplaceAll(strings.TrimSpace(values[0]), "\r\n", "\n")), - }, true, nil - } - return domain.Zettel{ - Meta: m, - Content: domain.NewContent(""), - }, false, nil -} - -func trimmedFormValue(r *http.Request, key string) (string, bool) { - if values, ok := r.PostForm[key]; ok && len(values) > 0 { - value := strings.TrimSpace(values[0]) - if len(value) > 0 { - return value, true - } - } - return "", false -} DELETED web/adapter/webui/get_info.go Index: web/adapter/webui/get_info.go ================================================================== --- web/adapter/webui/get_info.go +++ /dev/null @@ -1,185 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "fmt" - "net/http" - "strings" - - "zettelstore.de/z/ast" - "zettelstore.de/z/collect" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/encoder/encfun" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -type metaDataInfo struct { - Key string - Value string -} - -type matrixElement struct { - Text string - HasURL bool - URL string -} -type matrixLine struct { - Elements []matrixElement -} - -// MakeGetInfoHandler creates a new HTTP handler for the use case "get zettel". -func (wui *WebUI) MakeGetInfoHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - q := r.URL.Query() - if format := adapter.GetFormat(r, q, "html"); format != "html" { - wui.reportError(ctx, w, adapter.NewErrBadRequest( - fmt.Sprintf("Zettel info not available in format %q", format))) - return - } - - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - zn, err := parseZettel.Run(ctx, zid, q.Get("syntax")) - if err != nil { - wui.reportError(ctx, w, err) - return - } - - summary := collect.References(zn) - locLinks, extLinks := splitLocExtLinks(append(summary.Links, summary.Images...)) - - lang := config.GetLang(zn.InhMeta, wui.rtConfig) - env := encoder.Environment{Lang: lang} - pairs := zn.Meta.Pairs(true) - metaData := make([]metaDataInfo, len(pairs)) - getTitle := makeGetTitle(ctx, getMeta, &env) - for i, p := range pairs { - var html strings.Builder - wui.writeHTMLMetaValue(&html, zn.Meta, p.Key, getTitle, &env) - metaData[i] = metaDataInfo{p.Key, html.String()} - } - endnotes, err := formatBlocks(nil, "html", &env) - if err != nil { - endnotes = "" - } - - textTitle := encfun.MetaAsText(zn.InhMeta, meta.KeyTitle) - user := wui.getUser(ctx) - var base baseData - wui.makeBaseData(ctx, lang, textTitle, user, &base) - wui.renderTemplate(ctx, w, id.InfoTemplateZid, &base, struct { - Zid string - WebURL string - ContextURL string - CanWrite bool - EditURL string - CanFolge bool - FolgeURL string - CanCopy bool - CopyURL string - CanRename bool - RenameURL string - CanDelete bool - DeleteURL string - MetaData []metaDataInfo - HasLinks bool - HasLocLinks bool - LocLinks []localLink - HasExtLinks bool - ExtLinks []string - ExtNewWindow string - Matrix []matrixLine - Endnotes string - }{ - Zid: zid.String(), - WebURL: wui.NewURLBuilder('h').SetZid(zid).String(), - ContextURL: wui.NewURLBuilder('j').SetZid(zid).String(), - CanWrite: wui.canWrite(ctx, user, zn.Meta, zn.Content), - EditURL: wui.NewURLBuilder('e').SetZid(zid).String(), - CanFolge: base.CanCreate, - FolgeURL: wui.NewURLBuilder('f').SetZid(zid).String(), - CanCopy: base.CanCreate && !zn.Content.IsBinary(), - CopyURL: wui.NewURLBuilder('c').SetZid(zid).String(), - CanRename: wui.canRename(ctx, user, zn.Meta), - RenameURL: wui.NewURLBuilder('b').SetZid(zid).String(), - CanDelete: wui.canDelete(ctx, user, zn.Meta), - DeleteURL: wui.NewURLBuilder('d').SetZid(zid).String(), - MetaData: metaData, - HasLinks: len(extLinks)+len(locLinks) > 0, - HasLocLinks: len(locLinks) > 0, - LocLinks: locLinks, - HasExtLinks: len(extLinks) > 0, - ExtLinks: extLinks, - ExtNewWindow: htmlAttrNewWindow(len(extLinks) > 0), - Matrix: wui.infoAPIMatrix(zid), - Endnotes: endnotes, - }) - } -} - -type localLink struct { - Valid bool - Zid string -} - -func splitLocExtLinks(links []*ast.Reference) (locLinks []localLink, extLinks []string) { - if len(links) == 0 { - return nil, nil - } - for _, ref := range links { - if ref.State == ast.RefStateSelf { - continue - } - if ref.IsZettel() { - continue - } - if ref.IsExternal() { - extLinks = append(extLinks, ref.String()) - continue - } - locLinks = append(locLinks, localLink{ref.IsValid(), ref.String()}) - } - return locLinks, extLinks -} - -func (wui *WebUI) infoAPIMatrix(zid id.Zid) []matrixLine { - formats := encoder.GetFormats() - defFormat := encoder.GetDefaultFormat() - parts := []string{"zettel", "meta", "content"} - matrix := make([]matrixLine, 0, len(parts)) - u := wui.NewURLBuilder('z').SetZid(zid) - for _, part := range parts { - row := make([]matrixElement, 0, len(formats)+1) - row = append(row, matrixElement{part, false, ""}) - for _, format := range formats { - u.AppendQuery("_part", part) - if format != defFormat { - u.AppendQuery("_format", format) - } - row = append(row, matrixElement{format, true, u.String()}) - u.ClearQuery() - } - matrix = append(matrix, matrixLine{row}) - } - return matrix -} DELETED web/adapter/webui/get_zettel.go Index: web/adapter/webui/get_zettel.go ================================================================== --- web/adapter/webui/get_zettel.go +++ /dev/null @@ -1,203 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "bytes" - "net/http" - "strings" - - "zettelstore.de/z/ast" - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/encoder/encfun" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeGetHTMLZettelHandler creates a new HTTP handler for the use case "get zettel". -func (wui *WebUI) MakeGetHTMLZettelHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - syntax := r.URL.Query().Get("syntax") - zn, err := parseZettel.Run(ctx, zid, syntax) - if err != nil { - wui.reportError(ctx, w, err) - return - } - - lang := config.GetLang(zn.InhMeta, wui.rtConfig) - envHTML := encoder.Environment{ - LinkAdapter: adapter.MakeLinkAdapter(ctx, wui, 'h', getMeta, "", ""), - ImageAdapter: adapter.MakeImageAdapter(ctx, wui, getMeta), - CiteAdapter: nil, - Lang: lang, - Xhtml: false, - MarkerExternal: wui.rtConfig.GetMarkerExternal(), - NewWindow: true, - IgnoreMeta: map[string]bool{meta.KeyTitle: true, meta.KeyLang: true}, - } - metaHeader, err := formatMeta(zn.InhMeta, "html", &envHTML) - if err != nil { - wui.reportError(ctx, w, err) - return - } - htmlTitle, err := adapter.FormatInlines( - encfun.MetaAsInlineSlice(zn.InhMeta, meta.KeyTitle), "html", &envHTML) - if err != nil { - wui.reportError(ctx, w, err) - return - } - htmlContent, err := formatBlocks(zn.Ast, "html", &envHTML) - if err != nil { - wui.reportError(ctx, w, err) - return - } - textTitle := encfun.MetaAsText(zn.InhMeta, meta.KeyTitle) - user := wui.getUser(ctx) - roleText := zn.Meta.GetDefault(meta.KeyRole, "*") - tags := wui.buildTagInfos(zn.Meta) - getTitle := makeGetTitle(ctx, getMeta, &encoder.Environment{Lang: lang}) - extURL, hasExtURL := zn.Meta.Get(meta.KeyURL) - backLinks := wui.formatBackLinks(zn.InhMeta, getTitle) - var base baseData - wui.makeBaseData(ctx, lang, textTitle, user, &base) - base.MetaHeader = metaHeader - wui.renderTemplate(ctx, w, id.ZettelTemplateZid, &base, struct { - HTMLTitle string - CanWrite bool - EditURL string - Zid string - InfoURL string - RoleText string - RoleURL string - HasTags bool - Tags []simpleLink - CanCopy bool - CopyURL string - CanFolge bool - FolgeURL string - FolgeRefs string - PrecursorRefs string - HasExtURL bool - ExtURL string - ExtNewWindow string - Content string - HasBackLinks bool - BackLinks []simpleLink - }{ - HTMLTitle: htmlTitle, - CanWrite: wui.canWrite(ctx, user, zn.Meta, zn.Content), - EditURL: wui.NewURLBuilder('e').SetZid(zid).String(), - Zid: zid.String(), - InfoURL: wui.NewURLBuilder('i').SetZid(zid).String(), - RoleText: roleText, - RoleURL: wui.NewURLBuilder('h').AppendQuery("role", roleText).String(), - HasTags: len(tags) > 0, - Tags: tags, - CanCopy: base.CanCreate && !zn.Content.IsBinary(), - CopyURL: wui.NewURLBuilder('c').SetZid(zid).String(), - CanFolge: base.CanCreate, - FolgeURL: wui.NewURLBuilder('f').SetZid(zid).String(), - FolgeRefs: wui.formatMetaKey(zn.InhMeta, meta.KeyFolge, getTitle), - PrecursorRefs: wui.formatMetaKey(zn.InhMeta, meta.KeyPrecursor, getTitle), - ExtURL: extURL, - HasExtURL: hasExtURL, - ExtNewWindow: htmlAttrNewWindow(envHTML.NewWindow && hasExtURL), - Content: htmlContent, - HasBackLinks: len(backLinks) > 0, - BackLinks: backLinks, - }) - } -} - -func formatBlocks(bs ast.BlockSlice, format string, env *encoder.Environment) (string, error) { - enc := encoder.Create(format, env) - if enc == nil { - return "", adapter.ErrNoSuchFormat - } - - var content strings.Builder - _, err := enc.WriteBlocks(&content, bs) - if err != nil { - return "", err - } - return content.String(), nil -} - -func formatMeta(m *meta.Meta, format string, env *encoder.Environment) (string, error) { - enc := encoder.Create(format, env) - if enc == nil { - return "", adapter.ErrNoSuchFormat - } - - var content strings.Builder - _, err := enc.WriteMeta(&content, m) - if err != nil { - return "", err - } - return content.String(), nil -} - -func (wui *WebUI) buildTagInfos(m *meta.Meta) []simpleLink { - var tagInfos []simpleLink - if tags, ok := m.GetList(meta.KeyTags); ok { - ub := wui.NewURLBuilder('h') - tagInfos = make([]simpleLink, len(tags)) - for i, tag := range tags { - tagInfos[i] = simpleLink{Text: tag, URL: ub.AppendQuery("tags", tag).String()} - ub.ClearQuery() - } - } - return tagInfos -} - -func (wui *WebUI) formatMetaKey(m *meta.Meta, key string, getTitle getTitleFunc) string { - if _, ok := m.Get(key); ok { - var buf bytes.Buffer - wui.writeHTMLMetaValue(&buf, m, key, getTitle, nil) - return buf.String() - } - return "" -} - -func (wui *WebUI) formatBackLinks(m *meta.Meta, getTitle getTitleFunc) []simpleLink { - values, ok := m.GetList(meta.KeyBack) - if !ok || len(values) == 0 { - return nil - } - result := make([]simpleLink, 0, len(values)) - for _, val := range values { - zid, err := id.Parse(val) - if err != nil { - continue - } - if title, found := getTitle(zid, "text"); found > 0 { - url := wui.NewURLBuilder('h').SetZid(zid).String() - if title == "" { - result = append(result, simpleLink{Text: val, URL: url}) - } else { - result = append(result, simpleLink{Text: title, URL: url}) - } - } - } - return result -} DELETED web/adapter/webui/home.go Index: web/adapter/webui/home.go ================================================================== --- web/adapter/webui/home.go +++ /dev/null @@ -1,56 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "context" - "errors" - "net/http" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" -) - -type getRootStore interface { - // GetMeta retrieves just the meta data of a specific zettel. - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) -} - -// MakeGetRootHandler creates a new HTTP handler to show the root URL. -func (wui *WebUI) MakeGetRootHandler(s getRootStore) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - if r.URL.Path != "/" { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - homeZid := wui.rtConfig.GetHomeZettel() - if homeZid != id.DefaultHomeZid { - if _, err := s.GetMeta(ctx, homeZid); err == nil { - redirectFound(w, r, wui.NewURLBuilder('h').SetZid(homeZid)) - return - } - homeZid = id.DefaultHomeZid - } - _, err := s.GetMeta(ctx, homeZid) - if err == nil { - redirectFound(w, r, wui.NewURLBuilder('h').SetZid(homeZid)) - return - } - if errors.Is(err, &place.ErrNotAllowed{}) && wui.authz.WithAuth() && wui.getUser(ctx) == nil { - redirectFound(w, r, wui.NewURLBuilder('a')) - return - } - redirectFound(w, r, wui.NewURLBuilder('h')) - } -} DELETED web/adapter/webui/htmlmeta.go Index: web/adapter/webui/htmlmeta.go ================================================================== --- web/adapter/webui/htmlmeta.go +++ /dev/null @@ -1,205 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "context" - "errors" - "fmt" - "io" - "net/url" - "time" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/parser" - "zettelstore.de/z/place" - "zettelstore.de/z/strfun" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -var space = []byte{' '} - -func (wui *WebUI) writeHTMLMetaValue(w io.Writer, m *meta.Meta, key string, getTitle getTitleFunc, env *encoder.Environment) { - switch kt := m.Type(key); kt { - case meta.TypeBool: - wui.writeHTMLBool(w, key, m.GetBool(key)) - case meta.TypeCredential: - writeCredential(w, m.GetDefault(key, "???c")) - case meta.TypeEmpty: - writeEmpty(w, m.GetDefault(key, "???e")) - case meta.TypeID: - wui.writeIdentifier(w, m.GetDefault(key, "???i"), getTitle) - case meta.TypeIDSet: - if l, ok := m.GetList(key); ok { - wui.writeIdentifierSet(w, l, getTitle) - } - case meta.TypeNumber: - writeNumber(w, m.GetDefault(key, "???n")) - case meta.TypeString: - writeString(w, m.GetDefault(key, "???s")) - case meta.TypeTagSet: - if l, ok := m.GetList(key); ok { - wui.writeTagSet(w, key, l) - } - case meta.TypeTimestamp: - if ts, ok := m.GetTime(key); ok { - writeTimestamp(w, ts) - } - case meta.TypeURL: - writeURL(w, m.GetDefault(key, "???u")) - case meta.TypeWord: - wui.writeWord(w, key, m.GetDefault(key, "???w")) - case meta.TypeWordSet: - if l, ok := m.GetList(key); ok { - wui.writeWordSet(w, key, l) - } - case meta.TypeZettelmarkup: - writeZettelmarkup(w, m.GetDefault(key, "???z"), env) - case meta.TypeUnknown: - writeUnknown(w, m.GetDefault(key, "???u")) - default: - strfun.HTMLEscape(w, m.GetDefault(key, "???w"), false) - fmt.Fprintf(w, " <b>(Unhandled type: %v, key: %v)</b>", kt, key) - } -} - -func (wui *WebUI) writeHTMLBool(w io.Writer, key string, val bool) { - if val { - wui.writeLink(w, key, "true", "True") - } else { - wui.writeLink(w, key, "false", "False") - } -} - -func writeCredential(w io.Writer, val string) { - strfun.HTMLEscape(w, val, false) -} - -func writeEmpty(w io.Writer, val string) { - strfun.HTMLEscape(w, val, false) -} - -func (wui *WebUI) writeIdentifier(w io.Writer, val string, getTitle func(id.Zid, string) (string, int)) { - zid, err := id.Parse(val) - if err != nil { - strfun.HTMLEscape(w, val, false) - return - } - title, found := getTitle(zid, "text") - switch { - case found > 0: - if title == "" { - fmt.Fprintf(w, "<a href=\"%v\">%v</a>", wui.NewURLBuilder('h').SetZid(zid), zid) - } else { - fmt.Fprintf(w, "<a href=\"%v\" title=\"%v\">%v</a>", wui.NewURLBuilder('h').SetZid(zid), title, zid) - } - case found == 0: - fmt.Fprintf(w, "<s>%v</s>", val) - case found < 0: - io.WriteString(w, val) - } -} - -func (wui *WebUI) writeIdentifierSet(w io.Writer, vals []string, getTitle func(id.Zid, string) (string, int)) { - for i, val := range vals { - if i > 0 { - w.Write(space) - } - wui.writeIdentifier(w, val, getTitle) - } -} - -func writeNumber(w io.Writer, val string) { - strfun.HTMLEscape(w, val, false) -} - -func writeString(w io.Writer, val string) { - strfun.HTMLEscape(w, val, false) -} - -func writeUnknown(w io.Writer, val string) { - strfun.HTMLEscape(w, val, false) -} - -func (wui *WebUI) writeTagSet(w io.Writer, key string, tags []string) { - for i, tag := range tags { - if i > 0 { - w.Write(space) - } - wui.writeLink(w, key, tag, tag) - } -} - -func writeTimestamp(w io.Writer, ts time.Time) { - io.WriteString(w, ts.Format("2006-01-02 15:04:05")) -} - -func writeURL(w io.Writer, val string) { - u, err := url.Parse(val) - if err != nil { - strfun.HTMLEscape(w, val, false) - return - } - fmt.Fprintf(w, "<a href=\"%v\">", u) - strfun.HTMLEscape(w, val, false) - io.WriteString(w, "</a>") -} - -func (wui *WebUI) writeWord(w io.Writer, key, word string) { - wui.writeLink(w, key, word, word) -} - -func (wui *WebUI) writeWordSet(w io.Writer, key string, words []string) { - for i, word := range words { - if i > 0 { - w.Write(space) - } - wui.writeWord(w, key, word) - } -} -func writeZettelmarkup(w io.Writer, val string, env *encoder.Environment) { - title, err := adapter.FormatInlines(parser.ParseMetadata(val), "html", env) - if err != nil { - strfun.HTMLEscape(w, val, false) - return - } - io.WriteString(w, title) -} - -func (wui *WebUI) writeLink(w io.Writer, key, value, text string) { - fmt.Fprintf(w, "<a href=\"%v?%v=%v\">", wui.NewURLBuilder('h'), url.QueryEscape(key), url.QueryEscape(value)) - strfun.HTMLEscape(w, text, false) - io.WriteString(w, "</a>") -} - -type getTitleFunc func(id.Zid, string) (string, int) - -func makeGetTitle(ctx context.Context, getMeta usecase.GetMeta, env *encoder.Environment) getTitleFunc { - return func(zid id.Zid, format string) (string, int) { - m, err := getMeta.Run(place.NoEnrichContext(ctx), zid) - if err != nil { - if errors.Is(err, &place.ErrNotAllowed{}) { - return "", -1 - } - return "", 0 - } - astTitle := parser.ParseMetadata(m.GetDefault(meta.KeyTitle, "")) - title, err := adapter.FormatInlines(astTitle, format, env) - if err == nil { - return title, 1 - } - return "", 1 - } -} DELETED web/adapter/webui/lists.go Index: web/adapter/webui/lists.go ================================================================== --- web/adapter/webui/lists.go +++ /dev/null @@ -1,383 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "context" - "net/http" - "net/url" - "sort" - "strconv" - "strings" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/parser" - "zettelstore.de/z/place" - "zettelstore.de/z/search" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeListHTMLMetaHandler creates a HTTP handler for rendering the list of -// zettel as HTML. -func (wui *WebUI) MakeListHTMLMetaHandler( - listMeta usecase.ListMeta, - listRole usecase.ListRole, - listTags usecase.ListTags, -) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - query := r.URL.Query() - switch query.Get("_l") { - case "r": - wui.renderRolesList(w, r, listRole) - case "t": - wui.renderTagsList(w, r, listTags) - default: - wui.renderZettelList(w, r, listMeta) - } - } -} - -func (wui *WebUI) renderZettelList(w http.ResponseWriter, r *http.Request, listMeta usecase.ListMeta) { - query := r.URL.Query() - s := adapter.GetSearch(query, false) - ctx := r.Context() - title := wui.listTitleSearch("Filter", s) - wui.renderMetaList( - ctx, w, title, s, - func(s *search.Search) ([]*meta.Meta, error) { - if !s.HasComputedMetaKey() { - ctx = place.NoEnrichContext(ctx) - } - return listMeta.Run(ctx, s) - }, - func(offset int) string { - return wui.newPageURL('h', query, offset, "_offset", "_limit") - }) -} - -type roleInfo struct { - Text string - URL string -} - -func (wui *WebUI) renderRolesList(w http.ResponseWriter, r *http.Request, listRole usecase.ListRole) { - ctx := r.Context() - roleList, err := listRole.Run(ctx) - if err != nil { - adapter.ReportUsecaseError(w, err) - return - } - - roleInfos := make([]roleInfo, 0, len(roleList)) - for _, role := range roleList { - roleInfos = append( - roleInfos, - roleInfo{role, wui.NewURLBuilder('h').AppendQuery("role", role).String()}) - } - - user := wui.getUser(ctx) - var base baseData - wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base) - wui.renderTemplate(ctx, w, id.RolesTemplateZid, &base, struct { - Roles []roleInfo - }{ - Roles: roleInfos, - }) -} - -type countInfo struct { - Count string - URL string -} - -type tagInfo struct { - Name string - URL string - count int - Count string - Size string -} - -var fontSizes = [...]int{75, 83, 100, 117, 150, 200} - -func (wui *WebUI) renderTagsList(w http.ResponseWriter, r *http.Request, listTags usecase.ListTags) { - ctx := r.Context() - iMinCount, _ := strconv.Atoi(r.URL.Query().Get("min")) - tagData, err := listTags.Run(ctx, iMinCount) - if err != nil { - wui.reportError(ctx, w, err) - return - } - - user := wui.getUser(ctx) - tagsList := make([]tagInfo, 0, len(tagData)) - countMap := make(map[int]int) - baseTagListURL := wui.NewURLBuilder('h') - for tag, ml := range tagData { - count := len(ml) - countMap[count]++ - tagsList = append( - tagsList, - tagInfo{tag, baseTagListURL.AppendQuery("tags", tag).String(), count, "", ""}) - baseTagListURL.ClearQuery() - } - sort.Slice(tagsList, func(i, j int) bool { return tagsList[i].Name < tagsList[j].Name }) - - countList := make([]int, 0, len(countMap)) - for count := range countMap { - countList = append(countList, count) - } - sort.Ints(countList) - for pos, count := range countList { - countMap[count] = fontSizes[(pos*len(fontSizes))/len(countList)] - } - for i := 0; i < len(tagsList); i++ { - count := tagsList[i].count - tagsList[i].Count = strconv.Itoa(count) - tagsList[i].Size = strconv.Itoa(countMap[count]) - } - - var base baseData - wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base) - minCounts := make([]countInfo, 0, len(countList)) - for _, c := range countList { - sCount := strconv.Itoa(c) - minCounts = append(minCounts, countInfo{sCount, base.ListTagsURL + "&min=" + sCount}) - } - - wui.renderTemplate(ctx, w, id.TagsTemplateZid, &base, struct { - ListTagsURL string - MinCounts []countInfo - Tags []tagInfo - }{ - ListTagsURL: base.ListTagsURL, - MinCounts: minCounts, - Tags: tagsList, - }) -} - -// MakeSearchHandler creates a new HTTP handler for the use case "search". -func (wui *WebUI) MakeSearchHandler( - ucSearch usecase.Search, - getMeta usecase.GetMeta, - getZettel usecase.GetZettel, -) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - query := r.URL.Query() - ctx := r.Context() - s := adapter.GetSearch(query, true) - if s == nil { - redirectFound(w, r, wui.NewURLBuilder('h')) - return - } - - title := wui.listTitleSearch("Search", s) - wui.renderMetaList( - ctx, w, title, s, func(s *search.Search) ([]*meta.Meta, error) { - if !s.HasComputedMetaKey() { - ctx = place.NoEnrichContext(ctx) - } - return ucSearch.Run(ctx, s) - }, - func(offset int) string { - return wui.newPageURL('f', query, offset, "offset", "limit") - }) - } -} - -// MakeZettelContextHandler creates a new HTTP handler for the use case "zettel context". -func (wui *WebUI) MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - q := r.URL.Query() - dir := usecase.ParseZCDirection(q.Get("dir")) - depth := getIntParameter(q, "depth", 5) - limit := getIntParameter(q, "limit", 200) - metaList, err := getContext.Run(ctx, zid, dir, depth, limit) - if err != nil { - wui.reportError(ctx, w, err) - return - } - metaLinks, err := wui.buildHTMLMetaList(metaList) - if err != nil { - adapter.InternalServerError(w, "Build HTML meta list", err) - return - } - - depths := []string{"2", "3", "4", "5", "6", "7", "8", "9", "10"} - depthLinks := make([]simpleLink, len(depths)) - depthURL := wui.NewURLBuilder('j').SetZid(zid) - for i, depth := range depths { - depthURL.ClearQuery() - switch dir { - case usecase.ZettelContextBackward: - depthURL.AppendQuery("dir", "backward") - case usecase.ZettelContextForward: - depthURL.AppendQuery("dir", "forward") - } - depthURL.AppendQuery("depth", depth) - depthLinks[i].Text = depth - depthLinks[i].URL = depthURL.String() - } - var base baseData - user := wui.getUser(ctx) - wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base) - wui.renderTemplate(ctx, w, id.ContextTemplateZid, &base, struct { - Title string - InfoURL string - Depths []simpleLink - Start simpleLink - Metas []simpleLink - }{ - Title: "Zettel Context", - InfoURL: wui.NewURLBuilder('i').SetZid(zid).String(), - Depths: depthLinks, - Start: metaLinks[0], - Metas: metaLinks[1:], - }) - } -} - -func getIntParameter(q url.Values, key string, minValue int) int { - val, ok := adapter.GetInteger(q, key) - if !ok || val < 0 { - return minValue - } - return val -} - -func (wui *WebUI) renderMetaList( - ctx context.Context, - w http.ResponseWriter, - title string, - s *search.Search, - ucMetaList func(sorter *search.Search) ([]*meta.Meta, error), - pageURL func(int) string) { - - var metaList []*meta.Meta - var err error - var prevURL, nextURL string - if lps := wui.rtConfig.GetListPageSize(); lps > 0 { - if s.GetLimit() < lps { - s.SetLimit(lps + 1) - } - - metaList, err = ucMetaList(s) - if err != nil { - wui.reportError(ctx, w, err) - return - } - if offset := s.GetOffset(); offset > 0 { - offset -= lps - if offset < 0 { - offset = 0 - } - prevURL = pageURL(offset) - } - if len(metaList) >= s.GetLimit() { - nextURL = pageURL(s.GetOffset() + lps) - metaList = metaList[:len(metaList)-1] - } - } else { - metaList, err = ucMetaList(s) - if err != nil { - wui.reportError(ctx, w, err) - return - } - } - user := wui.getUser(ctx) - metas, err := wui.buildHTMLMetaList(metaList) - if err != nil { - wui.reportError(ctx, w, err) - return - } - var base baseData - wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base) - wui.renderTemplate(ctx, w, id.ListTemplateZid, &base, struct { - Title string - Metas []simpleLink - HasPrevNext bool - HasPrev bool - PrevURL string - HasNext bool - NextURL string - }{ - Title: title, - Metas: metas, - HasPrevNext: len(prevURL) > 0 || len(nextURL) > 0, - HasPrev: len(prevURL) > 0, - PrevURL: prevURL, - HasNext: len(nextURL) > 0, - NextURL: nextURL, - }) -} - -func (wui *WebUI) listTitleSearch(prefix string, s *search.Search) string { - if s == nil { - return wui.rtConfig.GetSiteName() - } - var sb strings.Builder - sb.WriteString(prefix) - if s != nil { - sb.WriteString(": ") - s.Print(&sb) - } - return sb.String() -} - -func (wui *WebUI) newPageURL(key byte, query url.Values, offset int, offsetKey, limitKey string) string { - ub := wui.NewURLBuilder(key) - for key, values := range query { - if key != offsetKey && key != limitKey { - for _, val := range values { - ub.AppendQuery(key, val) - } - } - } - if offset > 0 { - ub.AppendQuery(offsetKey, strconv.Itoa(offset)) - } - return ub.String() -} - -// buildHTMLMetaList builds a zettel list based on a meta list for HTML rendering. -func (wui *WebUI) buildHTMLMetaList(metaList []*meta.Meta) ([]simpleLink, error) { - defaultLang := wui.rtConfig.GetDefaultLang() - metas := make([]simpleLink, 0, len(metaList)) - for _, m := range metaList { - var lang string - if val, ok := m.Get(meta.KeyLang); ok { - lang = val - } else { - lang = defaultLang - } - title, _ := m.Get(meta.KeyTitle) - env := encoder.Environment{Lang: lang, Interactive: true} - htmlTitle, err := adapter.FormatInlines(parser.ParseMetadata(title), "html", &env) - if err != nil { - return nil, err - } - metas = append(metas, simpleLink{ - Text: htmlTitle, - URL: wui.NewURLBuilder('h').SetZid(m.Zid).String(), - }) - } - return metas, nil -} DELETED web/adapter/webui/login.go Index: web/adapter/webui/login.go ================================================================== --- web/adapter/webui/login.go +++ /dev/null @@ -1,77 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "context" - "net/http" - - "zettelstore.de/z/auth" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeGetLoginHandler creates a new HTTP handler to display the HTML login view. -func (wui *WebUI) MakeGetLoginHandler() http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - wui.renderLoginForm(wui.clearToken(r.Context(), w), w, false) - } -} - -func (wui *WebUI) renderLoginForm(ctx context.Context, w http.ResponseWriter, retry bool) { - var base baseData - wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), "Login", nil, &base) - wui.renderTemplate(ctx, w, id.LoginTemplateZid, &base, struct { - Title string - Retry bool - }{ - Title: base.Title, - Retry: retry, - }) -} - -// MakePostLoginHandlerHTML creates a new HTTP handler to authenticate the given user. -func (wui *WebUI) MakePostLoginHandlerHTML(ucAuth usecase.Authenticate) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - if !wui.authz.WithAuth() { - redirectFound(w, r, wui.NewURLBuilder('/')) - return - } - ctx := r.Context() - ident, cred, ok := adapter.GetCredentialsViaForm(r) - if !ok { - wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read login form")) - return - } - token, err := ucAuth.Run(ctx, ident, cred, wui.tokenLifetime, auth.KindHTML) - if err != nil { - wui.reportError(ctx, w, err) - return - } - if token == nil { - wui.renderLoginForm(wui.clearToken(ctx, w), w, true) - return - } - - wui.setToken(w, token) - redirectFound(w, r, wui.NewURLBuilder('/')) - } -} - -// MakeGetLogoutHandler creates a new HTTP handler to log out the current user -func (wui *WebUI) MakeGetLogoutHandler() http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - wui.clearToken(r.Context(), w) - redirectFound(w, r, wui.NewURLBuilder('/')) - } -} DELETED web/adapter/webui/rename_zettel.go Index: web/adapter/webui/rename_zettel.go ================================================================== --- web/adapter/webui/rename_zettel.go +++ /dev/null @@ -1,94 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "fmt" - "net/http" - "strings" - - "zettelstore.de/z/config" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/place" - "zettelstore.de/z/usecase" - "zettelstore.de/z/web/adapter" -) - -// MakeGetRenameZettelHandler creates a new HTTP handler to display the -// HTML rename view of a zettel. -func (wui *WebUI) MakeGetRenameZettelHandler(getMeta usecase.GetMeta) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - zid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - m, err := getMeta.Run(ctx, zid) - if err != nil { - wui.reportError(ctx, w, err) - return - } - - if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" { - wui.reportError(ctx, w, adapter.NewErrBadRequest( - fmt.Sprintf("Rename zettel %q not possible in format %q", zid.String(), format))) - return - } - - user := wui.getUser(ctx) - var base baseData - wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Rename Zettel "+zid.String(), user, &base) - wui.renderTemplate(ctx, w, id.RenameTemplateZid, &base, struct { - Zid string - MetaPairs []meta.Pair - }{ - Zid: zid.String(), - MetaPairs: m.Pairs(true), - }) - } -} - -// MakePostRenameZettelHandler creates a new HTTP handler to rename an existing zettel. -func (wui *WebUI) MakePostRenameZettelHandler(renameZettel usecase.RenameZettel) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - ctx := r.Context() - curZid, err := id.Parse(r.URL.Path[1:]) - if err != nil { - wui.reportError(ctx, w, place.ErrNotFound) - return - } - - if err = r.ParseForm(); err != nil { - wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read rename zettel form")) - return - } - if formCurZid, err1 := id.Parse( - r.PostFormValue("curzid")); err1 != nil || formCurZid != curZid { - wui.reportError(ctx, w, adapter.NewErrBadRequest("Invalid value for current zettel id in form")) - return - } - newZid, err := id.Parse(strings.TrimSpace(r.PostFormValue("newzid"))) - if err != nil { - wui.reportError(ctx, w, adapter.NewErrBadRequest(fmt.Sprintf("Invalid new zettel id %q", newZid))) - return - } - - if err := renameZettel.Run(r.Context(), curZid, newZid); err != nil { - wui.reportError(ctx, w, err) - return - } - redirectFound(w, r, wui.NewURLBuilder('h').SetZid(newZid)) - } -} DELETED web/adapter/webui/response.go Index: web/adapter/webui/response.go ================================================================== --- web/adapter/webui/response.go +++ /dev/null @@ -1,22 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "net/http" - - "zettelstore.de/z/web/server" -) - -func redirectFound(w http.ResponseWriter, r *http.Request, ub server.URLBuilder) { - http.Redirect(w, r, ub.String(), http.StatusFound) -} DELETED web/adapter/webui/webui.go Index: web/adapter/webui/webui.go ================================================================== --- web/adapter/webui/webui.go +++ /dev/null @@ -1,348 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package webui provides web-UI handlers for web requests. -package webui - -import ( - "bytes" - "context" - "log" - "net/http" - "sync" - "time" - - "zettelstore.de/z/auth" - "zettelstore.de/z/collect" - "zettelstore.de/z/config" - "zettelstore.de/z/domain" - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/encoder" - "zettelstore.de/z/input" - "zettelstore.de/z/kernel" - "zettelstore.de/z/parser" - "zettelstore.de/z/place" - "zettelstore.de/z/template" - "zettelstore.de/z/web/adapter" - "zettelstore.de/z/web/server" -) - -// WebUI holds all data for delivering the web ui. -type WebUI struct { - ab server.AuthBuilder - authz auth.AuthzManager - rtConfig config.Config - token auth.TokenManager - place webuiPlace - policy auth.Policy - - templateCache map[id.Zid]*template.Template - mxCache sync.RWMutex - - tokenLifetime time.Duration - stylesheetURL string - homeURL string - listZettelURL string - listRolesURL string - listTagsURL string - withAuth bool - loginURL string - searchURL string -} - -type webuiPlace interface { - CanCreateZettel(ctx context.Context) bool - GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) - GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) - CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool - AllowRenameZettel(ctx context.Context, zid id.Zid) bool - CanDeleteZettel(ctx context.Context, zid id.Zid) bool -} - -// New creates a new WebUI struct. -func New(ab server.AuthBuilder, authz auth.AuthzManager, rtConfig config.Config, token auth.TokenManager, - mgr place.Manager, pol auth.Policy) *WebUI { - wui := &WebUI{ - ab: ab, - rtConfig: rtConfig, - authz: authz, - token: token, - place: mgr, - policy: pol, - - tokenLifetime: kernel.Main.GetConfig(kernel.WebService, kernel.WebTokenLifetimeHTML).(time.Duration), - stylesheetURL: ab.NewURLBuilder('z').SetZid( - id.BaseCSSZid).AppendQuery("_format", "raw").AppendQuery( - "_part", "content").String(), - homeURL: ab.NewURLBuilder('/').String(), - listZettelURL: ab.NewURLBuilder('h').String(), - listRolesURL: ab.NewURLBuilder('h').AppendQuery("_l", "r").String(), - listTagsURL: ab.NewURLBuilder('h').AppendQuery("_l", "t").String(), - withAuth: authz.WithAuth(), - loginURL: ab.NewURLBuilder('a').String(), - searchURL: ab.NewURLBuilder('f').String(), - } - wui.observe(place.UpdateInfo{Place: mgr, Reason: place.OnReload, Zid: id.Invalid}) - mgr.RegisterObserver(wui.observe) - return wui -} - -func (wui *WebUI) observe(ci place.UpdateInfo) { - wui.mxCache.Lock() - if ci.Reason == place.OnReload || ci.Zid == id.BaseTemplateZid { - wui.templateCache = make(map[id.Zid]*template.Template, len(wui.templateCache)) - } else { - delete(wui.templateCache, ci.Zid) - } - wui.mxCache.Unlock() -} - -func (wui *WebUI) cacheSetTemplate(zid id.Zid, t *template.Template) { - wui.mxCache.Lock() - wui.templateCache[zid] = t - wui.mxCache.Unlock() -} - -func (wui *WebUI) cacheGetTemplate(zid id.Zid) (*template.Template, bool) { - wui.mxCache.RLock() - t, ok := wui.templateCache[zid] - wui.mxCache.RUnlock() - return t, ok -} - -func (wui *WebUI) canCreate(ctx context.Context, user *meta.Meta) bool { - m := meta.New(id.Invalid) - return wui.policy.CanCreate(user, m) && wui.place.CanCreateZettel(ctx) -} - -func (wui *WebUI) canWrite( - ctx context.Context, user, meta *meta.Meta, content domain.Content) bool { - return wui.policy.CanWrite(user, meta, meta) && - wui.place.CanUpdateZettel(ctx, domain.Zettel{Meta: meta, Content: content}) -} - -func (wui *WebUI) canRename(ctx context.Context, user, m *meta.Meta) bool { - return wui.policy.CanRename(user, m) && wui.place.AllowRenameZettel(ctx, m.Zid) -} - -func (wui *WebUI) canDelete(ctx context.Context, user, m *meta.Meta) bool { - return wui.policy.CanDelete(user, m) && wui.place.CanDeleteZettel(ctx, m.Zid) -} - -func (wui *WebUI) getTemplate( - ctx context.Context, templateID id.Zid) (*template.Template, error) { - if t, ok := wui.cacheGetTemplate(templateID); ok { - return t, nil - } - realTemplateZettel, err := wui.place.GetZettel(ctx, templateID) - if err != nil { - return nil, err - } - t, err := template.ParseString(realTemplateZettel.Content.AsString(), nil) - if err == nil { - // t.SetErrorOnMissing() - wui.cacheSetTemplate(templateID, t) - } - return t, err -} - -type simpleLink struct { - Text string - URL string -} - -type baseData struct { - Lang string - MetaHeader string - StylesheetURL string - Title string - HomeURL string - WithUser bool - WithAuth bool - UserIsValid bool - UserZettelURL string - UserIdent string - UserLogoutURL string - LoginURL string - ListZettelURL string - ListRolesURL string - ListTagsURL string - CanCreate bool - NewZettelURL string - NewZettelLinks []simpleLink - SearchURL string - Content string - FooterHTML string -} - -func (wui *WebUI) makeBaseData( - ctx context.Context, lang, title string, user *meta.Meta, data *baseData) { - var ( - newZettelLinks []simpleLink - userZettelURL string - userIdent string - userLogoutURL string - ) - canCreate := wui.canCreate(ctx, user) - if canCreate { - newZettelLinks = wui.fetchNewTemplates(ctx, user) - } - userIsValid := user != nil - if userIsValid { - userZettelURL = wui.NewURLBuilder('h').SetZid(user.Zid).String() - userIdent = user.GetDefault(meta.KeyUserID, "") - userLogoutURL = wui.NewURLBuilder('a').SetZid(user.Zid).String() - } - - data.Lang = lang - data.StylesheetURL = wui.stylesheetURL - data.Title = title - data.HomeURL = wui.homeURL - data.WithAuth = wui.withAuth - data.WithUser = data.WithAuth - data.UserIsValid = userIsValid - data.UserZettelURL = userZettelURL - data.UserIdent = userIdent - data.UserLogoutURL = userLogoutURL - data.LoginURL = wui.loginURL - data.ListZettelURL = wui.listZettelURL - data.ListRolesURL = wui.listRolesURL - data.ListTagsURL = wui.listTagsURL - data.CanCreate = canCreate - data.NewZettelLinks = newZettelLinks - data.SearchURL = wui.searchURL - data.FooterHTML = wui.rtConfig.GetFooterHTML() -} - -// htmlAttrNewWindow eturns HTML attribute string for opening a link in a new window. -// If hasURL is false an empty string is returned. -func htmlAttrNewWindow(hasURL bool) string { - if hasURL { - return " target=\"_blank\" ref=\"noopener noreferrer\"" - } - return "" -} - -func (wui *WebUI) fetchNewTemplates(ctx context.Context, user *meta.Meta) []simpleLink { - ctx = place.NoEnrichContext(ctx) - menu, err := wui.place.GetZettel(ctx, id.TOCNewTemplateZid) - if err != nil { - return nil - } - zn := parser.ParseZettel(menu, "", wui.rtConfig) - refs := collect.Order(zn) - result := make([]simpleLink, 0, len(refs)) - for _, ref := range refs { - zid, err := id.Parse(ref.URL.Path) - if err != nil { - continue - } - m, err := wui.place.GetMeta(ctx, zid) - if err != nil { - continue - } - if !wui.policy.CanRead(user, m) { - continue - } - title := config.GetTitle(m, wui.rtConfig) - astTitle := parser.ParseInlines(input.NewInput(title), meta.ValueSyntaxZmk) - env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)} - menuTitle, err := adapter.FormatInlines(astTitle, "html", &env) - if err != nil { - menuTitle, err = adapter.FormatInlines(astTitle, "text", nil) - if err != nil { - menuTitle = title - } - } - result = append(result, simpleLink{ - Text: menuTitle, - URL: wui.NewURLBuilder('g').SetZid(m.Zid).String(), - }) - } - return result -} - -func (wui *WebUI) renderTemplate( - ctx context.Context, - w http.ResponseWriter, - templateID id.Zid, - base *baseData, - data interface{}) { - wui.renderTemplateStatus(ctx, w, http.StatusOK, templateID, base, data) -} - -func (wui *WebUI) reportError(ctx context.Context, w http.ResponseWriter, err error) { - code, text := adapter.CodeMessageFromError(err) - if code == http.StatusInternalServerError { - log.Printf("%v: %v", text, err) - } - user := wui.getUser(ctx) - var base baseData - wui.makeBaseData(ctx, meta.ValueLangEN, "Error", user, &base) - wui.renderTemplateStatus(ctx, w, code, id.ErrorTemplateZid, &base, struct { - ErrorTitle string - ErrorText string - }{ - ErrorTitle: http.StatusText(code), - ErrorText: text, - }) -} - -func (wui *WebUI) renderTemplateStatus( - ctx context.Context, - w http.ResponseWriter, - code int, - templateID id.Zid, - base *baseData, - data interface{}) { - - bt, err := wui.getTemplate(ctx, id.BaseTemplateZid) - if err != nil { - adapter.InternalServerError(w, "Unable to get base template", err) - return - } - t, err := wui.getTemplate(ctx, templateID) - if err != nil { - adapter.InternalServerError(w, "Unable to get template", err) - return - } - if user := wui.getUser(ctx); user != nil { - if tok, err1 := wui.token.GetToken(user, wui.tokenLifetime, auth.KindHTML); err1 == nil { - wui.setToken(w, tok) - } - } - var content bytes.Buffer - err = t.Render(&content, data) - if err == nil { - base.Content = content.String() - w.Header().Set(adapter.ContentType, "text/html; charset=utf-8") - w.WriteHeader(code) - err = bt.Render(w, base) - } - if err != nil { - log.Println("Unable to render template", err) - } -} - -func (wui *WebUI) getUser(ctx context.Context) *meta.Meta { return wui.ab.GetUser(ctx) } - -// GetURLPrefix returns the configured URL prefix of the web server. -func (wui *WebUI) GetURLPrefix() string { return wui.ab.GetURLPrefix() } - -// NewURLBuilder creates a new URL builder object with the given key. -func (wui *WebUI) NewURLBuilder(key byte) server.URLBuilder { return wui.ab.NewURLBuilder(key) } - -func (wui *WebUI) clearToken(ctx context.Context, w http.ResponseWriter) context.Context { - return wui.ab.ClearToken(ctx, w) -} -func (wui *WebUI) setToken(w http.ResponseWriter, token []byte) { - wui.ab.SetToken(w, token, wui.tokenLifetime) -} DELETED web/server/impl/http.go Index: web/server/impl/http.go ================================================================== --- web/server/impl/http.go +++ /dev/null @@ -1,78 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package impl provides the Zettelstore web service. -package impl - -import ( - "context" - "net" - "net/http" - "time" -) - -// Server timeout values -const ( - shutdownTimeout = 5 * time.Second - readTimeout = 5 * time.Second - writeTimeout = 10 * time.Second - idleTimeout = 120 * time.Second -) - -// httpServer is a HTTP server. -type httpServer struct { - http.Server - waitStop chan struct{} -} - -// initializeHTTPServer creates a new HTTP server object. -func (srv *httpServer) initializeHTTPServer(addr string, handler http.Handler) { - if addr == "" { - addr = ":http" - } - srv.Server = http.Server{ - Addr: addr, - Handler: handler, - - // See: https://blog.cloudflare.com/exposing-go-on-the-internet/ - ReadTimeout: readTimeout, - WriteTimeout: writeTimeout, - IdleTimeout: idleTimeout, - } - srv.waitStop = make(chan struct{}) -} - -// SetDebug enables debugging goroutines that are started by the server. -// Basically, just the timeout values are reset. This method should be called -// before running the server. -func (srv *httpServer) SetDebug() { - srv.ReadTimeout = 0 - srv.WriteTimeout = 0 - srv.IdleTimeout = 0 -} - -// Run starts the web server, but does not wait for its completion. -func (srv *httpServer) Run() error { - ln, err := net.Listen("tcp", srv.Addr) - if err != nil { - return err - } - - go func() { srv.Serve(ln) }() - return nil -} - -// Stop the web server. -func (srv *httpServer) Stop() error { - ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout) - defer cancel() - - return srv.Shutdown(ctx) -} DELETED web/server/impl/impl.go Index: web/server/impl/impl.go ================================================================== --- web/server/impl/impl.go +++ /dev/null @@ -1,123 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package impl provides the Zettelstore web service. -package impl - -import ( - "context" - "net/http" - "time" - - "zettelstore.de/z/auth" - "zettelstore.de/z/domain/meta" - "zettelstore.de/z/web/server" -) - -type myServer struct { - server httpServer - router httpRouter - persistentCookie bool - secureCookie bool -} - -// New creates a new web server. -func New(listenAddr, urlPrefix string, persistentCookie, secureCookie bool, auth auth.TokenManager) server.Server { - srv := myServer{ - persistentCookie: persistentCookie, - secureCookie: secureCookie, - } - srv.router.initializeRouter(urlPrefix, auth) - srv.server.initializeHTTPServer(listenAddr, &srv.router) - return &srv -} - -func (srv *myServer) Handle(pattern string, handler http.Handler) { - srv.router.Handle(pattern, handler) -} -func (srv *myServer) AddListRoute(key byte, httpMethod string, handler http.Handler) { - srv.router.addListRoute(key, httpMethod, handler) -} -func (srv *myServer) AddZettelRoute(key byte, httpMethod string, handler http.Handler) { - srv.router.addZettelRoute(key, httpMethod, handler) -} -func (srv *myServer) SetUserRetriever(ur server.UserRetriever) { - srv.router.ur = ur -} -func (srv *myServer) GetUser(ctx context.Context) *meta.Meta { - if data := srv.GetAuthData(ctx); data != nil { - return data.User - } - return nil -} -func (srv *myServer) NewURLBuilder(key byte) server.URLBuilder { - return &URLBuilder{router: &srv.router, key: key} -} -func (srv *myServer) GetURLPrefix() string { - return srv.router.urlPrefix -} - -const sessionName = "zsession" - -func (srv *myServer) SetToken(w http.ResponseWriter, token []byte, d time.Duration) { - cookie := http.Cookie{ - Name: sessionName, - Value: string(token), - Path: srv.GetURLPrefix(), - Secure: srv.secureCookie, - HttpOnly: true, - SameSite: http.SameSiteStrictMode, - } - if srv.persistentCookie && d > 0 { - cookie.Expires = time.Now().Add(d).Add(30 * time.Second).UTC() - } - http.SetCookie(w, &cookie) -} - -// ClearToken invalidates the session cookie by sending an empty one. -func (srv *myServer) ClearToken(ctx context.Context, w http.ResponseWriter) context.Context { - if w != nil { - srv.SetToken(w, nil, 0) - } - return updateContext(ctx, nil, nil) -} - -// GetAuthData returns the full authentication data from the context. -func (srv *myServer) GetAuthData(ctx context.Context) *server.AuthData { - data, ok := ctx.Value(ctxKeySession).(*server.AuthData) - if ok { - return data - } - return nil -} - -type ctxKeyTypeSession struct{} - -var ctxKeySession ctxKeyTypeSession - -func updateContext(ctx context.Context, user *meta.Meta, data *auth.TokenData) context.Context { - if data == nil { - return context.WithValue(ctx, ctxKeySession, &server.AuthData{User: user}) - } - return context.WithValue( - ctx, - ctxKeySession, - &server.AuthData{ - User: user, - Token: data.Token, - Now: data.Now, - Issued: data.Issued, - Expires: data.Expires, - }) -} - -func (srv *myServer) SetDebug() { srv.server.SetDebug() } -func (srv *myServer) Run() error { return srv.server.Run() } -func (srv *myServer) Stop() error { return srv.server.Stop() } DELETED web/server/impl/router.go Index: web/server/impl/router.go ================================================================== --- web/server/impl/router.go +++ /dev/null @@ -1,173 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package impl provides the Zettelstore web service. -package impl - -import ( - "net/http" - "regexp" - "strings" - - "zettelstore.de/z/auth" - "zettelstore.de/z/web/server" -) - -type ( - methodHandler map[string]http.Handler - routingTable map[byte]methodHandler -) - -// httpRouter handles all routing for zettelstore. -type httpRouter struct { - urlPrefix string - auth auth.TokenManager - minKey byte - maxKey byte - reURL *regexp.Regexp - listTable routingTable - zettelTable routingTable - ur server.UserRetriever - mux *http.ServeMux -} - -// initializeRouter creates a new, empty router with the given root handler. -func (rt *httpRouter) initializeRouter(urlPrefix string, auth auth.TokenManager) { - rt.urlPrefix = urlPrefix - rt.auth = auth - rt.minKey = 255 - rt.maxKey = 0 - rt.reURL = regexp.MustCompile("^$") - rt.mux = http.NewServeMux() - rt.listTable = make(routingTable) - rt.zettelTable = make(routingTable) -} - -func (rt *httpRouter) addRoute(key byte, httpMethod string, handler http.Handler, table routingTable) { - // Set minKey and maxKey; re-calculate regexp. - if key < rt.minKey || rt.maxKey < key { - if key < rt.minKey { - rt.minKey = key - } - if rt.maxKey < key { - rt.maxKey = key - } - rt.reURL = regexp.MustCompile( - "^/(?:([" + string(rt.minKey) + "-" + string(rt.maxKey) + "])(?:/(?:([0-9]{14})/?)?)?)$") - } - - mh, hasKey := table[key] - if !hasKey { - mh = make(methodHandler) - table[key] = mh - } - mh[httpMethod] = handler - if httpMethod == http.MethodGet { - if _, hasHead := table[key][http.MethodHead]; !hasHead { - table[key][http.MethodHead] = handler - } - } -} - -// addListRoute adds a route for the given key and HTTP method to work with a list. -func (rt *httpRouter) addListRoute(key byte, httpMethod string, handler http.Handler) { - rt.addRoute(key, httpMethod, handler, rt.listTable) -} - -// addZettelRoute adds a route for the given key and HTTP method to work with a zettel. -func (rt *httpRouter) addZettelRoute(key byte, httpMethod string, handler http.Handler) { - rt.addRoute(key, httpMethod, handler, rt.zettelTable) -} - -// Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics. -func (rt *httpRouter) Handle(pattern string, handler http.Handler) { - rt.mux.Handle(pattern, handler) -} - -func (rt *httpRouter) ServeHTTP(w http.ResponseWriter, r *http.Request) { - if prefixLen := len(rt.urlPrefix); prefixLen > 1 { - if len(r.URL.Path) < prefixLen || r.URL.Path[:prefixLen] != rt.urlPrefix { - http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound) - return - } - r.URL.Path = r.URL.Path[prefixLen-1:] - } - match := rt.reURL.FindStringSubmatch(r.URL.Path) - if len(match) == 3 { - key := match[1][0] - table := rt.zettelTable - if match[2] == "" { - table = rt.listTable - } - if mh, ok := table[key]; ok { - if handler, ok := mh[r.Method]; ok { - r.URL.Path = "/" + match[2] - handler.ServeHTTP(w, rt.addUserContext(r)) - return - } - http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed) - return - } - } - rt.mux.ServeHTTP(w, rt.addUserContext(r)) -} - -func (rt *httpRouter) addUserContext(r *http.Request) *http.Request { - if rt.ur == nil { - return r - } - k := auth.KindJSON - t := getHeaderToken(r) - if len(t) == 0 { - k = auth.KindHTML - t = getSessionToken(r) - } - if len(t) == 0 { - return r - } - tokenData, err := rt.auth.CheckToken(t, k) - if err != nil { - return r - } - ctx := r.Context() - user, err := rt.ur.GetUser(ctx, tokenData.Zid, tokenData.Ident) - if err != nil { - return r - } - return r.WithContext(updateContext(ctx, user, &tokenData)) -} - -func getSessionToken(r *http.Request) []byte { - cookie, err := r.Cookie(sessionName) - if err != nil { - return nil - } - return []byte(cookie.Value) -} - -func getHeaderToken(r *http.Request) []byte { - h := r.Header["Authorization"] - if h == nil { - return nil - } - - // “Multiple message-header fields with the same field-name MAY be - // present in a message if and only if the entire field-value for that - // header field is defined as a comma-separated list.” - // — “Hypertext Transfer Protocol” RFC 2616, subsection 4.2 - auth := strings.Join(h, ", ") - - const prefix = "Bearer " - // RFC 2617, subsection 1.2 defines the scheme token as case-insensitive. - if len(auth) < len(prefix) || !strings.EqualFold(auth[:len(prefix)], prefix) { - return nil - } - return []byte(auth[len(prefix):]) -} DELETED web/server/impl/urlbuilder.go Index: web/server/impl/urlbuilder.go ================================================================== --- web/server/impl/urlbuilder.go +++ /dev/null @@ -1,110 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package impl provides the Zettelstore web service. -package impl - -import ( - "net/url" - "strings" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/web/server" -) - -type urlQuery struct{ key, val string } - -// URLBuilder should be used to create zettelstore URLs. -type URLBuilder struct { - router *httpRouter - key byte - path []string - query []urlQuery - fragment string -} - -// Clone an URLBuilder -func (ub *URLBuilder) Clone() server.URLBuilder { - cpy := new(URLBuilder) - cpy.key = ub.key - if len(ub.path) > 0 { - cpy.path = make([]string, 0, len(ub.path)) - cpy.path = append(cpy.path, ub.path...) - } - if len(ub.query) > 0 { - cpy.query = make([]urlQuery, 0, len(ub.query)) - cpy.query = append(cpy.query, ub.query...) - } - cpy.fragment = ub.fragment - return cpy -} - -// SetZid sets the zettel identifier. -func (ub *URLBuilder) SetZid(zid id.Zid) server.URLBuilder { - if len(ub.path) > 0 { - panic("Cannot add Zid") - } - ub.path = append(ub.path, zid.String()) - return ub -} - -// AppendPath adds a new path element -func (ub *URLBuilder) AppendPath(p string) server.URLBuilder { - ub.path = append(ub.path, p) - return ub -} - -// AppendQuery adds a new query parameter -func (ub *URLBuilder) AppendQuery(key, value string) server.URLBuilder { - ub.query = append(ub.query, urlQuery{key, value}) - return ub -} - -// ClearQuery removes all query parameters. -func (ub *URLBuilder) ClearQuery() server.URLBuilder { - ub.query = nil - ub.fragment = "" - return ub -} - -// SetFragment stores the fragment -func (ub *URLBuilder) SetFragment(s string) server.URLBuilder { - ub.fragment = s - return ub -} - -// String produces a string value. -func (ub *URLBuilder) String() string { - var sb strings.Builder - - sb.WriteString(ub.router.urlPrefix) - if ub.key != '/' { - sb.WriteByte(ub.key) - } - for _, p := range ub.path { - sb.WriteByte('/') - sb.WriteString(url.PathEscape(p)) - } - if len(ub.fragment) > 0 { - sb.WriteByte('#') - sb.WriteString(ub.fragment) - } - for i, q := range ub.query { - if i == 0 { - sb.WriteByte('?') - } else { - sb.WriteByte('&') - } - sb.WriteString(q.key) - sb.WriteByte('=') - sb.WriteString(url.QueryEscape(q.val)) - } - return sb.String() -} DELETED web/server/server.go Index: web/server/server.go ================================================================== --- web/server/server.go +++ /dev/null @@ -1,102 +0,0 @@ -//----------------------------------------------------------------------------- -// Copyright (c) 2020-2021 Detlef Stern -// -// This file is part of zettelstore. -// -// Zettelstore is licensed under the latest version of the EUPL (European Union -// Public License). Please see file LICENSE.txt for your rights and obligations -// under this license. -//----------------------------------------------------------------------------- - -// Package server provides the Zettelstore web service. -package server - -import ( - "context" - "net/http" - "time" - - "zettelstore.de/z/domain/id" - "zettelstore.de/z/domain/meta" -) - -// URLBuilder builds URLs. -type URLBuilder interface { - // Clone an URLBuilder - Clone() URLBuilder - - // SetZid sets the zettel identifier. - SetZid(zid id.Zid) URLBuilder - - // AppendPath adds a new path element - AppendPath(p string) URLBuilder - - // AppendQuery adds a new query parameter - AppendQuery(key, value string) URLBuilder - - // ClearQuery removes all query parameters. - ClearQuery() URLBuilder - - // SetFragment stores the fragment - SetFragment(s string) URLBuilder - - // String produces a string value. - String() string -} - -// UserRetriever allows to retrieve user data based on a given zettel identifier. -type UserRetriever interface { - GetUser(ctx context.Context, zid id.Zid, ident string) (*meta.Meta, error) -} - -// Router allows to state routes for various URL paths. -type Router interface { - Handle(pattern string, handler http.Handler) - AddListRoute(key byte, httpMethod string, handler http.Handler) - AddZettelRoute(key byte, httpMethod string, handler http.Handler) - SetUserRetriever(ur UserRetriever) -} - -// Builder allows to build new URLs for the web service. -type Builder interface { - GetURLPrefix() string - NewURLBuilder(key byte) URLBuilder -} - -// Auth is. -type Auth interface { - GetUser(context.Context) *meta.Meta - SetToken(w http.ResponseWriter, token []byte, d time.Duration) - - // ClearToken invalidates the session cookie by sending an empty one. - ClearToken(ctx context.Context, w http.ResponseWriter) context.Context - - // GetAuthData returns the full authentication data from the context. - GetAuthData(ctx context.Context) *AuthData -} - -// AuthData stores all relevant authentication data for a context. -type AuthData struct { - User *meta.Meta - Token []byte - Now time.Time - Issued time.Time - Expires time.Time -} - -// AuthBuilder is a Builder that also allows to execute authentication functions. -type AuthBuilder interface { - Auth - Builder -} - -// Server is the main web server for accessing Zettelstore via HTTP. -type Server interface { - Router - Auth - Builder - - SetDebug() - Run() error - Stop() error -} Index: www/build.md ================================================================== --- www/build.md +++ www/build.md @@ -1,59 +1,94 @@ -# How to build the Zettelstore +# How to build Zettelstore + ## Prerequisites + You must install the following software: -* A current, supported [release of Go](https://golang.org/doc/devel/release.html), -* [golint](https://github.com/golang/lint|golint), -* [Fossil](https://fossil-scm.org/). +* A current, supported [release of Go](https://go.dev/doc/devel/release), +* [staticcheck](https://staticcheck.io/), +* [shadow](https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/shadow), +* [unparam](https://mvdan.cc/unparam), +* [govulncheck](https://golang.org/x/vuln/cmd/govulncheck), +* [revive](https://revive.run/), +* [Fossil](https://fossil-scm.org/), +* [Git](https://git-scm.org) (so that Go can download some dependencies). + +See folder `docs/development` (a zettel box) for details. ## Clone the repository -Most of this is covered by the excellent Fossil documentation. +Most of this is covered by the excellent Fossil +[documentation](https://fossil-scm.org/home/doc/trunk/www/quickstart.wiki). 1. Create a directory to store your Fossil repositories. - Let's assume, you have created <tt>$HOME/fossil</tt>. -1. Clone the repository: `fossil clone https://zettelstore.de/ $HOME/fossil/zettelstore.fossil`. + Let's assume, you have created `$HOME/fossils`. +1. Clone the repository: `fossil clone https://zettelstore.de/ $HOME/fossils/zettelstore.fossil`. 1. Create a working directory. - Let's assume, you have created <tt>$HOME/zettelstore</tt>. + Let's assume, you have created `$HOME/zettelstore`. 1. Change into this directory: `cd $HOME/zettelstore`. -1. Open development: `fossil open $HOME/fossil/zettelstore.fossil`. - -(If you are not able to use Fossil, you could try the Git mirror -<https://github.com/zettelstore/zettelstore>.) - -## The build tool -In directory <tt>tools</tt> there is a Go file called <tt>build.go</tt>. -It automates most aspects, (hopefully) platform-independent. - -The script is called as: - -``` -go run tools/build.go [-v] COMMAND -``` +1. Open development: `fossil open $HOME/fossils/zettelstore.fossil`. + +## Tools to build, test, and manage +In the directory `tools` there are some Go files to automate most aspects of +building and testing, (hopefully) platform-independent. + +The build script is called as: + + go run tools/build/build.go [-v] COMMAND The flag `-v` enables the verbose mode. It outputs all commands called by the tool. -`COMMAND` is one of: +Some important `COMMAND`s are: -* `build`: builds the software with correct version information and places it - into a freshly created directory <tt>bin</tt>. +* `build`: builds the software with correct version information and puts it + into a freshly created directory `bin`. * `check`: checks the current state of the working directory to be ready for release (or commit). -* `release`: executes `check` command and if this was successful, builds the - software for various platforms, and creates ZIP files for each executable. - Everything is placed in the directory <tt>releases</tt>. -* `clean`: removes the directories <tt>bin</tt> and <tt>releases</tt>. * `version`: prints the current version information. Therefore, the easiest way to build your own version of the Zettelstore software is to execute the command -``` -go run tools/build.go build -``` + go run tools/build/build.go build In case of errors, please send the output of the verbose execution: -``` -go run tools/build.go -v build -``` + go run tools/build/build.go -v build + +Other tools are: + +* `go run tools/clean/clean.go` cleans your Go development workspace. +* `go run tools/check/check.go` executes all linters and unit tests. + If you add the option `-r` linters are more strict, to be used for a + release version. +* `go run tools/devtools/devtools.go` install all needed software (see above). +* `go run tools/htmllint/htmllint.go [URL]` checks all generated HTML of a + Zettelstore accessible at the given URL (default: http://localhost:23123). +* `go run tools/testapi/testapi.go` tests the API against a running + Zettelstore, which is started automatically. + +## A note on the use of Fossil + +Zettelstore is managed by the Fossil version control system. Fossil is an +alternative to the ubiquitous Git version control system. However, Go seems to +prefer Git and popular platforms that just support Git. + +Some dependencies of Zettelstore, namely [Zettelstore +client](https://t73f.de/r/zsc), [webs](https://t73f.de/r/webs), +[sx](https://t73f.de/r/sx), [sxwebs](https://t73f.de/r/sxwebs), +[zero](https://t73f.de/r/zero), and [zsx](https://t73f.de/r/zsx/) are also +managed by Fossil. Depending on your development setup, some error messages +might occur. + +If the error message mentions an environment variable called `GOVCS` you should +set it to the value `GOVCS=zettelstore.de:fossil` (alternatively more generous +to `GOVCS=*:all`). Since the Go build system is coupled with Git and some +special platforms, you must allow Go to download a Fossil repository from the +host `zettelstore.de`. The build tool sets `GOVCS` to the right value, but you +may use other `go` commands that try to download a Fossil repository. + +On some operating systems, namely Termux on Android, an error message might +state that an user cannot be determined (`cannot determine user`). In this +case, Fossil is allowed to download the repository, but cannot associate it +with an user name. Set the environment variable `USER` to any user name, like: +`USER=nobody go run tools/build.go build`. Index: www/changes.wiki ================================================================== --- www/changes.wiki +++ www/changes.wiki @@ -1,31 +1,1198 @@ <title>Change Log - -

Changes for Version 0.0.14 (pending)

+ +

Changes for Version 0.22.0 (pending)

+ * Sx builtin (bind-lookup ...) is replaced with + (resolve-symbol ...). If you maintain your own Sx code to + customize Zettelstore behaviour, you must update your code; otherwise it + will break. If your code use (ROLE-DEFAULT-action ...) + , infinite recursion might occur. This is now handled with the new startup + configuration sx-max-nesting. In such cases, you should omit + the call to ROLE-DEFAULT-action. For debugging purposes, use + the web:trace logging level, which logs all Sx computations. + (breaking: webui) + * Sx templates are changed: base.sxn, info.sxn, zettel.sxn. If you are using + a (self-) modified version, you should update your modifications. + (breaking: webui) + * Remove zettel for the Sx prelude. Fortunately, it was never documented, + so it was likely unused. The prelude is now a constant string whithin Sx's + code base. + (breaking: webui) + * Remove support for metadata keys predecessor and + successors. A Zettelstore is not a version control system. + Use precursor instead (if appropriate), use your own metadata + key (which should end with -zids, or get rid of manually + versioned zettel. In the WebUI, action “version” is removed. + (breaking) + * New query directives FOLGE, SEQUEL, and + THREAD support folge zettel and sequel zettel (and both). See + the manual for details. To make working with these directives a little bit + more easier, appropriate query links are placed on main zettel web user + interface and on info zettel page. + (major) + * If authentication is enabled, Zettelstore can now be accessed from the + loopback device without logging in or obtaining an access token. + (major: api, webui) + * Context directive allows DIRECTED. + (minor) + * Metadata with keys ending with -ref or -refs are + interpreted as zettel identifier. + (minor) + * The new startup configuration key sx-max-nesting allows + setting a limit on the nesting depth of Sx computations. This is primarily + useful to prevent unbounded recursion due to programming errors. + Previously, such issues would crash the Zettelstore. + (minor: webui) + * At logging level trace, the web user interface now logs all + Sx computations, mainly those used for rendering HTML templates. + (minor: webui) + * Move context link to zettel page. + (minor: webui) + * Make writing to zettel files more robust. When an error occurs, the state + before writing the file is restored. + (info) + + +

Changes for Version 0.21.0 (2025-04-17)

+ * Change zettel identifier for Zettelstore Log, Zettelstore Memory, and + Zettelstore Sx Engine. See manual for updated identifier values. + (breaking) + * Sz encodings of links were simplified into one LINK symbol. + Different link types are now specified by reference node. + (breaking: api) + * Sz encodings of lists, descriptions, tables, table cells, and block BLOBs + have now an additional attributes entry, directly after the initial + symbol. + (breaking: api) + * Sz encoding for table cell alignment (CELL-*) is now done via + table cell attribute (align . *), where * is the + alignment value ("center", "left", or + "right"). + (breaking: api) + * New API endpoint /r/{ZID} retrieves references of a zettel. + (major: api) + * New computed zettel Zettelstore Modules, which shows all Go modules + Zettelstore is dependent to. + (minor) + * Move most of the code to new internal package internal, to + make reuse by other software a little bit harder. + * Move some code to external packages, esp. to Zettelstore Client, webs. + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.20.0 (2025-03-07)

+ * Metadata with keys that have the suffix -title are no longer + interpreted as Zettelmarkup. This was a leftover from [#0_11|v0.11], when + type of metadata title was changed from Zettelmarkup to a + possibly empty string. + (breaking) + * Type of metadata key summary changed from Zettelmarkup to + plain text. You might need to update your zettel that contain this key. + (breaking) + * Remove support for metadata type Zettelmarkup. It made + implementation too complex, and it was seldom used (see above). + (breaking) + * Remove metadata key created-missing, which was used to + support the cancelled migration process into a four letter zettel + identifier format. + (breaking) + * Remove support for inline zettel snippets. Mostly used for some HTML + elements, which hinders portability and encoding in other formats. + (breaking: zettelmarkup) + * Remove support for inline HTML text within Markdown text. Such HTML code + (did I ever say that Markdown is just a super-set of HTML?) is now + translated into literal text. + (breaking: markdown/CommonMark) + * Query aggregates ATOM and RSS are removed, as + well as the accompanying TITLE query action (parameter). + Was announced as deprecated in version 0.19. + (breaking) + * Query data encoding and aggregate data encoding were changed to remove the + superfluous (list ...). Instead, the result list is now + spliced into the returned s-expression list. If you use the Zettelstore + Client, it will handle that for you. + (breaking: api) + * “Lists” menu is build by reading a zettel with menu items + (Default: 00000000080001) instead of being hard coded. Can be + customized with runtime and user configuration + lists-menu-zettel. + (major: webui) + * Show metadata values superior and subordinate + on WebUI (again). Partially reverses the removal of support for these + values in v0.19. However, creating child zettel is not supported. + (major: webui) + * Implement CONTEXT query more correctly, as stated in the + manual; add MIN directive. + (minor) + * Remove timeouts for API and WebUI. If faced to the public internet, + Zettelstore should run behind a full proxy, like Caddy server, Nginx, or + similar. + (minor: api, webui) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.19.0 (2024-12-13)

+ * Remove support for renaming zettel, i.e. changing zettel identifier. Was + announced as deprecated in version 0.18. + (breaking: api, webui) + * Format of zettel identifier will be not changed. The deprecation message + from version 0.18 is no longer valid. + (major) + * Zettel content for most predefined zettel (ID less or equal than + 0000099999899) is not indexed any more. If you search / query for zettel + content, these zettel will not be returned. However, their metadata is + still searchable. Content of predefined zettel with ID between + 00000999999900 and 00000999999999 will be indexed.. + (breaking: api, webui) + * Metadata keys superior and subordinate are not + supported on the WebUI any more. They are still typed as a set of zettel + identifier, but are treated as ordinary links. Zettel should not form a + hierarchy, in most cases. + (major: webui) + * Metadata keys sequel and prequel support a + sequence of zettel. They are supported through the WebUI also. + sequel is calculated based on prequel. While + folge zettel are a train of thought, zettel sequences form different train + of thoughts. + (major) + * Query aggregates ATOM and RSS will be removed in + the next version of Zettelstore. They were introduced in [#0_7|v0.7] and + [#0_8|v0.8]. Both are not needed for a digital zettelkasten. Their current + use is to transform Zettelstore into a blogging engine. RSS and Atom feed + can be provided by external software, much better than Zettelstore will + ever do. + (deprecation) + * Fix wrong quote translation for markdown encoder. + (minor) + * Generate <th> in table header (was: <td>). + Also applies to SHTML encoder. (minor: webui, api) + * External links are now generated in shtml and html with attribute + rel="external" (previously: class="external"). + (minor: webui, api) + * Allow to enable runtime profiling of the software, to be used by + developers. + (minor) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.18.0 (2024-07-11)

+ * Remove Sx macro defunconst. Use defun instead. + (breaking: webui) + * The sz encoding of zettel does not make use of (SPACE) + elements any more. Instead, space characters are encoded within the + (TEXT "...") element. This might affect any client that works + with the sz encoding to produce some output. + (breaking) + * Format of zettel identifier will be changed in the future to a new format, + instead of the current timestamp-based format. The usage of zettel + identifier that are before 1970-01-01T00:00:00 is not allowed any more + (with the exception of predefined identifier) + (deprecation) + * Due to the planned format of zettel identifier, the “rename” + operation is deprecated. It will be removed in version 0.19 or later. If + you have a significant use case for the rename operation, please contact + the maintainer immediate. + (deprecation) + * New zettel are now created with the permission for others to read/write + them. This is important especially for Unix-like systems. If you want the + previous behaviour, set umask accordingly, for example + umask 066. + (major: dirbox) + * Add expert-mode zettel “Zettelstore Warnings” to help + identifying zettel to upgrade for future migration to planned new zettel + identifier format. + (minor: webui) + * Add expert-mode zettel “Zettelstore Identifier Mapping” to + show a possible mapping from the old identifier format to the new one. + This should help users to possibly rename some zettel for a better + mapping. + (minor: webui) + * Add metadata key created-missing to list zettel without + stored metadata key created. Needed for migration to planned + new zettelstore identifier format, which is not based on timestamp of + zettel creation date. + (minor) + * Add zettel “Zettelstore Application Directory”, which contains + identifier for application specific zettel. Needed for planned new + identifier format. + (minor: webui) + * Update Sx prelude: make macros more robust / more general. This might + break your code in the future. + (minor: webui) + * Add computed expert-mode zettel “Zettelstore Memory” with + zettel identifier 00000000000008. It shows some statistics + about memory usage. + (minor: webui) + * Add computed expert-mode zettel “Zettelstore Sx Engine” with + zettel identifier 00000000000009. It shows some statistics + about the internal Sx engine. (Currently only the number of used symbols, + but this will change in the future.) + (minor: webui) + * Zettelstore client is now Go package t73f.de/r/zsc. + (minor) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.17.0 (2024-03-04)

+ * Context search operates only on explicit references. Add the directive + FULL to follow zettel tags additionally. + (breaking) + * Context cost calculation has been changed. Prepare to retrieve different + result. + (breaking) + * Remove metadata type WordSet. It was never implemented completely, and + nobody complained about this. + (breaking) + * Remove logging level “sense”, “warn”, + “fatal”, and “panic”. + (breaking) + * Add query action REDIRECT which redirects to zettel that is + the first in the query result list. + (minor: api, webui) + * Add link to CONTEXT FULL in the zettel info page. + (minor: webui) + * When generating HTML code to query set based metadata (esp. tags), also + generate a query that matches all values. + (minor: webui) + * Show all metadata with key ending “-url” on zettel view. + (minor: webui) + * Make WebUI form elements a little bit more accessible by using HTML + search tag and inputmode attribute. + (minor: webui) + * Add UI action for role zettel, similar to tag zettel. Obviously forgotten + in release 0.16.0, but thanks to the bug fix v0.16.1 detected. + (minor: webui) + * If an action, which is written in uppercase letters, results in an empty + list, the list of selected zettel is returned instead. This allows some + backward compatibility if a new action is introduced. + (minor) + * Only when query list is not empty, allow to show data and plain encoding, + an optionally show the “Save As Zettel” button. + (minor: webui) + * If query list is greater than three elements, show the number of elements + at bottom (before other encodings). + (minor: webui) + * Zettel with syntax “sxn” are pretty-printed during evaluation. + This allows to retrieve parsed zettel content, which checked for syntax, + but is not pretty-printed. + (minor) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.16.1 (2023-12-28)

+ * Fix some Sxn definitions to allow role-based UI customizations. + (minor: webui) + +

Changes for Version 0.16.0 (2023-11-30)

+ * Sx function define is removed, as announced for version + 0.15.0. Use defvar (to define variables) or + defun (to define functions) instead. In addition + defunconst defines a constant function, which ensures a fixed + binding of its name to its function body (performance optimization). + (breaking: webui) + * Allow to determine a role zettel for a given role. + (major: api, webui) + * Present user the option to create a (missing) role zettel (in list view). + Results in a new predefined zettel with identifier 00000000090004, which + is a template for new role zettel. + (minor: webui) + * Timestamp values can be abbreviated by omitting most of its components. + Previously, such values that are not in the format YYYYMMDDhhmmss were + ignored. Now the following formats are also allowed: YYYY, YYYYMM, + YYYYMMDD, YYYYMMDDhh, YYYYMMDDhhmm. Querying and sorting work accordingly. + Previously, only a sequences of zeroes were appended, resulting in illegal + timestamps, e.g. for YYYY or YYYYMM. + (minor) + * SHTML encoder fixed w.r.t inline quoting. Previously, an <q> tag was + used, which is inappropriate. Restored smart quotes from version 0.3, but + with new SxHTML infrastructure. This affect the html encoder and the WebUI + too. Now, an empty quote should not result in a warning by HTML linters. + (minor: api, webui) + * Add new zettelmarkup inline formatting: ##Text## will mark / + highlight the given Text. It is typically used to highlight some text, + which is important for you, but not for the original author. When rendered + as HTML, the <mark> tag is used. + (minor: zettelmarkup) + * Add configuration keys to show, not to show, or show the closed list of + referencing zettel in the web user interface. You can set these + configurations system-wide, per user, or per zettel. Often it is used to + ensure a “clean” home zettel. Affects the list of incoming + / back links, folge zettel, subordinate zettel, and successor zettel. + (minor: webui) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.15.0 (2023-10-26)

+ * Sx function define is now deprecated. It will be removed in + version 0.16. Use defvar or defun instead. + Otherwise the WebUI will not work in version 0.16. + (major: webui, deprecated) + * Zettel can be re-indexed via WebUI or API query action + REINDEX. The info page of a zettel contains a link to + re-index the zettel. In a query transclusion, this action is ignored. + (major: api, webui). + * Allow to determine a tag zettel for a given tag. + (major: api, webui) + * Present user the option to create a (missing) tag zettel (in list view). + Results in a new predefined zettel with identifier 00000000090003, which + is a template for new tag zettel. + (minor: webui) + * ZIP file with manual now contains a zettel 00001000000000 that contains + its build date (metadata key created) and version (in the + zettel content) + (minor) + * If an error page cannot be created due to template errors (or similar), a + plain text error page is delivered instead. It shows the original error + and the error that occurred during rendering the original error page. + (minor: webui) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.14.0 (2023-09-22)

+ * Remove support for JSON. This was marked deprecated in version 0.12.0. Use + the data encoding instead, a form of symbolic expressions. + (breaking: api; minor: webui) + * Remove deprecated syntax for a context list: CONTEXT zid. Use + zid CONTEXT instead. It was deprecated in version 0.13.0. + (breaking: api, webui, zettelmarkup) + * Replace CSS-role-map mechanism with a more general Sx-based one: user + specific code may generates parts of resulting HTML document. + (breaking: webui) + * Allow meta-tags, i.e. zettel for a specific tag. Meta-tags have the tag + name as a title and specify the role "tag". + (major: webui) + * Allow to load sx code from multiple zettel; dependencies are specified + using precursor metadata. + (major: webui) + * Allow sx code to change WebUI for zettel with specified role. + (major: webui) + * Some minor usability improvements. + (minor: webui) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.13.0 (2023-08-07)

+ * There are for new search operators: less, not less, greater, not greater. + These use the same syntax as the operators prefix, not prefix, suffix, not + suffix. The latter are now denoted as [, ![, + ], and !]. The first may operate numerically for + metadata like numbers, timestamps, and zettel identifier. They are not + supported for full-text search. + (breaking: api, webui) + * The API endpoint /o/{ID} (order of zettel ID) is no longer + available. Please use the query expression {ID} ITEMS + instead. + (breaking: api) + * The API endpoint /u/{ID} (unlinked references of zettel ID) + is no longer available. Please use the query expression {ID} + UNLINKED instead. + (breaking: api) + * All API endpoints allow to encode zettel data with the data + encodings, incl. creating, updating, retrieving, and querying zettel. + (major: api) + * Change syntax for context query to zid ... CONTEXT. This will + allow to add more directives that operate on zettel identifier. Old syntax + CONTEXT zid will be removed in 0.14. + (major, deprecated) + * Add query directive ITEMS that will produce a list of + metadata of all zettel that are referenced by the originating zettel in + a top-level list. It replaces the API endpoint /o/{ID} (and + makes it more useful). + (major: api, webui) + * Add query directive UNLINKED that will produce a list of + metadata of all zettel that are mentioning the originating zettel in + a top-level, but do not mention them. It replaces the API endpoint + /u/{ID} (and makes it more useful). + (major: api, webui) + * Add query directive IDENT to distinguish a search for + a zettel identifier (“{ID}”), that will list all metadata of + zettel containing that zettel identifier, and a request to just list the + metadata of given zettel (“{ID} IDENT”). The latter could be + filtered further. + (minor: api, webui) + * Add support for metadata key folge-role. + (minor) + * Allow to create a child from a given zettel. + (minor: webui) + * Make zettel entry/edit form a little friendlier: auto-prepend missing '#' + to tags; ensure that role and syntax receive just a word. + (minor: webui) + * Use a zettel that defines builtins for evaluating WebUI templates. + (minor: webui) + * Add links to retrieve result of a query in other formats. + (minor: webui) + * Always log the found configuration file. + (minor: server) + * The use of the json zettel encoding is deprecated (since + version 0.12.0). Support for this encoding will be removed in version + 0.14.0. Please use the new data encoding instead. + (deprecated: api) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.12.0 (2023-06-05)

+ * Syntax of templates for the web user interface are changed from Mustache + to Sxn (S-Expressions). Mustache is no longer supported, nowhere in the + software. Mustache was marked deprecated in version 0.11.0. If you + modified the template zettel, you must adapt to the new syntax. + (breaking: webui) + * Query expression is allowed to search for the "context" of a zettel. + Previously, this was a separate call, without adding a search expression + / action expression. + (breaking) + * "sexpr" encoding is renamed to "sz" encoding. This will affect mostly the + API. Additionally, all string "sexpr" are renamed to "sz" also. "Sz" is + the short form for "symbolic expression for zettel", similar to "shtml" + that is the short form for "symbolic expression for HTML". + (breaking) + * Render footer zettel on all WebUI pages. + (fix: webui) + * Query search operator "=" now compares for equality, ":" compares + depending on the value type. + (minor: api, webui) + * Search term PICK now respects the original sort order. This + makes it more useful and orthogonal to RANDOM and + LIMIT. As a side effect, zettel lists retrieved via the API + are no longer sorted. In case you want a specific order, you must specify + it explicit. + (minor: api, webui) + * New metadata key expire records a timestamp when a zettel + should be treated as, well, expired. + (minor) + * New metadata keys superior and subordinate + (calculated from superior) allow to specify a hierarchy + between zettel. + (minor) + * Metadata keys with suffix -date and -time are + treated as + timestamp values. + (minor) + * sexpr zettel encoding is now documented in the manual. + (minor: manual) + * Build tool allows to install / update external Go tools needed to build + the software. + (minor) + * Show only useful metadata on WebUI, not the internal metadata. + (minor: webui) + * The use of the json zettel encoding is deprecated. Support + for this encoding may be removed in future versions. Please use the new + data encoding instead. + (deprecated: api) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.11.2 (2023-04-16)

+ * Render footer zettel on all WebUI pages. Backported from 0.12.0. Many + thanks to HK for reporting it! + (fix: webui) + +

Changes for Version 0.11.1 (2023-03-28)

+ * Make PICK search term a little bit more deterministic so that + the “Save As Zettel” button produces the same list. + (fix: webui) + +

Changes for Version 0.11.0 (2023-03-27)

+ * Remove ZJSON encoding. It was announced in version 0.10.0. Use Sexpr + encoding instead. + (breaking) + * Title of a zettel is no longer interpreted as Zettelmarkup text. Now it is + just a plain string, possibly empty. Therefore, no inline formatting (like + bold text), no links, no footnotes, no citations (the latter made + rendering the title often questionable, in some contexts). If you used + special entities, please use the Unicode characters directly. However, as + a good practice, it is often the best to printable ASCII characters. + (breaking) + * Remove runtime configuration marker-external. It was added in + version [#0_0_6|0.0.6] and updated in [#0_0_10|0.0.10]. If you want to + change the marker for an external URL, you could modify zettel + 00000000020001 (Zettelstore Base CSS) or zettel 00000000025001 + (Zettelstore User CSS, preferred) by changing / adding a rule to add some + content after an external tag. + (breaking: webui) + * Add SHTML encoding. This allows to ensure the quality of generated HTML + code. In addition, clients might use it, because it is easier to parse and + manipulate than ordinary HTML. In the future, HTML template zettel will + probably also use SHTML, deprecating the current Mustache syntax (which + was added in [#0_0_9|0.0.9]). + (major) + * Search term PICK n, where n is an integer value + greater zero, will pick randomly n elements from the search + result list. Somehow similar (and faster) as RANDOM LIMIT n, + but allows also later ordering of the resulting list. + (minor) + * Changed cost model for zettel context: a zettel with more + outgoing/incoming references has higher cost than a zettel with less + references. Also added support for traversing tags, with a similar cost + model. As an effect, zettel hubs (in many cases your home zettel) will + less likely add its references. Same for often used tags. The cost model + might change in some details in the future, but the idea of a penalty + applied to zettel / tags with many references will hold. + (minor) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.10.1 (2023-01-30)

+ * Show button to save a query into a zettel only when the current user has + authorization to do it. + (fix: webui) + +

Changes for Version 0.10.0 (2023-01-24)

+ * Remove support for endpoints /j, /m, /q, /p, /v. Their + functions are merged into endpoint /z. This was announced in + version 0.9.0. Please use only client library with at least version 0.10.0 + too. + (breaking: api) + * Remove support for runtime configuration key footer-html. Use + footer-zettel instead. Deprecated in version 0.9.0. + (breaking: webui) + * Save a query into a zettel to freeze it. + (major: webui) + * Allow to show all used metadata keys, linked with their occurrences and + their values. + (minor: webui) + * Mark ZJSON encoding as deprecated for v0.11.0. Please use Sexpr encoding + instead. + (deprecated) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.9.0 (2022-12-12)

+ * Remove support syntax pikchr. Although it was a nice idea to + include it into Zettelstore, the implementation is too brittle (w.r.t. the + expected long lifetime of Zettelstore). There should be other ways to + support SVG front-ends. + (breaking) + * Allow to upload content when creating / updating a zettel. + (major: webui) + * Add syntax “draw” (again) + (minor: zettelmarkup) + * Allow to encode zettel in Markdown. Please note: not every aspect of + a zettel can be encoded in Markdown. Those aspects will be ignored. + (minor: api) + * Enhance zettel context by raising the importance of folge zettel (and + similar). + (minor: api, webui) + * Interpret zettel files with extension .webp as an binary + image file format. + (minor) + * Allow to specify service specific log level via startup configuration and + via command line. + (minor) + * Allow to specify a zettel to serve footer content via runtime + configuration footer-zettel. Can be overwritten by user + zettel. + (minor: webui) + * Footer data is automatically separated by a thematic break / horizontal + rule. If you do not like it, you have to update the base template. + (minor: webui) + * Allow to set runtime configuration home-zettel in the user + zettel to make it user-specific. + (minor: webui) + * Serve favicon.ico from the asset directory. + (minor: webui) + * Zettelmarkup cheat sheet + (minor: manual) + * Runtime configuration key footer-html will be removed in + Version 0.10.0. Please use footer-zettel instead. + (deprecated: webui) + * In the next version 0.10.0, the API endpoints for a zettel + (/j, /p, /v) will be merged with + endpoint /z. Basically, the previous endpoint will be + refactored as query parameter of endpoint /z. To reduce + errors, there will be no version, where the previous endpoint are still + available and the new functionality is still there. This is a warning to + prepare for some breaking changes in v0.10.0. This also affects the API + client implementation. + (warning: api) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.8.0 (2022-10-20)

+ * Remove support for tags within zettel content. Removes also property + metadata keys all-tags and computed-tags. + Deprecated in version 0.7.0. + (breaking: zettelmarkup, api, webui) + * Remove API endpoint /m, which retrieve aggregated (tags, + roles) zettel identifier. Deprecated in version 0.7.0. + (breaking: api) + * Remove support for URL query parameter starting with an underscore. + Deprecated in version 0.7.0. + (breaking: api, webui) + * Ignore HTML content by default, and allow HTML gradually by setting + startup value insecure-html. + (breaking: markup) + * Endpoint /q returns list of full metadata, if no query action + is specified. A HTTP call GET /z (retrieving metadata of all + or some zettel) is now an alias for GET /q. + (major: api) + * Allow to create a zettel that acts as the new version of an existing + zettel. Useful if you want to have access to older, outdated content. + (minor: webui) + * Allow transclusion to reference local image via URL. + (minor: zettelmarkup, webui) + * Add categories in RSS feed, based on zettel tags. + (minor: api, webui) + * Add support for creating an Atom 1.0 feed using a query action. + (minor: api, webui) + * Ignore entities with code point that is not allowed in HTML. + (minor: zettelmarkup) + * Enhance distribution of tag sizes when show a tag cloud. + (minor: webui) + * Warn user if zettelstore listens non-locally, but no authentication is + enabled. + (minor: server) + * Fix error that a manual zettel deletion was not always detected. + (bug: dirbox) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.7.1 (2022-09-18)

+ * Produce a RSS feed compatible to Miniflux. + (minor) + * Make sure to always produce a pubdata in RSS feed. + (bug) + * Prefix search for data that looks like a zettel identifier may end with a + 0. + (bug) + * Fix glitch on manual zettel. + (bug) + +

Changes for Version 0.7.0 (2022-09-17)

+ * Removes support for URL query parameter to search for metadata values, + sorting, offset, and limit a zettel list. Deprecated in version 0.6.0 + (breaking: api, webui) + * Allow to search for the existence / non-existence of a metadata key with + the "?" operator: key? and key!?. Previously, + the ":" operator was used for this by specifying an empty search value. + Now you can use the ":" operator to find empty / non-empty metadata + values. If you specify a search operator for metadata, the specified key + is assumed to exist. + (breaking: api, webui) + * Rename “search expression” into “query + expressions”. Similar, the reference prefix search: to + specify a query link or a query transclusion is renamed to + query: + (breaking: zettelmarkup) + * Rename query parameter for query expression from _s to + q. + (breaking: api, webui) + * Cleanup names for HTTP query parameters in WebUI. Update your bookmarks + if you used them. (For API: see below) + (breaking: webui) + * Allow search terms to be OR-ed. This allows to specify any search + expression in disjunctive normal form. Therefore, the NEGATE term is not + needed any more. + (breaking: api, webui) + * Replace runtime configuration default-lang with + lang. Additionally, lang set at the zettel of + the current user, will provide a default value for the current user, + overwriting the global default value. + (breaking) + * Add new syntax pikchr, a markup language for diagrams in + technical documentation. + (major) + * Add endpoint /q to query the zettelstore and aggregate + resulting values. This is done by extending the query syntax. + (major: api) + * Add support for query actions. Actions may aggregate w.r.t. some metadata + keys, or produce an RSS feed. + (major: api, webui) + * Query results can be ordered for more than one metadata key. Ordering by + zettel identifier is an implicit last order expression to produce stable + results. + (minor: api, webui) + * Add support for an asset directory, accessible via URL prefix + /assests/. + (minor: server) + * Add support for metadata key created, a time stamp when the + zettel was created. Since key published is now either + created or modified, it will now always contains + a valid time stamp. + (minor) + * Add support for metadata key author. It will be displayed on + a zettel, if set. + (minor: webui) + * Remove CSS for lists. The browsers default value for + padding-left will be used. + (minor: webui) + * Removed templates for rendering roles and tags lists. This is now done by + query actions. + (minor: webui) + * Tags within zettel content are deprecated in version 0.8. This affects the + computed metadata keys content-tags and + all-tags. They will be removed. The number sign of a content + tag introduces unintended tags, esp. in the English language; content tags + may occur within links → links within links, when rendered as HTML; + content tags may occur in the title of a zettel; naming of content tags, + zettel tags, and their union is confusing for many. Migration: use zettel + tags or replace content tag with a search. + (deprecated: zettelmarkup) + * Cleanup names for HTTP query parameter for API calls. Essentially, + underscore characters in front are removed. Please use new names, old + names will be deprecated in version 0.8. + (deprecated: api) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.6.2 (2022-08-22)

+ * Recognize renaming of zettel file external to Zettelstore. + (bug) + +

Changes for Version 0.6.1 (2022-08-22)

+ * Ignore empty tags when reading metadata. + (bug) + +

Changes for Version 0.6.0 (2022-08-11)

+ * Translating of "..." into horizontal ellipsis is no longer supported. Use + &hellip; instead. + (breaking: zettelmarkup) + * Allow to specify search expressions, which allow to specify search + criteria by using a simple syntax. Can be specified in WebUI's search box + and via the API by using query parameter "_s". + (major: api, webui) + * A link reference is allowed to be a search expression. The WebUI will + render this as a link to a list of zettel that satisfy the search + expression. + (major: zettelmarkup, webui) + * A block transclusion is allowed to specify a search expression. When + evaluated, the transclusion is replaced by a list of zettel that satisfy + the search expression. + (major: zettelmarkup) + * When presenting a zettel list, allow to change the search expression. + (minor: webui) + * When evaluating a zettel, ignore transclusions if current user is not + allowed to read transcluded zettel. + (minor) + * Added a small tutorial for Zettelmarkup. + (minor: manual) + * Using URL query parameter to search for metadata values, specify an + ordering, an offset, and a limit for the resulting list, will be removed + in version 0.7. Replace these with the more useable search expressions. + Please be aware that the = search operator is also deprecated. It was only + introduced to help the migration. + (deprecated: api, webui) + * Some smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.5.1 (2022-08-02)

+ * Log missing authentication tokens in debug level (was: sense level) + (major) + * Allow to use empty metadata values of string and zmk types. + (minor) + * Add IP address to some log messages, esp. when authentication fails. + (minor) + +

Changes for Version 0.5.0 (2022-07-29)

+ * Removed zettel syntax “draw”. The new default syntax for + inline zettel is now “text”. A drawing can now be made by + using the “evaluation block” syntax (see below) by setting + the generic attribute to “draw”. + (breaking: zettelmarkup, api, webui) + * If authentication is enabled, a secret of at least 16 bytes must be set in + the startup configuration. + (breaking) + * “Sexpr” encoding replaces “Native” encoding. Sexpr + encoding is much easier to parse, compared with native and ZJSON encoding. + In most cases it is smaller than ZJSON. + (breaking: api) + * Endpoint /r is changed to /m?_key=role and + returns now a map of role names to the list of zettel having this role. + Endpoint /t is changed to /m?_key=tags. It + already returned mapping described before. + (breaking: api) + * Remove support for a default value for metadata key title, role, and + syntax. Title and role are now allowed to be empty, an empty syntax value + defaults to “plain”. + (breaking) + * Add support for an “evaluation block” syntax in Zettelmarkup + to allow interpretation of content by external software. + (minor: zettelmarkup) + * Add initial support for a TeX-like math-mode to Zettelmarkup (both block- + and inline-structured elements). Currently, support only the syntax, + but WebUI does not render these elements in a special way. + (minor: zettelmarkup) + * For block-structured elements, attributes may now span more than one line. + If a line ending occurs within a quoted attribute value, the line ending + characters are part of the attributes value. + (minor: zettelmarkup) + * Zettel 00000000029000 acts as a map of zettel roles to identifier of + zettel that are included as additional CSS. Allows to display zettel + differently depending on their role. Use case: slides that are processed + by Zettel Presenter will use the same CSS if they are rendered by + Zettelstore WebUI. + (minor: webui) + * A zettel can be saved while creating / editing it. There is no need to + manually re-edit it by using the 'e' endpoint. + (minor: webui) + * Zettel role and zettel syntax are backed by a HTML5 data list element + which lists supported and used values to help to enter a valid value. + (minor: webui) + * Allow to use startup configuration, even if started in simple mode. + (minor) + * Log authentication issues in level "sense"; add caller IP address to some + web server log messages. + (minor: web server) + * New startup configuration key max-request-size to limit a web + request body to prevent client sending too large requests. + (minor: web server) + * Many smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.4 (2022-03-08)

+ * Encoding “djson” renamed to “zjson” (zettel + json). + (breaking: api; minor: webui) + * Remove inline quotation syntax <<...<<. Now, + ""..."" generates the equivalent code. + Typographical quotes are generated by the browser, not by Zettelstore. + (breaking: Zettelmarkup) + * Remove inline formatting for mono space. Its syntax is now used by the + similar syntax element of literal computer input. Mono space was just + a visual element with no semantic association. Now, the syntax + ++...++ is obsolete. + (breaking: Zettelmarkup). + * Remove API call to parse Zettelmarkup texts and encode it as text and + HTML. Was call “POST /v”. It was needed to separately encode + the titles of zettel. The same effect can be achieved by fetching the + ZJSON representation and encode it using the function in the Zettelstore + client software. + (breaking: api) + * Remove API call to retrieve all links of an zettel. This can be done more + easily on the client side by traversing the ZJSON encoding of a zettel. + (breaking: api) + * ZJSON will encode metadata value as pairs of a metadata type and metadata + value. This allows a client to decode the associated value more easily. + (minor: api) + * A sequence of inline-structured elements can be marked, not just a point + in the Zettelmarkup text. + (minor: zettelmarkup) + * Metadata keys with suffix -title force their value to be + interpreted as Zettelmarkup. Similar, the suffix -set denotes + a set/list of words and the suffix -zids a set/list of zettel + identifier. + (minor: api, webui) + * Change generated URLs for zettel-creation forms. If you have bookmarked + them, e.g. to create a new zettel, you should update. + (minor: webui) + * Remove support for metadata key no-index to suppress indexing + selected zettel. It was introduced in [#0_0_11|v0.0.11], but disallows + some future optimizations for searching zettel. + (minor: api, webui) + * Make some metadata-based searches a little bit faster by executing + a (in-memory-based) full-text search first. Now only those zettel are + loaded from file that contain the metadata value. + (minor: api, webui) + * Add an API call to retrieve the version of the Zettelstore. + (minor: api) + * Limit the amount of zettel and bytes to be stored in a memory box. Allows + to use it with public access. + (minor: box) + * Disallow to cache the authentication cookie. Will remove most unexpected + log-outs when using a mobile device. + (minor: webui) + * Many smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.3 (2022-02-09)

+ * Zettel files with extension .meta are now treated as content + files. Previously, they were interpreted as metadata files. The + interpretation as metadata files was deprecated in version 0.2. + (breaking: directory and file/zip box) + * Add syntax “draw” to produce some graphical representations. + (major) + * Add Zettelmarkup syntax to specify full transclusion of other zettel. + (major: Zettelmarkup) + * Add Zettelmarkup syntax to specify inline-zettel, both for + block-structured and for inline-structured elements. + (major: Zettelmarkup) + * Metadata-returning API calls additionally return an indication about + access rights for the given zettel. + (minor: api) + * A previously duplicate file that is now useful (because another file was + deleted) is now logged as such. + (minor: directory and file/zip box) + * Many smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.2 (2022-01-19)

+ * v0.2.1 (2021-02-01) updates the license year in some documents + * Remove support for ;;small text;; Zettelmarkup. + (breaking: Zettelmarkup) + * On macOS, the downloadable executable program is now called + “zettelstore”, as on all other Unix-like platforms. + (possibly breaking: macOS) + * External metadata (e.g. for zettel with file extension other than + .zettel) are stored in files without an extension. Metadata + files with extension .meta are still recognized, but result + in a warning message. In a future version (probably v0.3), + .meta files will be treated as ordinary content files, + possibly resulting in duplicate content. In other words: usage of + .meta files for storing metadata is deprecated. + (possibly breaking: directory and file box) + * Show unlinked references in info page of each zettel. Unlinked references + are phrases within zettel content that might reference another zettel with + the same title as the phase. + (major: webui) + * Add endpoint /u/{ID} to retrieve unlinked references. + (major: api) + * Provide a logging facility. + Log messages are written to standard output. Messages with level + “information” are also written to a circular buffer (of length + 8192) which can be retrieved via a computed zettel. There is a command + line flag -l LEVEL to specify an application global logging + level on startup (default: “information”). Logging level can + also be changed via the administrator console, even for specific (sub-) + services. + (major) + * The internal handling of zettel files is rewritten. This allows less + reloads ands detects when the directory containing the zettel files is + removed. The API, WebUI, and the admin console allow to manually refresh + the internal state on demand. + (major: box, webui) + * .zettel files with YAML header are now correctly written. + (bug) + * Selecting zettel based on their metadata allows the same syntax as + searching for zettel content. For example, you can list all zettel that + have an identifier not ending with 00 by using the query + id=!<00. + (minor: api, webui) + * Remove support for //deprecated emphasized// Zettelmarkup. + (minor: Zettelmarkup) + * Add options to profile the software. Profiling can be enabled at the + command line or via the administrator console. + (minor) + * Add computed zettel that lists all supported parser / recognized zettel + syntaxes. + (minor) + * Add API call to check for enabled authentication. + (minor: api) + * Renewing an API access token works even if authentication is not enabled. + This corresponds to the behaviour of obtaining an access token. + (minor: api) + * If there is nothing to return, use HTTP status code 204, instead of 200 + + Content-Length: 0. + (minor: api) + * Metadata key duplicates stores the duplicate file names, + instead of just a boolean value that there were duplicate file names. + (minor) + * Document auto starting Zettelstore on Windows, macOS, and Linux. + (minor) + * Many smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.1 (2021-11-11)

+ * v0.1.3 (2021-12-15) fixes a bug where the modification date could be set + when a new zettel is created. + * v0.1.2 (2021-11-18) fixes a bug when selecting zettel from a list when + more than one comparison is negated. + * v0.1.1 (2021-11-12) updates the documentation, mostly related to the + deprecation of the // markup. + * Remove visual Zettelmarkup (italic, underline). Semantic Zettelmarkup + (emphasize, insert) is still allowed, but got a different syntax. The new + syntax for inserted text is + >>inserted>>, while its previous syntax now + denotes emphasized text: __emphasized__. The + previous syntax for emphasized text is now deprecated: //deprecated + emphasized//. Starting with Version 0.2.0, the deprecated + syntax will not be supported. The reason is the collision with URLs that + also contain the characters //. The ZMK encoding of a zettel + may help with the transition + (/v/{ZettelID}?_part=zettel&_enc=zmk, on the Info page of + each zettel in the WebUI). Additionally, all deprecated uses of + // will be rendered with a dashed box within the WebUI. + (breaking: Zettelmarkup). + * API client software is now a separate project. + (breaking) + * Initial support for HTTP security headers (Content-Security-Policy, + Permissions-Policy, Referrer-Policy, X-Content-Type-Options, + X-Frame-Options). Header values are currently some constant values. + (possibly breaking: api, webui) + * Remove visual Zettelmarkup (bold, strike through). Semantic Zettelmarkup + (strong, delete) is still allowed and replaces the visual elements + syntactically. The visual appearance should not change (depends on your + changes / additions to CSS zettel). + (possibly breaking: Zettelmarkup). + * Add API endpoint POST /v to retrieve HTMl and text encoded + strings from given ZettelMarkup encoded values. This will be used to + render a HTML page from a given zettel: in many cases the title of + a zettel must be treated separately. + (minor: api) + * Add API endpoint /m to retrieve only the metadata of + a zettel. + (minor: api) + * New metadata value content-tags contains the tags that were + given in the zettel content. To put it simply, all-tags + = tags + content-tags. + (minor) + * Calculating the context of a zettel stops at the home zettel. + (minor: api, webui) + * When renaming or deleting a zettel, a warning will be given, if other + zettel references the given zettel, or when “deleting” will + uncover zettel in overlay box. + (minor: webui) + * Fix: do not allow control characters in JSON-based creating/updating API. + Otherwise, the created / updated zettel might not be parseable by the + software (but still by a human). In certain cases, even the WebUI might be + affected. + (minor: api, webui) + * Fix: when a very long word (longer than width of browser window) is given, + still allow to scroll horizontally. + (minor: webui) + * Separate repository for [https://zettelstore.de/contrib/|contributed] + software. First entry is a software for creating a presentation by using + zettel. + (info) + * Many smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.0.15 (2021-09-17)

+ * Move again endpoint characters for authentication to make room for future + features. WebUI authentication moves from /a to + /i (login) and /i?logout (logout). API + authentication moves from /v to /a. JSON-based + basic zettel handling moves from /z to /j and + /z/{ID} to /j/{ID}. Since the API client is + updated too, this should not be a breaking change for most users. + (minor: api, webui; possibly breaking) + * Add API endpoint /v/{ID} to retrieve an evaluated zettel in + various encodings. Mostly replaces endpoint /z/{ID} for other + encodings except “json” and “raw”. Endpoint + /j/{ID} now only returns JSON data, endpoint + /z/{ID} is used to retrieve plain zettel data (previously + called “raw”). See documentation for details. + (major: api; breaking) + * Metadata values of type tag set (the metadata with key + tags is its most prominent example), are now compared in + a case-insensitive manner. Tags that only differ in upper / lower case + character are now treated identical. This might break your workflow, if + you depend on case-sensitive comparison of tag values. Tag values are + translated to their lower case equivalent before comparing them and when + you edit a zettel through Zettelstore. If you just modify the zettel + files, your tag values remain unchanged. + (major; breaking) + * Endpoint /z/{ID} allows the same methods as endpoint + /j/{ID}: GET retrieves zettel (see above), + PUT updates a zettel, DELETE deletes a zettel, + MOVE renames a zettel. In addition, POST /z will + create a new zettel. When zettel data must be given, the format is plain + text, with metadata separated from content by an empty line. See + documentation for more details. + (major: api (plus WebUI for some details)) + * Allows to transclude / expand the content of another zettel into a target + zettel when the zettel is rendered. By using the syntax of embedding an + image (which is some kind of expansion too), the first top-level paragraph + of a zettel may be transcluded into the target zettel. Endless recursion + is checked, as well as a possible “transclusion bomb ” + (similar to a XML bomb). See manual for details. + (major: zettelmarkup) + * The endpoint /z allows to list zettel in a simpler format + than endpoint /j: one line per zettel, and only zettel + identifier plus zettel title. + (minor: api) + * Folgezettel are now displayed with full title at the bottom of a page. + (minor: webui) + * Add API endpoint /p/{ID} to retrieve a parsed, but not + evaluated zettel in various encodings. + (minor: api) + * Fix: do not list a shadowed zettel that matches the select criteria. + (minor) + * Many smaller bug fixes and improvements, to the software and to the + documentation. + + +

Changes for Version 0.0.14 (2021-07-23)

+ * Rename “place” into “box”. This also affects the + configuration keys to specify boxes box-uriX + (previously place-uri-X. Older changes documented + here are renamed too. + (breaking) + * Add API for creating, updating, renaming, and deleting zettel. + (major: api) + * Initial API client for Go. + (major: api) + * Remove support for paging of WebUI list. Runtime configuration key + list-page-size is removed. If you still specify it, it will + be ignored. + (major: webui) + * Use endpoint /v for user authentication via API. Endpoint + /a is now used for the web user interface only. Similar, + endpoint /y (“zettel context”) is renamed to + /x. + (minor, possibly breaking) + * Type of used-defined metadata is determined by suffix of key: + -number, -url, -zid will result the + values to be interpreted as a number, an URL, or a zettel identifier. + (minor, but possibly breaking if you already used a metadata key with + above suffixes, but as a string type) + * New user-role “creator”, which is only allowed to + create new zettel (except user zettel). This role may only read and update + public zettel or its own user zettel. Added to support future client + software (e.g. on a mobile device) that automatically creates new zettel + but, in case of a password loss, should not allow to read existing zettel. + (minor, possibly breaking, because new zettel template zettel must always + prepend the string new- before metadata keys that should be + transferred to the new zettel) + * New supported metadata key box-number, which gives an + indication from which box the zettel was loaded. + (minor) + * New supported syntax html. + (minor) + * New predefined zettel “User CSS” that can be used to redefine + some predefined CSS (without modifying the base CSS zettel). + (minor: webui) + * When a user moves a zettel file with additional characters into the box + directory, these characters are preserved when zettel is updated. + (bug) + * The phase “filtering a zettel list” is more precise + “selecting zettel” + (documentation) + * Many smaller bug fixes and improvements, to the software and to the + documentation. - +

Changes for Version 0.0.13 (2021-06-01)

- * Startup configuration place-X-uri (where X is a - number greater than zero) has been renamed to - place-uri-X. + * Startup configuration box-X-uri (where X is + a number greater than zero) has been renamed to + box-uri-X. (breaking) - * Web server processes startup configuration url-prefix. There is - no need for stripping the prefix by a front-end web server any more. + * Web server processes startup configuration url-prefix. There + is no need for stripping the prefix by a front-end web server any more. (breaking: webui, api) * Administrator console (only optional accessible locally). Enable it only on systems with a single user or with trusted users. It is disabled by default. (major: core) * Remove visibility value “simple-expert” introduced in [#0_0_8|version 0.0.8]. It was too complicated, esp. authorization. There - was a name collision with the “simple” directory place - sub-type. + was a name collision with the “simple” directory box sub-type. (major) * For security reasons, HTML blocks are not encoded as HTML if they contain - certain snippets, such as <script or <iframe. - These may be caused by using CommonMark as a zettel syntax. + certain snippets, such as <script or + <iframe. These may be caused by using CommonMark as + a zettel syntax. (major) * Full-text search can be a prefix search or a search for equal words, in addition to the search whether a word just contains word of the search term. (minor: api, webui) @@ -37,330 +1204,434 @@ values and about runtime metrics got new zettel identifier. This affects only user that referenced those zettel. (minor) * Local images that cannot be read (not found or no access rights) are substituted with the new default image, a spinning emoji. - See [/file?name=place/constplace/emoji_spin.gif]. + See [/file?name=box/constbox/emoji_spin.gif]. (minor: webui) * Add zettelmarkup syntax for a table row that should be ignored: - |%. This allows to paste output of the administrator console into - a zettel. + |%. This allows to paste output of the administrator console + into a zettel. (minor: zmk) - * Many smaller bug fixes and inprovements, to the software and to the + * Many smaller bug fixes and improvements, to the software and to the documentation. - +

Changes for Version 0.0.12 (2021-04-16)

* Raise the per-process limit of open files on macOS to 1.048.576. This allows most macOS users to use at least 500.000 zettel. That should be enough for the near future. (major) * Mitigate the shortcomings of the macOS version by introducing types of - directory places. The original directory place type is now called "notify" + directory boxes. The original directory box type is now called "notify" (the default value). There is a new type called "simple". This new type does not notify Zettelstore when some of the underlying Zettel files change. (major) - * Add new startup configuration default-dir-place-type, which gives - the default value for specifying a directory place type. The default value - is “notify”. On macOS, the default value may be changed + * Add new startup configuration default-dir-box-type, which + gives the default value for specifying a directory box type. The default + value is “notify”. On macOS, the default value may be changed “simple” if some errors occur while raising the per-process limit of open files. (minor) - +

Changes for Version 0.0.11 (2021-04-05)

- * New place schema "file" allows to read zettel from a ZIP file. + * New box schema "file" allows to read zettel from a ZIP file. A zettel collection can now be packaged and distributed easier. (major: server) - * Non-restricted search is a full-text search. The search string will - be normalized according to Unicode NFKD. Every character that is not a letter + * Non-restricted search is a full-text search. The search string will be + normalized according to Unicode NFKD. Every character that is not a letter or a number will be ignored for the search. It is sufficient if the words - to be searched are part of words inside a zettel, both content and metadata. + to be searched are part of words inside a zettel, both content and + metadata. (major: api, webui) * A zettel can be excluded from being indexed (and excluded from being found - in a search) if it contains the metadata no-index: true. + in a search) if it contains the metadata no-index: true. (minor: api, webui) * Menu bar is shown when displaying error messages. (minor: webui) - * When filtering a list of zettel, it can be specified that a given value should - not match. Previously, only the whole filter expression could be + * When selecting zettel, it can be specified that a given value should + not match. Previously, only the whole select criteria could be negated (which is still possible). (minor: api, webui) - * You can filter a zettel list by specifying that specific metadata keys must + * You can select a zettel by specifying that specific metadata keys must (or must not) be present. (minor: api, webui) - * Context of a zettel (introduced in version 0.0.10) does not take tags into account any more. - Using some tags for determining the context resulted into erratic, non-deterministic context lists. - (minor: api, webui) - * Filtering zettel depending on tag values can be both by comparing only the prefix - or the whole string. If a search value begins with '#', only zettel with the exact - tag will be returned. Otherwise a zettel will be returned if the search string - just matches the prefix of only one of its tags. - (minor: api, webui) - * Many smaller bug fixes and inprovements, to the software and to the documentation. - -A note for users of macOS: in the current release and with macOS's default values, -a zettel directory place must not contain more than approx. 250 files. There are three options -to mitigate this limitation temporarily: - # You update the per-process limit of open files on macOS. - # You setup a virtualization environment to run Zettelstore on Linux or Windows. + * Context of a zettel (introduced in version 0.0.10) does not take tags into + account any more. Using some tags for determining the context resulted + into erratic, non-deterministic context lists. + (minor: api, webui) + * Selecting zettel depending on tag values can be both by comparing only the + prefix or the whole string. If a search value begins with '#', only zettel + with the exact tag will be returned. Otherwise a zettel will be returned + if the search string just matches the prefix of only one of its tags. + (minor: api, webui) + * Many smaller bug fixes and improvements, to the software and to the + documentation. + +A note for users of macOS: in the current release and with macOS's default +values, a zettel directory must not contain more than approx. 250 files. There +are three options to mitigate this limitation temporarily: + # You update the per-process limit of open files on macOS. + # You setup a virtualisation environment to run Zettelstore on Linux or + Windows. # You wait for version 0.0.12 which addresses this issue. - +

Changes for Version 0.0.10 (2021-02-26)

* Menu item “Home” now redirects to a home zettel. - Its default identifier is 000100000000. - The identifier can be changed with configuration key home-zettel, which supersedes key start. - The default home zettel contains some welcoming information for the new user. + Its default identifier is 000100000000. The identifier can be + changed with configuration key home-zettel, which supersedes + key start. The default home zettel contains some welcoming + information for the new user. (major: webui) - * Show context of a zettel by following all backward and/or forward reference - up to a defined depth and list the resulting zettel. Additionally, some zettel - with similar tags as the initial zettel are also taken into account. + * Show context of a zettel by following all backward and/or forward + reference up to a defined depth and list the resulting zettel. + Additionally, some zettel with similar tags as the initial zettel are also + taken into account. (major: api, webui) - * A zettel that references other zettel within first-level list items, can act - as a “table of contents” zettel. - The API endpoint /o/{ID} allows to retrieve the referenced zettel in - the same order as they occur in the zettel. + * A zettel that references other zettel within first-level list items, can + act as a “table of contents” zettel. The API endpoint + /o/{ID} allows to retrieve the referenced zettel in the same + order as they occur in the zettel. (major: api) - * The zettel “New Menu” with identifier 00000000090000 contains - a list of all zettel that should act as a template for new zettel. - They are listed in the WebUIs ”New“ menu. - This is an application of the previous item. - It supersedes the usage of a role new-template introduced in [#0_0_6|version 0.0.6]. - Please update your zettel if you make use of the now deprecated feature. + * The zettel “New Menu” with identifier + 00000000090000 contains a list of all zettel that should act + as a template for new zettel. They are listed in the WebUIs + ”New“ menu. This is an application of the previous item. It + supersedes the usage of a role new-template introduced in + [#0_0_6|version 0.0.6]. Please update your zettel if you make use of + the now deprecated feature. (major: webui) - * A reference that starts with two slash characters (“//”) - it will be interpreted relative to the value of url-prefix. - For example, if url-prefix has the value /manual/, - the reference [[Zettel list|//h]] will render as - <a href="/manual/h">Zettel list</a>. (minor: syntax) - * Searching/filtering ignores the leading '#' character of tags. + * A reference that starts with two slash characters + (“//”) it will be interpreted relative to the + value of url-prefix. For example, if url-prefix + has the value /manual/, the reference + [[Zettel list|//h]] will render as <a + href="/manual/h">Zettel list</a>. + (minor: syntax) + * Searching/selecting ignores the leading '#' character of tags. (minor: api, webui) - * When result of filtering or searching is presented, the query is written as the page heading. + * When result of selecting or searching is presented, the query is written + as the page heading. (minor: webui) - * A reference to a zettel that contains a URL fragment, will now be processed by the indexer. + * A reference to a zettel that contains a URL fragment, will now be + processed by the indexer. (bug: server) - * Runtime configuration key marker-external now defaults to + * Runtime configuration key marker-external now defaults to “&#10138;” (“➚”). It is more beautiful than the previous “&#8599;&#xfe0e;” (“↗︎”), which also needed the additional - “&#xfe0e;” to disable the conversion to an emoji on iPadOS. + “&#xfe0e;” to disable the conversion to an emoji on + iPadOS. (minor: webui) - * A pre-build binary for macOS ARM64 (also known as Apple silicon) is available. + * A pre-build binary for macOS ARM64 (also known as Apple silicon) is + available. (minor: infrastructure) - * Many smaller bug fixes and inprovements, to the software and to the documentation. + * Many smaller bug fixes and improvements, to the software and to the + documentation. - +

Changes for Version 0.0.9 (2021-01-29)

-This is the first version that is managed by [https://fossil-scm.org|Fossil] instead -of GitHub. To access older versions, use the Git repository under +This is the first version that is managed by [https://fossil-scm.org|Fossil] +instead of GitHub. To access older versions, use the Git repository under [https://github.com/zettelstore/zettelstore-github|zettelstore-github].

Server / API

* (major) Support for property metadata. - Metadata key published is the first example of such a property. - * (major) A background activity (called indexer) continuously monitors - zettel changes to establish the reverse direction of found internal links. - This affects the new metadata keys precursor and folge. - A user specifies the precursor of a zettel and the indexer computes the - property metadata for [https://forum.zettelkasten.de/discussion/996/definition-folgezettel|Folgezettel]. - Metadata keys with type “Identifier” or “IdentifierSet” - that have no inverse key (like precursor and folge - with add to the key forward that also collects all internal - links within the content. The computed inverse is backward, which provides all backlinks. - The key back is computed as the value of backward, but without forward links. - Therefore, back is something like the list of “smart backlinks”. - * (minor) If Zettelstore is being stopped, an appropriate message is written in the console log. - * (minor) New computed zettel with environmental data, the list of supported meta data keys, - and statistics about all configured zettel places. - Some other computed zettel got a new identifier (to make room for other variant). - * (minor) Remove zettel 00000000000004, which contained the Go version that produced the Zettelstore executable. - It was too specific to the current implementation. - This information is now included in zettel 00000000000006 (Zettelstore Environment Values). - * (minor) Predefined templates for new zettel do not contain any value for attribute visibility any more. + Metadata key published is the first example of such + a property. + * (major) A background activity (called indexer) continuously + monitors zettel changes to establish the reverse direction of + found internal links. This affects the new metadata keys + precursor and folge. A user specifies + the precursor of a zettel and the indexer computes the property + metadata for + [https://forum.zettelkasten.de/discussion/996/definition-folgezettel|Folgezettel]. + Metadata keys with type “Identifier” or + “IdentifierSet” that have no inverse key (like + precursor and folge with add to the key + forward that also collects all internal links within + the content. The computed inverse is backward, which + provides all backlinks. The key back is computed as + the value of backward, but without forward links. + Therefore, back is something like the list of + “smart backlinks”. + * (minor) If Zettelstore is being stopped, an appropriate message is written + in the console log. + * (minor) New computed zettel with environmental data, the list of supported + meta data keys, and statistics about all configured zettel boxes. + Some other computed zettel got a new identifier (to make room for + other variant). + * (minor) Remove zettel 00000000000004, which contained the Go + version that produced the Zettelstore executable. It was too + specific to the current implementation. This information is now + included in zettel 00000000000006 (Zettelstore + Environment Values). + * (minor) Predefined templates for new zettel do not contain any value for + attribute visibility any more. * (minor) Add a new metadata key type called “Zettelmarkup”. - It is a non-empty string, that will be formatted with Zettelmarkup. - title and default-title have this type. + It is a non-empty string, that will be formatted with + Zettelmarkup. title and default-title + have this type. * (major) Rename zettel syntax “meta” to “none”. - Please update the Zettelstore Runtime Configuration and all other zettel that previously used the value “meta”. + Please update the Zettelstore Runtime Configuration and all + other zettel that previously used the value “meta”. Other zettel are typically user zettel, used for authentication. However, there is no real harm, if you do not update these zettel. In this case, the metadata is just not presented when rendered. Zettelstore will still work. - * (minor) Login will take at least 500 milliseconds to mitigate login attacks. - This affects both the API and the WebUI. - * (minor) Add a sort option “_random” to produce a zettel list in random order. - _order / order are now an aliases for the query parameters _sort / sort. + * (minor) Login will take at least 500 milliseconds to mitigate login + attacks. This affects both the API and the WebUI. + * (minor) Add a sort option “_random” to produce a zettel list + in random order. _order / order are now + an aliases for the query parameters _sort + / sort.

WebUI

- * (major) HTML template zettel for WebUI now use [https://mustache.github.io/|Mustache] - syntax instead of previously used [https://golang.org/pkg/html/template/|Go template] syntax. - This allows these zettel to be used, even when there is another Zettelstore implementation, in another programming language. - Mustache is available for approx. 48 programming languages, instead of only one for Go templates. - If you modified your templates, you must adapt them to the new syntax. - Otherwise the WebUI will not work. - * (major) Show zettel identifier of folgezettel and precursor zettel in the header of a rendered zettel. - If a zettel has real backlinks, they are shown at the botton of the page - (“Additional links to this zettel”). - * (minor) All property metadata, even computed metadata is shown in the info page of a zettel. - * (minor) Rendering of metadata keys title and default-title in info page changed to a full HTML output for these Zettelmarkup encoded values. + * (major) HTML template zettel for WebUI now use + [https://mustache.github.io/|Mustache] syntax instead of + previously used [https://golang.org/pkg/html/template/|Go + template] syntax. This allows these zettel to be used, even when + there is another Zettelstore implementation, in another + programming language. Mustache is available for approx. 48 + programming languages, instead of only one for Go templates. If + you modified your templates, you must adapt them to the new + syntax. Otherwise the WebUI will not work. + * (major) Show zettel identifier of folgezettel and precursor zettel in the + header of a rendered zettel. If a zettel has real backlinks, they + are shown at the botton of the page (“Additional links to + this zettel”). + * (minor) All property metadata, even computed metadata is shown in the info + page of a zettel. + * (minor) Rendering of metadata keys title and + default-title in info page changed to a full HTML + output for these Zettelmarkup encoded values. * (minor) Always show the zettel identifier on the zettel detail view. - Previously, the identifier was not shown if the zettel was not editable. + Previously, the identifier was not shown if the zettel was not + editable. * (minor) Do not show computed metadata in edit forms anymore. - +

Changes for Version 0.0.8 (2020-12-23)

Server / API

- * (bug) Zettel files with extension .jpg and without metadata will get a syntax value “jpg”. - The internal data structure got the same value internally, instead of “jpeg”. - This has been fixed for all possible alternative syntax values. - * (bug) If a file, e.g. an image file like 20201130190200.jpg, is added to the directory place, - * its metadata are just calculated from the information available. - Updated metadata did not find its way into the place, because the .meta file was not written. - This has been fixed. - * (bug) If just the .meta file was deleted manually, the zettel was assumed to be missing. - A workaround is to restart the software. - This has been fixed. - If the .meta file is deleted, metadata is now calculated in the same way when the .meta file is non-existing at the start of the software. - * (bug) A link to the current zettel, only using a fragment (e.g. [[Title|#title]]) is now handled correctly as a zettel link (and not as a link to external material). + * (bug) Zettel files with extension .jpg and without metadata + will get a syntax value “jpg”. The internal + data structure got the same value internally, instead of + “jpeg”. This has been fixed for all possible alternative + syntax values. + * (bug) If a file, e.g. an image file like 20201130190200.jpg, + is added to the directory box, its metadata are just calculated from + the information available. Updated metadata did not find its way + into the zettel box, because the .meta file was not + written. + * (bug) If just the .meta file was deleted manually, the zettel + was assumed to be missing. A workaround is to restart the software. + If the .meta file is deleted, metadata is now + calculated in the same way when the .meta file is + non-existing at the start of the software. + * (bug) A link to the current zettel, only using a fragment (e.g. + [[Title|#title]]) is now handled correctly as + a zettel link (and not as a link to external material). * (minor) Allow zettel to be marked as “read only”. - This is done through the metadata key read-only. - * (bug) When renaming a zettel, check all places for the new zettel identifier, not just the first place. - Otherwise it will be possible to shadow a read-only zettel from a next place, effectively modifying it. - * (minor) Add support for a configurable default value for metadata key visibility. - * (bug) If list-page-size is set to a relatively small value and the authenticated user is not the owner, - * some zettel were not shown in the list of zettel or were not returned by the API. + This is done through the metadata key read-only. + * (bug) When renaming a zettel, check all boxes for the new zettel + identifier, not just the first one. Otherwise it will be possible to + shadow a read-only zettel from a next box, effectively modifying it. + * (minor) Add support for a configurable default value for metadata key + visibility. + * (bug) If list-page-size is set to a relatively small value + and the authenticated user is not the owner, some zettel were + not shown in the list of zettel or were not returned by the API. * (minor) Add support for new visibility “expert”. - An owner becomes an expert, if the runtime configuration key expert-mode is set to true. + An owner becomes an expert, if the runtime configuration key + expert-mode is set to true. * (major) Add support for computed zettel. - These zettel have an identifier less than 0000000000100. - Most of them are only visible, if expert-mode is enabled. - * (bug) Fixes a memory leak that results in too many open files after approx. 125 reload operations. - * (major) Predefined templates for new zettel got an explicit value for visibility: “login”. - Please update these zettel if you modified them. - * (major) Rename key readonly of Zettelstore Startup Configuration to read-only-mode. - This was done to avoid some confusion with the the zettel metadata key read-only. - Please adapt your startup configuration. - Otherwise your Zettelstore will be accidentally writable. - * (minor) References starting with “./” and “../” are treated as a local reference. - Previously, only the prefix “/” was treated as a local reference. - * (major) Metadata key modified will be set automatically to the current local time if a zettel is updated through Zettelstore. - If you used that key previously for your own, you should rename it before you upgrade. - * (minor) The new visibility value “simple-expert” ensures that many computed zettel are shown for new users. - This is to enable them to send useful bug reports. - * (minor) When a zettel is stored as a file, its identifier is additionally stored within the metadata. - This helps for better robustness in case the file names were corrupted. - In addition, there could be a tool that compares the identifier with the file name. + These zettel have an identifier less than + 0000000000100. Most of them are only visible, if + expert-mode is enabled. + * (bug) Fixes a memory leak that results in too many open files after + approx. 125 reload operations. + * (major) Predefined templates for new zettel got an explicit value for + visibility: “login”. Please update these zettel if you + modified them. + * (major) Rename key readonly of Zettelstore Startup + Configuration to read-only-mode. This was done to + avoid some confusion with the zettel metadata key + read-only. Please adapt your startup + configuration. Otherwise your Zettelstore will be accidentally + writable. + * (minor) References starting with “./” and “../” + are treated as a local reference. Previously, only the prefix + “/” was treated as a local reference. + * (major) Metadata key modified will be set automatically to + the current local time if a zettel is updated through Zettelstore. + If you used that key previously for your own, you should rename + it before you upgrade. + * (minor) The new visibility value “simple-expert” ensures that + many computed zettel are shown for new users. This is to enable + them to send useful bug reports. + * (minor) When a zettel is stored as a file, its identifier is additionally + stored within the metadata. This helps for better robustness in + case the file names were corrupted. In addition, there could be + a tool that compares the identifier with the file name.

WebUI

- * (minor) Remove list of tags in “List Zettel” and search results. - There was some feedback that the additional tags were not helpful. - * (minor) Move zettel field "role" above "tags" and move "syntax" more to "content". + * (minor) Remove list of tags in “List Zettel” and search + results. There was some feedback that the additional tags were not + helpful. + * (minor) Move zettel field "role" above "tags" and move "syntax" more to + "content". * (minor) Rename zettel operation “clone” to “copy”. - * (major) All predefined HTML templates have now a visibility value “expert”. - If you want to see them as an non-expert owner, you must temporary enable expert-mode and change the visibility metadata value. - * (minor) Initial support for [https://zettelkasten.de/posts/tags/folgezettel/|Folgezettel]. - If you click on “Folge” (detail view or info view), a new zettel is created with a reference (precursor) to the original zettel. - Title, role, tags, and syntax are copied from the original zettel. - * (major) Most predefined zettel have a title prefix of “Zettelstore”. - * (minor) If started in simple mode, e.g. via double click or without any command, some information for the new user is presented. - In the terminal, there is a hint about opening the web browser and use a specific URL. - A Welcome zettel is created, to give some more information. - (This change also applies to the server itself, but it is more suited to the WebUI user.) - - + * (major) All predefined HTML templates have now a visibility value + “expert”. If you want to see them as an non-expert + owner, you must temporary enable expert-mode and + change the visibility metadata value. + * (minor) Initial support for + [https://zettelkasten.de/posts/tags/folgezettel/|Folgezettel]. If + you click on “Folge” (detail view or info view), a new + zettel is created with a reference (precursor) to the + original zettel. Title, role, tags, and syntax are copied from the + original zettel. + * (major) Most predefined zettel have a title prefix of + “Zettelstore”. + * (minor) If started in simple mode, e.g. via double click or without any + command, some information for the new user is presented. In the + terminal, there is a hint about opening the web browser and use + a specific URL. A Welcome zettel is created, to give some + more information. (This change also applies to the server itself, + but it is more suited to the WebUI user.) + +

Changes for Version 0.0.7 (2020-11-24)

* With this version, Zettelstore and this manual got a new license, the - [https://joinup.ec.europa.eu/collection/eupl|European Union Public Licence] (EUPL), version 1.2 or later. - Nothing else changed. - If you want to stay with the old licenses (AGPLv3+, CC BY-SA 4.0), - you are free to fork from the previous version. + [https://joinup.ec.europa.eu/collection/eupl|European Union Public + Licence] (EUPL), version 1.2 or later. Nothing else changed. If you want + to stay with the old licenses (AGPLv3+, CC BY-SA 4.0), you are free to + fork from the previous version. - +

Changes for Version 0.0.6 (2020-11-23)

Server

- * (major) Rename identifier of Zettelstore Runtime Configuration to 00000000000100 (previously 00000000000001). - This is done to gain some free identifier with smaller number to be used internally. - If you customized this zettel, please make sure to rename it to the new identifier. - * (major) Rename the two essential metadata keys of a user zettel to credential and user-id. - The previous values were cred and ident. - If you enabled user authentication and added some user zettel, make sure to change them accordingly. - Otherwise these users will not authenticated any more. - * (minor) Rename the scheme of the place URL where predefined zettel are stored to “const”. - The previous value was “globals”. + * (major) Rename identifier of Zettelstore Runtime Configuration to + 00000000000100 (previously + 00000000000001). This is done to gain some free + identifier with smaller number to be used internally. If you + customized this zettel, please make sure to rename it to the new + identifier. + * (major) Rename the two essential metadata keys of a user zettel to + credential and user-id. The previous + values were cred and ident. If you + enabled user authentication and added some user zettel, make sure + to change them accordingly. Otherwise these users will not + authenticated any more. + * (minor) Rename the scheme of the box URL where predefined zettel are + stored to “const”. The previous value was + “globals”.

Zettelmarkup

* (bug) Allow to specify a fragment in a reference to a zettel. Used to link to an internal position within a zettel. This applies to CommonMark too.

API

- * (bug) Encoding binary content in format “json” now results in valid JSON content. - * (bug) All query parameter of filtering a list must be true, regardless if a specific key occurs more than one or not. - * (minor) Encode all inherited meta values in all formats except “raw”. - A meta value is called inherited if there is a key starting with default- in the Zettelstore Runtime Configuration. - Applies to WebUI also. - * (minor) Automatic calculated identifier for headings (only for “html”, “djson”, “native” format and for the Web user interface). - You can use this to provide a zettel reference that links to the heading, without specifying an explicit mark ([!mark]). + * (bug) Encoding binary content in format “json” now results + in valid JSON content. + * (bug) All query parameters of selecting zettel must be true, regardless + if a specific key occurs more than one or not. + * (minor) Encode all inherited meta values in all formats except + “raw”. A meta value is called inherited if + there is a key starting with default- in the + Zettelstore Runtime Configuration. Applies to WebUI also. + * (minor) Automatic calculated identifier for headings (only for + “html”, “djson”, “native” + format and for the Web user interface). You can use this to + provide a zettel reference that links to the heading, without + specifying an explicit mark ([!mark]). * (major) Allow to retrieve all references of a given zettel.

Web user interface (WebUI)

- * (minor) Focus on the first text field on some forms (new zettel, edit zettel, rename zettel, login) + * (minor) Focus on the first text field on some forms (new zettel, edit + zettel, rename zettel, login) * (major) Adapt all HTML templates to a simpler structure. - * (bug) Rendered wrong URLs for internal links on info page. - * (bug) If a zettel contains binary content it cannot be cloned. - For such a zettel only the metadata can be changed. - * (minor) Non-zettel references that neither have an URL scheme, user info, nor host name, - are considered “local references” (in contrast to “zettel references” and “external references”). - When a local reference is displayed as an URL on the WebUI, it will not opened in a new window/tab. - They will receive a local marker, when encoded as “djson” or “native”. - Local references are listed on the Info page of each zettel. - * (minor) Change the default value for some visual sugar placed after an external URL to &\#8599;&\#xfe0e; (“↗︎”). - This affects the former key icon-material of the Zettelstore Runtime Configuration, which is renamed to marker-external. + * (bug) Rendered wrong URLs for internal links on info page. + * (bug) If a zettel contains binary content it cannot be cloned. + For such a zettel only the metadata can be changed. + * (minor) Non-zettel references that neither have an URL scheme, user info, + nor host name, are considered “local references” (in + contrast to “zettel references” and “external + references”). When a local reference is displayed as an URL + on the WebUI, it will not opened in a new window/tab. They will + receive a local marker, when encoded as “djson” + or “native”. Local references are listed on the + Info page of each zettel. + * (minor) Change the default value for some visual sugar put after an + external URL to &\#8599;&\#xfe0e; + (“↗︎”). This affects the former key + icon-material of the Zettelstore Runtime + Configuration, which is renamed to + marker-external. * (major) Allow multiple zettel to act as templates for creating new zettel. - All zettel with a role value “new-template” act as a template to create a new zettel. - The WebUI menu item “New” changed to a drop-down list with all those zettel, ordered by their identifier. - All metadata keys with the prefix new- will be translated to a new or updated keys/value without that prefix. - You can use this mechanism to specify a role for the new zettel, or a different title. - The title of the template zettel is used in the drop-down list. - The initial template zettel “New Zettel” has now a different zettel identifier - (now: 00000000091001, was: 00000000040001). - Please update it, if you changed that zettel. -
Note: this feature was superseded in [#0_0_10|version 0.0.10] by the “New Menu” zettel. - * (minor) When a page should be opened in a new windows (e.g. for external references), - the web browser is instructed to decouple the new page from the previous one for privacy and security reasons. - In detail, the web browser is instructed to omit referrer information and to omit a JS object linking to the page that contained the external link. - * (minor) If the value of the Zettelstore Runtime Configuration key list-page-size is greater than zero, - the number of WebUI list elements will be restricted and it is possible to change to the next/previous page to list more elements. - * (minor) Change CSS to enhance reading: make line-height a little smaller (previous: 1.6, now 1.4) and move list items to the left. - - -

Changes for Version 0.0.5 (2020-10-22)

- * Application Programming Interface (API) to allow external software to retrieve zettel data from the Zettelstore. - * Specify places, where zettel are stored, via an URL. + All zettel with a role value “new-template” act as + a template to create a new zettel. The WebUI menu item + “New” changed to a drop-down list with all those + zettel, ordered by their identifier. All metadata keys with the + prefix new- will be translated to a new or updated + keys/value without that prefix. You can use this mechanism to + specify a role for the new zettel, or a different title. The title + of the template zettel is used in the drop-down list. The initial + template zettel “New Zettel” has now a different + zettel identifier (now: 00000000091001, was: + 00000000040001). Please update it, if you changed + that zettel. +
Note: this feature was superseded in [#0_0_10|version 0.0.10] + by the “New Menu” zettel. + * (minor) When a page should be opened in a new windows (e.g. for external + references), the web browser is instructed to decouple the new + page from the previous one for privacy and security reasons. In + detail, the web browser is instructed to omit referrer information + and to omit a JS object linking to the page that contained the + external link. + * (minor) If the value of the Zettelstore Runtime Configuration key + list-page-size is greater than zero, the number of + WebUI list elements will be restricted and it is possible to + change to the next/previous page to list more elements. + * (minor) Change CSS to enhance reading: make line-height + a little smaller (previous: 1.6, now 1.4) and move list items to + the left. + + +

Changes for Version 0.0.5 (2020-10-22)

+ * Application Programming Interface (API) to allow external software to + retrieve zettel data from the Zettelstore. + * Specify boxes, where zettel are stored, via an URL. * Add support for a custom footer. - +

Changes for Version 0.0.4 (2020-09-11)

* Optional user authentication/authorization. - * New sub-commands file (use Zettelstore as a command line filter), password (for authentication), and config. + * New sub-commands file (use Zettelstore as a command line + filter), password (for authentication), and + config. - +

Changes for Version 0.0.3 (2020-08-31)

* Starting Zettelstore has been changed by introducing sub-commands. This change is also reflected on the server installation procedures. * Limitations on renaming zettel has been relaxed. - +

Changes for Version 0.0.2 (2020-08-28)

- * Configuration zettel now has ID 00000000000001 (previously: 00000000000000). - * The zettel with ID 00000000000000 is no longer shown in any zettel list. - If you changed the configuration zettel, you should rename it manually in its file directory. + * Configuration zettel now has ID 00000000000001 (previously: + 00000000000000). + * The zettel with ID 00000000000000 is no longer shown in any + zettel list. If you changed the configuration zettel, you should rename it + manually in its file directory. * Creating a new zettel is now done by cloning an existing zettel. - To mimic the previous behaviour, a zettel with ID 00000000040001 is introduced. - You can change it if you need a different template zettel. + To mimic the previous behaviour, a zettel with ID + 00000000040001 is introduced. You can change it if you need + a different template zettel. - +

Changes for Version 0.0.1 (2020-08-21)

* Initial public release. - Index: www/download.wiki ================================================================== --- www/download.wiki +++ www/download.wiki @@ -1,25 +1,38 @@ Download

Download of Zettelstore Software

Foreword

- * Zettelstore is free/libre open source software, licensed under EUPL-1.2-or-later. - * The software is provided as-is. - * There is no guarantee that it will not damage your system. - * However, it is in use by the main developer since March 2020 without any damage. - * It may be useful for you. It is useful for me. - * Take a look at the [https://zettelstore.de/manual/|manual] to know how to start and use it. + +Zettelstore is free/libre open source software, licensed under the +[https://zettelstore.de/home/file?name=LICENSE.txt&ci=trunk|EUPL-1.2 or later]. + +The software is provided as is, without any warranty. While there's no +guarantee it won't cause issues, it has been in daily use by the main developer +since March 2020, without any damage or data loss. +Since 2021, others have also been using Zettelstore reliably, and my students +have used it since 2023, all without encountering any problems. + +It’s useful for me and many others. It might be useful for you too. + +Check out the [https://zettelstore.de/manual/|manual] to get started and learn +how to make the most of it. +

ZIP-ped Executables

-Build: v0.0.13 (2021-06-01). - - * [/uv/zettelstore-0.0.13-linux-amd64.zip|Linux] (amd64) - * [/uv/zettelstore-0.0.13-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi) - * [/uv/zettelstore-0.0.13-windows-amd64.zip|Windows] (amd64) - * [/uv/zettelstore-0.0.13-darwin-amd64.zip|macOS] (amd64) - * [/uv/zettelstore-0.0.13-darwin-arm64.zip|macOS] (arm64, aka Apple silicon) - -Unzip the appropriate file, install and execute Zettelstore according to the manual. +Build: v0.21.0 (2025-03-07). + + * [/uv/zettelstore-0.21.0-android-arm64.zip|Android] (arm64) + * [/uv/zettelstore-0.21.0-linux-amd64.zip|Linux] (amd64) + * [/uv/zettelstore-0.21.0-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi) + * [/uv/zettelstore-0.21.0-darwin-arm64.zip|macOS] (arm64) + * [/uv/zettelstore-0.21.0-darwin-amd64.zip|macOS] (amd64) + * [/uv/zettelstore-0.21.0-windows-amd64.zip|Windows] (amd64) + +Unzip the appropriate file, install and execute Zettelstore according to the +manual.

Zettel for the manual

-As a starter, you can download the zettel for the manual [/uv/manual-0.0.13.zip|here]. -Just unzip the contained files and put them into your zettel folder or configure -a file place to read the zettel directly from the ZIP file. +As a starter, you can download the zettel for the manual +[/uv/manual-0.21.0.zip|here]. + +Just unzip the contained files and put them into your zettel folder or +configure a file box to read the zettel directly from the ZIP file. Index: www/impri.wiki ================================================================== --- www/impri.wiki +++ www/impri.wiki @@ -6,13 +6,13 @@ Phone: +49 (15678) 386566
Mail: ds (at) zettelstore.de

Privacy

If you do not log into this site, or login as the user "anonymous", -the only personal data this web service will process is your IP adress. It will +the only personal data this web service will process is your IP address. It will be used to send the data of the website you requested to you and to mitigate possible attacks against this website. This website is hosted by [https://ionos.de|1&1 IONOS SE]. According to -[https://www.ionos.de/hilfe/datenschutz/datenverarbeitung-von-webseitenbesuchern-ihres-11-ionos-produktes/andere-11-ionos-produkte/|their information], +[https://www.ionos.de/hilfe/datenschutz/datenverarbeitung-von-webseitenbesuchern-ihres-ionos-produktes/andere-ionos-produkte/|their information], no processing of personal data is done by them. Index: www/index.wiki ================================================================== --- www/index.wiki +++ www/index.wiki @@ -1,37 +1,47 @@ Home Zettelstore is a software that collects and relates your notes (“zettel”) to represent and enhance your knowledge. It helps with many tasks of personal knowledge management by explicitly supporting the -[https://en.wikipedia.org/wiki/Zettelkasten|Zettelkasten method]. The -method is based on creating many individual notes, each with one idea or -information, that are related to each other. Since knowledge is typically build -up gradually, one major focus is a long-term store of these notes, hence the +[https://en.wikipedia.org/wiki/Zettelkasten|Zettelkasten method]. The method is +based on creating many individual notes, each with one idea or information, +that are related to each other. Since knowledge is typically built up +gradually, one major focus is a long-term storage of these notes, hence the name “Zettelstore”. -To get an initial impression, take a look at the [https://zettelstore.de/manual/|manual]. -It is a live example of the zettelstore software, running in read-only mode. +To get an initial impression, take a look at the +[https://zettelstore.de/manual/|manual]. It is a live example of the +zettelstore software, running in read-only mode. The software, including the manual, is licensed under the [/file?name=LICENSE.txt&ci=trunk|European Union Public License 1.2 (or later)]. -[https://twitter.com/zettelstore|Stay tuned]… + * [https://t73f.de/r/zsc|Zettelstore Client] provides client software to + access Zettelstore via its API more easily. + * [https://zettelstore.de/contrib|Zettelstore Contrib] contains contributed + software, which often connects to Zettelstore via its API. Some of the + software packages may be experimental. + * [https://t73f.de/r/sx|Sx] provides an evaluator for symbolic + expressions, which is used for HTML templates and more. + +[https://mastodon.social/tags/Zettelstore|Stay tuned] …
-

Latest Release: 0.0.13 (2021-06-01)

+

Latest Release: 0.21.0 (2025-03-07)

* [./download.wiki|Download] - * [./changes.wiki#0_0_13|Change summary] - * [/timeline?p=version-0.0.13&bt=version-0.0.12&y=ci|Check-ins for version 0.0.13], - [/vdiff?to=version-0.0.13&from=version-0.0.12|content diff] - * [/timeline?df=version-0.0.13&y=ci|Check-ins derived from the 0.0.13 release], - [/vdiff?from=version-0.0.13&to=trunk|content diff] + * [./changes.wiki#0_21|Change summary] + * [/timeline?p=v0.21.0&bt=v0.20.0&y=ci|Check-ins for version 0.21], + [/vdiff?to=v0.21.0&from=v0.20.0|content diff] + * [/timeline?df=v0.21.0&y=ci|Check-ins derived from the 0.21 release], + [/vdiff?from=v0.21.0&to=trunk|content diff] * [./plan.wiki|Limitations and planned improvements] * [/timeline?t=release|Timeline of all past releases]

Build instructions

-Just install [https://golang.org/dl/|Go] and some Go-based tools. Please read + +Just install [https://go.dev/dl/|Go] and some Go-based tools. Please read the [./build.md|instructions] for details. * [/dir?ci=trunk|Source code] * [/download|Download the source code] as a tarball or a ZIP file (you must [/login|login] as user "anonymous"). Index: www/plan.wiki ================================================================== --- www/plan.wiki +++ www/plan.wiki @@ -1,34 +1,21 @@ Limitations and planned improvements Here is a list of some shortcomings of Zettelstore. They are planned to be solved. -

Serious limitations

- * Content with binary data (e.g. a GIF, PNG, or JPG file) cannot be created - nor modified via the standard web interface. As a workaround, you should - place your file into the directory where your zettel are stored. Make sure - that the file name starts with unique 14 digits that make up the zettel - identifier. - * Automatic lists and transclusions are not supported in Zettelmarkup. - * … - -

Smaller limitations

+ * Zettelstore must have indexed all zettel to make use of queries. + Otherwise not all zettel may be returned. * Quoted attribute values are not yet supported in Zettelmarkup: {key="value with space"}. - * The file sub-command currently does not support output format - “json”. - * The horizontal tab character (U+0009) is not supported. + * The horizontal tab character (U+0009) is not fully supported by + the parser. * Missing support for citation keys. * Changing the content syntax is not reflected in file extension. * File names with additional text besides the zettel identifier are not always preserved. - * Backspace character in links does not always work, esp. for \| or - \]. - * … - -

Planned improvements

- * Support for mathematical content is missing, e.g. $$F(x) &= - \\int^a_b \\frac{1}{3}x^3$$. - * Render zettel in [https://pandoc.org|pandoc's] JSON version of - their native AST to make pandoc an external renderer for Zettelstore. + * Some file systems differentiate filenames with different cases (e.g. some + on Linux, sometimes on macOS), others do not (default on macOS, most on + Windows). Zettelstore is not able to detect these differences. Do not put + files in your directory boxes and in files boxes that differ only by upper + / lower case letters. * …