Zettelstore

Check-in Differences
Login

Check-in Differences

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Difference From v0.16.0 To trunk

2024-07-11
17:10
Remove "rename" operation (both webui and api; plus internal zettel etc) ... (Leaf check-in: b61264a18b user: stern tags: trunk)
15:34
Increase version to 0.19.0-dev to begin next development cycle ... (check-in: 1d1cd5e637 user: stern tags: trunk)
2023-12-28
16:41
Fix sxn code that removed role-based customization, esp. for an additional action if role is "tag" ... (check-in: e721174596 user: t73fde tags: release-0.16)
2023-11-30
18:10
Increase version to 0.17.0-dev to begin next development cycle ... (check-in: 9d654c5606 user: stern tags: trunk)
16:08
Version 0.16.0 ... (check-in: a5afffaf5f user: stern tags: trunk, release, v0.16.0)
2023-11-29
16:53
Add predefined role zettel ... (check-in: 514e400b28 user: stern tags: trunk)

Changes to .fossil-settings/ignore-glob.

1
2
3
bin/*
releases/*
parser/pikchr/*.out


<
1
2

bin/*
releases/*

Changes to Makefile.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

## Copyright (c) 2020-present Detlef Stern
##
## This file is part of Zettelstore.
##
## Zettelstore is licensed under the latest version of the EUPL (European Union
## Public License). Please see file LICENSE.txt for your rights and obligations
## under this license.

.PHONY:  check relcheck api build release clean

check:
	go run tools/build.go check

relcheck:
	go run tools/build.go relcheck

api:
	go run tools/build.go testapi

version:
	@echo $(shell go run tools/build.go version)

build:
	go run tools/build.go build

release:
	go run tools/build.go release

clean:
	go run tools/build.go clean









|


|


|


|


|


|


|


|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

## Copyright (c) 2020-present Detlef Stern
##
## This file is part of Zettelstore.
##
## Zettelstore is licensed under the latest version of the EUPL (European Union
## Public License). Please see file LICENSE.txt for your rights and obligations
## under this license.

.PHONY:  check relcheck api version build release clean

check:
	go run tools/check/check.go

relcheck:
	go run tools/check/check.go -r

api:
	go run tools/testapi/testapi.go

version:
	@echo $(shell go run tools/build/build.go version)

build:
	go run tools/build/build.go build

release:
	go run tools/build/build.go release

clean:
	go run tools/clean/clean.go

Changes to README.md.

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
gradually, one major focus is a long-term store of these notes, hence the name
&ldquo;Zettelstore&rdquo;.

To get an initial impression, take a look at the
[manual](https://zettelstore.de/manual/). It is a live example of the
zettelstore software, running in read-only mode.

[Zettelstore Client](https://zettelstore.de/client) provides client
software to access Zettelstore via its API more easily, [Zettelstore
Contrib](https://zettelstore.de/contrib) contains contributed software, which
often connects to Zettelstore via its API. Some of the software packages may be
experimental.

The software, including the manual, is licensed
under the [European Union Public License 1.2 (or
later)](https://zettelstore.de/home/file?name=LICENSE.txt&ci=trunk).







|
|







9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
gradually, one major focus is a long-term store of these notes, hence the name
&ldquo;Zettelstore&rdquo;.

To get an initial impression, take a look at the
[manual](https://zettelstore.de/manual/). It is a live example of the
zettelstore software, running in read-only mode.

[Zettelstore Client](https://t73f.de/r/zsc) provides client software to access
Zettelstore via its API more easily, [Zettelstore
Contrib](https://zettelstore.de/contrib) contains contributed software, which
often connects to Zettelstore via its API. Some of the software packages may be
experimental.

The software, including the manual, is licensed
under the [European Union Public License 1.2 (or
later)](https://zettelstore.de/home/file?name=LICENSE.txt&ci=trunk).

Changes to VERSION.

1
0.16.0
|
1
0.19.0-dev

Changes to ast/ast.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package ast provides the abstract syntax tree for parsed zettel content.
package ast

import (
	"net/url"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package ast provides the abstract syntax tree for parsed zettel content.
package ast

import (
	"net/url"

Changes to ast/block.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package ast

import "zettelstore.de/client.fossil/attrs"

// Definition of Block nodes.

// BlockSlice is a slice of BlockNodes.
type BlockSlice []BlockNode

func (*BlockSlice) blockNode() { /* Just a marker */ }








>
>
>




|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package ast

import "t73f.de/r/zsc/attrs"

// Definition of Block nodes.

// BlockSlice is a slice of BlockNodes.
type BlockSlice []BlockNode

func (*BlockSlice) blockNode() { /* Just a marker */ }

Changes to ast/inline.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package ast

import (
	"unicode/utf8"

	"zettelstore.de/client.fossil/attrs"
)

// Definitions of inline nodes.

// InlineSlice is a list of BlockNodes.
type InlineSlice []InlineNode

func (*InlineSlice) inlineNode() { /* Just a marker */ }

// CreateInlineSliceFromWords makes a new inline list from words,
// that will be space-separated.
func CreateInlineSliceFromWords(words ...string) InlineSlice {
	inl := make(InlineSlice, 0, 2*len(words)-1)
	for i, word := range words {
		if i > 0 {
			inl = append(inl, &SpaceNode{Lexeme: " "})
		}
		inl = append(inl, &TextNode{Text: word})
	}
	return inl
}

// WalkChildren walks down to the list.
func (is *InlineSlice) WalkChildren(v Visitor) {
	for _, in := range *is {
		Walk(v, in)
	}
}

// --------------------------------------------------------------------------

// TextNode just contains some text.
type TextNode struct {
	Text string // The text itself.
}

func (*TextNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*TextNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// SpaceNode tracks inter-word space characters.
type SpaceNode struct {
	Lexeme string
}

func (*SpaceNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*SpaceNode) WalkChildren(Visitor) { /* No children*/ }

// Count returns the number of space runes.
func (sn *SpaceNode) Count() int {
	return utf8.RuneCountInString(sn.Lexeme)
}

// --------------------------------------------------------------------------

// BreakNode signals a new line that must / should be interpreted as a new line break.
type BreakNode struct {
	Hard bool // Hard line break?
}









>
>
>





<
|
<









<
<
<
<
<
<
<
<
<
<
<
<
<



















<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

17

18
19
20
21
22
23
24
25
26













27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

















46
47
48
49
50
51
52
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package ast

import (

	"t73f.de/r/zsc/attrs"

)

// Definitions of inline nodes.

// InlineSlice is a list of BlockNodes.
type InlineSlice []InlineNode

func (*InlineSlice) inlineNode() { /* Just a marker */ }














// WalkChildren walks down to the list.
func (is *InlineSlice) WalkChildren(v Visitor) {
	for _, in := range *is {
		Walk(v, in)
	}
}

// --------------------------------------------------------------------------

// TextNode just contains some text.
type TextNode struct {
	Text string // The text itself.
}

func (*TextNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*TextNode) WalkChildren(Visitor) { /* No children*/ }


















// --------------------------------------------------------------------------

// BreakNode signals a new line that must / should be interpreted as a new line break.
type BreakNode struct {
	Hard bool // Hard line break?
}

Changes to ast/ref.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package ast

import (
	"net/url"
	"strings"


	"zettelstore.de/z/zettel/id"
)

// QueryPrefix is the prefix that denotes a query expression.
const QueryPrefix = "query:"

// ParseReference parses a string and returns a reference.
func ParseReference(s string) *Reference {
	if invalidReference(s) {
		return &Reference{URL: nil, Value: s, State: RefStateInvalid}
	}
	if strings.HasPrefix(s, QueryPrefix) {








>
>
>








>




|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package ast

import (
	"net/url"
	"strings"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/zettel/id"
)

// QueryPrefix is the prefix that denotes a query expression.
const QueryPrefix = api.QueryPrefix

// ParseReference parses a string and returns a reference.
func ParseReference(s string) *Reference {
	if invalidReference(s) {
		return &Reference{URL: nil, Value: s, State: RefStateInvalid}
	}
	if strings.HasPrefix(s, QueryPrefix) {

Changes to ast/ref_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package ast_test

import (
	"testing"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package ast_test

import (
	"testing"

Changes to ast/walk.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package ast

// Visitor is a visitor for walking the AST.
type Visitor interface {
	Visit(node Node) Visitor








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package ast

// Visitor is a visitor for walking the AST.
type Visitor interface {
	Visit(node Node) Visitor

Changes to ast/walk_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package ast_test

import (
	"testing"

	"zettelstore.de/client.fossil/attrs"
	"zettelstore.de/z/ast"
)

func BenchmarkWalk(b *testing.B) {
	root := ast.BlockSlice{
		&ast.HeadingNode{
			Inlines: ast.CreateInlineSliceFromWords("A", "Simple", "Heading"),
		},
		&ast.ParaNode{
			Inlines: ast.CreateInlineSliceFromWords("This", "is", "the", "introduction."),
		},
		&ast.NestedListNode{
			Kind: ast.NestedListUnordered,
			Items: []ast.ItemSlice{
				[]ast.ItemNode{
					&ast.ParaNode{
						Inlines: ast.CreateInlineSliceFromWords("Item", "1"),
					},
				},
				[]ast.ItemNode{
					&ast.ParaNode{
						Inlines: ast.CreateInlineSliceFromWords("Item", "2"),
					},
				},
			},
		},
		&ast.ParaNode{
			Inlines: ast.CreateInlineSliceFromWords("This", "is", "some", "intermediate", "text."),
		},
		ast.CreateParaNode(
			&ast.FormatNode{
				Kind: ast.FormatEmph,
				Attrs: attrs.Attributes(map[string]string{
					"":      "class",
					"color": "green",
				}),
				Inlines: ast.CreateInlineSliceFromWords("This", "is", "some", "emphasized", "text."),
			},
			&ast.SpaceNode{Lexeme: " "},
			&ast.LinkNode{
				Ref:     &ast.Reference{Value: "http://zettelstore.de"},
				Inlines: ast.CreateInlineSliceFromWords("URL", "text."),
			},
		),
	}
	v := benchVisitor{}
	b.ResetTimer()
	for n := 0; n < b.N; n++ {
		ast.Walk(&v, &root)
	}
}

type benchVisitor struct{}

func (bv *benchVisitor) Visit(ast.Node) ast.Visitor { return bv }








>
>
>







|






|


|






|




|





|








|

|


|





|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package ast_test

import (
	"testing"

	"t73f.de/r/zsc/attrs"
	"zettelstore.de/z/ast"
)

func BenchmarkWalk(b *testing.B) {
	root := ast.BlockSlice{
		&ast.HeadingNode{
			Inlines: ast.InlineSlice{&ast.TextNode{Text: "A Simple Heading"}},
		},
		&ast.ParaNode{
			Inlines: ast.InlineSlice{&ast.TextNode{Text: "This is the introduction."}},
		},
		&ast.NestedListNode{
			Kind: ast.NestedListUnordered,
			Items: []ast.ItemSlice{
				[]ast.ItemNode{
					&ast.ParaNode{
						Inlines: ast.InlineSlice{&ast.TextNode{Text: "Item 1"}},
					},
				},
				[]ast.ItemNode{
					&ast.ParaNode{
						Inlines: ast.InlineSlice{&ast.TextNode{Text: "Item 2"}},
					},
				},
			},
		},
		&ast.ParaNode{
			Inlines: ast.InlineSlice{&ast.TextNode{Text: "This is some intermediate text."}},
		},
		ast.CreateParaNode(
			&ast.FormatNode{
				Kind: ast.FormatEmph,
				Attrs: attrs.Attributes(map[string]string{
					"":      "class",
					"color": "green",
				}),
				Inlines: ast.InlineSlice{&ast.TextNode{Text: "This is some emphasized text."}},
			},
			&ast.TextNode{Text: " "},
			&ast.LinkNode{
				Ref:     &ast.Reference{Value: "http://zettelstore.de"},
				Inlines: ast.InlineSlice{&ast.TextNode{Text: "URL text."}},
			},
		),
	}
	v := benchVisitor{}
	b.ResetTimer()
	for range b.N {
		ast.Walk(&v, &root)
	}
}

type benchVisitor struct{}

func (bv *benchVisitor) Visit(ast.Node) ast.Visitor { return bv }

Changes to auth/auth.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package auth provides services for authentification / authorization.
package auth

import (
	"time"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package auth provides services for authentification / authorization.
package auth

import (
	"time"
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103

	// User is allowed to read zettel
	CanRead(user, m *meta.Meta) bool

	// User is allowed to write zettel.
	CanWrite(user, oldMeta, newMeta *meta.Meta) bool

	// User is allowed to rename zettel
	CanRename(user, m *meta.Meta) bool

	// User is allowed to delete zettel.
	CanDelete(user, m *meta.Meta) bool

	// User is allowed to refresh box data.
	CanRefresh(user *meta.Meta) bool
}







<
<
<






91
92
93
94
95
96
97



98
99
100
101
102
103

	// User is allowed to read zettel
	CanRead(user, m *meta.Meta) bool

	// User is allowed to write zettel.
	CanWrite(user, oldMeta, newMeta *meta.Meta) bool




	// User is allowed to delete zettel.
	CanDelete(user, m *meta.Meta) bool

	// User is allowed to refresh box data.
	CanRefresh(user *meta.Meta) bool
}

Changes to auth/cred/cred.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package cred provides some function for handling credentials.
package cred

import (
	"bytes"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package cred provides some function for handling credentials.
package cred

import (
	"bytes"

Changes to auth/impl/digest.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29



30
31
32
33
34
35
36
37
38
39
40
41
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package impl

import (
	"bytes"
	"crypto"
	"crypto/hmac"
	"encoding/base64"

	"zettelstore.de/sx.fossil"
	"zettelstore.de/sx.fossil/sxreader"
)

var encoding = base64.RawURLEncoding

const digestAlg = crypto.SHA384

func sign(claim sx.Object, secret []byte) ([]byte, error) {
	var buf bytes.Buffer
	sx.Print(&buf, claim)



	token := make([]byte, encoding.EncodedLen(buf.Len()))
	encoding.Encode(token, buf.Bytes())

	digest := hmac.New(digestAlg.New, secret)
	_, err := digest.Write(buf.Bytes())
	if err != nil {
		return nil, err
	}
	dig := digest.Sum(nil)
	encDig := make([]byte, encoding.EncodedLen(len(dig)))
	encoding.Encode(encDig, dig)









>
>
>










|
|








|
>
>
>




|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2023-present Detlef Stern
//-----------------------------------------------------------------------------

package impl

import (
	"bytes"
	"crypto"
	"crypto/hmac"
	"encoding/base64"

	"t73f.de/r/sx"
	"t73f.de/r/sx/sxreader"
)

var encoding = base64.RawURLEncoding

const digestAlg = crypto.SHA384

func sign(claim sx.Object, secret []byte) ([]byte, error) {
	var buf bytes.Buffer
	_, err := sx.Print(&buf, claim)
	if err != nil {
		return nil, err
	}
	token := make([]byte, encoding.EncodedLen(buf.Len()))
	encoding.Encode(token, buf.Bytes())

	digest := hmac.New(digestAlg.New, secret)
	_, err = digest.Write(buf.Bytes())
	if err != nil {
		return nil, err
	}
	dig := digest.Sum(nil)
	encDig := make([]byte, encoding.EncodedLen(len(dig)))
	encoding.Encode(encDig, dig)

Changes to auth/impl/impl.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package impl provides services for authentification / authorization.
package impl

import (
	"errors"
	"hash/fnv"
	"io"
	"time"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/client.fossil/sexp"
	"zettelstore.de/sx.fossil"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/policy"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"








>
>
>











|
|
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package impl provides services for authentification / authorization.
package impl

import (
	"errors"
	"hash/fnv"
	"io"
	"time"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/sexp"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/policy"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
	if !ok || subject == "" {
		return nil, ErrNoIdent
	}

	now := time.Now().Round(time.Second)
	sClaim := sx.MakeList(
		sx.Int64(kind),
		sx.String(subject),
		sx.Int64(now.Unix()),
		sx.Int64(now.Add(d).Unix()),
		sx.Int64(ident.Zid),
	)
	return sign(sClaim, a.secret)
}








|







88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
	if !ok || subject == "" {
		return nil, ErrNoIdent
	}

	now := time.Now().Round(time.Second)
	sClaim := sx.MakeList(
		sx.Int64(kind),
		sx.MakeString(subject),
		sx.Int64(now.Unix()),
		sx.Int64(now.Add(d).Unix()),
		sx.Int64(ident.Zid),
	)
	return sign(sClaim, a.secret)
}

118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
	vals, err := sexp.ParseList(obj, "isiii")
	if err != nil {
		return ErrMalformedToken
	}
	if auth.TokenKind(vals[0].(sx.Int64)) != k {
		return ErrOtherKind
	}
	ident := vals[1].(sx.String)
	if ident == "" {
		return ErrNoIdent
	}
	issued := time.Unix(int64(vals[2].(sx.Int64)), 0)
	expires := time.Unix(int64(vals[3].(sx.Int64)), 0)
	now := time.Now().Round(time.Second)
	if expires.Before(now) {
		return ErrTokenExpired
	}
	zid := id.Zid(vals[4].(sx.Int64))
	if !zid.IsValid() {
		return ErrNoZid
	}

	tokenData.Ident = ident.String()
	tokenData.Issued = issued
	tokenData.Now = now
	tokenData.Expires = expires
	tokenData.Zid = zid
	return nil
}








|














|







121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
	vals, err := sexp.ParseList(obj, "isiii")
	if err != nil {
		return ErrMalformedToken
	}
	if auth.TokenKind(vals[0].(sx.Int64)) != k {
		return ErrOtherKind
	}
	ident := vals[1].(sx.String).GetValue()
	if ident == "" {
		return ErrNoIdent
	}
	issued := time.Unix(int64(vals[2].(sx.Int64)), 0)
	expires := time.Unix(int64(vals[3].(sx.Int64)), 0)
	now := time.Now().Round(time.Second)
	if expires.Before(now) {
		return ErrTokenExpired
	}
	zid := id.Zid(vals[4].(sx.Int64))
	if !zid.IsValid() {
		return ErrNoZid
	}

	tokenData.Ident = string(ident)
	tokenData.Issued = issued
	tokenData.Now = now
	tokenData.Expires = expires
	tokenData.Zid = zid
	return nil
}

Changes to auth/policy/anon.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package policy

import (
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package policy

import (
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
	return ap.pre.CanRead(user, m) && ap.checkVisibility(m)
}

func (ap *anonPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool {
	return ap.pre.CanWrite(user, oldMeta, newMeta) && ap.checkVisibility(oldMeta)
}

func (ap *anonPolicy) CanRename(user, m *meta.Meta) bool {
	return ap.pre.CanRename(user, m) && ap.checkVisibility(m)
}

func (ap *anonPolicy) CanDelete(user, m *meta.Meta) bool {
	return ap.pre.CanDelete(user, m) && ap.checkVisibility(m)
}

func (ap *anonPolicy) CanRefresh(user *meta.Meta) bool {
	if ap.authConfig.GetExpertMode() || ap.authConfig.GetSimpleMode() {
		return true







<
<
<
<







32
33
34
35
36
37
38




39
40
41
42
43
44
45
	return ap.pre.CanRead(user, m) && ap.checkVisibility(m)
}

func (ap *anonPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool {
	return ap.pre.CanWrite(user, oldMeta, newMeta) && ap.checkVisibility(oldMeta)
}





func (ap *anonPolicy) CanDelete(user, m *meta.Meta) bool {
	return ap.pre.CanDelete(user, m) && ap.checkVisibility(m)
}

func (ap *anonPolicy) CanRefresh(user *meta.Meta) bool {
	if ap.authConfig.GetExpertMode() || ap.authConfig.GetSimpleMode() {
		return true

Changes to auth/policy/box.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package policy

import (
	"context"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package policy

import (
	"context"

71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
	return zettel.Zettel{}, box.NewErrNotAllowed("GetZettel", user, zid)
}

func (pp *polBox) GetAllZettel(ctx context.Context, zid id.Zid) ([]zettel.Zettel, error) {
	return pp.box.GetAllZettel(ctx, zid)
}

func (pp *polBox) FetchZids(ctx context.Context) (id.Set, error) {
	return nil, box.NewErrNotAllowed("fetch-zids", server.GetUser(ctx), id.Invalid)
}

func (pp *polBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	m, err := pp.box.GetMeta(ctx, zid)
	if err != nil {
		return nil, err







|







74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
	return zettel.Zettel{}, box.NewErrNotAllowed("GetZettel", user, zid)
}

func (pp *polBox) GetAllZettel(ctx context.Context, zid id.Zid) ([]zettel.Zettel, error) {
	return pp.box.GetAllZettel(ctx, zid)
}

func (pp *polBox) FetchZids(ctx context.Context) (*id.Set, error) {
	return nil, box.NewErrNotAllowed("fetch-zids", server.GetUser(ctx), id.Invalid)
}

func (pp *polBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	m, err := pp.box.GetMeta(ctx, zid)
	if err != nil {
		return nil, err
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
	}
	if pp.policy.CanWrite(user, oldZettel.Meta, zettel.Meta) {
		return pp.box.UpdateZettel(ctx, zettel)
	}
	return box.NewErrNotAllowed("Write", user, zid)
}

func (pp *polBox) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
	return pp.box.AllowRenameZettel(ctx, zid)
}

func (pp *polBox) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	z, err := pp.box.GetZettel(ctx, curZid)
	if err != nil {
		return err
	}
	user := server.GetUser(ctx)
	if pp.policy.CanRename(user, z.Meta) {
		return pp.box.RenameZettel(ctx, curZid, newZid)
	}
	return box.NewErrNotAllowed("Rename", user, curZid)
}

func (pp *polBox) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	return pp.box.CanDeleteZettel(ctx, zid)
}

func (pp *polBox) DeleteZettel(ctx context.Context, zid id.Zid) error {
	z, err := pp.box.GetZettel(ctx, zid)
	if err != nil {







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







118
119
120
121
122
123
124
















125
126
127
128
129
130
131
	}
	if pp.policy.CanWrite(user, oldZettel.Meta, zettel.Meta) {
		return pp.box.UpdateZettel(ctx, zettel)
	}
	return box.NewErrNotAllowed("Write", user, zid)
}

















func (pp *polBox) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	return pp.box.CanDeleteZettel(ctx, zid)
}

func (pp *polBox) DeleteZettel(ctx context.Context, zid id.Zid) error {
	z, err := pp.box.GetZettel(ctx, zid)
	if err != nil {

Changes to auth/policy/default.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package policy

import (
	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/zettel/meta"
)

type defaultPolicy struct {
	manager auth.AuthzManager
}

func (*defaultPolicy) CanCreate(_, _ *meta.Meta) bool { return true }
func (*defaultPolicy) CanRead(_, _ *meta.Meta) bool   { return true }
func (d *defaultPolicy) CanWrite(user, oldMeta, _ *meta.Meta) bool {
	return d.canChange(user, oldMeta)
}
func (d *defaultPolicy) CanRename(user, m *meta.Meta) bool { return d.canChange(user, m) }
func (d *defaultPolicy) CanDelete(user, m *meta.Meta) bool { return d.canChange(user, m) }

func (*defaultPolicy) CanRefresh(user *meta.Meta) bool { return user != nil }

func (d *defaultPolicy) canChange(user, m *meta.Meta) bool {
	metaRo, ok := m.Get(api.KeyReadOnly)
	if !ok {








>
>
>





|













<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

31
32
33
34
35
36
37
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package policy

import (
	"t73f.de/r/zsc/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/zettel/meta"
)

type defaultPolicy struct {
	manager auth.AuthzManager
}

func (*defaultPolicy) CanCreate(_, _ *meta.Meta) bool { return true }
func (*defaultPolicy) CanRead(_, _ *meta.Meta) bool   { return true }
func (d *defaultPolicy) CanWrite(user, oldMeta, _ *meta.Meta) bool {
	return d.canChange(user, oldMeta)
}

func (d *defaultPolicy) CanDelete(user, m *meta.Meta) bool { return d.canChange(user, m) }

func (*defaultPolicy) CanRefresh(user *meta.Meta) bool { return user != nil }

func (d *defaultPolicy) canChange(user, m *meta.Meta) bool {
	metaRo, ok := m.Get(api.KeyReadOnly)
	if !ok {

Changes to auth/policy/owner.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package policy

import (
	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/zettel/meta"
)

type ownerPolicy struct {
	manager    auth.AuthzManager








>
>
>





|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package policy

import (
	"t73f.de/r/zsc/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/zettel/meta"
)

type ownerPolicy struct {
	manager    auth.AuthzManager
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
	switch userRole := o.manager.GetUserRole(user); userRole {
	case meta.UserRoleReader, meta.UserRoleCreator:
		return false
	}
	return o.userCanCreate(user, newMeta)
}

func (o *ownerPolicy) CanRename(user, m *meta.Meta) bool {
	if user == nil || !o.pre.CanRename(user, m) {
		return false
	}
	if res, ok := o.checkVisibility(user, o.authConfig.GetVisibility(m)); ok {
		return res
	}
	return o.userIsOwner(user)
}

func (o *ownerPolicy) CanDelete(user, m *meta.Meta) bool {
	if user == nil || !o.pre.CanDelete(user, m) {
		return false
	}
	if res, ok := o.checkVisibility(user, o.authConfig.GetVisibility(m)); ok {
		return res
	}







<
<
<
<
<
<
<
<
<
<







111
112
113
114
115
116
117










118
119
120
121
122
123
124
	switch userRole := o.manager.GetUserRole(user); userRole {
	case meta.UserRoleReader, meta.UserRoleCreator:
		return false
	}
	return o.userCanCreate(user, newMeta)
}











func (o *ownerPolicy) CanDelete(user, m *meta.Meta) bool {
	if user == nil || !o.pre.CanDelete(user, m) {
		return false
	}
	if res, ok := o.checkVisibility(user, o.authConfig.GetVisibility(m)); ok {
		return res
	}

Changes to auth/policy/policy.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"zettelstore.de/z/auth"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"zettelstore.de/z/auth"
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
}

func (p *prePolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool {
	return oldMeta != nil && newMeta != nil && oldMeta.Zid == newMeta.Zid &&
		p.post.CanWrite(user, oldMeta, newMeta)
}

func (p *prePolicy) CanRename(user, m *meta.Meta) bool {
	return m != nil && p.post.CanRename(user, m)
}

func (p *prePolicy) CanDelete(user, m *meta.Meta) bool {
	return m != nil && p.post.CanDelete(user, m)
}

func (p *prePolicy) CanRefresh(user *meta.Meta) bool {
	return p.post.CanRefresh(user)
}







<
<
<
<







56
57
58
59
60
61
62




63
64
65
66
67
68
69
}

func (p *prePolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool {
	return oldMeta != nil && newMeta != nil && oldMeta.Zid == newMeta.Zid &&
		p.post.CanWrite(user, oldMeta, newMeta)
}





func (p *prePolicy) CanDelete(user, m *meta.Meta) bool {
	return m != nil && p.post.CanDelete(user, m)
}

func (p *prePolicy) CanRefresh(user *meta.Meta) bool {
	return p.post.CanRefresh(user)
}

Changes to auth/policy/policy_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package policy

import (
	"fmt"
	"testing"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func TestPolicies(t *testing.T) {
	t.Parallel()








>
>
>








|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package policy

import (
	"fmt"
	"testing"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func TestPolicies(t *testing.T) {
	t.Parallel()
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
		)
		name := fmt.Sprintf("readonly=%v/withauth=%v/expert=%v/simple=%v",
			ts.readonly, ts.withAuth, ts.expert, ts.simple)
		t.Run(name, func(tt *testing.T) {
			testCreate(tt, pol, ts.withAuth, ts.readonly)
			testRead(tt, pol, ts.withAuth, ts.expert)
			testWrite(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testRename(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testDelete(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testRefresh(tt, pol, ts.withAuth, ts.expert, ts.simple)
		})
	}
}

type testAuthzManager struct {







<







55
56
57
58
59
60
61

62
63
64
65
66
67
68
		)
		name := fmt.Sprintf("readonly=%v/withauth=%v/expert=%v/simple=%v",
			ts.readonly, ts.withAuth, ts.expert, ts.simple)
		t.Run(name, func(tt *testing.T) {
			testCreate(tt, pol, ts.withAuth, ts.readonly)
			testRead(tt, pol, ts.withAuth, ts.expert)
			testWrite(tt, pol, ts.withAuth, ts.readonly, ts.expert)

			testDelete(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testRefresh(tt, pol, ts.withAuth, ts.expert, ts.simple)
		})
	}
}

type testAuthzManager struct {
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
		{owner, roTrue, roTrue, false},
		{owner2, roTrue, roTrue, false},
	}
	for _, tc := range testCases {
		t.Run("Write", func(tt *testing.T) {
			got := pol.CanWrite(tc.user, tc.old, tc.new)
			if tc.exp != got {
				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testRename(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {
	t.Helper()
	anonUser := newAnon()
	creator := newCreator()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()
	zettel := newZettel()
	expertZettel := newExpertZettel()
	roFalse := newRoFalseZettel()
	roTrue := newRoTrueZettel()
	roReader := newRoReaderZettel()
	roWriter := newRoWriterZettel()
	roOwner := newRoOwnerZettel()
	notAuthNotReadonly := !withAuth && !readonly
	testCases := []struct {
		user *meta.Meta
		meta *meta.Meta
		exp  bool
	}{
		// No meta
		{anonUser, nil, false},
		{creator, nil, false},
		{reader, nil, false},
		{writer, nil, false},
		{owner, nil, false},
		{owner2, nil, false},
		// Any zettel
		{anonUser, zettel, notAuthNotReadonly},
		{creator, zettel, notAuthNotReadonly},
		{reader, zettel, notAuthNotReadonly},
		{writer, zettel, notAuthNotReadonly},
		{owner, zettel, !readonly},
		{owner2, zettel, !readonly},
		// Expert zettel
		{anonUser, expertZettel, notAuthNotReadonly && expert},
		{creator, expertZettel, notAuthNotReadonly && expert},
		{reader, expertZettel, notAuthNotReadonly && expert},
		{writer, expertZettel, notAuthNotReadonly && expert},
		{owner, expertZettel, !readonly && expert},
		{owner2, expertZettel, !readonly && expert},
		// No r/o zettel
		{anonUser, roFalse, notAuthNotReadonly},
		{creator, roFalse, notAuthNotReadonly},
		{reader, roFalse, notAuthNotReadonly},
		{writer, roFalse, notAuthNotReadonly},
		{owner, roFalse, !readonly},
		{owner2, roFalse, !readonly},
		// Reader r/o zettel
		{anonUser, roReader, false},
		{creator, roReader, false},
		{reader, roReader, false},
		{writer, roReader, notAuthNotReadonly},
		{owner, roReader, !readonly},
		{owner2, roReader, !readonly},
		// Writer r/o zettel
		{anonUser, roWriter, false},
		{creator, roWriter, false},
		{reader, roWriter, false},
		{writer, roWriter, false},
		{owner, roWriter, !readonly},
		{owner2, roWriter, !readonly},
		// Owner r/o zettel
		{anonUser, roOwner, false},
		{creator, roOwner, false},
		{reader, roOwner, false},
		{writer, roOwner, false},
		{owner, roOwner, false},
		{owner2, roOwner, false},
		// r/o = true zettel
		{anonUser, roTrue, false},
		{creator, roTrue, false},
		{reader, roTrue, false},
		{writer, roTrue, false},
		{owner, roTrue, false},
		{owner2, roTrue, false},
	}
	for _, tc := range testCases {
		t.Run("Rename", func(tt *testing.T) {
			got := pol.CanRename(tc.user, tc.meta)
			if tc.exp != got {
				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testDelete(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







390
391
392
393
394
395
396
























































































397
398
399
400
401
402
403
		{owner, roTrue, roTrue, false},
		{owner2, roTrue, roTrue, false},
	}
	for _, tc := range testCases {
		t.Run("Write", func(tt *testing.T) {
			got := pol.CanWrite(tc.user, tc.old, tc.new)
			if tc.exp != got {
























































































				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testDelete(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {

Changes to auth/policy/readonly.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package policy

import "zettelstore.de/z/zettel/meta"

type roPolicy struct{}

func (*roPolicy) CanCreate(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanRead(_, _ *meta.Meta) bool     { return true }
func (*roPolicy) CanWrite(_, _, _ *meta.Meta) bool { return false }
func (*roPolicy) CanRename(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanDelete(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanRefresh(user *meta.Meta) bool  { return user != nil }








>
>
>











<


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package policy

import "zettelstore.de/z/zettel/meta"

type roPolicy struct{}

func (*roPolicy) CanCreate(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanRead(_, _ *meta.Meta) bool     { return true }
func (*roPolicy) CanWrite(_, _, _ *meta.Meta) bool { return false }

func (*roPolicy) CanDelete(_, _ *meta.Meta) bool   { return false }
func (*roPolicy) CanRefresh(user *meta.Meta) bool  { return user != nil }

Changes to box/box.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package box provides a generic interface to zettel boxes.
package box

import (
	"context"
	"errors"
	"fmt"
	"io"
	"time"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// BaseBox is implemented by all Zettel boxes.
type BaseBox interface {
	// Location returns some information where the box is located.
	// Format is dependent of the box.
	Location() string

	// GetZettel retrieves a specific zettel.
	GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error)

	// AllowRenameZettel returns true, if box will not disallow renaming the zettel.
	AllowRenameZettel(ctx context.Context, zid id.Zid) bool

	// RenameZettel changes the current Zid to a new Zid.
	RenameZettel(ctx context.Context, curZid, newZid id.Zid) error

	// CanDeleteZettel returns true, if box could possibly delete the given zettel.
	CanDeleteZettel(ctx context.Context, zid id.Zid) bool

	// DeleteZettel removes the zettel from the box.
	DeleteZettel(ctx context.Context, zid id.Zid) error
}









>
>
>












|















<
<
<
<
<
<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39






40
41
42
43
44
45
46
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package box provides a generic interface to zettel boxes.
package box

import (
	"context"
	"errors"
	"fmt"
	"io"
	"time"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// BaseBox is implemented by all Zettel boxes.
type BaseBox interface {
	// Location returns some information where the box is located.
	// Format is dependent of the box.
	Location() string

	// GetZettel retrieves a specific zettel.
	GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error)







	// CanDeleteZettel returns true, if box could possibly delete the given zettel.
	CanDeleteZettel(ctx context.Context, zid id.Zid) bool

	// DeleteZettel removes the zettel from the box.
	DeleteZettel(ctx context.Context, zid id.Zid) error
}

131
132
133
134
135
136
137
138
139
140
141
142
143
144
145

// Box is to be used outside the box package and its descendants.
type Box interface {
	BaseBox
	WriteBox

	// FetchZids returns the set of all zettel identifer managed by the box.
	FetchZids(ctx context.Context) (id.Set, error)

	// GetMeta returns the metadata of the zettel with the given identifier.
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)

	// SelectMeta returns a list of metadata that comply to the given selection criteria.
	// If `metaSeq` is `nil`, the box assumes metadata of all available zettel.
	SelectMeta(ctx context.Context, metaSeq []*meta.Meta, q *query.Query) ([]*meta.Meta, error)







|







128
129
130
131
132
133
134
135
136
137
138
139
140
141
142

// Box is to be used outside the box package and its descendants.
type Box interface {
	BaseBox
	WriteBox

	// FetchZids returns the set of all zettel identifer managed by the box.
	FetchZids(ctx context.Context) (*id.Set, error)

	// GetMeta returns the metadata of the zettel with the given identifier.
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)

	// SelectMeta returns a list of metadata that comply to the given selection criteria.
	// If `metaSeq` is `nil`, the box assumes metadata of all available zettel.
	SelectMeta(ctx context.Context, metaSeq []*meta.Meta, q *query.Query) ([]*meta.Meta, error)
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
	return context.WithValue(ctx, ctxNoEnrichKey, &ctxNoEnrichKey)
}

type ctxNoEnrichType struct{}

var ctxNoEnrichKey ctxNoEnrichType

// DoNotEnrich determines if the context is marked to not enrich metadata.
func DoNotEnrich(ctx context.Context) bool {
	_, ok := ctx.Value(ctxNoEnrichKey).(*ctxNoEnrichType)
	return ok
}

// NoEnrichQuery provides a context that signals not to enrich, if the query does not need this.
func NoEnrichQuery(ctx context.Context, q *query.Query) context.Context {
	if q.EnrichNeeded() {
		return ctx
	}







|
|

|







240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
	return context.WithValue(ctx, ctxNoEnrichKey, &ctxNoEnrichKey)
}

type ctxNoEnrichType struct{}

var ctxNoEnrichKey ctxNoEnrichType

// DoEnrich determines if the context is not marked to not enrich metadata.
func DoEnrich(ctx context.Context) bool {
	_, ok := ctx.Value(ctxNoEnrichKey).(*ctxNoEnrichType)
	return !ok
}

// NoEnrichQuery provides a context that signals not to enrich, if the query does not need this.
func NoEnrichQuery(ctx context.Context, q *query.Query) context.Context {
	if q.EnrichNeeded() {
		return ctx
	}

Changes to box/compbox/compbox.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51






52


53
54
55


56
57
58
59
60
61
62
63
64

65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package compbox provides zettel that have computed content.
package compbox

import (
	"context"
	"net/url"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	manager.Register(
		" comp",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return getCompBox(cdata.Number, cdata.Enricher), nil
		})
}

type compBox struct {
	log      *logger.Logger
	number   int
	enricher box.Enricher

}

var myConfig *meta.Meta
var myZettel = map[id.Zid]struct {
	meta    func(id.Zid) *meta.Meta
	content func(*meta.Meta) []byte
}{
	id.MustParse(api.ZidVersion):              {genVersionBuildM, genVersionBuildC},
	id.MustParse(api.ZidHost):                 {genVersionHostM, genVersionHostC},
	id.MustParse(api.ZidOperatingSystem):      {genVersionOSM, genVersionOSC},
	id.MustParse(api.ZidLog):                  {genLogM, genLogC},






	id.MustParse(api.ZidBoxManager):           {genManagerM, genManagerC},


	id.MustParse(api.ZidMetadataKey):          {genKeysM, genKeysC},
	id.MustParse(api.ZidParser):               {genParserM, genParserC},
	id.MustParse(api.ZidStartupConfiguration): {genConfigZettelM, genConfigZettelC},


}

// Get returns the one program box.
func getCompBox(boxNumber int, mf box.Enricher) box.ManagedBox {
	return &compBox{
		log: kernel.Main.GetLogger(kernel.BoxService).Clone().
			Str("box", "comp").Int("boxnum", int64(boxNumber)).Child(),
		number:   boxNumber,
		enricher: mf,

	}
}

// Setup remembers important values.
func Setup(cfg *meta.Meta) { myConfig = cfg.Clone() }

func (*compBox) Location() string { return "" }

func (cb *compBox) GetZettel(_ context.Context, zid id.Zid) (zettel.Zettel, error) {
	if gen, ok := myZettel[zid]; ok && gen.meta != nil {
		if m := gen.meta(zid); m != nil {
			updateMeta(m)
			if genContent := gen.content; genContent != nil {
				cb.log.Trace().Msg("GetZettel/Content")
				return zettel.Zettel{
					Meta:    m,
					Content: zettel.NewContent(genContent(m)),
				}, nil
			}
			cb.log.Trace().Msg("GetZettel/NoContent")
			return zettel.Zettel{Meta: m}, nil
		}
	}
	err := box.ErrZettelNotFound{Zid: zid}








>
>
>









|














|







>





|

|
|
|
|
>
>
>
>
>
>
|
>
>



>
>



|





>








|







|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package compbox provides zettel that have computed content.
package compbox

import (
	"context"
	"net/url"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	manager.Register(
		" comp",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return getCompBox(cdata.Number, cdata.Enricher, cdata.Mapper), nil
		})
}

type compBox struct {
	log      *logger.Logger
	number   int
	enricher box.Enricher
	mapper   manager.Mapper
}

var myConfig *meta.Meta
var myZettel = map[id.Zid]struct {
	meta    func(id.Zid) *meta.Meta
	content func(context.Context, *compBox) []byte
}{
	id.MustParse(api.ZidVersion):         {genVersionBuildM, genVersionBuildC},
	id.MustParse(api.ZidHost):            {genVersionHostM, genVersionHostC},
	id.MustParse(api.ZidOperatingSystem): {genVersionOSM, genVersionOSC},
	id.MustParse(api.ZidLog):             {genLogM, genLogC},
	id.MustParse(api.ZidMemory):          {genMemoryM, genMemoryC},
	id.MustParse(api.ZidSx):              {genSxM, genSxC},
	// id.MustParse(api.ZidHTTP):                 {genHttpM, genHttpC},
	// id.MustParse(api.ZidAPI):                  {genApiM, genApiC},
	// id.MustParse(api.ZidWebUI):                {genWebUiM, genWebUiC},
	// id.MustParse(api.ZidConsole):              {genConsoleM, genConsoleC},
	id.MustParse(api.ZidBoxManager): {genManagerM, genManagerC},
	// id.MustParse(api.ZidIndex):                {genIndexM, genIndexC},
	// id.MustParse(api.ZidQuery):                {genQueryM, genQueryC},
	id.MustParse(api.ZidMetadataKey):          {genKeysM, genKeysC},
	id.MustParse(api.ZidParser):               {genParserM, genParserC},
	id.MustParse(api.ZidStartupConfiguration): {genConfigZettelM, genConfigZettelC},
	id.MustParse(api.ZidWarnings):             {genWarningsM, genWarningsC},
	id.MustParse(api.ZidMapping):              {genMappingM, genMappingC},
}

// Get returns the one program box.
func getCompBox(boxNumber int, mf box.Enricher, mapper manager.Mapper) *compBox {
	return &compBox{
		log: kernel.Main.GetLogger(kernel.BoxService).Clone().
			Str("box", "comp").Int("boxnum", int64(boxNumber)).Child(),
		number:   boxNumber,
		enricher: mf,
		mapper:   mapper,
	}
}

// Setup remembers important values.
func Setup(cfg *meta.Meta) { myConfig = cfg.Clone() }

func (*compBox) Location() string { return "" }

func (cb *compBox) GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error) {
	if gen, ok := myZettel[zid]; ok && gen.meta != nil {
		if m := gen.meta(zid); m != nil {
			updateMeta(m)
			if genContent := gen.content; genContent != nil {
				cb.log.Trace().Msg("GetZettel/Content")
				return zettel.Zettel{
					Meta:    m,
					Content: zettel.NewContent(genContent(ctx, cb)),
				}, nil
			}
			cb.log.Trace().Msg("GetZettel/NoContent")
			return zettel.Zettel{Meta: m}, nil
		}
	}
	err := box.ErrZettelNotFound{Zid: zid}
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161






162
163
164
165
166
167



168
169
170
171
172
173
				handle(m)
			}
		}
	}
	return nil
}

func (*compBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	_, ok := myZettel[zid]
	return !ok
}

func (cb *compBox) RenameZettel(_ context.Context, curZid, _ id.Zid) (err error) {
	if _, ok := myZettel[curZid]; ok {
		err = box.ErrReadOnly
	} else {
		err = box.ErrZettelNotFound{Zid: curZid}
	}
	cb.log.Trace().Err(err).Msg("RenameZettel")
	return err
}

func (*compBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (cb *compBox) DeleteZettel(_ context.Context, zid id.Zid) (err error) {
	if _, ok := myZettel[zid]; ok {
		err = box.ErrReadOnly
	} else {
		err = box.ErrZettelNotFound{Zid: zid}
	}
	cb.log.Trace().Err(err).Msg("DeleteZettel")
	return err
}

func (cb *compBox) ReadStats(st *box.ManagedBoxStats) {
	st.ReadOnly = true
	st.Zettel = len(myZettel)
	cb.log.Trace().Int("zettel", int64(st.Zettel)).Msg("ReadStats")
}







func updateMeta(m *meta.Meta) {
	if _, ok := m.Get(api.KeySyntax); !ok {
		m.Set(api.KeySyntax, meta.SyntaxZmk)
	}
	m.Set(api.KeyRole, api.ValueRoleConfiguration)



	m.Set(api.KeyLang, api.ValueLangEN)
	m.Set(api.KeyReadOnly, api.ValueTrue)
	if _, ok := m.Get(api.KeyVisibility); !ok {
		m.Set(api.KeyVisibility, api.ValueVisibilityExpert)
	}
}







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<

















>
>
>
>
>
>






>
>
>






138
139
140
141
142
143
144















145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
				handle(m)
			}
		}
	}
	return nil
}
















func (*compBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (cb *compBox) DeleteZettel(_ context.Context, zid id.Zid) (err error) {
	if _, ok := myZettel[zid]; ok {
		err = box.ErrReadOnly
	} else {
		err = box.ErrZettelNotFound{Zid: zid}
	}
	cb.log.Trace().Err(err).Msg("DeleteZettel")
	return err
}

func (cb *compBox) ReadStats(st *box.ManagedBoxStats) {
	st.ReadOnly = true
	st.Zettel = len(myZettel)
	cb.log.Trace().Int("zettel", int64(st.Zettel)).Msg("ReadStats")
}

func getTitledMeta(zid id.Zid, title string) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, title)
	return m
}

func updateMeta(m *meta.Meta) {
	if _, ok := m.Get(api.KeySyntax); !ok {
		m.Set(api.KeySyntax, meta.SyntaxZmk)
	}
	m.Set(api.KeyRole, api.ValueRoleConfiguration)
	if _, ok := m.Get(api.KeyCreated); !ok {
		m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))
	}
	m.Set(api.KeyLang, api.ValueLangEN)
	m.Set(api.KeyReadOnly, api.ValueTrue)
	if _, ok := m.Get(api.KeyVisibility); !ok {
		m.Set(api.KeyVisibility, api.ValueVisibilityExpert)
	}
}

Changes to box/compbox/config.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"


	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genConfigZettelM(zid id.Zid) *meta.Meta {
	if myConfig == nil {
		return nil
	}
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Startup Configuration")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityExpert)
	return m
}

func genConfigZettelC(*meta.Meta) []byte {
	var buf bytes.Buffer
	for i, p := range myConfig.Pairs() {
		if i > 0 {
			buf.WriteByte('\n')
		}
		buf.WriteString("; ''")
		buf.WriteString(p.Key)








>
>
>






>

<
<








<
|
<
<
<


|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


20
21
22
23
24
25
26
27

28



29
30
31
32
33
34
35
36
37
38
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"



	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genConfigZettelM(zid id.Zid) *meta.Meta {
	if myConfig == nil {
		return nil
	}

	return getTitledMeta(zid, "Zettelstore Startup Configuration")



}

func genConfigZettelC(context.Context, *compBox) []byte {
	var buf bytes.Buffer
	for i, p := range myConfig.Pairs() {
		if i > 0 {
			buf.WriteByte('\n')
		}
		buf.WriteString("; ''")
		buf.WriteString(p.Key)

Changes to box/compbox/keys.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"

	"fmt"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genKeysM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Supported Metadata Keys")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)
	return m
}

func genKeysC(*meta.Meta) []byte {
	keys := meta.GetSortedKeyDescriptions()
	var buf bytes.Buffer
	buf.WriteString("|=Name<|=Type<|=Computed?:|=Property?:\n")
	for _, kd := range keys {
		fmt.Fprintf(&buf,
			"|[[%v|query:%v?]]|%v|%v|%v\n", kd.Name, kd.Name, kd.Type.Name, kd.IsComputed(), kd.IsProperty())
	}








>
>
>






>


|






<
|





|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

28
29
30
31
32
33
34
35
36
37
38
39
40
41
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"
	"fmt"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genKeysM(zid id.Zid) *meta.Meta {

	m := getTitledMeta(zid, "Zettelstore Supported Metadata Keys")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)
	return m
}

func genKeysC(context.Context, *compBox) []byte {
	keys := meta.GetSortedKeyDescriptions()
	var buf bytes.Buffer
	buf.WriteString("|=Name<|=Type<|=Computed?:|=Property?:\n")
	for _, kd := range keys {
		fmt.Fprintf(&buf,
			"|[[%v|query:%v?]]|%v|%v|%v\n", kd.Name, kd.Name, kd.Type.Name, kd.IsComputed(), kd.IsProperty())
	}

Changes to box/compbox/log.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"


	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genLogM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Log")
	m.Set(api.KeySyntax, meta.SyntaxText)
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))
	m.Set(api.KeyModified, kernel.Main.GetLastLogTime().Local().Format(id.TimestampLayout))
	return m
}

func genLogC(*meta.Meta) []byte {
	const tsFormat = "2006-01-02 15:04:05.999999"
	entries := kernel.Main.RetrieveLogEntries()
	var buf bytes.Buffer
	for _, entry := range entries {
		ts := entry.TS.Format(tsFormat)
		buf.WriteString(ts)
		for j := len(ts); j < len(tsFormat); j++ {








>
>
>






>

|






<
|

<




|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28

29
30
31
32
33
34
35
36
37
38
39
40
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genLogM(zid id.Zid) *meta.Meta {

	m := getTitledMeta(zid, "Zettelstore Log")
	m.Set(api.KeySyntax, meta.SyntaxText)

	m.Set(api.KeyModified, kernel.Main.GetLastLogTime().Local().Format(id.TimestampLayout))
	return m
}

func genLogC(context.Context, *compBox) []byte {
	const tsFormat = "2006-01-02 15:04:05.999999"
	entries := kernel.Main.RetrieveLogEntries()
	var buf bytes.Buffer
	for _, entry := range entries {
		ts := entry.TS.Format(tsFormat)
		buf.WriteString(ts)
		for j := len(ts); j < len(tsFormat); j++ {

Changes to box/compbox/manager.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"

	"fmt"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genManagerM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Box Manager")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))
	return m
}

func genManagerC(*meta.Meta) []byte {
	kvl := kernel.Main.GetServiceStatistics(kernel.BoxService)
	if len(kvl) == 0 {
		return nil
	}
	var buf bytes.Buffer
	buf.WriteString("|=Name|=Value>\n")
	for _, kv := range kvl {








>
>
>






>


<






<
|
<
<


|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26

27


28
29
30
31
32
33
34
35
36
37
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"
	"fmt"


	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genManagerM(zid id.Zid) *meta.Meta {

	return getTitledMeta(zid, "Zettelstore Box Manager")


}

func genManagerC(context.Context, *compBox) []byte {
	kvl := kernel.Main.GetServiceStatistics(kernel.BoxService)
	if len(kvl) == 0 {
		return nil
	}
	var buf bytes.Buffer
	buf.WriteString("|=Name|=Value>\n")
	for _, kv := range kvl {

Added box/compbox/mapping.go.



































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
//-----------------------------------------------------------------------------
// Copyright (c) 2024-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2024-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"

	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// Zettelstore Identifier Mapping.
//
// In the first stage of migration process, it is a computed zettel showing a
// hypothetical mapping. In later stages, it will be stored as a normal zettel
// that is updated when a new zettel is created or an old zettel is deleted.

func genMappingM(zid id.Zid) *meta.Meta {
	return getTitledMeta(zid, "Zettelstore Identifier Mapping")
}

func genMappingC(ctx context.Context, cb *compBox) []byte {
	var buf bytes.Buffer
	toNew, err := cb.mapper.OldToNewMapping(ctx)
	if err != nil {
		buf.WriteString("**Error while fetching: ")
		buf.WriteString(err.Error())
		buf.WriteString("**\n")
		return buf.Bytes()
	}
	oldZids := id.NewSetCap(len(toNew))
	for zidO := range toNew {
		oldZids.Add(zidO)
	}
	first := true
	oldZids.ForEach(func(zidO id.Zid) {
		if first {
			buf.WriteString("**Note**: this mapping is preliminary.\n")
			buf.WriteString("It only shows you how it could look if the migration is done.\n")
			buf.WriteString("Use this page to update your zettel if something strange is shown.\n")
			buf.WriteString("```\n")
			first = false
		}
		buf.WriteString(zidO.String())
		buf.WriteByte(' ')
		buf.WriteString(toNew[zidO].String())
		buf.WriteByte('\n')
	})
	if !first {
		buf.WriteString("```")
	}
	return buf.Bytes()
}

Added box/compbox/memory.go.











































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
//-----------------------------------------------------------------------------
// Copyright (c) 2024-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2024-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"
	"fmt"
	"os"
	"runtime"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genMemoryM(zid id.Zid) *meta.Meta {
	return getTitledMeta(zid, "Zettelstore Memory")
}

func genMemoryC(context.Context, *compBox) []byte {
	pageSize := os.Getpagesize()
	var m runtime.MemStats
	runtime.GC()
	runtime.ReadMemStats(&m)

	var buf bytes.Buffer
	buf.WriteString("|=Name|=Value>\n")
	fmt.Fprintf(&buf, "|Page Size|%d\n", pageSize)
	fmt.Fprintf(&buf, "|Pages|%d\n", m.HeapSys/uint64(pageSize))
	fmt.Fprintf(&buf, "|Heap Objects|%d\n", m.HeapObjects)
	fmt.Fprintf(&buf, "|Heap Sys (KiB)|%d\n", m.HeapSys/1024)
	fmt.Fprintf(&buf, "|Heap Inuse (KiB)|%d\n", m.HeapInuse/1024)
	debug := kernel.Main.GetConfig(kernel.CoreService, kernel.CoreDebug).(bool)
	if debug {
		for i, bysize := range m.BySize {
			fmt.Fprintf(&buf, "|Size %2d: %d|%d - %d &rarr; %d\n",
				i, bysize.Size, bysize.Mallocs, bysize.Frees, bysize.Mallocs-bysize.Frees)
		}
	}
	return buf.Bytes()
}

Changes to box/compbox/parser.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"

	"fmt"
	"sort"
	"strings"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genParserM(zid id.Zid) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, "Zettelstore Supported Parser")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)
	return m
}

func genParserC(*meta.Meta) []byte {
	var buf bytes.Buffer
	buf.WriteString("|=Syntax<|=Alt. Value(s):|=Text Parser?:|=Text Format?:|=Image Format?:\n")
	syntaxes := parser.GetSyntaxes()
	sort.Strings(syntaxes)
	for _, syntax := range syntaxes {
		info := parser.Get(syntax)
		if info.Name != syntax {
			continue
		}
		altNames := info.AltNames
		sort.Strings(altNames)
		fmt.Fprintf(
			&buf, "|%v|%v|%v|%v|%v\n",
			syntax, strings.Join(altNames, ", "), info.IsASTParser, info.IsTextFormat, info.IsImageFormat)
	}
	return buf.Bytes()
}








>
>
>






>

|


|







<
|





|



|






|






1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"
	"fmt"
	"slices"
	"strings"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genParserM(zid id.Zid) *meta.Meta {

	m := getTitledMeta(zid, "Zettelstore Supported Parser")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)
	return m
}

func genParserC(context.Context, *compBox) []byte {
	var buf bytes.Buffer
	buf.WriteString("|=Syntax<|=Alt. Value(s):|=Text Parser?:|=Text Format?:|=Image Format?:\n")
	syntaxes := parser.GetSyntaxes()
	slices.Sort(syntaxes)
	for _, syntax := range syntaxes {
		info := parser.Get(syntax)
		if info.Name != syntax {
			continue
		}
		altNames := info.AltNames
		slices.Sort(altNames)
		fmt.Fprintf(
			&buf, "|%v|%v|%v|%v|%v\n",
			syntax, strings.Join(altNames, ", "), info.IsASTParser, info.IsTextFormat, info.IsImageFormat)
	}
	return buf.Bytes()
}

Added box/compbox/sx.go.







































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
//-----------------------------------------------------------------------------
// Copyright (c) 2024-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2024-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"
	"fmt"

	"t73f.de/r/sx"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genSxM(zid id.Zid) *meta.Meta {
	return getTitledMeta(zid, "Zettelstore Sx Engine")
}

func genSxC(context.Context, *compBox) []byte {
	var buf bytes.Buffer
	buf.WriteString("|=Name|=Value>\n")
	fmt.Fprintf(&buf, "|Symbols|%d\n", sx.MakeSymbol("NIL").Factory().Size())
	return buf.Bytes()
}

Changes to box/compbox/version.go.

1
2
3
4
5
6
7
8



9
10
11
12
13

14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package compbox

import (

	"zettelstore.de/client.fossil/api"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func getVersionMeta(zid id.Zid, title string) *meta.Meta {
	m := meta.New(zid)
	m.Set(api.KeyTitle, title)
	m.Set(api.KeyVisibility, api.ValueVisibilityExpert)
	return m
}

func genVersionBuildM(zid id.Zid) *meta.Meta {
	m := getVersionMeta(zid, "Zettelstore Version")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)
	return m
}
func genVersionBuildC(*meta.Meta) []byte {
	return []byte(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string))
}

func genVersionHostM(zid id.Zid) *meta.Meta {
	m := getVersionMeta(zid, "Zettelstore Host")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))
	return m
}
func genVersionHostC(*meta.Meta) []byte {
	return []byte(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreHostname).(string))
}

func genVersionOSM(zid id.Zid) *meta.Meta {
	m := getVersionMeta(zid, "Zettelstore Operating System")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreStarted).(string))
	return m
}
func genVersionOSC(*meta.Meta) []byte {
	goOS := kernel.Main.GetConfig(kernel.CoreService, kernel.CoreGoOS).(string)
	goArch := kernel.Main.GetConfig(kernel.CoreService, kernel.CoreGoArch).(string)
	result := make([]byte, 0, len(goOS)+len(goArch)+1)
	result = append(result, goOS...)
	result = append(result, '/')
	return append(result, goArch...)
}








>
>
>





>
|
>





<
<
<
<
<
<
<

|




|




|
<
<

|




|
<
<

|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24







25
26
27
28
29
30
31
32
33
34
35
36


37
38
39
40
41
42
43


44
45
46
47
48
49
50
51
52
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"context"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)








func genVersionBuildM(zid id.Zid) *meta.Meta {
	m := getTitledMeta(zid, "Zettelstore Version")
	m.Set(api.KeyCreated, kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVTime).(string))
	m.Set(api.KeyVisibility, api.ValueVisibilityLogin)
	return m
}
func genVersionBuildC(context.Context, *compBox) []byte {
	return []byte(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string))
}

func genVersionHostM(zid id.Zid) *meta.Meta {
	return getTitledMeta(zid, "Zettelstore Host")


}
func genVersionHostC(context.Context, *compBox) []byte {
	return []byte(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreHostname).(string))
}

func genVersionOSM(zid id.Zid) *meta.Meta {
	return getTitledMeta(zid, "Zettelstore Operating System")


}
func genVersionOSC(context.Context, *compBox) []byte {
	goOS := kernel.Main.GetConfig(kernel.CoreService, kernel.CoreGoOS).(string)
	goArch := kernel.Main.GetConfig(kernel.CoreService, kernel.CoreGoArch).(string)
	result := make([]byte, 0, len(goOS)+len(goArch)+1)
	result = append(result, goOS...)
	result = append(result, '/')
	return append(result, goArch...)
}

Added box/compbox/warnings.go.









































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
//-----------------------------------------------------------------------------
// Copyright (c) 2024-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2024-present Detlef Stern
//-----------------------------------------------------------------------------

package compbox

import (
	"bytes"
	"context"

	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func genWarningsM(zid id.Zid) *meta.Meta {
	return getTitledMeta(zid, "Zettelstore Warnings")
}

func genWarningsC(ctx context.Context, cb *compBox) []byte {
	var buf bytes.Buffer
	buf.WriteString("* [[Zettel without stored creation date|query:created-missing:true]]\n")
	buf.WriteString("* [[Zettel with strange creation date|query:created<19700101000000]]\n")

	ws, err := cb.mapper.Warnings(ctx)
	if err != nil {
		buf.WriteString("**Error while fetching: ")
		buf.WriteString(err.Error())
		buf.WriteString("**\n")
		return buf.Bytes()
	}
	first := true
	ws.ForEach(func(zid id.Zid) {
		if first {
			first = false
			buf.WriteString("=== Mapper Warnings\n")
		}
		buf.WriteString("* [[")
		buf.WriteString(zid.String())
		buf.WriteString("]]\n")
	})

	return buf.Bytes()
}

Changes to box/constbox/base.css.















1
2
3
4
5
6
7














*,*::before,*::after {
    box-sizing: border-box;
  }
  html {
    font-size: 1rem;
    font-family: serif;
    scroll-behavior: smooth;
>
>
>
>
>
>
>
>
>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
/*-----------------------------------------------------------------------------
 * Copyright (c) 2020-present Detlef Stern
 *
 * This file is part of Zettelstore.
 *
 * Zettelstore is licensed under the latest version of the EUPL (European Union
 * Public License). Please see file LICENSE.txt for your rights and obligations
 * under this license.
 *
 * SPDX-License-Identifier: EUPL-1.2
 * SPDX-FileCopyrightText: 2020-present Detlef Stern
 *-----------------------------------------------------------------------------
 */

*,*::before,*::after {
    box-sizing: border-box;
  }
  html {
    font-size: 1rem;
    font-family: serif;
    scroll-behavior: smooth;

Changes to box/constbox/base.sxn.














1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18













`(@@@@
(html ,@(if lang `((@ (lang ,lang))))
(head
  (meta (@ (charset "utf-8")))
  (meta (@ (name "viewport") (content "width=device-width, initial-scale=1.0")))
  (meta (@ (name "generator") (content "Zettelstore")))
  (meta (@ (name "format-detection") (content "telephone=no")))
  ,@META-HEADER
  (link (@ (rel "stylesheet") (href ,css-base-url)))
  (link (@ (rel "stylesheet") (href ,css-user-url)))
  ,@(ROLE-DEFAULT-meta (current-environment))
  (title ,title))
(body
  (nav (@ (class "zs-menu"))
    (a (@ (href ,home-url)) "Home")
    ,@(if with-auth
      `((div (@ (class "zs-dropdown"))
        (button "User")
>
>
>
>
>
>
>
>
>
>
>
>
>










|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(@@@@
(html ,@(if lang `((@ (lang ,lang))))
(head
  (meta (@ (charset "utf-8")))
  (meta (@ (name "viewport") (content "width=device-width, initial-scale=1.0")))
  (meta (@ (name "generator") (content "Zettelstore")))
  (meta (@ (name "format-detection") (content "telephone=no")))
  ,@META-HEADER
  (link (@ (rel "stylesheet") (href ,css-base-url)))
  (link (@ (rel "stylesheet") (href ,css-user-url)))
  ,@(ROLE-DEFAULT-meta (current-binding))
  (title ,title))
(body
  (nav (@ (class "zs-menu"))
    (a (@ (href ,home-url)) "Home")
    ,@(if with-auth
      `((div (@ (class "zs-dropdown"))
        (button "User")
35
36
37
38
39
40
41
42


43
44
45
46
47
48
    ,@(if new-zettel-links
      `((div (@ (class "zs-dropdown"))
        (button "New")
        (nav (@ (class "zs-dropdown-content"))
          ,@(map wui-link new-zettel-links)
       )))
    )
    (form (@ (action ,search-url))


      (input (@ (type "text") (placeholder "Search..") (name ,query-key-query) (dir "auto"))))
  )
  (main (@ (class "content")) ,DETAIL)
  ,@(if FOOTER `((footer (hr) ,@FOOTER)))
  ,@(if debug-mode '((div (b "WARNING: Debug mode is enabled. DO NOT USE IN PRODUCTION!"))))
)))







|
>
>
|





48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
    ,@(if new-zettel-links
      `((div (@ (class "zs-dropdown"))
        (button "New")
        (nav (@ (class "zs-dropdown-content"))
          ,@(map wui-link new-zettel-links)
       )))
    )
    (search (form (@ (action ,search-url))
      (input (@ (type "search") (inputmode "search") (name ,query-key-query)
                (title "General search field, with same behaviour as search field in search result list")
                (placeholder "Search..") (dir "auto")))))
  )
  (main (@ (class "content")) ,DETAIL)
  ,@(if FOOTER `((footer (hr) ,@FOOTER)))
  ,@(if debug-mode '((div (b "WARNING: Debug mode is enabled. DO NOT USE IN PRODUCTION!"))))
)))

Changes to box/constbox/constbox.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package constbox puts zettel inside the executable.
package constbox

import (
	"context"
	_ "embed" // Allow to embed file content
	"net/url"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"








>
>
>










|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package constbox puts zettel inside the executable.
package constbox

import (
	"context"
	_ "embed" // Allow to embed file content
	"net/url"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
			cb.enricher.Enrich(ctx, m, cb.number)
			handle(m)
		}
	}
	return nil
}

func (cb *constBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	_, ok := cb.zettel[zid]
	return !ok
}

func (cb *constBox) RenameZettel(_ context.Context, curZid, _ id.Zid) (err error) {
	if _, ok := cb.zettel[curZid]; ok {
		err = box.ErrReadOnly
	} else {
		err = box.ErrZettelNotFound{Zid: curZid}
	}
	cb.log.Trace().Err(err).Msg("RenameZettel")
	return err
}

func (*constBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (cb *constBox) DeleteZettel(_ context.Context, zid id.Zid) (err error) {
	if _, ok := cb.zettel[zid]; ok {
		err = box.ErrReadOnly
	} else {
		err = box.ErrZettelNotFound{Zid: zid}







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







93
94
95
96
97
98
99















100
101
102
103
104
105
106
			cb.enricher.Enrich(ctx, m, cb.number)
			handle(m)
		}
	}
	return nil
}
















func (*constBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (cb *constBox) DeleteZettel(_ context.Context, zid id.Zid) (err error) {
	if _, ok := cb.zettel[zid]; ok {
		err = box.ErrReadOnly
	} else {
		err = box.ErrZettelNotFound{Zid: zid}
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270

271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
			api.KeyTitle:      "Zettelstore Dependencies",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyLang:       api.ValueLangEN,
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityPublic,
			api.KeyCreated:    "20210504135842",
			api.KeyModified:   "20230601163100",
		},
		zettel.NewContent(contentDependencies)},
	id.BaseTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Base HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230510155100",
			api.KeyModified:   "20230827212200",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentBaseSxn)},
	id.LoginTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Login Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230527144100",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentLoginSxn)},
	id.ZettelTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230510155300",
			api.KeyModified:   "20231126180500",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentZettelSxn)},
	id.InfoTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Info HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20231023152000",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentInfoSxn)},
	id.FormTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230621132600",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentFormSxn)},
	id.RenameTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Rename Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230707190246",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentRenameSxn)},
	id.DeleteTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Delete HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20230621133100",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentDeleteSxn)},
	id.ListTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore List Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230704122100",
			api.KeyModified:   "20231129112800",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentListZettelSxn)},
	id.ErrorTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Error HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20210305133215",
			api.KeyModified:   "20230527224800",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentErrorSxn)},
	id.StartSxnZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Sxn Start Code",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230824160700",

			api.KeyVisibility: api.ValueVisibilityExpert,
			api.KeyPrecursor:  string(api.ZidSxnBase),
		},
		zettel.NewContent(contentStartCodeSxn)},
	id.BaseSxnZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Sxn Base Code",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230619132800",
			api.KeyModified:   "20231012154500",
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
			api.KeyPrecursor:  string(api.ZidSxnPrelude),
		},
		zettel.NewContent(contentBaseCodeSxn)},
	id.PreludeSxnZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Sxn Prelude",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20231006181700",
			api.KeyModified:   "20231019140400",
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentPreludeSxn)},
	id.MustParse(api.ZidBaseCSS): {
		constHeader{
			api.KeyTitle:      "Zettelstore Base CSS",







|








|









|









|









|









|



<
<
<
<
<
<
<
<
<
<






|









|









|









>










|











|







153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212










213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
			api.KeyTitle:      "Zettelstore Dependencies",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyLang:       api.ValueLangEN,
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityPublic,
			api.KeyCreated:    "20210504135842",
			api.KeyModified:   "20240418095500",
		},
		zettel.NewContent(contentDependencies)},
	id.BaseTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Base HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230510155100",
			api.KeyModified:   "20240219145300",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentBaseSxn)},
	id.LoginTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Login Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20240219145200",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentLoginSxn)},
	id.ZettelTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230510155300",
			api.KeyModified:   "20240219145100",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentZettelSxn)},
	id.InfoTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Info HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20240618170000",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentInfoSxn)},
	id.FormTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20240219145200",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentFormSxn)},










	id.DeleteTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Delete HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20200804111624",
			api.KeyModified:   "20240219145200",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentDeleteSxn)},
	id.ListTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore List Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230704122100",
			api.KeyModified:   "20240219145200",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentListZettelSxn)},
	id.ErrorTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Error HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20210305133215",
			api.KeyModified:   "20240219145200",
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentErrorSxn)},
	id.StartSxnZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Sxn Start Code",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230824160700",
			api.KeyModified:   "20240219145200",
			api.KeyVisibility: api.ValueVisibilityExpert,
			api.KeyPrecursor:  string(api.ZidSxnBase),
		},
		zettel.NewContent(contentStartCodeSxn)},
	id.BaseSxnZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Sxn Base Code",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20230619132800",
			api.KeyModified:   "20240618170100",
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
			api.KeyPrecursor:  string(api.ZidSxnPrelude),
		},
		zettel.NewContent(contentBaseCodeSxn)},
	id.PreludeSxnZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Sxn Prelude",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxSxn,
			api.KeyCreated:    "20231006181700",
			api.KeyModified:   "20240222121200",
			api.KeyReadOnly:   api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		zettel.NewContent(contentPreludeSxn)},
	id.MustParse(api.ZidBaseCSS): {
		constHeader{
			api.KeyTitle:      "Zettelstore Base CSS",
416
417
418
419
420
421
422










423
424
425
426
427
428
429
			api.KeyRole:       api.ValueRoleRole,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyCreated:    "20231129162000",
			api.KeyLang:       api.ValueLangEN,
			api.KeyVisibility: api.ValueVisibilityLogin,
		},
		zettel.NewContent(contentRoleTag)},










	id.DefaultHomeZid: {
		constHeader{
			api.KeyTitle:   "Home",
			api.KeyRole:    api.ValueRoleZettel,
			api.KeySyntax:  meta.SyntaxZmk,
			api.KeyLang:    api.ValueLangEN,
			api.KeyCreated: "20210210190757",







>
>
>
>
>
>
>
>
>
>







395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
			api.KeyRole:       api.ValueRoleRole,
			api.KeySyntax:     meta.SyntaxZmk,
			api.KeyCreated:    "20231129162000",
			api.KeyLang:       api.ValueLangEN,
			api.KeyVisibility: api.ValueVisibilityLogin,
		},
		zettel.NewContent(contentRoleTag)},
	id.MustParse(api.ZidAppDirectory): {
		constHeader{
			api.KeyTitle:      "Zettelstore Application Directory",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     meta.SyntaxNone,
			api.KeyLang:       api.ValueLangEN,
			api.KeyCreated:    "20240703235900",
			api.KeyVisibility: api.ValueVisibilityLogin,
		},
		zettel.NewContent(nil)},
	id.DefaultHomeZid: {
		constHeader{
			api.KeyTitle:   "Home",
			api.KeyRole:    api.ValueRoleZettel,
			api.KeySyntax:  meta.SyntaxZmk,
			api.KeyLang:    api.ValueLangEN,
			api.KeyCreated: "20210210190757",
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467

//go:embed info.sxn
var contentInfoSxn []byte

//go:embed form.sxn
var contentFormSxn []byte

//go:embed rename.sxn
var contentRenameSxn []byte

//go:embed delete.sxn
var contentDeleteSxn []byte

//go:embed listzettel.sxn
var contentListZettelSxn []byte

//go:embed error.sxn







<
<
<







440
441
442
443
444
445
446



447
448
449
450
451
452
453

//go:embed info.sxn
var contentInfoSxn []byte

//go:embed form.sxn
var contentFormSxn []byte




//go:embed delete.sxn
var contentDeleteSxn []byte

//go:embed listzettel.sxn
var contentListZettelSxn []byte

//go:embed error.sxn

Changes to box/constbox/delete.sxn.














1
2
3
4
5
6
7













`(article
  (header (h1 "Delete Zettel " ,zid))
  (p "Do you really want to delete this zettel?")
  ,@(if shadowed-box
    `((div (@ (class "zs-info"))
      (h2 "Information")
      (p "If you delete this zettel, the previously shadowed zettel from overlayed box " ,shadowed-box " becomes available.")
>
>
>
>
>
>
>
>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(article
  (header (h1 "Delete Zettel " ,zid))
  (p "Do you really want to delete this zettel?")
  ,@(if shadowed-box
    `((div (@ (class "zs-info"))
      (h2 "Information")
      (p "If you delete this zettel, the previously shadowed zettel from overlayed box " ,shadowed-box " becomes available.")

Changes to box/constbox/dependencies.zettel.

126
127
128
129
130
131
132
133
134
135
136
137
138


139


140
141
142
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```

=== sx, zettelstore-client
These are companion projects, written by the current main developer of Zettelstore.
They are published under the same license, [[EUPL v1.2, or later|00000000000004]].

; URL & Source sx
: [[https://zettelstore.de/sx]]


; URL & Source zettelstore-client


: [[https://zettelstore.de/client/]]
; License:
: European Union Public License, version 1.2 (EUPL v1.2), or later.







|
|


|
|
>
>
|
>
>
|


126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```

=== Sx, SxWebs, Webs, Zettelstore-Client
These are companion projects, written by the main developer of Zettelstore.
They are published under the same license, [[EUPL v1.2, or later|00000000000004]].

; URL & Source Sx
: [[https://t73f.de/r/sx]]
; URL & Source SxWebs
: [[https://t73f.de/r/sxwebs]]
; URL & Source Webs
: [[https://t73f.de/r/webs]]
; URL & Source Zettelstore-Client
: [[https://t73f.de/r/zsc]]
; License:
: European Union Public License, version 1.2 (EUPL v1.2), or later.

Changes to box/constbox/error.sxn.














1
2
3
4













`(article
  (header (h1 ,heading))
  ,message
)
>
>
>
>
>
>
>
>
>
>
>
>
>




1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(article
  (header (h1 ,heading))
  ,message
)

Changes to box/constbox/form.sxn.














1
2
3
4
5
6


7
8


9
10
11
12
13
14
15
16


17
18
19


20
21


22
23
24
25
26
27
28
29
30


31
32
33
34
35
36
37













`(article
  (header (h1 ,heading))
  (form (@ (action ,form-action-url) (method "POST") (enctype "multipart/form-data"))
  (div
    (label (@ (for "zs-title")) "Title " (a (@ (title "Main heading of this zettel.")) (@H "&#9432;")))
    (input (@ (class "zs-input") (type "text") (id "zs-title") (name "title") (placeholder "Title..") (value ,meta-title) (dir "auto") (autofocus))))


  (div
    (label (@ (for "zs-role")) "Role " (a (@ (title "One word, without spaces, to set the main role of this zettel.")) (@H "&#9432;")))


    (input (@ (class "zs-input") (type "text") (id "zs-role") (name "role") (placeholder "role..") (value ,meta-role) (dir "auto")
      ,@(if role-data '((list "zs-role-data")))
    ))
    ,@(wui-datalist "zs-role-data" role-data)
  )
  (div
    (label (@ (for "zs-tags")) "Tags " (a (@ (title "Tags must begin with an '#' sign. They are separated by spaces.")) (@H "&#9432;")))
    (input (@ (class "zs-input") (type "text") (id "zs-tags") (name "tags") (placeholder "#tag") (value ,meta-tags) (dir "auto"))))


  (div
    (label (@ (for "zs-meta")) "Metadata " (a (@ (title "Other metadata for this zettel. Each line contains a key/value pair, separated by a colon ':'.")) (@H "&#9432;")))
    (textarea (@ (class "zs-input") (id "zs-meta") (name "meta") (rows "4") (placeholder "metakey: metavalue") (dir "auto")) ,meta))


  (div
    (label (@ (for "zs-syntax")) "Syntax " (a (@ (title "Syntax of zettel content below, one word. Typically 'zmk' (for zettelmarkup).")) (@H "&#9432;")))


    (input (@ (class "zs-input") (type "text") (id "zs-syntax") (name "syntax") (placeholder "syntax..") (value ,meta-syntax) (dir "auto")
      ,@(if syntax-data '((list "zs-syntax-data")))
    ))
    ,@(wui-datalist "zs-syntax-data" syntax-data)
  )
  ,@(if (bound? 'content)
    `((div
      (label (@ (for "zs-content")) "Content " (a (@ (title "Content for this zettel, according to above syntax.")) (@H "&#9432;")))
      (textarea (@ (class "zs-input zs-content") (id "zs-content") (name "content") (rows "20") (placeholder "Zettel content..") (dir "auto")) ,content)


    ))
  )
  (div
    (input (@ (class "zs-primary") (type "submit") (value "Submit")))
    (input (@ (class "zs-secondary") (type "submit") (value "Save") (formaction "?save")))
    (input (@ (class "zs-upload") (type "file") (id "zs-file") (name "file")))
  ))
>
>
>
>
>
>
>
>
>
>
>
>
>





|
>
>


>
>
|






|
>
>


|
>
>


>
>
|







|
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(article
  (header (h1 ,heading))
  (form (@ (action ,form-action-url) (method "POST") (enctype "multipart/form-data"))
  (div
    (label (@ (for "zs-title")) "Title " (a (@ (title "Main heading of this zettel.")) (@H "&#9432;")))
    (input (@ (class "zs-input") (type "text") (id "zs-title") (name "title")
              (title "Title of this zettel")
              (placeholder "Title..") (value ,meta-title) (dir "auto") (autofocus))))
  (div
    (label (@ (for "zs-role")) "Role " (a (@ (title "One word, without spaces, to set the main role of this zettel.")) (@H "&#9432;")))
    (input (@ (class "zs-input") (type "text") (pattern "\\w*") (id "zs-role") (name "role")
              (title "One word, letters and digits, but no spaces, to set the main role of the zettel.")
              (placeholder "role..") (value ,meta-role) (dir "auto")
      ,@(if role-data '((list "zs-role-data")))
    ))
    ,@(wui-datalist "zs-role-data" role-data)
  )
  (div
    (label (@ (for "zs-tags")) "Tags " (a (@ (title "Tags must begin with an '#' sign. They are separated by spaces.")) (@H "&#9432;")))
    (input (@ (class "zs-input") (type "text") (id "zs-tags") (name "tags")
              (title "Tags/keywords to categorize the zettel. Each tags is a word that begins with a '#' character; they are separated by spaces")
              (placeholder "#tag") (value ,meta-tags) (dir "auto"))))
  (div
    (label (@ (for "zs-meta")) "Metadata " (a (@ (title "Other metadata for this zettel. Each line contains a key/value pair, separated by a colon ':'.")) (@H "&#9432;")))
    (textarea (@ (class "zs-input") (id "zs-meta") (name "meta") (rows "4")
                 (title "Additional metadata about the zettel")
                 (placeholder "metakey: metavalue") (dir "auto")) ,meta))
  (div
    (label (@ (for "zs-syntax")) "Syntax " (a (@ (title "Syntax of zettel content below, one word. Typically 'zmk' (for zettelmarkup).")) (@H "&#9432;")))
    (input (@ (class "zs-input") (type "text") (pattern "\\w*") (id "zs-syntax") (name "syntax")
              (title "Syntax/format of zettel content below, one word, letters and digits, no spaces.")
              (placeholder "syntax..") (value ,meta-syntax) (dir "auto")
      ,@(if syntax-data '((list "zs-syntax-data")))
    ))
    ,@(wui-datalist "zs-syntax-data" syntax-data)
  )
  ,@(if (bound? 'content)
    `((div
      (label (@ (for "zs-content")) "Content " (a (@ (title "Content for this zettel, according to above syntax.")) (@H "&#9432;")))
      (textarea (@ (class "zs-input zs-content") (id "zs-content") (name "content") (rows "20")
                   (title "Zettel content, according to the given syntax")
                   (placeholder "Zettel content..") (dir "auto")) ,content)
    ))
  )
  (div
    (input (@ (class "zs-primary") (type "submit") (value "Submit")))
    (input (@ (class "zs-secondary") (type "submit") (value "Save") (formaction "?save")))
    (input (@ (class "zs-upload") (type "file") (id "zs-file") (name "file")))
  ))

Changes to box/constbox/info.sxn.














1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23













`(article
  (header (h1 "Information for Zettel " ,zid)
    (p
      (a (@ (href ,web-url)) "Web")
      (@H " &#183; ") (a (@ (href ,context-url)) "Context")

      ,@(if (bound? 'edit-url) `((@H " &#183; ") (a (@ (href ,edit-url)) "Edit")))
      ,@(ROLE-DEFAULT-actions (current-environment))
      ,@(if (bound? 'reindex-url) `((@H " &#183; ") (a (@ (href ,reindex-url)) "Reindex")))
      ,@(if (bound? 'rename-url) `((@H " &#183; ") (a (@ (href ,rename-url)) "Rename")))
      ,@(if (bound? 'delete-url) `((@H " &#183; ") (a (@ (href ,delete-url)) "Delete")))
    )
  )
  (h2 "Interpreted Metadata")
  (table ,@(map wui-table-row metadata))
  (h2 "References")
  ,@(if local-links `((h3 "Local")    (ul ,@(map wui-valid-link local-links))))
  ,@(if query-links `((h3 "Queries")  (ul ,@(map wui-item-link query-links))))
  ,@(if ext-links   `((h3 "External") (ul ,@(map wui-item-popup-link ext-links))))
  (h3 "Unlinked")
  ,@unlinked-content
  (form
    (label (@ (for "phrase")) "Search Phrase")
    (input (@ (class "zs-input") (type "text") (id "phrase") (name ,query-key-phrase) (placeholder "Phrase..") (value ,phrase)))
>
>
>
>
>
>
>
>
>
>
>
>
>





>

|

<




|

|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

23
24
25
26
27
28
29
30
31
32
33
34
35
36
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(article
  (header (h1 "Information for Zettel " ,zid)
    (p
      (a (@ (href ,web-url)) "Web")
      (@H " &#183; ") (a (@ (href ,context-url)) "Context")
      (@H " / ") (a (@ (href ,context-full-url)) "Full")
      ,@(if (bound? 'edit-url) `((@H " &#183; ") (a (@ (href ,edit-url)) "Edit")))
      ,@(ROLE-DEFAULT-actions (current-binding))
      ,@(if (bound? 'reindex-url) `((@H " &#183; ") (a (@ (href ,reindex-url)) "Reindex")))

      ,@(if (bound? 'delete-url) `((@H " &#183; ") (a (@ (href ,delete-url)) "Delete")))
    )
  )
  (h2 "Interpreted Metadata")
  (table ,@(map wui-info-meta-table-row metadata))
  (h2 "References")
  ,@(if local-links `((h3 "Local")    (ul ,@(map wui-local-link local-links)))) 
  ,@(if query-links `((h3 "Queries")  (ul ,@(map wui-item-link query-links))))
  ,@(if ext-links   `((h3 "External") (ul ,@(map wui-item-popup-link ext-links))))
  (h3 "Unlinked")
  ,@unlinked-content
  (form
    (label (@ (for "phrase")) "Search Phrase")
    (input (@ (class "zs-input") (type "text") (id "phrase") (name ,query-key-phrase) (placeholder "Phrase..") (value ,phrase)))

Changes to box/constbox/listzettel.sxn.














1
2
3


4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

20

21
22
23


24
25
26
27
28
29
30
31













`(article
  (header (h1 ,heading))
  (form (@ (action ,search-url))


    (input (@ (class "zs-input") (type "text") (placeholder "Search..") (name ,query-key-query) (value ,query-value) (dir "auto"))))
  ,@(if (bound? 'tag-zettel)
     `((p (@ (class "zs-meta-zettel")) "Tag zettel: " ,@tag-zettel))
    )
  ,@(if (bound? 'create-tag-zettel)
     `((p (@ (class "zs-meta-zettel")) "Create tag zettel: " ,@create-tag-zettel))
    )
  ,@(if (bound? 'role-zettel)
     `((p (@ (class "zs-meta-zettel")) "Role zettel: " ,@role-zettel))
    )
  ,@(if (bound? 'create-role-zettel)
     `((p (@ (class "zs-meta-zettel")) "Create role zettel: " ,@create-role-zettel))
    )
  ,@content
  ,@endnotes
  (form (@ (action ,(if (bound? 'create-url) create-url)))

      "Other encodings: "

      (a (@ (href ,data-url)) "data")
      ", "
      (a (@ (href ,plain-url)) "plain")


      ,@(if (bound? 'create-url)
        `((input (@ (type "hidden") (name ,query-key-query) (value ,query-value)))
          (input (@ (type "hidden") (name ,query-key-seed) (value ,seed)))
          (input (@ (class "zs-primary") (type "submit") (value "Save As Zettel")))
        )
      )
  )
)
>
>
>
>
>
>
>
>
>
>
>
>
>


|
>
>
|















>
|
>
|
|
|
>
>








1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(article
  (header (h1 ,heading))
  (search (form (@ (action ,search-url))
    (input (@ (class "zs-input") (type "search") (inputmode "search") (name ,query-key-query)
              (title "Contains the search that leads to the list below. You're allowed to modify it")
              (placeholder "Search..") (value ,query-value) (dir "auto")))))
  ,@(if (bound? 'tag-zettel)
     `((p (@ (class "zs-meta-zettel")) "Tag zettel: " ,@tag-zettel))
    )
  ,@(if (bound? 'create-tag-zettel)
     `((p (@ (class "zs-meta-zettel")) "Create tag zettel: " ,@create-tag-zettel))
    )
  ,@(if (bound? 'role-zettel)
     `((p (@ (class "zs-meta-zettel")) "Role zettel: " ,@role-zettel))
    )
  ,@(if (bound? 'create-role-zettel)
     `((p (@ (class "zs-meta-zettel")) "Create role zettel: " ,@create-role-zettel))
    )
  ,@content
  ,@endnotes
  (form (@ (action ,(if (bound? 'create-url) create-url)))
      ,(if (bound? 'data-url)
          `(@L "Other encodings"
               ,(if (> num-entries 3) `(@L " of these " ,num-entries " entries: ") ": ")
               (a (@ (href ,data-url)) "data")
               ", "
               (a (@ (href ,plain-url)) "plain")
           )
      )
      ,@(if (bound? 'create-url)
        `((input (@ (type "hidden") (name ,query-key-query) (value ,query-value)))
          (input (@ (type "hidden") (name ,query-key-seed) (value ,seed)))
          (input (@ (class "zs-primary") (type "submit") (value "Save As Zettel")))
        )
      )
  )
)

Changes to box/constbox/login.sxn.














1
2
3
4
5
6
7













`(article
  (header (h1 "Login"))
  ,@(if retry '((div (@ (class "zs-indication zs-error")) "Wrong user name / password. Try again.")))
  (form (@ (method "POST") (action ""))
    (div
      (label (@ (for "username")) "User name:")
      (input (@ (class "zs-input") (type "text") (id "username") (name "username") (placeholder "Your user name..") (autofocus))))
>
>
>
>
>
>
>
>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(article
  (header (h1 "Login"))
  ,@(if retry '((div (@ (class "zs-indication zs-error")) "Wrong user name / password. Try again.")))
  (form (@ (method "POST") (action ""))
    (div
      (label (@ (for "username")) "User name:")
      (input (@ (class "zs-input") (type "text") (id "username") (name "username") (placeholder "Your user name..") (autofocus))))

Changes to box/constbox/prelude.sxn.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44













45
46
47
48
49
50
51
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.



;;;----------------------------------------------------------------------------

;;; This zettel contains all sxn definitions that are independent of specific
;;; subsystems, such as WebUI, API, or other. It just contains generic code to
;;; be used elsewhere.

;; Constants NIL and T
(defconst NIL ())
(defconst T   'T)

;; Function not
(defun not (x) (if x NIL T))
(defconst not not)

;; defunconst macro to define functions that are bound as a constant.
;;
;; (defunconst NAME ARGS EXPR ...)
(defmacro defunconst (name args . body)
    `(begin (defun ,name ,args ,@body) (defconst ,name ,name)))

;; let macro
;;
;; (let (BINDING ...) EXPR ...), where BINDING is a list of two elements
;;   (SYMBOL EXPR)
(defmacro let (bindings . body)
    `((lambda ,(map car bindings) ,@body) ,@(map cadr bindings)))

;; let* macro
;;
;; (let* (BINDING ...) EXPR ...), where SYMBOL may occur in later bindings.
(defmacro let* (bindings . body)
    (if (null? bindings)
        `((lambda () ,@body))
        `((lambda (,(caar bindings))
                  (let* ,(cdr bindings) ,@body))
                  ,(cadar bindings))))














;; and macro
;;
;; (and EXPR ...)
(defmacro and args
    (cond ((null? args)       T)
          ((null? (cdr args)) (car args))








>
>
>


|

|

<
<
<
|
<
|
<

<
<
<
<
<
|
<
<
<
<
|
<






|
|
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17



18

19

20





21




22

23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

;;; This zettel contains sxn definitions that are independent of specific
;;; subsystems, such as WebUI, API, or other. It just contains generic code to
;;; be used in all places. It asumes that the symbols NIL and T are defined.




;; not macro

(defmacro not (x) `(if ,x NIL T))







;; not= macro, to negate an equivalence




(defmacro not= args `(not (= ,@args)))


;; let* macro
;;
;; (let* (BINDING ...) EXPR ...), where SYMBOL may occur in later bindings.
(defmacro let* (bindings . body)
    (if (null? bindings)
        `(begin ,@body)
        `(let ((,(caar bindings) ,(cadar bindings)))
               (let* ,(cdr bindings) ,@body))))

;; cond macro
;;
;; (cond ((COND EXPR) ...))
(defmacro cond clauses
    (if (null? clauses)
        ()
        (let* ((clause (car clauses))
               (the-cond (car clause)))
              (if (= the-cond T)
                  `(begin ,@(cdr clause))
                  `(if ,the-cond
                       (begin ,@(cdr clause))
                       (cond ,@(cdr clauses)))))))

;; and macro
;;
;; (and EXPR ...)
(defmacro and args
    (cond ((null? args)       T)
          ((null? (cdr args)) (car args))

Deleted box/constbox/rename.sxn.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
`(article
  (header (h1 "Rename Zettel " ,zid))
  (p "Do you really want to rename this zettel?")
  ,@(if incoming
    `((div (@ (class "zs-warning"))
      (h2 "Warning!")
      (p "If you rename this zettel, incoming references from the following zettel will become invalid.")
      (ul ,@(map wui-item-link incoming))
    ))
  )
  ,@(if (and (bound? 'useless) useless)
    `((div (@ (class "zs-warning"))
      (h2 "Warning!")
      (p "Renaming this zettel will also delete the following files, so that they will not be interpreted as content for this zettel.")
      (ul ,@(map wui-item useless))
    ))
  )
  (form (@ (method "POST"))
    (input (@ (type "hidden") (id "curzid") (name "curzid") (value ,zid)))
    (div
      (label (@ (for "newzid")) "New zettel id")
      (input (@ (class "zs-input") (type "text") (id "newzid") (name "newzid") (placeholder "ZID..") (value ,zid) (autofocus))))
    (div (input (@ (class "zs-primary") (type "submit") (value "Rename"))))
  )
  ,(wui-meta-desc metapairs)
)
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<




















































Changes to box/constbox/start.sxn.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.



;;;----------------------------------------------------------------------------

;;; This zettel is the start of the loading sequence for Sx code used in the
;;; Zettelstore. Via the precursor metadata, dependend zettel are evaluated
;;; before this zettel. You must always depend, directly or indirectly on the
;;; "Zettelstore Sxn Base Code" zettel. It provides the base definitions.








>
>
>






1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

;;; This zettel is the start of the loading sequence for Sx code used in the
;;; Zettelstore. Via the precursor metadata, dependend zettel are evaluated
;;; before this zettel. You must always depend, directly or indirectly on the
;;; "Zettelstore Sxn Base Code" zettel. It provides the base definitions.

Changes to box/constbox/wuicode.sxn.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109











110
111
112
113
114
115
116
117
118
119
120





121
122
123
124
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.



;;;----------------------------------------------------------------------------

;; Contains WebUI specific code, but not related to a specific template.

;; wui-list-item returns the argument as a HTML list item.
(defunconst wui-item (s) `(li ,s))

;; wui-table-row takes a pair and translates it into a HTML table row with
;; two columns.
(defunconst wui-table-row (p)
    `(tr (td ,(car p)) (td ,(cdr p))))

;; wui-valid-link translates a local link into a HTML link. A link is a pair
;; (valid . url). If valid is not truish, only the invalid url is returned.
(defunconst wui-valid-link (l)
    (if (car l)
        `(li (a (@ (href ,(cdr l))) ,(cdr l)))
        `(li ,(cdr l))))

;; wui-link takes a link (title . url) and returns a HTML reference.
(defunconst wui-link (q)
    `(a (@ (href ,(cdr q))) ,(car q)))

;; wui-item-link taks a pair (text . url) and returns a HTML link inside
;; a list item.
(defunconst wui-item-link (q) `(li ,(wui-link q)))

;; wui-tdata-link taks a pair (text . url) and returns a HTML link inside
;; a table data item.
(defunconst wui-tdata-link (q) `(td ,(wui-link q)))

;; wui-item-popup-link is like 'wui-item-link, but the HTML link will open
;; a new tab / window.
(defunconst wui-item-popup-link (e)
    `(li (a (@ (href ,e) (target "_blank") (rel "noopener noreferrer")) ,e)))

;; wui-option-value returns a value for an HTML option element.
(defunconst wui-option-value (v) `(option (@ (value ,v))))

;; wui-datalist returns a HTML datalist with the given HTML identifier and a
;; list of values.
(defunconst wui-datalist (id lst)
    (if lst
        `((datalist (@ (id ,id)) ,@(map wui-option-value lst)))))

;; wui-pair-desc-item takes a pair '(term . text) and returns a list with
;; a HTML description term and a HTML description data. 
(defunconst wui-pair-desc-item (p) `((dt ,(car p)) (dd ,(cdr p))))

;; wui-meta-desc returns a HTML description list made from the list of pairs
;; given.
(defunconst wui-meta-desc (l)
    `(dl ,@(apply append (map wui-pair-desc-item l))))

;; wui-enc-matrix returns the HTML table of all encodings and parts.
(defunconst wui-enc-matrix (matrix)
    `(table
      ,@(map
         (lambda (row) `(tr (th ,(car row)) ,@(map wui-tdata-link (cdr row))))
         matrix)))

;; CSS-ROLE-map is a mapping (pair list, assoc list) of role names to zettel
;; identifier. It is used in the base template to update the metadata of the
;; HTML page to include some role specific CSS code.
;; Referenced in function "ROLE-DEFAULT-meta".
(defvar CSS-ROLE-map '())

;; ROLE-DEFAULT-meta returns some metadata for the base template. Any role
;; specific code should include the returned list of this function.
(defunconst ROLE-DEFAULT-meta (env)
    `(,@(let* ((meta-role (environment-lookup 'meta-role env))
               (entry (assoc CSS-ROLE-map meta-role)))
              (if (pair? entry)
                  `((link (@ (rel "stylesheet") (href ,(zid-content-path (cdr entry))))))
              )
        )
    )
)

;;; ACTION-SEPARATOR defines a HTML value that separates actions links.
(defvar ACTION-SEPARATOR '(@H " &#183; "))

;;; ROLE-DEFAULT-actions returns the default text for actions.
(defunconst ROLE-DEFAULT-actions (env)
    `(,@(let ((copy-url (environment-lookup 'copy-url env)))
             (if (defined? copy-url) `((@H " &#183; ") (a (@ (href ,copy-url)) "Copy"))))
      ,@(let ((version-url (environment-lookup 'version-url env)))
             (if (defined? version-url) `((@H " &#183; ") (a (@ (href ,version-url)) "Version"))))
      ,@(let ((child-url (environment-lookup 'child-url env)))
             (if (defined? child-url) `((@H " &#183; ") (a (@ (href ,child-url)) "Child"))))
      ,@(let ((folge-url (environment-lookup 'folge-url env)))
             (if (defined? folge-url) `((@H " &#183; ") (a (@ (href ,folge-url)) "Folge"))))
    )
)

;;; ROLE-tag-actions returns an additional action "Zettel" for zettel with role "tag".
(defunconst ROLE-tag-actions (env)
    `(,@(ROLE-DEFAULT-actions env)
      ,@(let ((title (environment-lookup 'title env)))
             (if (and (defined? title) title)
                 `(,ACTION-SEPARATOR (a (@ (href ,(query->url (string-append "tags:" title)))) "Zettel"))











             )
        )
    )
)

;;; ROLE-DEFAULT-heading returns the default text for headings, below the
;;; references of a zettel. In most cases it should be called from an
;;; overwriting function.
(defunconst ROLE-DEFAULT-heading (env)
    `(,@(let ((meta-url (environment-lookup 'meta-url env)))
           (if (defined? meta-url) `((br) "URL: " ,(url-to-html meta-url))))





      ,@(let ((meta-author (environment-lookup 'meta-author env)))
           (if (and (defined? meta-author) meta-author) `((br) "By " ,meta-author)))
    )
)








>
>
>





|

|
|
|
|

|
<
<
<
|
<


<
|



|



|



|



|



|





|



|



|













|
|








|


|
|
|

|

|

|




|
|
|
|

|
>
>
>
>
>
>
>
>
>
>
>





|
|
|
|
|

>
>
>
>
>
|



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24



25

26
27

28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

;; Contains WebUI specific code, but not related to a specific template.

;; wui-list-item returns the argument as a HTML list item.
(defun wui-item (s) `(li ,s))

;; wui-info-meta-table-row takes a pair and translates it into a HTML table row
;; with two columns.
(defun wui-info-meta-table-row (p)
    `(tr (td (@ (class zs-info-meta-key)) ,(car p)) (td (@ (class zs-info-meta-value)) ,(cdr p))))

;; wui-local-link translates a local link into HTML.



(defun wui-local-link (l) `(li (a (@ (href ,l )) ,l)))


;; wui-link takes a link (title . url) and returns a HTML reference.

(defun wui-link (q) `(a (@ (href ,(cdr q))) ,(car q)))

;; wui-item-link taks a pair (text . url) and returns a HTML link inside
;; a list item.
(defun wui-item-link (q) `(li ,(wui-link q)))

;; wui-tdata-link taks a pair (text . url) and returns a HTML link inside
;; a table data item.
(defun wui-tdata-link (q) `(td ,(wui-link q)))

;; wui-item-popup-link is like 'wui-item-link, but the HTML link will open
;; a new tab / window.
(defun wui-item-popup-link (e)
    `(li (a (@ (href ,e) (target "_blank") (rel "noopener noreferrer")) ,e)))

;; wui-option-value returns a value for an HTML option element.
(defun wui-option-value (v) `(option (@ (value ,v))))

;; wui-datalist returns a HTML datalist with the given HTML identifier and a
;; list of values.
(defun wui-datalist (id lst)
    (if lst
        `((datalist (@ (id ,id)) ,@(map wui-option-value lst)))))

;; wui-pair-desc-item takes a pair '(term . text) and returns a list with
;; a HTML description term and a HTML description data. 
(defun wui-pair-desc-item (p) `((dt ,(car p)) (dd ,(cdr p))))

;; wui-meta-desc returns a HTML description list made from the list of pairs
;; given.
(defun wui-meta-desc (l)
    `(dl ,@(apply append (map wui-pair-desc-item l))))

;; wui-enc-matrix returns the HTML table of all encodings and parts.
(defun wui-enc-matrix (matrix)
    `(table
      ,@(map
         (lambda (row) `(tr (th ,(car row)) ,@(map wui-tdata-link (cdr row))))
         matrix)))

;; CSS-ROLE-map is a mapping (pair list, assoc list) of role names to zettel
;; identifier. It is used in the base template to update the metadata of the
;; HTML page to include some role specific CSS code.
;; Referenced in function "ROLE-DEFAULT-meta".
(defvar CSS-ROLE-map '())

;; ROLE-DEFAULT-meta returns some metadata for the base template. Any role
;; specific code should include the returned list of this function.
(defun ROLE-DEFAULT-meta (binding)
    `(,@(let* ((meta-role (binding-lookup 'meta-role binding))
               (entry (assoc CSS-ROLE-map meta-role)))
              (if (pair? entry)
                  `((link (@ (rel "stylesheet") (href ,(zid-content-path (cdr entry))))))
              )
        )
    )
)

;; ACTION-SEPARATOR defines a HTML value that separates actions links.
(defvar ACTION-SEPARATOR '(@H " &#183; "))

;; ROLE-DEFAULT-actions returns the default text for actions.
(defun ROLE-DEFAULT-actions (binding)
    `(,@(let ((copy-url (binding-lookup 'copy-url binding)))
             (if (defined? copy-url) `((@H " &#183; ") (a (@ (href ,copy-url)) "Copy"))))
      ,@(let ((version-url (binding-lookup 'version-url binding)))
             (if (defined? version-url) `((@H " &#183; ") (a (@ (href ,version-url)) "Version"))))
      ,@(let ((child-url (binding-lookup 'child-url binding)))
             (if (defined? child-url) `((@H " &#183; ") (a (@ (href ,child-url)) "Child"))))
      ,@(let ((folge-url (binding-lookup 'folge-url binding)))
             (if (defined? folge-url) `((@H " &#183; ") (a (@ (href ,folge-url)) "Folge"))))
    )
)

;; ROLE-tag-actions returns an additional action "Zettel" for zettel with role "tag".
(defun ROLE-tag-actions (binding)
    `(,@(ROLE-DEFAULT-actions binding)
      ,@(let ((title (binding-lookup 'title binding)))
             (if (and (defined? title) title)
                 `(,ACTION-SEPARATOR (a (@ (href ,(query->url (concat "tags:" title)))) "Zettel"))
             )
        )
    )
)

;; ROLE-role-actions returns an additional action "Zettel" for zettel with role "role".
(defun ROLE-role-actions (binding)
    `(,@(ROLE-DEFAULT-actions binding)
      ,@(let ((title (binding-lookup 'title binding)))
             (if (and (defined? title) title)
                 `(,ACTION-SEPARATOR (a (@ (href ,(query->url (concat "role:" title)))) "Zettel"))
             )
        )
    )
)

;; ROLE-DEFAULT-heading returns the default text for headings, below the
;; references of a zettel. In most cases it should be called from an
;; overwriting function.
(defun ROLE-DEFAULT-heading (binding)
    `(,@(let ((meta-url (binding-lookup 'meta-url binding)))
           (if (defined? meta-url) `((br) "URL: " ,(url-to-html meta-url))))
      ,@(let ((urls (binding-lookup 'urls binding)))
           (if (defined? urls)
               (map (lambda (u) `(@L (br) ,(car u) ": " ,(url-to-html (cdr u)))) urls)
           )
        )
      ,@(let ((meta-author (binding-lookup 'meta-author binding)))
           (if (and (defined? meta-author) meta-author) `((br) "By " ,meta-author)))
    )
)

Changes to box/constbox/zettel.sxn.














1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24













`(article
  (header
    (h1 ,heading)
    (div (@ (class "zs-meta"))
      ,@(if (bound? 'edit-url) `((a (@ (href ,edit-url)) "Edit") (@H " &#183; ")))
      ,zid (@H " &#183; ")
      (a (@ (href ,info-url)) "Info") (@H " &#183; ")
      "(" ,@(if (bound? 'role-url) `((a (@ (href ,role-url)) ,meta-role)))
          ,@(if (and (bound? 'folge-role-url) (bound? 'meta-folge-role))
                `((@H " &rarr; ") (a (@ (href ,folge-role-url)) ,meta-folge-role)))
      ")"
      ,@(if tag-refs `((@H " &#183; ") ,@tag-refs))
      ,@(ROLE-DEFAULT-actions (current-environment))
      ,@(if predecessor-refs `((br) "Predecessor: " ,predecessor-refs))
      ,@(if precursor-refs `((br) "Precursor: " ,precursor-refs))
      ,@(if superior-refs `((br) "Superior: " ,superior-refs))
      ,@(ROLE-DEFAULT-heading (current-environment))
    )
  )
  ,@content
  ,endnotes
  ,@(if (or folge-links subordinate-links back-links successor-links)
    `((nav
      ,@(if folge-links `((details (@ (,folge-open)) (summary "Folgezettel") (ul ,@(map wui-item-link folge-links)))))
>
>
>
>
>
>
>
>
>
>
>
>
>












|



|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
;;;----------------------------------------------------------------------------
;;; Copyright (c) 2023-present Detlef Stern
;;;
;;; This file is part of Zettelstore.
;;;
;;; Zettelstore is licensed under the latest version of the EUPL (European
;;; Union Public License). Please see file LICENSE.txt for your rights and
;;; obligations under this license.
;;;
;;; SPDX-License-Identifier: EUPL-1.2
;;; SPDX-FileCopyrightText: 2023-present Detlef Stern
;;;----------------------------------------------------------------------------

`(article
  (header
    (h1 ,heading)
    (div (@ (class "zs-meta"))
      ,@(if (bound? 'edit-url) `((a (@ (href ,edit-url)) "Edit") (@H " &#183; ")))
      ,zid (@H " &#183; ")
      (a (@ (href ,info-url)) "Info") (@H " &#183; ")
      "(" ,@(if (bound? 'role-url) `((a (@ (href ,role-url)) ,meta-role)))
          ,@(if (and (bound? 'folge-role-url) (bound? 'meta-folge-role))
                `((@H " &rarr; ") (a (@ (href ,folge-role-url)) ,meta-folge-role)))
      ")"
      ,@(if tag-refs `((@H " &#183; ") ,@tag-refs))
      ,@(ROLE-DEFAULT-actions (current-binding))
      ,@(if predecessor-refs `((br) "Predecessor: " ,predecessor-refs))
      ,@(if precursor-refs `((br) "Precursor: " ,precursor-refs))
      ,@(if superior-refs `((br) "Superior: " ,superior-refs))
      ,@(ROLE-DEFAULT-heading (current-binding))
    )
  )
  ,@content
  ,endnotes
  ,@(if (or folge-links subordinate-links back-links successor-links)
    `((nav
      ,@(if folge-links `((details (@ (,folge-open)) (summary "Folgezettel") (ul ,@(map wui-item-link folge-links)))))

Changes to box/dirbox/dirbox.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package dirbox provides a directory-based zettel box.
package dirbox

import (
	"context"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package dirbox provides a directory-based zettel box.
package dirbox

import (
	"context"
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
	_ notifyTypeSpec = iota
	dirNotifyAny
	dirNotifySimple
	dirNotifyFS
)

func getDirSrvInfo(log *logger.Logger, notifyType string) notifyTypeSpec {
	for count := 0; count < 2; count++ {
		switch notifyType {
		case kernel.BoxDirTypeNotify:
			return dirNotifyFS
		case kernel.BoxDirTypeSimple:
			return dirNotifySimple
		default:
			notifyType = kernel.Main.GetConfig(kernel.BoxService, kernel.BoxDefaultDirType).(string)







|







88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
	_ notifyTypeSpec = iota
	dirNotifyAny
	dirNotifySimple
	dirNotifyFS
)

func getDirSrvInfo(log *logger.Logger, notifyType string) notifyTypeSpec {
	for range 2 {
		switch notifyType {
		case kernel.BoxDirTypeNotify:
			return dirNotifyFS
		case kernel.BoxDirTypeSimple:
			return dirNotifySimple
		default:
			notifyType = kernel.Main.GetConfig(kernel.BoxService, kernel.BoxDefaultDirType).(string)
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
	return box.StartStateStopped
}

func (dp *dirBox) Start(context.Context) error {
	dp.mxCmds.Lock()
	defer dp.mxCmds.Unlock()
	dp.fCmds = make([]chan fileCmd, 0, dp.fSrvs)
	for i := uint32(0); i < dp.fSrvs; i++ {
		cc := make(chan fileCmd)
		go fileService(i, dp.log.Clone().Str("sub", "file").Uint("fn", uint64(i)).Child(), dp.dir, cc)
		dp.fCmds = append(dp.fCmds, cc)
	}

	var notifier notify.Notifier
	var err error
	switch dp.notifySpec {
	case dirNotifySimple:
		notifier, err = notify.NewSimpleDirNotifier(dp.log.Clone().Str("notify", "simple").Child(), dp.dir)
	default:
		notifier, err = notify.NewFSDirNotifier(dp.log.Clone().Str("notify", "fs").Child(), dp.dir)
	}
	if err != nil {
		dp.log.Fatal().Err(err).Msg("Unable to create directory supervisor")
		dp.stopFileServices()
		return err
	}
	dp.dirSrv = notify.NewDirService(
		dp,
		dp.log.Clone().Str("sub", "dirsrv").Child(),
		notifier,







|














|







150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
	return box.StartStateStopped
}

func (dp *dirBox) Start(context.Context) error {
	dp.mxCmds.Lock()
	defer dp.mxCmds.Unlock()
	dp.fCmds = make([]chan fileCmd, 0, dp.fSrvs)
	for i := range dp.fSrvs {
		cc := make(chan fileCmd)
		go fileService(i, dp.log.Clone().Str("sub", "file").Uint("fn", uint64(i)).Child(), dp.dir, cc)
		dp.fCmds = append(dp.fCmds, cc)
	}

	var notifier notify.Notifier
	var err error
	switch dp.notifySpec {
	case dirNotifySimple:
		notifier, err = notify.NewSimpleDirNotifier(dp.log.Clone().Str("notify", "simple").Child(), dp.dir)
	default:
		notifier, err = notify.NewFSDirNotifier(dp.log.Clone().Str("notify", "fs").Child(), dp.dir)
	}
	if err != nil {
		dp.log.Error().Err(err).Msg("Unable to create directory supervisor")
		dp.stopFileServices()
		return err
	}
	dp.dirSrv = notify.NewDirService(
		dp,
		dp.log.Clone().Str("sub", "dirsrv").Child(),
		notifier,
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
	return err
}

func (dp *dirBox) updateEntryFromMetaContent(entry *notify.DirEntry, m *meta.Meta, content zettel.Content) {
	entry.SetupFromMetaContent(m, content, dp.cdata.Config.GetZettelFileSyntax)
}

func (dp *dirBox) AllowRenameZettel(context.Context, id.Zid) bool {
	return !dp.readonly
}

func (dp *dirBox) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	if curZid == newZid {
		return nil
	}
	curEntry := dp.dirSrv.GetDirEntry(curZid)
	if !curEntry.IsValid() {
		return box.ErrZettelNotFound{Zid: curZid}
	}
	if dp.readonly {
		return box.ErrReadOnly
	}

	// Check whether zettel with new ID already exists in this box.
	if dp.HasZettel(ctx, newZid) {
		return box.ErrInvalidZid{Zid: newZid.String()}
	}

	oldMeta, oldContent, err := dp.srvGetMetaContent(ctx, curEntry, curZid)
	if err != nil {
		return err
	}

	newEntry, err := dp.dirSrv.RenameDirEntry(curEntry, newZid)
	if err != nil {
		return err
	}
	oldMeta.Zid = newZid
	newZettel := zettel.Zettel{Meta: oldMeta, Content: zettel.NewContent(oldContent)}
	if err = dp.srvSetZettel(ctx, &newEntry, newZettel); err != nil {
		// "Rollback" rename. No error checking...
		dp.dirSrv.RenameDirEntry(&newEntry, curZid)
		return err
	}
	err = dp.srvDeleteZettel(ctx, curEntry, curZid)
	if err == nil {
		dp.notifyChanged(curZid)
		dp.notifyChanged(newZid)
	}
	dp.log.Trace().Zid(curZid).Zid(newZid).Err(err).Msg("RenameZettel")
	return err
}

func (dp *dirBox) CanDeleteZettel(_ context.Context, zid id.Zid) bool {
	if dp.readonly {
		return false
	}
	entry := dp.dirSrv.GetDirEntry(zid)
	return entry.IsValid()
}







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







322
323
324
325
326
327
328














































329
330
331
332
333
334
335
	return err
}

func (dp *dirBox) updateEntryFromMetaContent(entry *notify.DirEntry, m *meta.Meta, content zettel.Content) {
	entry.SetupFromMetaContent(m, content, dp.cdata.Config.GetZettelFileSyntax)
}















































func (dp *dirBox) CanDeleteZettel(_ context.Context, zid id.Zid) bool {
	if dp.readonly {
		return false
	}
	entry := dp.dirSrv.GetDirEntry(zid)
	return entry.IsValid()
}

Changes to box/dirbox/dirbox_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package dirbox

import "testing"

func TestIsPrime(t *testing.T) {








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package dirbox

import "testing"

func TestIsPrime(t *testing.T) {
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
		if got != tc.exp {
			t.Errorf("isPrime(%d)=%v, but got %v", tc.n, tc.exp, got)
		}
	}
}

func TestMakePrime(t *testing.T) {
	for i := uint32(0); i < 1500; i++ {
		np := makePrime(i)
		if np < i {
			t.Errorf("makePrime(%d) < %d", i, np)
			continue
		}
		if !isPrime(np) {
			t.Errorf("makePrime(%d) == %d is not prime", i, np)







|







31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
		if got != tc.exp {
			t.Errorf("isPrime(%d)=%v, but got %v", tc.n, tc.exp, got)
		}
	}
}

func TestMakePrime(t *testing.T) {
	for i := range uint32(1500) {
		np := makePrime(i)
		if np < i {
			t.Errorf("makePrime(%d) < %d", i, np)
			continue
		}
		if !isPrime(np) {
			t.Errorf("makePrime(%d) == %d is not prime", i, np)

Changes to box/dirbox/service.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14

15
16
17
18
19

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package dirbox

import (
	"context"

	"io"
	"os"
	"path/filepath"
	"time"


	"zettelstore.de/z/box/filebox"
	"zettelstore.de/z/box/notify"
	"zettelstore.de/z/input"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func fileService(i uint32, log *logger.Logger, dirPath string, cmds <-chan fileCmd) {
	// Something may panic. Ensure a running service.
	defer func() {
		if ri := recover(); ri != nil {
			kernel.Main.LogRecover("FileService", ri)
			go fileService(i, log, dirPath, cmds)
		}
	}()

	log.Trace().Uint("i", uint64(i)).Str("dirpath", dirPath).Msg("File service started")
	for cmd := range cmds {
		cmd.run(log, dirPath)
	}
	log.Trace().Uint("i", uint64(i)).Str("dirpath", dirPath).Msg("File service stopped")
}

type fileCmd interface {
	run(*logger.Logger, string)
}

const serviceTimeout = 5 * time.Second // must be shorter than the web servers timeout values for reading+writing.

// COMMAND: srvGetMeta ----------------------------------------
//
// Retrieves the meta data from a zettel.








>
>
>






>





>


<
















|

|

|



|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package dirbox

import (
	"context"
	"fmt"
	"io"
	"os"
	"path/filepath"
	"time"

	"t73f.de/r/zsc/input"
	"zettelstore.de/z/box/filebox"
	"zettelstore.de/z/box/notify"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

func fileService(i uint32, log *logger.Logger, dirPath string, cmds <-chan fileCmd) {
	// Something may panic. Ensure a running service.
	defer func() {
		if ri := recover(); ri != nil {
			kernel.Main.LogRecover("FileService", ri)
			go fileService(i, log, dirPath, cmds)
		}
	}()

	log.Debug().Uint("i", uint64(i)).Str("dirpath", dirPath).Msg("File service started")
	for cmd := range cmds {
		cmd.run(dirPath)
	}
	log.Debug().Uint("i", uint64(i)).Str("dirpath", dirPath).Msg("File service stopped")
}

type fileCmd interface {
	run(string)
}

const serviceTimeout = 5 * time.Second // must be shorter than the web servers timeout values for reading+writing.

// COMMAND: srvGetMeta ----------------------------------------
//
// Retrieves the meta data from a zettel.
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
	rc    chan<- resGetMeta
}
type resGetMeta struct {
	meta *meta.Meta
	err  error
}

func (cmd *fileGetMeta) run(log *logger.Logger, dirPath string) {
	var m *meta.Meta
	var err error

	entry := cmd.entry
	zid := entry.Zid
	if metaName := entry.MetaName; metaName == "" {
		contentName := entry.ContentName
		contentExt := entry.ContentExt
		if contentName == "" || contentExt == "" {
			log.Panic().Zid(zid).Msg("No meta, no content in getMeta")
		}
		if entry.HasMetaInContent() {
			m, _, err = parseMetaContentFile(zid, filepath.Join(dirPath, contentName))
		} else {
			m = filebox.CalcDefaultMeta(zid, contentExt)
		}
	} else {
		m, err = parseMetaFile(zid, filepath.Join(dirPath, metaName))
	}







|









|
<
|







75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

93
94
95
96
97
98
99
100
	rc    chan<- resGetMeta
}
type resGetMeta struct {
	meta *meta.Meta
	err  error
}

func (cmd *fileGetMeta) run(dirPath string) {
	var m *meta.Meta
	var err error

	entry := cmd.entry
	zid := entry.Zid
	if metaName := entry.MetaName; metaName == "" {
		contentName := entry.ContentName
		contentExt := entry.ContentExt
		if contentName == "" || contentExt == "" {
			err = fmt.Errorf("no meta, no content in getMeta, zid=%v", zid)

		} else if entry.HasMetaInContent() {
			m, _, err = parseMetaContentFile(zid, filepath.Join(dirPath, contentName))
		} else {
			m = filebox.CalcDefaultMeta(zid, contentExt)
		}
	} else {
		m, err = parseMetaFile(zid, filepath.Join(dirPath, metaName))
	}
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
}
type resGetMetaContent struct {
	meta    *meta.Meta
	content []byte
	err     error
}

func (cmd *fileGetMetaContent) run(log *logger.Logger, dirPath string) {
	var m *meta.Meta
	var content []byte
	var err error

	entry := cmd.entry
	zid := entry.Zid
	contentName := entry.ContentName
	contentExt := entry.ContentExt
	contentPath := filepath.Join(dirPath, contentName)
	if metaName := entry.MetaName; metaName == "" {
		if contentName == "" || contentExt == "" {
			log.Panic().Zid(zid).Msg("No meta, no content in getMetaContent")
		}
		if entry.HasMetaInContent() {
			m, content, err = parseMetaContentFile(zid, contentPath)
		} else {
			m = filebox.CalcDefaultMeta(zid, contentExt)
			content, err = os.ReadFile(contentPath)
		}
	} else {
		m, err = parseMetaFile(zid, filepath.Join(dirPath, metaName))







|











|
<
|







127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146

147
148
149
150
151
152
153
154
}
type resGetMetaContent struct {
	meta    *meta.Meta
	content []byte
	err     error
}

func (cmd *fileGetMetaContent) run(dirPath string) {
	var m *meta.Meta
	var content []byte
	var err error

	entry := cmd.entry
	zid := entry.Zid
	contentName := entry.ContentName
	contentExt := entry.ContentExt
	contentPath := filepath.Join(dirPath, contentName)
	if metaName := entry.MetaName; metaName == "" {
		if contentName == "" || contentExt == "" {
			err = fmt.Errorf("no meta, no content in getMetaContent, zid=%v", zid)

		} else if entry.HasMetaInContent() {
			m, content, err = parseMetaContentFile(zid, contentPath)
		} else {
			m = filebox.CalcDefaultMeta(zid, contentExt)
			content, err = os.ReadFile(contentPath)
		}
	} else {
		m, err = parseMetaFile(zid, filepath.Join(dirPath, metaName))
184
185
186
187
188
189
190
191

192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208

209
210
211
212
213
214
215
216
217
218
219
220
type fileSetZettel struct {
	entry  *notify.DirEntry
	zettel zettel.Zettel
	rc     chan<- resSetZettel
}
type resSetZettel = error

func (cmd *fileSetZettel) run(log *logger.Logger, dirPath string) {

	entry := cmd.entry
	zid := entry.Zid
	contentName := entry.ContentName
	m := cmd.zettel.Meta
	content := cmd.zettel.Content.AsBytes()
	metaName := entry.MetaName
	if metaName == "" {
		if contentName == "" {
			log.Panic().Zid(zid).Msg("No meta, no content in setZettel")
		}
		contentPath := filepath.Join(dirPath, contentName)
		if entry.HasMetaInContent() {
			err := writeZettelFile(contentPath, m, content)
			cmd.rc <- err
			return
		}
		err := writeFileContent(contentPath, content)

		cmd.rc <- err
		return
	}

	err := writeMetaFile(filepath.Join(dirPath, metaName), m)
	if err == nil && contentName != "" {
		err = writeFileContent(filepath.Join(dirPath, contentName), content)
	}
	cmd.rc <- err
}

func writeMetaFile(metaPath string, m *meta.Meta) error {







|
>








|
|
|
|
|
|
|
|
|
>




|







186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
type fileSetZettel struct {
	entry  *notify.DirEntry
	zettel zettel.Zettel
	rc     chan<- resSetZettel
}
type resSetZettel = error

func (cmd *fileSetZettel) run(dirPath string) {
	var err error
	entry := cmd.entry
	zid := entry.Zid
	contentName := entry.ContentName
	m := cmd.zettel.Meta
	content := cmd.zettel.Content.AsBytes()
	metaName := entry.MetaName
	if metaName == "" {
		if contentName == "" {
			err = fmt.Errorf("no meta, no content in setZettel, zid=%v", zid)
		} else {
			contentPath := filepath.Join(dirPath, contentName)
			if entry.HasMetaInContent() {
				err = writeZettelFile(contentPath, m, content)
				cmd.rc <- err
				return
			}
			err = writeFileContent(contentPath, content)
		}
		cmd.rc <- err
		return
	}

	err = writeMetaFile(filepath.Join(dirPath, metaName), m)
	if err == nil && contentName != "" {
		err = writeFileContent(filepath.Join(dirPath, contentName), content)
	}
	cmd.rc <- err
}

func writeMetaFile(metaPath string, m *meta.Meta) error {
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
}

func writeZettelFile(contentPath string, m *meta.Meta, content []byte) error {
	zettelFile, err := openFileWrite(contentPath)
	if err != nil {
		return err
	}
	if err == nil {
		err = writeMetaHeader(zettelFile, m)
	}
	if err == nil {
		_, err = zettelFile.Write(content)
	}
	if err1 := zettelFile.Close(); err == nil {
		err = err1
	}
	return err







<
|
<







237
238
239
240
241
242
243

244

245
246
247
248
249
250
251
}

func writeZettelFile(contentPath string, m *meta.Meta, content []byte) error {
	zettelFile, err := openFileWrite(contentPath)
	if err != nil {
		return err
	}

	err = writeMetaHeader(zettelFile, m)

	if err == nil {
		_, err = zettelFile.Write(content)
	}
	if err1 := zettelFile.Close(); err == nil {
		err = err1
	}
	return err
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313

314
315
316
317
318
319
320

type fileDeleteZettel struct {
	entry *notify.DirEntry
	rc    chan<- resDeleteZettel
}
type resDeleteZettel = error

func (cmd *fileDeleteZettel) run(log *logger.Logger, dirPath string) {
	var err error

	entry := cmd.entry
	contentName := entry.ContentName
	contentPath := filepath.Join(dirPath, contentName)
	if metaName := entry.MetaName; metaName == "" {
		if contentName == "" {
			log.Panic().Zid(entry.Zid).Msg("No meta, no content in getMetaContent")
		}
		err = os.Remove(contentPath)

	} else {
		if contentName != "" {
			err = os.Remove(contentPath)
		}
		err1 := os.Remove(filepath.Join(dirPath, metaName))
		if err == nil {
			err = err1







|







|
|
|
>







298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323

type fileDeleteZettel struct {
	entry *notify.DirEntry
	rc    chan<- resDeleteZettel
}
type resDeleteZettel = error

func (cmd *fileDeleteZettel) run(dirPath string) {
	var err error

	entry := cmd.entry
	contentName := entry.ContentName
	contentPath := filepath.Join(dirPath, contentName)
	if metaName := entry.MetaName; metaName == "" {
		if contentName == "" {
			err = fmt.Errorf("no meta, no content in deleteZettel, zid=%v", entry.Zid)
		} else {
			err = os.Remove(contentPath)
		}
	} else {
		if contentName != "" {
			err = os.Remove(contentPath)
		}
		err1 := os.Remove(filepath.Join(dirPath, metaName))
		if err == nil {
			err = err1
355
356
357
358
359
360
361
362






363
364
365
366
367
368
369
370
371
		m,
		entry.Zid,
		entry.ContentExt,
		entry.MetaName != "",
		entry.UselessFiles,
	)
}







func openFileWrite(path string) (*os.File, error) {
	return os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
}

func writeFileZid(w io.Writer, zid id.Zid) error {
	_, err := io.WriteString(w, "id: ")
	if err == nil {
		_, err = w.Write(zid.Bytes())
		if err == nil {








>
>
>
>
>
>

|







358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
		m,
		entry.Zid,
		entry.ContentExt,
		entry.MetaName != "",
		entry.UselessFiles,
	)
}

// fileMode to create a new file: user, group, and all are allowed to read and write.
//
// If you want to forbid others or the group to read or to write, you must set
// umask(1) accordingly.
const fileMode os.FileMode = 0666 //

func openFileWrite(path string) (*os.File, error) {
	return os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fileMode)
}

func writeFileZid(w io.Writer, zid id.Zid) error {
	_, err := io.WriteString(w, "id: ")
	if err == nil {
		_, err = w.Write(zid.Bytes())
		if err == nil {

Changes to box/filebox/filebox.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package filebox provides boxes that are stored in a file.
package filebox

import (
	"errors"
	"net/url"
	"path/filepath"
	"strings"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)









>
>
>











|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package filebox provides boxes that are stored in a file.
package filebox

import (
	"errors"
	"net/url"
	"path/filepath"
	"strings"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

Changes to box/filebox/zipbox.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15

16
17
18

19
20
21
22
23
24
25
26
27
28
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package filebox

import (
	"archive/zip"
	"context"

	"io"
	"strings"


	"zettelstore.de/z/box"
	"zettelstore.de/z/box/notify"
	"zettelstore.de/z/input"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)









>
>
>







>



>


<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package filebox

import (
	"archive/zip"
	"context"
	"fmt"
	"io"
	"strings"

	"t73f.de/r/zsc/input"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/notify"

	"zettelstore.de/z/logger"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

96
97
98
99
100
101
102
103

104
105
106
107
108
109
110
	var m *meta.Meta
	var src []byte
	var inMeta bool

	contentName := entry.ContentName
	if metaName := entry.MetaName; metaName == "" {
		if contentName == "" {
			zb.log.Panic().Zid(zid).Msg("No meta, no content in zipBox.GetZettel")

		}
		src, err = readZipFileContent(reader, entry.ContentName)
		if err != nil {
			return zettel.Zettel{}, err
		}
		if entry.HasMetaInContent() {
			inp := input.NewInput(src)







|
>







100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
	var m *meta.Meta
	var src []byte
	var inMeta bool

	contentName := entry.ContentName
	if metaName := entry.MetaName; metaName == "" {
		if contentName == "" {
			err = fmt.Errorf("no meta, no content in getZettel, zid=%v", zid)
			return zettel.Zettel{}, err
		}
		src, err = readZipFileContent(reader, entry.ContentName)
		if err != nil {
			return zettel.Zettel{}, err
		}
		if entry.HasMetaInContent() {
			inp := input.NewInput(src)
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
		}
		zb.enricher.Enrich(ctx, m, zb.number)
		handle(m)
	}
	return nil
}

func (zb *zipBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	entry := zb.dirSrv.GetDirEntry(zid)
	return !entry.IsValid()
}

func (zb *zipBox) RenameZettel(_ context.Context, curZid, newZid id.Zid) error {
	err := box.ErrReadOnly
	if curZid == newZid {
		err = nil
	}
	curEntry := zb.dirSrv.GetDirEntry(curZid)
	if !curEntry.IsValid() {
		err = box.ErrZettelNotFound{Zid: curZid}
	}
	zb.log.Trace().Err(err).Msg("RenameZettel")
	return err
}

func (*zipBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (zb *zipBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	err := box.ErrReadOnly
	entry := zb.dirSrv.GetDirEntry(zid)
	if !entry.IsValid() {
		err = box.ErrZettelNotFound{Zid: zid}







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







168
169
170
171
172
173
174


















175
176
177
178
179
180
181
		}
		zb.enricher.Enrich(ctx, m, zb.number)
		handle(m)
	}
	return nil
}



















func (*zipBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (zb *zipBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	err := box.ErrReadOnly
	entry := zb.dirSrv.GetDirEntry(zid)
	if !entry.IsValid() {
		err = box.ErrZettelNotFound{Zid: zid}
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221

func (zb *zipBox) readZipMeta(reader *zip.ReadCloser, zid id.Zid, entry *notify.DirEntry) (m *meta.Meta, err error) {
	var inMeta bool
	if metaName := entry.MetaName; metaName == "" {
		contentName := entry.ContentName
		contentExt := entry.ContentExt
		if contentName == "" || contentExt == "" {
			zb.log.Panic().Zid(zid).Msg("No meta, no content in getMeta")
		}
		if entry.HasMetaInContent() {
			m, err = readZipMetaFile(reader, zid, contentName)
		} else {
			m = CalcDefaultMeta(zid, contentExt)
		}
	} else {
		m, err = readZipMetaFile(reader, zid, metaName)
	}







|
<
|







192
193
194
195
196
197
198
199

200
201
202
203
204
205
206
207

func (zb *zipBox) readZipMeta(reader *zip.ReadCloser, zid id.Zid, entry *notify.DirEntry) (m *meta.Meta, err error) {
	var inMeta bool
	if metaName := entry.MetaName; metaName == "" {
		contentName := entry.ContentName
		contentExt := entry.ContentExt
		if contentName == "" || contentExt == "" {
			err = fmt.Errorf("no meta, no content in getMeta, zid=%v", zid)

		} else if entry.HasMetaInContent() {
			m, err = readZipMetaFile(reader, zid, contentName)
		} else {
			m = CalcDefaultMeta(zid, contentExt)
		}
	} else {
		m, err = readZipMetaFile(reader, zid, metaName)
	}

Changes to box/helper.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package box

import (
	"net/url"
	"strconv"
	"time"

	"zettelstore.de/z/zettel/id"
)

// GetNewZid calculates a new and unused zettel identifier, based on the current date and time.
func GetNewZid(testZid func(id.Zid) (bool, error)) (id.Zid, error) {
	withSeconds := false
	for i := 0; i < 90; i++ { // Must be completed within 9 seconds (less than web/server.writeTimeout)
		zid := id.New(withSeconds)
		found, err := testZid(zid)
		if err != nil {
			return id.Invalid, err
		}
		if found {
			return zid, nil








>
>
>















|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package box

import (
	"net/url"
	"strconv"
	"time"

	"zettelstore.de/z/zettel/id"
)

// GetNewZid calculates a new and unused zettel identifier, based on the current date and time.
func GetNewZid(testZid func(id.Zid) (bool, error)) (id.Zid, error) {
	withSeconds := false
	for range 90 { // Must be completed within 9 seconds (less than web/server.writeTimeout)
		zid := id.New(withSeconds)
		found, err := testZid(zid)
		if err != nil {
			return id.Invalid, err
		}
		if found {
			return zid, nil

Changes to box/manager/anteroom.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package manager

import (
	"sync"

	"zettelstore.de/z/zettel/id"
)

type arAction int

const (
	arNothing arAction = iota
	arReload
	arZettel
)

type anteroom struct {
	next    *anteroom
	waiting id.Set
	curLoad int
	reload  bool
}

type anteroomQueue struct {
	mx      sync.Mutex
	first   *anteroom








>
>
>




















|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package manager

import (
	"sync"

	"zettelstore.de/z/zettel/id"
)

type arAction int

const (
	arNothing arAction = iota
	arReload
	arZettel
)

type anteroom struct {
	next    *anteroom
	waiting *id.Set
	curLoad int
	reload  bool
}

type anteroomQueue struct {
	mx      sync.Mutex
	first   *anteroom
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
		ar.last = ar.first
		return
	}
	for room := ar.first; room != nil; room = room.next {
		if room.reload {
			continue // Do not put zettel in reload room
		}
		if _, ok := room.waiting[zid]; ok {
			// Zettel is already waiting. Nothing to do.
			return
		}
	}
	if room := ar.last; !room.reload && (ar.maxLoad == 0 || room.curLoad < ar.maxLoad) {
		room.waiting.Add(zid)
		room.curLoad++







|







54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
		ar.last = ar.first
		return
	}
	for room := ar.first; room != nil; room = room.next {
		if room.reload {
			continue // Do not put zettel in reload room
		}
		if room.waiting.Contains(zid) {
			// Zettel is already waiting. Nothing to do.
			return
		}
	}
	if room := ar.last; !room.reload && (ar.maxLoad == 0 || room.curLoad < ar.maxLoad) {
		room.waiting.Add(zid)
		room.curLoad++
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
func (ar *anteroomQueue) Reset() {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	ar.first = &anteroom{next: nil, waiting: nil, curLoad: 0, reload: true}
	ar.last = ar.first
}

func (ar *anteroomQueue) Reload(allZids id.Set) {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	ar.deleteReloadedRooms()

	if ns := len(allZids); ns > 0 {
		ar.first = &anteroom{next: ar.first, waiting: allZids, curLoad: ns, reload: true}
		if ar.first.next == nil {
			ar.last = ar.first
		}
	} else {
		ar.first = nil
		ar.last = nil
	}







|




|
|







84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
func (ar *anteroomQueue) Reset() {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	ar.first = &anteroom{next: nil, waiting: nil, curLoad: 0, reload: true}
	ar.last = ar.first
}

func (ar *anteroomQueue) Reload(allZids *id.Set) {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	ar.deleteReloadedRooms()

	if !allZids.IsEmpty() {
		ar.first = &anteroom{next: ar.first, waiting: allZids, curLoad: allZids.Length(), reload: true}
		if ar.first.next == nil {
			ar.last = ar.first
		}
	} else {
		ar.first = nil
		ar.last = nil
	}
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
	defer ar.mx.Unlock()
	first := ar.first
	if first != nil {
		if first.waiting == nil && first.reload {
			ar.removeFirst()
			return arReload, id.Invalid, false
		}
		for zid := range first.waiting {
			delete(first.waiting, zid)
			if len(first.waiting) == 0 {
				ar.removeFirst()
			}
			return arZettel, zid, first.reload
		}
		ar.removeFirst()
	}
	return arNothing, id.Invalid, false







|
|
<







120
121
122
123
124
125
126
127
128

129
130
131
132
133
134
135
	defer ar.mx.Unlock()
	first := ar.first
	if first != nil {
		if first.waiting == nil && first.reload {
			ar.removeFirst()
			return arReload, id.Invalid, false
		}
		if zid, found := first.waiting.Pop(); found {
			if first.waiting.IsEmpty() {

				ar.removeFirst()
			}
			return arZettel, zid, first.reload
		}
		ar.removeFirst()
	}
	return arNothing, id.Invalid, false

Changes to box/manager/anteroom_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package manager

import (
	"testing"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package manager

import (
	"testing"

Changes to box/manager/box.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"errors"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"errors"
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120




121






122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188

// Location returns some information where the box is located.
func (mgr *Manager) Location() string {
	if len(mgr.boxes) <= 2 {
		return "NONE"
	}
	var sb strings.Builder
	for i := 0; i < len(mgr.boxes)-2; i++ {
		if i > 0 {
			sb.WriteString(", ")
		}
		sb.WriteString(mgr.boxes[i].Location())
	}
	return sb.String()
}

// CanCreateZettel returns true, if box could possibly create a new zettel.
func (mgr *Manager) CanCreateZettel(ctx context.Context) bool {
	if mgr.State() != box.StartStateStarted {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		return box.CanCreateZettel(ctx)
	}
	return false
}

// CreateZettel creates a new zettel.
func (mgr *Manager) CreateZettel(ctx context.Context, zettel zettel.Zettel) (id.Zid, error) {
	mgr.mgrLog.Debug().Msg("CreateZettel")
	if mgr.State() != box.StartStateStarted {
		return id.Invalid, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		zettel.Meta = mgr.cleanMetaProperties(zettel.Meta)
		zid, err := box.CreateZettel(ctx, zettel)
		if err == nil {
			mgr.idxUpdateZettel(ctx, zettel)
		}
		return zid, err
	}
	return id.Invalid, box.ErrReadOnly
}

// GetZettel retrieves a specific zettel.
func (mgr *Manager) GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetZettel")
	if mgr.State() != box.StartStateStarted {
		return zettel.Zettel{}, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for i, p := range mgr.boxes {
		var errZNF box.ErrZettelNotFound
		if z, err := p.GetZettel(ctx, zid); !errors.As(err, &errZNF) {
			if err == nil {
				mgr.Enrich(ctx, z.Meta, i+1)
			}
			return z, err
		}
	}
	return zettel.Zettel{}, box.ErrZettelNotFound{Zid: zid}
}

// GetAllZettel retrieves a specific zettel from all managed boxes.
func (mgr *Manager) GetAllZettel(ctx context.Context, zid id.Zid) ([]zettel.Zettel, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetAllZettel")
	if mgr.State() != box.StartStateStarted {
		return nil, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	var result []zettel.Zettel
	for i, p := range mgr.boxes {
		if z, err := p.GetZettel(ctx, zid); err == nil {
			mgr.Enrich(ctx, z.Meta, i+1)
			result = append(result, z)
		}
	}
	return result, nil
}

// FetchZids returns the set of all zettel identifer managed by the box.
func (mgr *Manager) FetchZids(ctx context.Context) (id.Set, error) {
	mgr.mgrLog.Debug().Msg("FetchZids")
	if mgr.State() != box.StartStateStarted {
		return nil, box.ErrStopped
	}
	result := id.Set{}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()




	for _, p := range mgr.boxes {






		err := p.ApplyZid(ctx, func(zid id.Zid) { result.Add(zid) }, func(id.Zid) bool { return true })
		if err != nil {
			return nil, err
		}
	}
	return result, nil
}

func (mgr *Manager) HasZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mgrLog.Debug().Zid(zid).Msg("HasZettel")
	if mgr.State() != box.StartStateStarted {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, bx := range mgr.boxes {
		if bx.HasZettel(ctx, zid) {
			return true
		}
	}
	return false
}

func (mgr *Manager) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetMeta")
	if mgr.State() != box.StartStateStarted {
		return nil, box.ErrStopped
	}

	m, err := mgr.idxStore.GetMeta(ctx, zid)
	if err != nil {
		return nil, err
	}
	mgr.Enrich(ctx, m, 0)
	return m, nil
}

// SelectMeta returns all zettel meta data that match the selection
// criteria. The result is ordered by descending zettel id.
func (mgr *Manager) SelectMeta(ctx context.Context, metaSeq []*meta.Meta, q *query.Query) ([]*meta.Meta, error) {
	if msg := mgr.mgrLog.Debug(); msg.Enabled() {
		msg.Str("query", q.String()).Msg("SelectMeta")
	}
	if mgr.State() != box.StartStateStarted {
		return nil, box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()

	compSearch := q.RetrieveAndCompile(ctx, mgr, metaSeq)
	if result := compSearch.Result(); result != nil {
		mgr.mgrLog.Trace().Int("count", int64(len(result))).Msg("found without ApplyMeta")
		return result, nil
	}
	selected := map[id.Zid]*meta.Meta{}
	for _, term := range compSearch.Terms {
		rejected := id.Set{}
		handleMeta := func(m *meta.Meta) {
			zid := m.Zid
			if rejected.ContainsOrNil(zid) {
				mgr.mgrLog.Trace().Zid(zid).Msg("SelectMeta/alreadyRejected")
				return
			}
			if _, ok := selected[zid]; ok {
				mgr.mgrLog.Trace().Zid(zid).Msg("SelectMeta/alreadySelected")
				return
			}







|










|













|
|

















|
|


















|
|














|

|
|

<


>
>
>
>

>
>
>
>
>
>
|







|

|














|
|
















|
|











|


|







29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200

// Location returns some information where the box is located.
func (mgr *Manager) Location() string {
	if len(mgr.boxes) <= 2 {
		return "NONE"
	}
	var sb strings.Builder
	for i := range len(mgr.boxes) - 2 {
		if i > 0 {
			sb.WriteString(", ")
		}
		sb.WriteString(mgr.boxes[i].Location())
	}
	return sb.String()
}

// CanCreateZettel returns true, if box could possibly create a new zettel.
func (mgr *Manager) CanCreateZettel(ctx context.Context) bool {
	if err := mgr.checkContinue(ctx); err != nil {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		return box.CanCreateZettel(ctx)
	}
	return false
}

// CreateZettel creates a new zettel.
func (mgr *Manager) CreateZettel(ctx context.Context, zettel zettel.Zettel) (id.Zid, error) {
	mgr.mgrLog.Debug().Msg("CreateZettel")
	if err := mgr.checkContinue(ctx); err != nil {
		return id.Invalid, err
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		zettel.Meta = mgr.cleanMetaProperties(zettel.Meta)
		zid, err := box.CreateZettel(ctx, zettel)
		if err == nil {
			mgr.idxUpdateZettel(ctx, zettel)
		}
		return zid, err
	}
	return id.Invalid, box.ErrReadOnly
}

// GetZettel retrieves a specific zettel.
func (mgr *Manager) GetZettel(ctx context.Context, zid id.Zid) (zettel.Zettel, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetZettel")
	if err := mgr.checkContinue(ctx); err != nil {
		return zettel.Zettel{}, err
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for i, p := range mgr.boxes {
		var errZNF box.ErrZettelNotFound
		if z, err := p.GetZettel(ctx, zid); !errors.As(err, &errZNF) {
			if err == nil {
				mgr.Enrich(ctx, z.Meta, i+1)
			}
			return z, err
		}
	}
	return zettel.Zettel{}, box.ErrZettelNotFound{Zid: zid}
}

// GetAllZettel retrieves a specific zettel from all managed boxes.
func (mgr *Manager) GetAllZettel(ctx context.Context, zid id.Zid) ([]zettel.Zettel, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetAllZettel")
	if err := mgr.checkContinue(ctx); err != nil {
		return nil, err
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	var result []zettel.Zettel
	for i, p := range mgr.boxes {
		if z, err := p.GetZettel(ctx, zid); err == nil {
			mgr.Enrich(ctx, z.Meta, i+1)
			result = append(result, z)
		}
	}
	return result, nil
}

// FetchZids returns the set of all zettel identifer managed by the box.
func (mgr *Manager) FetchZids(ctx context.Context) (*id.Set, error) {
	mgr.mgrLog.Debug().Msg("FetchZids")
	if err := mgr.checkContinue(ctx); err != nil {
		return nil, err
	}

	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.fetchZids(ctx)
}
func (mgr *Manager) fetchZids(ctx context.Context) (*id.Set, error) {
	numZettel := 0
	for _, p := range mgr.boxes {
		var mbstats box.ManagedBoxStats
		p.ReadStats(&mbstats)
		numZettel += mbstats.Zettel
	}
	result := id.NewSetCap(numZettel)
	for _, p := range mgr.boxes {
		err := p.ApplyZid(ctx, func(zid id.Zid) { result.Add(zid) }, query.AlwaysIncluded)
		if err != nil {
			return nil, err
		}
	}
	return result, nil
}

func (mgr *Manager) hasZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mgrLog.Debug().Zid(zid).Msg("HasZettel")
	if err := mgr.checkContinue(ctx); err != nil {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, bx := range mgr.boxes {
		if bx.HasZettel(ctx, zid) {
			return true
		}
	}
	return false
}

func (mgr *Manager) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	mgr.mgrLog.Debug().Zid(zid).Msg("GetMeta")
	if err := mgr.checkContinue(ctx); err != nil {
		return nil, err
	}

	m, err := mgr.idxStore.GetMeta(ctx, zid)
	if err != nil {
		return nil, err
	}
	mgr.Enrich(ctx, m, 0)
	return m, nil
}

// SelectMeta returns all zettel meta data that match the selection
// criteria. The result is ordered by descending zettel id.
func (mgr *Manager) SelectMeta(ctx context.Context, metaSeq []*meta.Meta, q *query.Query) ([]*meta.Meta, error) {
	if msg := mgr.mgrLog.Debug(); msg.Enabled() {
		msg.Str("query", q.String()).Msg("SelectMeta")
	}
	if err := mgr.checkContinue(ctx); err != nil {
		return nil, err
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()

	compSearch := q.RetrieveAndCompile(ctx, mgr, metaSeq)
	if result := compSearch.Result(); result != nil {
		mgr.mgrLog.Trace().Int("count", int64(len(result))).Msg("found without ApplyMeta")
		return result, nil
	}
	selected := map[id.Zid]*meta.Meta{}
	for _, term := range compSearch.Terms {
		rejected := id.NewSet()
		handleMeta := func(m *meta.Meta) {
			zid := m.Zid
			if rejected.Contains(zid) {
				mgr.mgrLog.Trace().Zid(zid).Msg("SelectMeta/alreadyRejected")
				return
			}
			if _, ok := selected[zid]; ok {
				mgr.mgrLog.Trace().Zid(zid).Msg("SelectMeta/alreadySelected")
				return
			}
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282

283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
	result = compSearch.AfterSearch(result)
	mgr.mgrLog.Trace().Int("count", int64(len(result))).Msg("found with ApplyMeta")
	return result, nil
}

// CanUpdateZettel returns true, if box could possibly update the given zettel.
func (mgr *Manager) CanUpdateZettel(ctx context.Context, zettel zettel.Zettel) bool {
	if mgr.State() != box.StartStateStarted {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		return box.CanUpdateZettel(ctx, zettel)
	}
	return false

}

// UpdateZettel updates an existing zettel.
func (mgr *Manager) UpdateZettel(ctx context.Context, zettel zettel.Zettel) error {
	mgr.mgrLog.Debug().Zid(zettel.Meta.Zid).Msg("UpdateZettel")
	if mgr.State() != box.StartStateStarted {
		return box.ErrStopped
	}
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		zettel.Meta = mgr.cleanMetaProperties(zettel.Meta)
		if err := box.UpdateZettel(ctx, zettel); err != nil {
			return err
		}
		mgr.idxUpdateZettel(ctx, zettel)
		return nil
	}
	return box.ErrReadOnly
}

// AllowRenameZettel returns true, if box will not disallow renaming the zettel.
func (mgr *Manager) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
	if mgr.State() != box.StartStateStarted {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		if !p.AllowRenameZettel(ctx, zid) {
			return false
		}
	}
	return true
}

// RenameZettel changes the current zid to a new zid.
func (mgr *Manager) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	mgr.mgrLog.Debug().Zid(curZid).Zid(newZid).Msg("RenameZettel")
	if mgr.State() != box.StartStateStarted {
		return box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for i, p := range mgr.boxes {
		err := p.RenameZettel(ctx, curZid, newZid)
		var errZNF box.ErrZettelNotFound
		if err != nil && !errors.As(err, &errZNF) {
			for j := 0; j < i; j++ {
				mgr.boxes[j].RenameZettel(ctx, newZid, curZid)
			}
			return err
		}
	}
	mgr.idxRenameZettel(ctx, curZid, newZid)
	return nil
}

// CanDeleteZettel returns true, if box could possibly delete the given zettel.
func (mgr *Manager) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	if mgr.State() != box.StartStateStarted {

		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		if p.CanDeleteZettel(ctx, zid) {
			return true
		}
	}
	return false
}

// DeleteZettel removes the zettel from the box.
func (mgr *Manager) DeleteZettel(ctx context.Context, zid id.Zid) error {
	mgr.mgrLog.Debug().Zid(zid).Msg("DeleteZettel")
	if mgr.State() != box.StartStateStarted {
		return box.ErrStopped
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		err := p.DeleteZettel(ctx, zid)
		if err == nil {
			mgr.idxDeleteZettel(ctx, zid)







|














|
|












<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<


<
>















|
|







219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254





































255
256

257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
	result = compSearch.AfterSearch(result)
	mgr.mgrLog.Trace().Int("count", int64(len(result))).Msg("found with ApplyMeta")
	return result, nil
}

// CanUpdateZettel returns true, if box could possibly update the given zettel.
func (mgr *Manager) CanUpdateZettel(ctx context.Context, zettel zettel.Zettel) bool {
	if err := mgr.checkContinue(ctx); err != nil {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		return box.CanUpdateZettel(ctx, zettel)
	}
	return false

}

// UpdateZettel updates an existing zettel.
func (mgr *Manager) UpdateZettel(ctx context.Context, zettel zettel.Zettel) error {
	mgr.mgrLog.Debug().Zid(zettel.Meta.Zid).Msg("UpdateZettel")
	if err := mgr.checkContinue(ctx); err != nil {
		return err
	}
	if box, isWriteBox := mgr.boxes[0].(box.WriteBox); isWriteBox {
		zettel.Meta = mgr.cleanMetaProperties(zettel.Meta)
		if err := box.UpdateZettel(ctx, zettel); err != nil {
			return err
		}
		mgr.idxUpdateZettel(ctx, zettel)
		return nil
	}
	return box.ErrReadOnly
}






































// CanDeleteZettel returns true, if box could possibly delete the given zettel.
func (mgr *Manager) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {

	if err := mgr.checkContinue(ctx); err != nil {
		return false
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		if p.CanDeleteZettel(ctx, zid) {
			return true
		}
	}
	return false
}

// DeleteZettel removes the zettel from the box.
func (mgr *Manager) DeleteZettel(ctx context.Context, zid id.Zid) error {
	mgr.mgrLog.Debug().Zid(zid).Msg("DeleteZettel")
	if err := mgr.checkContinue(ctx); err != nil {
		return err
	}
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	for _, p := range mgr.boxes {
		err := p.DeleteZettel(ctx, zid)
		if err == nil {
			mgr.idxDeleteZettel(ctx, zid)

Changes to box/manager/collect.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package manager

import (
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
)

type collectData struct {
	refs  id.Set
	words store.WordSet
	urls  store.WordSet
}

func (data *collectData) initialize() {
	data.refs = id.NewSet()
	data.words = store.NewWordSet()








>
>
>














|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package manager

import (
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
)

type collectData struct {
	refs  *id.Set
	words store.WordSet
	urls  store.WordSet
}

func (data *collectData) initialize() {
	data.refs = id.NewSet()
	data.words = store.NewWordSet()

Changes to box/manager/enrich.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

28
29
30
31
32
33
34
35
36
37
38
39
40





41
42
43
44
45
46
47
48
49
50
51
52
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"strconv"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// Enrich computes additional properties and updates the given metadata.
func (mgr *Manager) Enrich(ctx context.Context, m *meta.Meta, boxNumber int) {

	// Calculate computed, but stored values.
	if _, ok := m.Get(api.KeyCreated); !ok {

		m.Set(api.KeyCreated, computeCreated(m.Zid))
	}

	if box.DoNotEnrich(ctx) {
		// Enrich is called indirectly via indexer or enrichment is not requested
		// because of other reasons -> ignore this call, do not update metadata
		return
	}
	computePublished(m)
	if boxNumber > 0 {
		m.Set(api.KeyBoxNumber, strconv.Itoa(boxNumber))
	}
	mgr.idxStore.Enrich(ctx, m)





}

func computeCreated(zid id.Zid) string {
	if zid <= 10101000000 {
		// A year 0000 is not allowed and therefore an artificaial Zid.
		// In the year 0001, the month must be > 0.
		// In the month 000101, the day must be > 0.
		return "00010101000000"
	}
	seconds := zid % 100
	if seconds > 59 {
		seconds = 59








>
>
>








|









|
>



|
<
<
<
<
|
|
|
|
|
>
>
>
>
>




|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35




36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"strconv"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// Enrich computes additional properties and updates the given metadata.
func (mgr *Manager) Enrich(ctx context.Context, m *meta.Meta, boxNumber int) {

	// Calculate computed, but stored values.
	_, hasCreated := m.Get(api.KeyCreated)
	if !hasCreated {
		m.Set(api.KeyCreated, computeCreated(m.Zid))
	}

	if box.DoEnrich(ctx) {




		computePublished(m)
		if boxNumber > 0 {
			m.Set(api.KeyBoxNumber, strconv.Itoa(boxNumber))
		}
		mgr.idxStore.Enrich(ctx, m)
	}

	if !hasCreated {
		m.Set(meta.KeyCreatedMissing, api.ValueTrue)
	}
}

func computeCreated(zid id.Zid) string {
	if zid <= 10101000000 {
		// A year 0000 is not allowed and therefore an artificial Zid.
		// In the year 0001, the month must be > 0.
		// In the month 000101, the day must be > 0.
		return "00010101000000"
	}
	seconds := zid % 100
	if seconds > 59 {
		seconds = 59

Changes to box/manager/indexer.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"fmt"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"fmt"
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// SearchEqual returns all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchEqual(word string) id.Set {
	found := mgr.idxStore.SearchEqual(word)
	mgr.idxLog.Debug().Str("word", word).Int("found", int64(len(found))).Msg("SearchEqual")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// SearchPrefix returns all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchPrefix(prefix string) id.Set {
	found := mgr.idxStore.SearchPrefix(prefix)
	mgr.idxLog.Debug().Str("prefix", prefix).Int("found", int64(len(found))).Msg("SearchPrefix")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// SearchSuffix returns all zettel that have a word with the given suffix.
// The suffix must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchSuffix(suffix string) id.Set {
	found := mgr.idxStore.SearchSuffix(suffix)
	mgr.idxLog.Debug().Str("suffix", suffix).Int("found", int64(len(found))).Msg("SearchSuffix")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// SearchContains returns all zettel that contains the given string.
// The string must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchContains(s string) id.Set {
	found := mgr.idxStore.SearchContains(s)
	mgr.idxLog.Debug().Str("s", s).Int("found", int64(len(found))).Msg("SearchContains")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// idxIndexer runs in the background and updates the index data structures.







|

|








|

|








|

|








|

|







27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// SearchEqual returns all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchEqual(word string) *id.Set {
	found := mgr.idxStore.SearchEqual(word)
	mgr.idxLog.Debug().Str("word", word).Int("found", int64(found.Length())).Msg("SearchEqual")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// SearchPrefix returns all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchPrefix(prefix string) *id.Set {
	found := mgr.idxStore.SearchPrefix(prefix)
	mgr.idxLog.Debug().Str("prefix", prefix).Int("found", int64(found.Length())).Msg("SearchPrefix")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// SearchSuffix returns all zettel that have a word with the given suffix.
// The suffix must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchSuffix(suffix string) *id.Set {
	found := mgr.idxStore.SearchSuffix(suffix)
	mgr.idxLog.Debug().Str("suffix", suffix).Int("found", int64(found.Length())).Msg("SearchSuffix")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// SearchContains returns all zettel that contains the given string.
// The string must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SearchContains(s string) *id.Set {
	found := mgr.idxStore.SearchContains(s)
	mgr.idxLog.Debug().Str("s", s).Int("found", int64(found.Length())).Msg("SearchContains")
	if msg := mgr.idxLog.Trace(); msg.Enabled() {
		msg.Str("ids", fmt.Sprint(found)).Msg("IDs")
	}
	return found
}

// idxIndexer runs in the background and updates the index data structures.
136
137
138
139
140
141
142

143
144
145
146
147
148
149
		if !ok {
			return false
		}
	case _, ok := <-timer.C:
		if !ok {
			return false
		}

		timer.Reset(timerDuration)
	case <-mgr.done:
		if !timer.Stop() {
			<-timer.C
		}
		return false
	}







>







139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
		if !ok {
			return false
		}
	case _, ok := <-timer.C:
		if !ok {
			return false
		}
		// mgr.idxStore.Optimize() // TODO: make it less often, for example once per 10 minutes
		timer.Reset(timerDuration)
	case <-mgr.done:
		if !timer.Stop() {
			<-timer.C
		}
		return false
	}
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
		}
	} else {
		stWords.Add(value)
	}
}

func (mgr *Manager) idxProcessData(ctx context.Context, zi *store.ZettelIndex, cData *collectData) {
	for ref := range cData.refs {
		if mgr.HasZettel(ctx, ref) {
			zi.AddBackRef(ref)
		} else {
			zi.AddDeadRef(ref)
		}
	}
	zi.SetWords(cData.words)
	zi.SetUrls(cData.urls)
}

func (mgr *Manager) idxUpdateValue(ctx context.Context, inverseKey, value string, zi *store.ZettelIndex) {
	zid, err := id.Parse(value)
	if err != nil {
		return
	}
	if !mgr.HasZettel(ctx, zid) {
		zi.AddDeadRef(zid)
		return
	}
	if inverseKey == "" {
		zi.AddBackRef(zid)
		return
	}
	zi.AddInverseRef(inverseKey, zid)
}

func (mgr *Manager) idxRenameZettel(ctx context.Context, curZid, newZid id.Zid) {
	toCheck := mgr.idxStore.RenameZettel(ctx, curZid, newZid)
	mgr.idxCheckZettel(toCheck)
}

func (mgr *Manager) idxDeleteZettel(ctx context.Context, zid id.Zid) {
	toCheck := mgr.idxStore.DeleteZettel(ctx, zid)
	mgr.idxCheckZettel(toCheck)
}

func (mgr *Manager) idxCheckZettel(s id.Set) {
	for zid := range s {
		mgr.idxAr.EnqueueZettel(zid)
	}
}







|
|




|









|










<
<
<
<
<





|
|

|

206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239





240
241
242
243
244
245
246
247
248
249
		}
	} else {
		stWords.Add(value)
	}
}

func (mgr *Manager) idxProcessData(ctx context.Context, zi *store.ZettelIndex, cData *collectData) {
	cData.refs.ForEach(func(ref id.Zid) {
		if mgr.hasZettel(ctx, ref) {
			zi.AddBackRef(ref)
		} else {
			zi.AddDeadRef(ref)
		}
	})
	zi.SetWords(cData.words)
	zi.SetUrls(cData.urls)
}

func (mgr *Manager) idxUpdateValue(ctx context.Context, inverseKey, value string, zi *store.ZettelIndex) {
	zid, err := id.Parse(value)
	if err != nil {
		return
	}
	if !mgr.hasZettel(ctx, zid) {
		zi.AddDeadRef(zid)
		return
	}
	if inverseKey == "" {
		zi.AddBackRef(zid)
		return
	}
	zi.AddInverseRef(inverseKey, zid)
}






func (mgr *Manager) idxDeleteZettel(ctx context.Context, zid id.Zid) {
	toCheck := mgr.idxStore.DeleteZettel(ctx, zid)
	mgr.idxCheckZettel(toCheck)
}

func (mgr *Manager) idxCheckZettel(s *id.Set) {
	s.ForEach(func(zid id.Zid) {
		mgr.idxAr.EnqueueZettel(zid)
	})
}

Changes to box/manager/manager.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39








40
41
42
43
44
45
46
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package manager coordinates the various boxes and indexes of a Zettelstore.
package manager

import (
	"context"
	"io"
	"net/url"
	"sync"
	"time"

	"zettelstore.de/client.fossil/maps"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager/memstore"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/config"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ConnectData contains all administration related values.
type ConnectData struct {
	Number   int // number of the box, starting with 1.
	Config   config.Config
	Enricher box.Enricher
	Notify   chan<- box.UpdateInfo








}

// Connect returns a handle to the specified box.
func Connect(u *url.URL, authManager auth.BaseManager, cdata *ConnectData) (box.ManagedBox, error) {
	if authManager.IsReadonly() {
		rawURL := u.String()
		// TODO: the following is wrong under some circumstances:








>
>
>












<


|















>
>
>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package manager coordinates the various boxes and indexes of a Zettelstore.
package manager

import (
	"context"
	"io"
	"net/url"
	"sync"
	"time"


	"zettelstore.de/z/auth"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager/mapstore"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/config"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ConnectData contains all administration related values.
type ConnectData struct {
	Number   int // number of the box, starting with 1.
	Config   config.Config
	Enricher box.Enricher
	Notify   chan<- box.UpdateInfo
	Mapper   Mapper
}

// Mapper allows to inspect the mapping between old-style and new-style zettel identifier.
type Mapper interface {
	Warnings(context.Context) (*id.Set, error) // Fetch problematic zettel identifier

	OldToNewMapping(ctx context.Context) (map[id.Zid]id.ZidN, error)
}

// Connect returns a handle to the specified box.
func Connect(u *url.URL, authManager auth.BaseManager, cdata *ConnectData) (box.ManagedBox, error) {
	if authManager.IsReadonly() {
		rawURL := u.String()
		// TODO: the following is wrong under some circumstances:
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
func Register(scheme string, create createFunc) {
	if _, ok := registry[scheme]; ok {
		panic(scheme)
	}
	registry[scheme] = create
}

// GetSchemes returns all registered scheme, ordered by scheme string.
func GetSchemes() []string { return maps.Keys(registry) }

// Manager is a coordinating box.
type Manager struct {
	mgrLog       *logger.Logger
	stateMx      sync.RWMutex
	state        box.StartState
	mgrMx        sync.RWMutex
	rtConfig     config.Config
	boxes        []box.ManagedBox
	observers    []box.UpdateFunc
	mxObserver   sync.RWMutex
	done         chan struct{}
	infos        chan box.UpdateInfo
	propertyKeys strfun.Set // Set of property key names


	// Indexer data
	idxLog   *logger.Logger
	idxStore store.Store
	idxAr    *anteroomQueue
	idxReady chan struct{} // Signal a non-empty anteroom to background task








<
<
<













>







85
86
87
88
89
90
91



92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
func Register(scheme string, create createFunc) {
	if _, ok := registry[scheme]; ok {
		panic(scheme)
	}
	registry[scheme] = create
}




// Manager is a coordinating box.
type Manager struct {
	mgrLog       *logger.Logger
	stateMx      sync.RWMutex
	state        box.StartState
	mgrMx        sync.RWMutex
	rtConfig     config.Config
	boxes        []box.ManagedBox
	observers    []box.UpdateFunc
	mxObserver   sync.RWMutex
	done         chan struct{}
	infos        chan box.UpdateInfo
	propertyKeys strfun.Set // Set of property key names
	zidMapper    *zidMapper

	// Indexer data
	idxLog   *logger.Logger
	idxStore store.Store
	idxAr    *anteroomQueue
	idxReady chan struct{} // Signal a non-empty anteroom to background task

135
136
137
138
139
140
141
142
143
144
145


146
147
148
149
150
151
152
153
	mgr := &Manager{
		mgrLog:       boxLog.Clone().Str("box", "manager").Child(),
		rtConfig:     rtConfig,
		infos:        make(chan box.UpdateInfo, len(boxURIs)*10),
		propertyKeys: propertyKeys,

		idxLog:   boxLog.Clone().Str("box", "index").Child(),
		idxStore: memstore.New(),
		idxAr:    newAnteroomQueue(1000),
		idxReady: make(chan struct{}, 1),
	}


	cdata := ConnectData{Number: 1, Config: rtConfig, Enricher: mgr, Notify: mgr.infos}
	boxes := make([]box.ManagedBox, 0, len(boxURIs)+2)
	for _, uri := range boxURIs {
		p, err := Connect(uri, authManager, &cdata)
		if err != nil {
			return nil, err
		}
		if p != nil {







|



>
>
|







143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
	mgr := &Manager{
		mgrLog:       boxLog.Clone().Str("box", "manager").Child(),
		rtConfig:     rtConfig,
		infos:        make(chan box.UpdateInfo, len(boxURIs)*10),
		propertyKeys: propertyKeys,

		idxLog:   boxLog.Clone().Str("box", "index").Child(),
		idxStore: createIdxStore(rtConfig),
		idxAr:    newAnteroomQueue(1000),
		idxReady: make(chan struct{}, 1),
	}
	mgr.zidMapper = NewZidMapper(mgr)

	cdata := ConnectData{Number: 1, Config: rtConfig, Enricher: mgr, Notify: mgr.infos, Mapper: mgr.zidMapper}
	boxes := make([]box.ManagedBox, 0, len(boxURIs)+2)
	for _, uri := range boxURIs {
		p, err := Connect(uri, authManager, &cdata)
		if err != nil {
			return nil, err
		}
		if p != nil {
165
166
167
168
169
170
171




172
173
174
175
176
177
178
		return nil, err
	}
	cdata.Number++
	boxes = append(boxes, constbox, compbox)
	mgr.boxes = boxes
	return mgr, nil
}





// RegisterObserver registers an observer that will be notified
// if a zettel was found to be changed.
func (mgr *Manager) RegisterObserver(f box.UpdateFunc) {
	if f != nil {
		mgr.mxObserver.Lock()
		mgr.observers = append(mgr.observers, f)







>
>
>
>







175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
		return nil, err
	}
	cdata.Number++
	boxes = append(boxes, constbox, compbox)
	mgr.boxes = boxes
	return mgr, nil
}

func createIdxStore(_ config.Config) store.Store {
	return mapstore.New()
}

// RegisterObserver registers an observer that will be notified
// if a zettel was found to be changed.
func (mgr *Manager) RegisterObserver(f box.UpdateFunc) {
	if f != nil {
		mgr.mxObserver.Lock()
		mgr.observers = append(mgr.observers, f)
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
	case box.OnReady:
		return
	case box.OnReload:
		mgr.idxAr.Reset()
	case box.OnZettel:
		mgr.idxAr.EnqueueZettel(zid)
	default:
		mgr.mgrLog.Warn().Uint("reason", uint64(reason)).Zid(zid).Msg("Unknown notification reason")
		return
	}
	select {
	case mgr.idxReady <- struct{}{}:
	default:
	}
}







|







262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
	case box.OnReady:
		return
	case box.OnReload:
		mgr.idxAr.Reset()
	case box.OnZettel:
		mgr.idxAr.EnqueueZettel(zid)
	default:
		mgr.mgrLog.Error().Uint("reason", uint64(reason)).Zid(zid).Msg("Unknown notification reason")
		return
	}
	select {
	case mgr.idxReady <- struct{}{}:
	default:
	}
}
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325

func (mgr *Manager) waitBoxesAreStarted() {
	const waitTime = 10 * time.Millisecond
	const waitLoop = int(1 * time.Second / waitTime)
	for i := 1; !mgr.allBoxesStarted(); i++ {
		if i%waitLoop == 0 {
			if time.Duration(i)*waitTime > time.Minute {
				mgr.mgrLog.Warn().Msg("Waiting for more than one minute to start")
			} else {
				mgr.mgrLog.Trace().Msg("Wait for boxes to start")
			}
		}
		time.Sleep(waitTime)
	}
}







|







325
326
327
328
329
330
331
332
333
334
335
336
337
338
339

func (mgr *Manager) waitBoxesAreStarted() {
	const waitTime = 10 * time.Millisecond
	const waitLoop = int(1 * time.Second / waitTime)
	for i := 1; !mgr.allBoxesStarted(); i++ {
		if i%waitLoop == 0 {
			if time.Duration(i)*waitTime > time.Minute {
				mgr.mgrLog.Info().Msg("Waiting for more than one minute to start")
			} else {
				mgr.mgrLog.Trace().Msg("Wait for boxes to start")
			}
		}
		time.Sleep(waitTime)
	}
}
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
	return true
}

// Stop the started box. Now only the Start() function is allowed.
func (mgr *Manager) Stop(ctx context.Context) {
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	if mgr.State() != box.StartStateStarted {
		return
	}
	mgr.setState(box.StartStateStopping)
	close(mgr.done)
	for _, p := range mgr.boxes {
		if ss, ok := p.(box.StartStopper); ok {
			ss.Stop(ctx)
		}
	}
	mgr.setState(box.StartStateStopped)
}

// Refresh internal box data.
func (mgr *Manager) Refresh(ctx context.Context) error {
	mgr.mgrLog.Debug().Msg("Refresh")
	if mgr.State() != box.StartStateStarted {
		return box.ErrStopped
	}
	mgr.infos <- box.UpdateInfo{Reason: box.OnReload, Zid: id.Invalid}
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	for _, bx := range mgr.boxes {
		if rb, ok := bx.(box.Refresher); ok {
			rb.Refresh(ctx)
		}
	}
	return nil
}

// ReIndex data of the given zettel.
func (mgr *Manager) ReIndex(_ context.Context, zid id.Zid) error {
	mgr.mgrLog.Debug().Msg("ReIndex")
	if mgr.State() != box.StartStateStarted {
		return box.ErrStopped
	}
	mgr.infos <- box.UpdateInfo{Reason: box.OnReload, Zid: zid}
	return nil
}

// ReadStats populates st with box statistics.
func (mgr *Manager) ReadStats(st *box.Stats) {
	mgr.mgrLog.Debug().Msg("ReadStats")
	mgr.mgrMx.RLock()







|















|
|













|

|
|

|







347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
	return true
}

// Stop the started box. Now only the Start() function is allowed.
func (mgr *Manager) Stop(ctx context.Context) {
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	if err := mgr.checkContinue(ctx); err != nil {
		return
	}
	mgr.setState(box.StartStateStopping)
	close(mgr.done)
	for _, p := range mgr.boxes {
		if ss, ok := p.(box.StartStopper); ok {
			ss.Stop(ctx)
		}
	}
	mgr.setState(box.StartStateStopped)
}

// Refresh internal box data.
func (mgr *Manager) Refresh(ctx context.Context) error {
	mgr.mgrLog.Debug().Msg("Refresh")
	if err := mgr.checkContinue(ctx); err != nil {
		return err
	}
	mgr.infos <- box.UpdateInfo{Reason: box.OnReload, Zid: id.Invalid}
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	for _, bx := range mgr.boxes {
		if rb, ok := bx.(box.Refresher); ok {
			rb.Refresh(ctx)
		}
	}
	return nil
}

// ReIndex data of the given zettel.
func (mgr *Manager) ReIndex(ctx context.Context, zid id.Zid) error {
	mgr.mgrLog.Debug().Msg("ReIndex")
	if err := mgr.checkContinue(ctx); err != nil {
		return err
	}
	mgr.infos <- box.UpdateInfo{Reason: box.OnZettel, Zid: zid}
	return nil
}

// ReadStats populates st with box statistics.
func (mgr *Manager) ReadStats(st *box.Stats) {
	mgr.mgrLog.Debug().Msg("ReadStats")
	mgr.mgrMx.RLock()
412
413
414
415
416
417
418







	st.IndexedUrls = storeSt.Urls
}

// Dump internal data structures to a Writer.
func (mgr *Manager) Dump(w io.Writer) {
	mgr.idxStore.Dump(w)
}














>
>
>
>
>
>
>
426
427
428
429
430
431
432
433
434
435
436
437
438
439
	st.IndexedUrls = storeSt.Urls
}

// Dump internal data structures to a Writer.
func (mgr *Manager) Dump(w io.Writer) {
	mgr.idxStore.Dump(w)
}

func (mgr *Manager) checkContinue(ctx context.Context) error {
	if mgr.State() != box.StartStateStarted {
		return box.ErrStopped
	}
	return ctx.Err()
}

Added box/manager/mapstore/mapstore.go.



































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package mapstore stored the index in main memory via a Go map.
package mapstore

import (
	"context"
	"fmt"
	"io"
	"slices"
	"strings"
	"sync"

	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/maps"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

type zettelData struct {
	meta      *meta.Meta // a local copy of the metadata, without computed keys
	dead      *id.Set    // set of dead references in this zettel
	forward   *id.Set    // set of forward references in this zettel
	backward  *id.Set    // set of zettel that reference with zettel
	otherRefs map[string]bidiRefs
	words     []string // list of words of this zettel
	urls      []string // list of urls of this zettel
}

type bidiRefs struct {
	forward  *id.Set
	backward *id.Set
}

func (zd *zettelData) optimize() {
	zd.dead.Optimize()
	zd.forward.Optimize()
	zd.backward.Optimize()
	for _, bidi := range zd.otherRefs {
		bidi.forward.Optimize()
		bidi.backward.Optimize()
	}
}

type mapStore struct {
	mx     sync.RWMutex
	intern map[string]string // map to intern strings
	idx    map[id.Zid]*zettelData
	dead   map[id.Zid]*id.Set // map dead refs where they occur
	words  stringRefs
	urls   stringRefs

	// Stats
	mxStats sync.Mutex
	updates uint64
}
type stringRefs map[string]*id.Set

// New returns a new memory-based index store.
func New() store.Store {
	return &mapStore{
		intern: make(map[string]string, 1024),
		idx:    make(map[id.Zid]*zettelData),
		dead:   make(map[id.Zid]*id.Set),
		words:  make(stringRefs),
		urls:   make(stringRefs),
	}
}

func (ms *mapStore) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	if zi, found := ms.idx[zid]; found && zi.meta != nil {
		// zi.meta is nil, if zettel was referenced, but is not indexed yet.
		return zi.meta.Clone(), nil
	}
	return nil, box.ErrZettelNotFound{Zid: zid}
}

func (ms *mapStore) Enrich(_ context.Context, m *meta.Meta) {
	if ms.doEnrich(m) {
		ms.mxStats.Lock()
		ms.updates++
		ms.mxStats.Unlock()
	}
}

func (ms *mapStore) doEnrich(m *meta.Meta) bool {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	zi, ok := ms.idx[m.Zid]
	if !ok {
		return false
	}
	var updated bool
	if !zi.dead.IsEmpty() {
		m.Set(api.KeyDead, zi.dead.MetaString())
		updated = true
	}
	back := removeOtherMetaRefs(m, zi.backward.Clone())
	if !zi.backward.IsEmpty() {
		m.Set(api.KeyBackward, zi.backward.MetaString())
		updated = true
	}
	if !zi.forward.IsEmpty() {
		m.Set(api.KeyForward, zi.forward.MetaString())
		back.ISubstract(zi.forward)
		updated = true
	}
	for k, refs := range zi.otherRefs {
		if !refs.backward.IsEmpty() {
			m.Set(k, refs.backward.MetaString())
			back.ISubstract(refs.backward)
			updated = true
		}
	}
	if !back.IsEmpty() {
		m.Set(api.KeyBack, back.MetaString())
		updated = true
	}
	return updated
}

// SearchEqual returns all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *mapStore) SearchEqual(word string) *id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := id.NewSet()
	if refs, ok := ms.words[word]; ok {
		result = result.IUnion(refs)
	}
	if refs, ok := ms.urls[word]; ok {
		result = result.IUnion(refs)
	}
	zid, err := id.Parse(word)
	if err != nil {
		return result
	}
	zi, ok := ms.idx[zid]
	if !ok {
		return result
	}

	return addBackwardZids(result, zid, zi)
}

// SearchPrefix returns all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *mapStore) SearchPrefix(prefix string) *id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(prefix, strings.HasPrefix)
	l := len(prefix)
	if l > 14 {
		return result
	}
	maxZid, err := id.Parse(prefix + "99999999999999"[:14-l])
	if err != nil {
		return result
	}
	var minZid id.Zid
	if l < 14 && prefix == "0000000000000"[:l] {
		minZid = id.Zid(1)
	} else {
		minZid, err = id.Parse(prefix + "00000000000000"[:14-l])
		if err != nil {
			return result
		}
	}
	for zid, zi := range ms.idx {
		if minZid <= zid && zid <= maxZid {
			result = addBackwardZids(result, zid, zi)
		}
	}
	return result
}

// SearchSuffix returns all zettel that have a word with the given suffix.
// The suffix must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *mapStore) SearchSuffix(suffix string) *id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(suffix, strings.HasSuffix)
	l := len(suffix)
	if l > 14 {
		return result
	}
	val, err := id.ParseUint(suffix)
	if err != nil {
		return result
	}
	modulo := uint64(1)
	for range l {
		modulo *= 10
	}
	for zid, zi := range ms.idx {
		if uint64(zid)%modulo == val {
			result = addBackwardZids(result, zid, zi)
		}
	}
	return result
}

// SearchContains returns all zettel that contains the given string.
// The string must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *mapStore) SearchContains(s string) *id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(s, strings.Contains)
	if len(s) > 14 {
		return result
	}
	if _, err := id.ParseUint(s); err != nil {
		return result
	}
	for zid, zi := range ms.idx {
		if strings.Contains(zid.String(), s) {
			result = addBackwardZids(result, zid, zi)
		}
	}
	return result
}

func (ms *mapStore) selectWithPred(s string, pred func(string, string) bool) *id.Set {
	// Must only be called if ms.mx is read-locked!
	result := id.NewSet()
	for word, refs := range ms.words {
		if !pred(word, s) {
			continue
		}
		result.IUnion(refs)
	}
	for u, refs := range ms.urls {
		if !pred(u, s) {
			continue
		}
		result.IUnion(refs)
	}
	return result
}

func addBackwardZids(result *id.Set, zid id.Zid, zi *zettelData) *id.Set {
	// Must only be called if ms.mx is read-locked!
	result = result.Add(zid)
	result = result.IUnion(zi.backward)
	for _, mref := range zi.otherRefs {
		result = result.IUnion(mref.backward)
	}
	return result
}

func removeOtherMetaRefs(m *meta.Meta, back *id.Set) *id.Set {
	for _, p := range m.PairsRest() {
		switch meta.Type(p.Key) {
		case meta.TypeID:
			if zid, err := id.Parse(p.Value); err == nil {
				back = back.Remove(zid)
			}
		case meta.TypeIDSet:
			for _, val := range meta.ListFromValue(p.Value) {
				if zid, err := id.Parse(val); err == nil {
					back = back.Remove(zid)
				}
			}
		}
	}
	return back
}

func (ms *mapStore) UpdateReferences(_ context.Context, zidx *store.ZettelIndex) *id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()
	m := ms.makeMeta(zidx)
	zi, ziExist := ms.idx[zidx.Zid]
	if !ziExist || zi == nil {
		zi = &zettelData{}
		ziExist = false
	}

	// Is this zettel an old dead reference mentioned in other zettel?
	var toCheck *id.Set
	if refs, ok := ms.dead[zidx.Zid]; ok {
		// These must be checked later again
		toCheck = refs
		delete(ms.dead, zidx.Zid)
	}

	zi.meta = m
	ms.updateDeadReferences(zidx, zi)
	ids := ms.updateForwardBackwardReferences(zidx, zi)
	toCheck = toCheck.IUnion(ids)
	ids = ms.updateMetadataReferences(zidx, zi)
	toCheck = toCheck.IUnion(ids)
	zi.words = updateStrings(zidx.Zid, ms.words, zi.words, zidx.GetWords())
	zi.urls = updateStrings(zidx.Zid, ms.urls, zi.urls, zidx.GetUrls())

	// Check if zi must be inserted into ms.idx
	if !ziExist {
		ms.idx[zidx.Zid] = zi
	}
	zi.optimize()
	return toCheck
}

var internableKeys = map[string]bool{
	api.KeyRole:      true,
	api.KeySyntax:    true,
	api.KeyFolgeRole: true,
	api.KeyLang:      true,
	api.KeyReadOnly:  true,
}

func isInternableValue(key string) bool {
	if internableKeys[key] {
		return true
	}
	return strings.HasSuffix(key, meta.SuffixKeyRole)
}

func (ms *mapStore) internString(s string) string {
	if is, found := ms.intern[s]; found {
		return is
	}
	ms.intern[s] = s
	return s
}

func (ms *mapStore) makeMeta(zidx *store.ZettelIndex) *meta.Meta {
	origM := zidx.GetMeta()
	copyM := meta.New(origM.Zid)
	for _, p := range origM.Pairs() {
		key := ms.internString(p.Key)
		if isInternableValue(key) {
			copyM.Set(key, ms.internString(p.Value))
		} else if key == api.KeyBoxNumber || !meta.IsComputed(key) {
			copyM.Set(key, p.Value)
		}
	}
	return copyM
}

func (ms *mapStore) updateDeadReferences(zidx *store.ZettelIndex, zi *zettelData) {
	// Must only be called if ms.mx is write-locked!
	drefs := zidx.GetDeadRefs()
	newRefs, remRefs := zi.dead.Diff(drefs)
	zi.dead = drefs
	remRefs.ForEach(func(ref id.Zid) {
		ms.dead[ref] = ms.dead[ref].Remove(zidx.Zid)
	})
	newRefs.ForEach(func(ref id.Zid) {
		ms.dead[ref] = ms.dead[ref].Add(zidx.Zid)
	})
}

func (ms *mapStore) updateForwardBackwardReferences(zidx *store.ZettelIndex, zi *zettelData) *id.Set {
	// Must only be called if ms.mx is write-locked!
	brefs := zidx.GetBackRefs()
	newRefs, remRefs := zi.forward.Diff(brefs)
	zi.forward = brefs

	var toCheck *id.Set
	remRefs.ForEach(func(ref id.Zid) {
		bzi := ms.getOrCreateEntry(ref)
		bzi.backward = bzi.backward.Remove(zidx.Zid)
		if bzi.meta == nil {
			toCheck = toCheck.Add(ref)
		}
	})
	newRefs.ForEach(func(ref id.Zid) {
		bzi := ms.getOrCreateEntry(ref)
		bzi.backward = bzi.backward.Add(zidx.Zid)
		if bzi.meta == nil {
			toCheck = toCheck.Add(ref)
		}
	})
	return toCheck
}

func (ms *mapStore) updateMetadataReferences(zidx *store.ZettelIndex, zi *zettelData) *id.Set {
	// Must only be called if ms.mx is write-locked!
	inverseRefs := zidx.GetInverseRefs()
	for key, mr := range zi.otherRefs {
		if _, ok := inverseRefs[key]; ok {
			continue
		}
		ms.removeInverseMeta(zidx.Zid, key, mr.forward)
	}
	if zi.otherRefs == nil {
		zi.otherRefs = make(map[string]bidiRefs)
	}
	var toCheck *id.Set
	for key, mrefs := range inverseRefs {
		mr := zi.otherRefs[key]
		newRefs, remRefs := mr.forward.Diff(mrefs)
		mr.forward = mrefs
		zi.otherRefs[key] = mr

		newRefs.ForEach(func(ref id.Zid) {
			bzi := ms.getOrCreateEntry(ref)
			if bzi.otherRefs == nil {
				bzi.otherRefs = make(map[string]bidiRefs)
			}
			bmr := bzi.otherRefs[key]
			bmr.backward = bmr.backward.Add(zidx.Zid)
			bzi.otherRefs[key] = bmr
			if bzi.meta == nil {
				toCheck = toCheck.Add(ref)
			}
		})

		ms.removeInverseMeta(zidx.Zid, key, remRefs)
	}
	return toCheck
}

func updateStrings(zid id.Zid, srefs stringRefs, prev []string, next store.WordSet) []string {
	newWords, removeWords := next.Diff(prev)
	for _, word := range newWords {
		srefs[word] = srefs[word].Add(zid)
	}
	for _, word := range removeWords {
		refs, ok := srefs[word]
		if !ok {
			continue
		}
		refs = refs.Remove(zid)
		if refs.IsEmpty() {
			delete(srefs, word)
			continue
		}
		srefs[word] = refs
	}
	return next.Words()
}

func (ms *mapStore) getOrCreateEntry(zid id.Zid) *zettelData {
	// Must only be called if ms.mx is write-locked!
	if zi, ok := ms.idx[zid]; ok {
		return zi
	}
	zi := &zettelData{}
	ms.idx[zid] = zi
	return zi
}

func (ms *mapStore) DeleteZettel(_ context.Context, zid id.Zid) *id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()
	return ms.doDeleteZettel(zid)
}

func (ms *mapStore) doDeleteZettel(zid id.Zid) *id.Set {
	// Must only be called if ms.mx is write-locked!
	zi, ok := ms.idx[zid]
	if !ok {
		return nil
	}

	ms.deleteDeadSources(zid, zi)
	toCheck := ms.deleteForwardBackward(zid, zi)
	for key, mrefs := range zi.otherRefs {
		ms.removeInverseMeta(zid, key, mrefs.forward)
	}
	deleteStrings(ms.words, zi.words, zid)
	deleteStrings(ms.urls, zi.urls, zid)
	delete(ms.idx, zid)
	return toCheck
}

func (ms *mapStore) deleteDeadSources(zid id.Zid, zi *zettelData) {
	// Must only be called if ms.mx is write-locked!
	zi.dead.ForEach(func(ref id.Zid) {
		if drefs, ok := ms.dead[ref]; ok {
			if drefs = drefs.Remove(zid); drefs.IsEmpty() {
				delete(ms.dead, ref)
			} else {
				ms.dead[ref] = drefs
			}
		}
	})
}

func (ms *mapStore) deleteForwardBackward(zid id.Zid, zi *zettelData) *id.Set {
	// Must only be called if ms.mx is write-locked!
	zi.forward.ForEach(func(ref id.Zid) {
		if fzi, ok := ms.idx[ref]; ok {
			fzi.backward = fzi.backward.Remove(zid)
		}
	})

	var toCheck *id.Set
	zi.backward.ForEach(func(ref id.Zid) {
		if bzi, ok := ms.idx[ref]; ok {
			bzi.forward = bzi.forward.Remove(zid)
			toCheck = toCheck.Add(ref)
		}
	})
	return toCheck
}

func (ms *mapStore) removeInverseMeta(zid id.Zid, key string, forward *id.Set) {
	// Must only be called if ms.mx is write-locked!
	forward.ForEach(func(ref id.Zid) {
		bzi, ok := ms.idx[ref]
		if !ok || bzi.otherRefs == nil {
			return
		}
		bmr, ok := bzi.otherRefs[key]
		if !ok {
			return
		}
		bmr.backward = bmr.backward.Remove(zid)
		if !bmr.backward.IsEmpty() || !bmr.forward.IsEmpty() {
			bzi.otherRefs[key] = bmr
		} else {
			delete(bzi.otherRefs, key)
			if len(bzi.otherRefs) == 0 {
				bzi.otherRefs = nil
			}
		}
	})
}

func deleteStrings(msStringMap stringRefs, curStrings []string, zid id.Zid) {
	// Must only be called if ms.mx is write-locked!
	for _, word := range curStrings {
		refs, ok := msStringMap[word]
		if !ok {
			continue
		}
		refs = refs.Remove(zid)
		if refs.IsEmpty() {
			delete(msStringMap, word)
			continue
		}
		msStringMap[word] = refs
	}
}

func (ms *mapStore) Optimize() {
	ms.mx.Lock()
	defer ms.mx.Unlock()

	// No need to optimize ms.idx: is already done via ms.UpdateReferences
	for _, dead := range ms.dead {
		dead.Optimize()
	}
	for _, s := range ms.words {
		s.Optimize()
	}
	for _, s := range ms.urls {
		s.Optimize()
	}
}

func (ms *mapStore) ReadStats(st *store.Stats) {
	ms.mx.RLock()
	st.Zettel = len(ms.idx)
	st.Words = uint64(len(ms.words))
	st.Urls = uint64(len(ms.urls))
	ms.mx.RUnlock()
	ms.mxStats.Lock()
	st.Updates = ms.updates
	ms.mxStats.Unlock()
}

func (ms *mapStore) Dump(w io.Writer) {
	ms.mx.RLock()
	defer ms.mx.RUnlock()

	io.WriteString(w, "=== Dump\n")
	ms.dumpIndex(w)
	ms.dumpDead(w)
	dumpStringRefs(w, "Words", "", "", ms.words)
	dumpStringRefs(w, "URLs", "[[", "]]", ms.urls)
}

func (ms *mapStore) dumpIndex(w io.Writer) {
	if len(ms.idx) == 0 {
		return
	}
	io.WriteString(w, "==== Zettel Index\n")
	zids := make(id.Slice, 0, len(ms.idx))
	for id := range ms.idx {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, "=====", id)
		zi := ms.idx[id]
		if !zi.dead.IsEmpty() {
			fmt.Fprintln(w, "* Dead:", zi.dead)
		}
		dumpSet(w, "* Forward:", zi.forward)
		dumpSet(w, "* Backward:", zi.backward)

		otherRefs := make([]string, 0, len(zi.otherRefs))
		for k := range zi.otherRefs {
			otherRefs = append(otherRefs, k)
		}
		slices.Sort(otherRefs)
		for _, k := range otherRefs {
			fmt.Fprintln(w, "* Meta", k)
			dumpSet(w, "** Forward:", zi.otherRefs[k].forward)
			dumpSet(w, "** Backward:", zi.otherRefs[k].backward)
		}
		dumpStrings(w, "* Words", "", "", zi.words)
		dumpStrings(w, "* URLs", "[[", "]]", zi.urls)
	}
}

func (ms *mapStore) dumpDead(w io.Writer) {
	if len(ms.dead) == 0 {
		return
	}
	fmt.Fprintf(w, "==== Dead References\n")
	zids := make(id.Slice, 0, len(ms.dead))
	for id := range ms.dead {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, ";", id)
		fmt.Fprintln(w, ":", ms.dead[id])
	}
}

func dumpSet(w io.Writer, prefix string, s *id.Set) {
	if !s.IsEmpty() {
		io.WriteString(w, prefix)
		s.ForEach(func(zid id.Zid) {
			io.WriteString(w, " ")
			w.Write(zid.Bytes())
		})
		fmt.Fprintln(w)
	}
}
func dumpStrings(w io.Writer, title, preString, postString string, slice []string) {
	if len(slice) > 0 {
		sl := make([]string, len(slice))
		copy(sl, slice)
		slices.Sort(sl)
		fmt.Fprintln(w, title)
		for _, s := range sl {
			fmt.Fprintf(w, "** %s%s%s\n", preString, s, postString)
		}
	}
}

func dumpStringRefs(w io.Writer, title, preString, postString string, srefs stringRefs) {
	if len(srefs) == 0 {
		return
	}
	fmt.Fprintln(w, "====", title)
	for _, s := range maps.Keys(srefs) {
		fmt.Fprintf(w, "; %s%s%s\n", preString, s, postString)
		fmt.Fprintln(w, ":", srefs[s])
	}
}

Deleted box/manager/memstore/memstore.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package memstore stored the index in main memory.
package memstore

import (
	"context"
	"fmt"
	"io"
	"sort"
	"strings"
	"sync"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/client.fossil/maps"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

type bidiRefs struct {
	forward  id.Slice
	backward id.Slice
}

type zettelData struct {
	meta      *meta.Meta // a local copy of the metadata, without computed keys
	dead      id.Slice   // list of dead references in this zettel
	forward   id.Slice   // list of forward references in this zettel
	backward  id.Slice   // list of zettel that reference with zettel
	otherRefs map[string]bidiRefs
	words     []string // list of words of this zettel
	urls      []string // list of urls of this zettel
}

type stringRefs map[string]id.Slice

type memStore struct {
	mx     sync.RWMutex
	intern map[string]string // map to intern strings
	idx    map[id.Zid]*zettelData
	dead   map[id.Zid]id.Slice // map dead refs where they occur
	words  stringRefs
	urls   stringRefs

	// Stats
	mxStats sync.Mutex
	updates uint64
}

// New returns a new memory-based index store.
func New() store.Store {
	return &memStore{
		intern: make(map[string]string, 1024),
		idx:    make(map[id.Zid]*zettelData),
		dead:   make(map[id.Zid]id.Slice),
		words:  make(stringRefs),
		urls:   make(stringRefs),
	}
}

func (ms *memStore) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	if zi, found := ms.idx[zid]; found && zi.meta != nil {
		// zi.meta is nil, if zettel was referenced, but is not indexed yet.
		return zi.meta.Clone(), nil
	}
	return nil, box.ErrZettelNotFound{Zid: zid}
}

func (ms *memStore) Enrich(_ context.Context, m *meta.Meta) {
	if ms.doEnrich(m) {
		ms.mxStats.Lock()
		ms.updates++
		ms.mxStats.Unlock()
	}
}

func (ms *memStore) doEnrich(m *meta.Meta) bool {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	zi, ok := ms.idx[m.Zid]
	if !ok {
		return false
	}
	var updated bool
	if len(zi.dead) > 0 {
		m.Set(api.KeyDead, zi.dead.String())
		updated = true
	}
	back := removeOtherMetaRefs(m, zi.backward.Clone())
	if len(zi.backward) > 0 {
		m.Set(api.KeyBackward, zi.backward.String())
		updated = true
	}
	if len(zi.forward) > 0 {
		m.Set(api.KeyForward, zi.forward.String())
		back = remRefs(back, zi.forward)
		updated = true
	}
	for k, refs := range zi.otherRefs {
		if len(refs.backward) > 0 {
			m.Set(k, refs.backward.String())
			back = remRefs(back, refs.backward)
			updated = true
		}
	}
	if len(back) > 0 {
		m.Set(api.KeyBack, back.String())
		updated = true
	}
	return updated
}

// SearchEqual returns all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SearchEqual(word string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := id.NewSet()
	if refs, ok := ms.words[word]; ok {
		result.CopySlice(refs)
	}
	if refs, ok := ms.urls[word]; ok {
		result.CopySlice(refs)
	}
	zid, err := id.Parse(word)
	if err != nil {
		return result
	}
	zi, ok := ms.idx[zid]
	if !ok {
		return result
	}

	addBackwardZids(result, zid, zi)
	return result
}

// SearchPrefix returns all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SearchPrefix(prefix string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(prefix, strings.HasPrefix)
	l := len(prefix)
	if l > 14 {
		return result
	}
	maxZid, err := id.Parse(prefix + "99999999999999"[:14-l])
	if err != nil {
		return result
	}
	var minZid id.Zid
	if l < 14 && prefix == "0000000000000"[:l] {
		minZid = id.Zid(1)
	} else {
		minZid, err = id.Parse(prefix + "00000000000000"[:14-l])
		if err != nil {
			return result
		}
	}
	for zid, zi := range ms.idx {
		if minZid <= zid && zid <= maxZid {
			addBackwardZids(result, zid, zi)
		}
	}
	return result
}

// SearchSuffix returns all zettel that have a word with the given suffix.
// The suffix must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SearchSuffix(suffix string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(suffix, strings.HasSuffix)
	l := len(suffix)
	if l > 14 {
		return result
	}
	val, err := id.ParseUint(suffix)
	if err != nil {
		return result
	}
	modulo := uint64(1)
	for i := 0; i < l; i++ {
		modulo *= 10
	}
	for zid, zi := range ms.idx {
		if uint64(zid)%modulo == val {
			addBackwardZids(result, zid, zi)
		}
	}
	return result
}

// SearchContains returns all zettel that contains the given string.
// The string must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SearchContains(s string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(s, strings.Contains)
	if len(s) > 14 {
		return result
	}
	if _, err := id.ParseUint(s); err != nil {
		return result
	}
	for zid, zi := range ms.idx {
		if strings.Contains(zid.String(), s) {
			addBackwardZids(result, zid, zi)
		}
	}
	return result
}

func (ms *memStore) selectWithPred(s string, pred func(string, string) bool) id.Set {
	// Must only be called if ms.mx is read-locked!
	result := id.NewSet()
	for word, refs := range ms.words {
		if !pred(word, s) {
			continue
		}
		result.CopySlice(refs)
	}
	for u, refs := range ms.urls {
		if !pred(u, s) {
			continue
		}
		result.CopySlice(refs)
	}
	return result
}

func addBackwardZids(result id.Set, zid id.Zid, zi *zettelData) {
	// Must only be called if ms.mx is read-locked!
	result.Add(zid)
	result.CopySlice(zi.backward)
	for _, mref := range zi.otherRefs {
		result.CopySlice(mref.backward)
	}
}

func removeOtherMetaRefs(m *meta.Meta, back id.Slice) id.Slice {
	for _, p := range m.PairsRest() {
		switch meta.Type(p.Key) {
		case meta.TypeID:
			if zid, err := id.Parse(p.Value); err == nil {
				back = remRef(back, zid)
			}
		case meta.TypeIDSet:
			for _, val := range meta.ListFromValue(p.Value) {
				if zid, err := id.Parse(val); err == nil {
					back = remRef(back, zid)
				}
			}
		}
	}
	return back
}

func (ms *memStore) UpdateReferences(_ context.Context, zidx *store.ZettelIndex) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()
	m := ms.makeMeta(zidx)
	zi, ziExist := ms.idx[zidx.Zid]
	if !ziExist || zi == nil {
		zi = &zettelData{}
		ziExist = false
	}

	// Is this zettel an old dead reference mentioned in other zettel?
	var toCheck id.Set
	if refs, ok := ms.dead[zidx.Zid]; ok {
		// These must be checked later again
		toCheck = id.NewSet(refs...)
		delete(ms.dead, zidx.Zid)
	}

	zi.meta = m
	ms.updateDeadReferences(zidx, zi)
	ids := ms.updateForwardBackwardReferences(zidx, zi)
	toCheck = toCheck.Copy(ids)
	ids = ms.updateMetadataReferences(zidx, zi)
	toCheck = toCheck.Copy(ids)
	zi.words = updateStrings(zidx.Zid, ms.words, zi.words, zidx.GetWords())
	zi.urls = updateStrings(zidx.Zid, ms.urls, zi.urls, zidx.GetUrls())

	// Check if zi must be inserted into ms.idx
	if !ziExist {
		ms.idx[zidx.Zid] = zi
	}

	return toCheck
}

var internableKeys = map[string]bool{
	api.KeyRole:      true,
	api.KeySyntax:    true,
	api.KeyFolgeRole: true,
	api.KeyLang:      true,
	api.KeyReadOnly:  true,
}

func isInternableValue(key string) bool {
	if internableKeys[key] {
		return true
	}
	return strings.HasSuffix(key, "-role")
}

func (ms *memStore) internString(s string) string {
	if is, found := ms.intern[s]; found {
		return is
	}
	ms.intern[s] = s
	return s
}

func (ms *memStore) makeMeta(zidx *store.ZettelIndex) *meta.Meta {
	origM := zidx.GetMeta()
	copyM := meta.New(origM.Zid)
	for _, p := range origM.Pairs() {
		key := ms.internString(p.Key)
		if isInternableValue(key) {
			copyM.Set(key, ms.internString(p.Value))
		} else if key == api.KeyBoxNumber || !meta.IsComputed(key) {
			copyM.Set(key, p.Value)
		}
	}
	return copyM
}

func (ms *memStore) updateDeadReferences(zidx *store.ZettelIndex, zi *zettelData) {
	// Must only be called if ms.mx is write-locked!
	drefs := zidx.GetDeadRefs()
	newRefs, remRefs := refsDiff(drefs, zi.dead)
	zi.dead = drefs
	for _, ref := range remRefs {
		ms.dead[ref] = remRef(ms.dead[ref], zidx.Zid)
	}
	for _, ref := range newRefs {
		ms.dead[ref] = addRef(ms.dead[ref], zidx.Zid)
	}
}

func (ms *memStore) updateForwardBackwardReferences(zidx *store.ZettelIndex, zi *zettelData) id.Set {
	// Must only be called if ms.mx is write-locked!
	brefs := zidx.GetBackRefs()
	newRefs, remRefs := refsDiff(brefs, zi.forward)
	zi.forward = brefs

	var toCheck id.Set
	for _, ref := range remRefs {
		bzi := ms.getOrCreateEntry(ref)
		bzi.backward = remRef(bzi.backward, zidx.Zid)
		if bzi.meta == nil {
			toCheck = toCheck.Add(ref)
		}
	}
	for _, ref := range newRefs {
		bzi := ms.getOrCreateEntry(ref)
		bzi.backward = addRef(bzi.backward, zidx.Zid)
		if bzi.meta == nil {
			toCheck = toCheck.Add(ref)
		}
	}
	return toCheck
}

func (ms *memStore) updateMetadataReferences(zidx *store.ZettelIndex, zi *zettelData) id.Set {
	// Must only be called if ms.mx is write-locked!
	inverseRefs := zidx.GetInverseRefs()
	for key, mr := range zi.otherRefs {
		if _, ok := inverseRefs[key]; ok {
			continue
		}
		ms.removeInverseMeta(zidx.Zid, key, mr.forward)
	}
	if zi.otherRefs == nil {
		zi.otherRefs = make(map[string]bidiRefs)
	}
	var toCheck id.Set
	for key, mrefs := range inverseRefs {
		mr := zi.otherRefs[key]
		newRefs, remRefs := refsDiff(mrefs, mr.forward)
		mr.forward = mrefs
		zi.otherRefs[key] = mr

		for _, ref := range newRefs {
			bzi := ms.getOrCreateEntry(ref)
			if bzi.otherRefs == nil {
				bzi.otherRefs = make(map[string]bidiRefs)
			}
			bmr := bzi.otherRefs[key]
			bmr.backward = addRef(bmr.backward, zidx.Zid)
			bzi.otherRefs[key] = bmr
			if bzi.meta == nil {
				toCheck = toCheck.Add(ref)
			}
		}
		ms.removeInverseMeta(zidx.Zid, key, remRefs)
	}
	return toCheck
}

func updateStrings(zid id.Zid, srefs stringRefs, prev []string, next store.WordSet) []string {
	newWords, removeWords := next.Diff(prev)
	for _, word := range newWords {
		if refs, ok := srefs[word]; ok {
			srefs[word] = addRef(refs, zid)
			continue
		}
		srefs[word] = id.Slice{zid}
	}
	for _, word := range removeWords {
		refs, ok := srefs[word]
		if !ok {
			continue
		}
		refs2 := remRef(refs, zid)
		if len(refs2) == 0 {
			delete(srefs, word)
			continue
		}
		srefs[word] = refs2
	}
	return next.Words()
}

func (ms *memStore) getOrCreateEntry(zid id.Zid) *zettelData {
	// Must only be called if ms.mx is write-locked!
	if zi, ok := ms.idx[zid]; ok {
		return zi
	}
	zi := &zettelData{}
	ms.idx[zid] = zi
	return zi
}

func (ms *memStore) RenameZettel(_ context.Context, curZid, newZid id.Zid) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()

	curZi, curFound := ms.idx[curZid]
	_, newFound := ms.idx[newZid]
	if !curFound || newFound {
		return nil
	}
	newZi := &zettelData{
		meta:      copyMeta(curZi.meta, newZid),
		dead:      ms.copyDeadReferences(curZi.dead),
		forward:   ms.copyForward(curZi.forward, newZid),
		backward:  nil, // will be done through tocheck
		otherRefs: nil, // TODO: check if this will be done through toCheck
		words:     copyStrings(ms.words, curZi.words, newZid),
		urls:      copyStrings(ms.urls, curZi.urls, newZid),
	}

	ms.idx[newZid] = newZi
	toCheck := ms.doDeleteZettel(curZid)
	toCheck = toCheck.CopySlice(ms.dead[newZid])
	delete(ms.dead, newZid)
	toCheck = toCheck.Add(newZid) // should update otherRefs
	return toCheck
}
func copyMeta(m *meta.Meta, newZid id.Zid) *meta.Meta {
	result := m.Clone()
	result.Zid = newZid
	return result
}
func (ms *memStore) copyDeadReferences(curDead id.Slice) id.Slice {
	// Must only be called if ms.mx is write-locked!
	if l := len(curDead); l > 0 {
		result := make(id.Slice, l)
		for i, ref := range curDead {
			result[i] = ref
			ms.dead[ref] = addRef(ms.dead[ref], ref)
		}
		return result
	}
	return nil
}
func (ms *memStore) copyForward(curForward id.Slice, newZid id.Zid) id.Slice {
	// Must only be called if ms.mx is write-locked!
	if l := len(curForward); l > 0 {
		result := make(id.Slice, l)
		for i, ref := range curForward {
			result[i] = ref
			if fzi, found := ms.idx[ref]; found {
				fzi.backward = addRef(fzi.backward, newZid)
			}
		}
		return result
	}
	return nil
}
func copyStrings(msStringMap stringRefs, curStrings []string, newZid id.Zid) []string {
	// Must only be called if ms.mx is write-locked!
	if l := len(curStrings); l > 0 {
		result := make([]string, l)
		for i, s := range curStrings {
			result[i] = s
			msStringMap[s] = addRef(msStringMap[s], newZid)
		}
		return result
	}
	return nil
}

func (ms *memStore) DeleteZettel(_ context.Context, zid id.Zid) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()
	return ms.doDeleteZettel(zid)
}

func (ms *memStore) doDeleteZettel(zid id.Zid) id.Set {
	// Must only be called if ms.mx is write-locked!
	zi, ok := ms.idx[zid]
	if !ok {
		return nil
	}

	ms.deleteDeadSources(zid, zi)
	toCheck := ms.deleteForwardBackward(zid, zi)
	for key, mrefs := range zi.otherRefs {
		ms.removeInverseMeta(zid, key, mrefs.forward)
	}
	deleteStrings(ms.words, zi.words, zid)
	deleteStrings(ms.urls, zi.urls, zid)
	delete(ms.idx, zid)
	return toCheck
}

func (ms *memStore) deleteDeadSources(zid id.Zid, zi *zettelData) {
	// Must only be called if ms.mx is write-locked!
	for _, ref := range zi.dead {
		if drefs, ok := ms.dead[ref]; ok {
			drefs = remRef(drefs, zid)
			if len(drefs) > 0 {
				ms.dead[ref] = drefs
			} else {
				delete(ms.dead, ref)
			}
		}
	}
}

func (ms *memStore) deleteForwardBackward(zid id.Zid, zi *zettelData) id.Set {
	// Must only be called if ms.mx is write-locked!
	for _, ref := range zi.forward {
		if fzi, ok := ms.idx[ref]; ok {
			fzi.backward = remRef(fzi.backward, zid)
		}
	}
	var toCheck id.Set
	for _, ref := range zi.backward {
		if bzi, ok := ms.idx[ref]; ok {
			bzi.forward = remRef(bzi.forward, zid)
			toCheck = toCheck.Add(ref)
		}
	}
	return toCheck
}

func (ms *memStore) removeInverseMeta(zid id.Zid, key string, forward id.Slice) {
	// Must only be called if ms.mx is write-locked!
	for _, ref := range forward {
		bzi, ok := ms.idx[ref]
		if !ok || bzi.otherRefs == nil {
			continue
		}
		bmr, ok := bzi.otherRefs[key]
		if !ok {
			continue
		}
		bmr.backward = remRef(bmr.backward, zid)
		if len(bmr.backward) > 0 || len(bmr.forward) > 0 {
			bzi.otherRefs[key] = bmr
		} else {
			delete(bzi.otherRefs, key)
			if len(bzi.otherRefs) == 0 {
				bzi.otherRefs = nil
			}
		}
	}
}

func deleteStrings(msStringMap stringRefs, curStrings []string, zid id.Zid) {
	// Must only be called if ms.mx is write-locked!
	for _, word := range curStrings {
		refs, ok := msStringMap[word]
		if !ok {
			continue
		}
		refs2 := remRef(refs, zid)
		if len(refs2) == 0 {
			delete(msStringMap, word)
			continue
		}
		msStringMap[word] = refs2
	}
}

func (ms *memStore) ReadStats(st *store.Stats) {
	ms.mx.RLock()
	st.Zettel = len(ms.idx)
	st.Words = uint64(len(ms.words))
	st.Urls = uint64(len(ms.urls))
	ms.mx.RUnlock()
	ms.mxStats.Lock()
	st.Updates = ms.updates
	ms.mxStats.Unlock()
}

func (ms *memStore) Dump(w io.Writer) {
	ms.mx.RLock()
	defer ms.mx.RUnlock()

	io.WriteString(w, "=== Dump\n")
	ms.dumpIndex(w)
	ms.dumpDead(w)
	dumpStringRefs(w, "Words", "", "", ms.words)
	dumpStringRefs(w, "URLs", "[[", "]]", ms.urls)
}

func (ms *memStore) dumpIndex(w io.Writer) {
	if len(ms.idx) == 0 {
		return
	}
	io.WriteString(w, "==== Zettel Index\n")
	zids := make(id.Slice, 0, len(ms.idx))
	for id := range ms.idx {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, "=====", id)
		zi := ms.idx[id]
		if len(zi.dead) > 0 {
			fmt.Fprintln(w, "* Dead:", zi.dead)
		}
		dumpZids(w, "* Forward:", zi.forward)
		dumpZids(w, "* Backward:", zi.backward)
		for k, fb := range zi.otherRefs {
			fmt.Fprintln(w, "* Meta", k)
			dumpZids(w, "** Forward:", fb.forward)
			dumpZids(w, "** Backward:", fb.backward)
		}
		dumpStrings(w, "* Words", "", "", zi.words)
		dumpStrings(w, "* URLs", "[[", "]]", zi.urls)
	}
}

func (ms *memStore) dumpDead(w io.Writer) {
	if len(ms.dead) == 0 {
		return
	}
	fmt.Fprintf(w, "==== Dead References\n")
	zids := make(id.Slice, 0, len(ms.dead))
	for id := range ms.dead {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, ";", id)
		fmt.Fprintln(w, ":", ms.dead[id])
	}
}

func dumpZids(w io.Writer, prefix string, zids id.Slice) {
	if len(zids) > 0 {
		io.WriteString(w, prefix)
		for _, zid := range zids {
			io.WriteString(w, " ")
			w.Write(zid.Bytes())
		}
		fmt.Fprintln(w)
	}
}

func dumpStrings(w io.Writer, title, preString, postString string, slice []string) {
	if len(slice) > 0 {
		sl := make([]string, len(slice))
		copy(sl, slice)
		sort.Strings(sl)
		fmt.Fprintln(w, title)
		for _, s := range sl {
			fmt.Fprintf(w, "** %s%s%s\n", preString, s, postString)
		}
	}

}

func dumpStringRefs(w io.Writer, title, preString, postString string, srefs stringRefs) {
	if len(srefs) == 0 {
		return
	}
	fmt.Fprintln(w, "====", title)
	for _, s := range maps.Keys(srefs) {
		fmt.Fprintf(w, "; %s%s%s\n", preString, s, postString)
		fmt.Fprintln(w, ":", srefs[s])
	}
}
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<




















































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































Deleted box/manager/memstore/refs.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package memstore

import (
	"slices"

	"zettelstore.de/z/zettel/id"
)

func refsDiff(refsN, refsO id.Slice) (newRefs, remRefs id.Slice) {
	npos, opos := 0, 0
	for npos < len(refsN) && opos < len(refsO) {
		rn, ro := refsN[npos], refsO[opos]
		if rn == ro {
			npos++
			opos++
			continue
		}
		if rn < ro {
			newRefs = append(newRefs, rn)
			npos++
			continue
		}
		remRefs = append(remRefs, ro)
		opos++
	}
	if npos < len(refsN) {
		newRefs = append(newRefs, refsN[npos:]...)
	}
	if opos < len(refsO) {
		remRefs = append(remRefs, refsO[opos:]...)
	}
	return newRefs, remRefs
}

func addRef(refs id.Slice, ref id.Zid) id.Slice {
	hi := len(refs)
	for lo := 0; lo < hi; {
		m := lo + (hi-lo)/2
		if r := refs[m]; r == ref {
			return refs
		} else if r < ref {
			lo = m + 1
		} else {
			hi = m
		}
	}
	refs = slices.Insert(refs, hi, ref)
	return refs
}

func remRefs(refs, rem id.Slice) id.Slice {
	if len(refs) == 0 || len(rem) == 0 {
		return refs
	}
	result := make(id.Slice, 0, len(refs))
	rpos, dpos := 0, 0
	for rpos < len(refs) && dpos < len(rem) {
		rr, dr := refs[rpos], rem[dpos]
		if rr < dr {
			result = append(result, rr)
			rpos++
			continue
		}
		if dr < rr {
			dpos++
			continue
		}
		rpos++
		dpos++
	}
	if rpos < len(refs) {
		result = append(result, refs[rpos:]...)
	}
	return result
}

func remRef(refs id.Slice, ref id.Zid) id.Slice {
	hi := len(refs)
	for lo := 0; lo < hi; {
		m := lo + (hi-lo)/2
		if r := refs[m]; r == ref {
			copy(refs[m:], refs[m+1:])
			refs = refs[:len(refs)-1]
			return refs
		} else if r < ref {
			lo = m + 1
		} else {
			hi = m
		}
	}
	return refs
}
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<












































































































































































































Deleted box/manager/memstore/refs_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package memstore

import (
	"testing"

	"zettelstore.de/z/zettel/id"
)

func assertRefs(t *testing.T, i int, got, exp id.Slice) {
	t.Helper()
	if got == nil && exp != nil {
		t.Errorf("%d: got nil, but expected %v", i, exp)
		return
	}
	if got != nil && exp == nil {
		t.Errorf("%d: expected nil, but got %v", i, got)
		return
	}
	if len(got) != len(exp) {
		t.Errorf("%d: expected len(%v)==%d, but got len(%v)==%d", i, exp, len(exp), got, len(got))
		return
	}
	for p, n := range exp {
		if got := got[p]; got != id.Zid(n) {
			t.Errorf("%d: pos %d: expected %d, but got %d", i, p, n, got)
		}
	}
}

func TestRefsDiff(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		in1, in2   id.Slice
		exp1, exp2 id.Slice
	}{
		{nil, nil, nil, nil},
		{id.Slice{1}, nil, id.Slice{1}, nil},
		{nil, id.Slice{1}, nil, id.Slice{1}},
		{id.Slice{1}, id.Slice{1}, nil, nil},
		{id.Slice{1, 2}, id.Slice{1}, id.Slice{2}, nil},
		{id.Slice{1, 2}, id.Slice{1, 3}, id.Slice{2}, id.Slice{3}},
		{id.Slice{1, 4}, id.Slice{1, 3}, id.Slice{4}, id.Slice{3}},
	}
	for i, tc := range testcases {
		got1, got2 := refsDiff(tc.in1, tc.in2)
		assertRefs(t, i, got1, tc.exp1)
		assertRefs(t, i, got2, tc.exp2)
	}
}

func TestAddRef(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		ref id.Slice
		zid uint
		exp id.Slice
	}{
		{nil, 5, id.Slice{5}},
		{id.Slice{1}, 5, id.Slice{1, 5}},
		{id.Slice{10}, 5, id.Slice{5, 10}},
		{id.Slice{5}, 5, id.Slice{5}},
		{id.Slice{1, 10}, 5, id.Slice{1, 5, 10}},
		{id.Slice{1, 5, 10}, 5, id.Slice{1, 5, 10}},
	}
	for i, tc := range testcases {
		got := addRef(tc.ref, id.Zid(tc.zid))
		assertRefs(t, i, got, tc.exp)
	}
}

func TestRemRefs(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		in1, in2 id.Slice
		exp      id.Slice
	}{
		{nil, nil, nil},
		{nil, id.Slice{}, nil},
		{id.Slice{}, nil, id.Slice{}},
		{id.Slice{}, id.Slice{}, id.Slice{}},
		{id.Slice{1}, id.Slice{5}, id.Slice{1}},
		{id.Slice{10}, id.Slice{5}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{5}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{5}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{5}, id.Slice{1, 10}},
		{id.Slice{1}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 2, 5}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{2, 5, 10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{2, 5}, id.Slice{1, 10}},
		{id.Slice{1}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 5, 9}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{5, 9, 10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{5, 9}, id.Slice{1, 10}},
	}
	for i, tc := range testcases {
		got := remRefs(tc.in1, tc.in2)
		assertRefs(t, i, got, tc.exp)
	}
}

func TestRemRef(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		ref id.Slice
		zid uint
		exp id.Slice
	}{
		{nil, 5, nil},
		{id.Slice{}, 5, id.Slice{}},
		{id.Slice{5}, 5, id.Slice{}},
		{id.Slice{1}, 5, id.Slice{1}},
		{id.Slice{10}, 5, id.Slice{10}},
		{id.Slice{1, 5}, 5, id.Slice{1}},
		{id.Slice{5, 10}, 5, id.Slice{10}},
		{id.Slice{1, 5, 10}, 5, id.Slice{1, 10}},
	}
	for i, tc := range testcases {
		got := remRef(tc.ref, id.Zid(tc.zid))
		assertRefs(t, i, got, tc.exp)
	}
}
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<


















































































































































































































































































Changes to box/manager/store/store.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package store contains general index data for storing a zettel index.
package store

import (
	"context"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package store contains general index data for storing a zettel index.
package store

import (
	"context"
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59



60
61
62
63
64
65
66
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)

	// Entrich metadata with data from store.
	Enrich(ctx context.Context, m *meta.Meta)

	// UpdateReferences for a specific zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	UpdateReferences(context.Context, *ZettelIndex) id.Set

	// RenameZettel changes all references of current zettel identifier to new
	// zettel identifier.
	RenameZettel(_ context.Context, curZid, newZid id.Zid) id.Set

	// DeleteZettel removes index data for given zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	DeleteZettel(context.Context, id.Zid) id.Set




	// ReadStats populates st with store statistics.
	ReadStats(st *Stats)

	// Dump the content to a Writer.
	Dump(io.Writer)
}







|
<
<
<
<



|
>
>
>







47
48
49
50
51
52
53
54




55
56
57
58
59
60
61
62
63
64
65
66
67
68
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)

	// Entrich metadata with data from store.
	Enrich(ctx context.Context, m *meta.Meta)

	// UpdateReferences for a specific zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	UpdateReferences(context.Context, *ZettelIndex) *id.Set





	// DeleteZettel removes index data for given zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	DeleteZettel(context.Context, id.Zid) *id.Set

	// Optimize removes unneeded space.
	Optimize()

	// ReadStats populates st with store statistics.
	ReadStats(st *Stats)

	// Dump the content to a Writer.
	Dump(io.Writer)
}

Changes to box/manager/store/wordset.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package store

// WordSet contains the set of all words, with the count of their occurrences.
type WordSet map[string]int









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package store

// WordSet contains the set of all words, with the count of their occurrences.
type WordSet map[string]int

Changes to box/manager/store/wordset_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package store_test

import (
	"sort"
	"testing"

	"zettelstore.de/z/box/manager/store"
)

func equalWordList(exp, got []string) bool {
	if len(exp) != len(got) {
		return false
	}
	if len(got) == 0 {
		return len(exp) == 0
	}
	sort.Strings(got)
	for i, w := range exp {
		if w != got[i] {
			return false
		}
	}
	return true
}








>
>
>





|












|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package store_test

import (
	"slices"
	"testing"

	"zettelstore.de/z/box/manager/store"
)

func equalWordList(exp, got []string) bool {
	if len(exp) != len(got) {
		return false
	}
	if len(got) == 0 {
		return len(exp) == 0
	}
	slices.Sort(got)
	for i, w := range exp {
		if w != got[i] {
			return false
		}
	}
	return true
}

Changes to box/manager/store/zettel.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package store

import (
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ZettelIndex contains all index data of a zettel.
type ZettelIndex struct {
	Zid         id.Zid            // zid of the indexed zettel
	meta        *meta.Meta        // full metadata
	backrefs    id.Set            // set of back references
	inverseRefs map[string]id.Set // references of inverse keys
	deadrefs    id.Set            // set of dead references
	words       WordSet
	urls        WordSet
}

// NewZettelIndex creates a new zettel index.
func NewZettelIndex(m *meta.Meta) *ZettelIndex {
	return &ZettelIndex{
		Zid:         m.Zid,
		meta:        m,
		backrefs:    id.NewSet(),
		inverseRefs: make(map[string]id.Set),
		deadrefs:    id.NewSet(),
	}
}

// AddBackRef adds a reference to a zettel where the current zettel links to
// without any more information.
func (zi *ZettelIndex) AddBackRef(zid id.Zid) {
	zi.backrefs.Add(zid)
}

// AddInverseRef adds a named reference to a zettel. On that zettel, the given
// metadata key should point back to the current zettel.
func (zi *ZettelIndex) AddInverseRef(key string, zid id.Zid) {
	if zids, ok := zi.inverseRefs[key]; ok {
		zids.Add(zid)
		return








>
>
>











|
|
|
|
|










|






|
<
<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45


46
47
48
49
50
51
52
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package store

import (
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ZettelIndex contains all index data of a zettel.
type ZettelIndex struct {
	Zid         id.Zid             // zid of the indexed zettel
	meta        *meta.Meta         // full metadata
	backrefs    *id.Set            // set of back references
	inverseRefs map[string]*id.Set // references of inverse keys
	deadrefs    *id.Set            // set of dead references
	words       WordSet
	urls        WordSet
}

// NewZettelIndex creates a new zettel index.
func NewZettelIndex(m *meta.Meta) *ZettelIndex {
	return &ZettelIndex{
		Zid:         m.Zid,
		meta:        m,
		backrefs:    id.NewSet(),
		inverseRefs: make(map[string]*id.Set),
		deadrefs:    id.NewSet(),
	}
}

// AddBackRef adds a reference to a zettel where the current zettel links to
// without any more information.
func (zi *ZettelIndex) AddBackRef(zid id.Zid) { zi.backrefs.Add(zid) }



// AddInverseRef adds a named reference to a zettel. On that zettel, the given
// metadata key should point back to the current zettel.
func (zi *ZettelIndex) AddInverseRef(key string, zid id.Zid) {
	if zids, ok := zi.inverseRefs[key]; ok {
		zids.Add(zid)
		return
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
// SetWords sets the words to the given value.
func (zi *ZettelIndex) SetWords(words WordSet) { zi.words = words }

// SetUrls sets the words to the given value.
func (zi *ZettelIndex) SetUrls(urls WordSet) { zi.urls = urls }

// GetDeadRefs returns all dead references as a sorted list.
func (zi *ZettelIndex) GetDeadRefs() id.Slice { return zi.deadrefs.Sorted() }

// GetMeta return just the raw metadata.
func (zi *ZettelIndex) GetMeta() *meta.Meta { return zi.meta }

// GetBackRefs returns all back references as a sorted list.
func (zi *ZettelIndex) GetBackRefs() id.Slice { return zi.backrefs.Sorted() }

// GetInverseRefs returns all inverse meta references as a map of strings to a sorted list of references
func (zi *ZettelIndex) GetInverseRefs() map[string]id.Slice {
	if len(zi.inverseRefs) == 0 {
		return nil
	}
	result := make(map[string]id.Slice, len(zi.inverseRefs))
	for key, refs := range zi.inverseRefs {
		result[key] = refs.Sorted()
	}
	return result
}

// GetWords returns a reference to the set of words. It must not be modified.
func (zi *ZettelIndex) GetWords() WordSet { return zi.words }

// GetUrls returns a reference to the set of URLs. It must not be modified.
func (zi *ZettelIndex) GetUrls() WordSet { return zi.urls }







|





|


|



|

|









62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
// SetWords sets the words to the given value.
func (zi *ZettelIndex) SetWords(words WordSet) { zi.words = words }

// SetUrls sets the words to the given value.
func (zi *ZettelIndex) SetUrls(urls WordSet) { zi.urls = urls }

// GetDeadRefs returns all dead references as a sorted list.
func (zi *ZettelIndex) GetDeadRefs() *id.Set { return zi.deadrefs }

// GetMeta return just the raw metadata.
func (zi *ZettelIndex) GetMeta() *meta.Meta { return zi.meta }

// GetBackRefs returns all back references as a sorted list.
func (zi *ZettelIndex) GetBackRefs() *id.Set { return zi.backrefs }

// GetInverseRefs returns all inverse meta references as a map of strings to a sorted list of references
func (zi *ZettelIndex) GetInverseRefs() map[string]*id.Set {
	if len(zi.inverseRefs) == 0 {
		return nil
	}
	result := make(map[string]*id.Set, len(zi.inverseRefs))
	for key, refs := range zi.inverseRefs {
		result[key] = refs
	}
	return result
}

// GetWords returns a reference to the set of words. It must not be modified.
func (zi *ZettelIndex) GetWords() WordSet { return zi.words }

// GetUrls returns a reference to the set of URLs. It must not be modified.
func (zi *ZettelIndex) GetUrls() WordSet { return zi.urls }

Added box/manager/zidmapper.go.













































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"maps"
	"sync"
	"time"

	"zettelstore.de/z/zettel/id"
)

// zidMapper transforms old-style zettel identifier (14 digits) into new one (4 alphanums).
//
// Since there are no new-style identifier defined, there is only support for old-style
// identifier by checking, whether they are suported as new-style or not.
//
// This will change in later versions.
type zidMapper struct {
	fetcher   zidfetcher
	defined   map[id.Zid]id.ZidN // predefined mapping, constant after creation
	mx        sync.RWMutex       // protect toNew ... nextZidN
	toNew     map[id.Zid]id.ZidN // working mapping old->new
	toOld     map[id.ZidN]id.Zid // working mapping new->old
	nextZidM  id.ZidN            // next zid for manual
	hadManual bool
	nextZidN  id.ZidN // next zid for normal zettel
}

type zidfetcher interface {
	fetchZids(context.Context) (*id.Set, error)
}

// NewZidMapper creates a new ZipMapper.
func NewZidMapper(fetcher zidfetcher) *zidMapper {
	defined := map[id.Zid]id.ZidN{
		id.Invalid: id.InvalidN,
		1:          id.MustParseN("0001"), // ZidVersion
		2:          id.MustParseN("0002"), // ZidHost
		3:          id.MustParseN("0003"), // ZidOperatingSystem
		4:          id.MustParseN("0004"), // ZidLicense
		5:          id.MustParseN("0005"), // ZidAuthors
		6:          id.MustParseN("0006"), // ZidDependencies
		7:          id.MustParseN("0007"), // ZidLog
		8:          id.MustParseN("0008"), // ZidMemory
		9:          id.MustParseN("0009"), // ZidSx
		10:         id.MustParseN("000a"), // ZidHTTP
		11:         id.MustParseN("000b"), // ZidAPI
		12:         id.MustParseN("000c"), // ZidWebUI
		13:         id.MustParseN("000d"), // ZidConsole
		20:         id.MustParseN("000e"), // ZidBoxManager
		21:         id.MustParseN("000f"), // ZidZettel
		22:         id.MustParseN("000g"), // ZidIndex
		23:         id.MustParseN("000h"), // ZidQuery
		90:         id.MustParseN("000i"), // ZidMetadataKey
		92:         id.MustParseN("000j"), // ZidParser
		96:         id.MustParseN("000k"), // ZidStartupConfiguration
		100:        id.MustParseN("000l"), // ZidRuntimeConfiguration
		101:        id.MustParseN("000m"), // ZidDirectory
		102:        id.MustParseN("000n"), // ZidWarnings
		10100:      id.MustParseN("000s"), // Base HTML Template
		10200:      id.MustParseN("000t"), // Login Form Template
		10300:      id.MustParseN("000u"), // List Zettel Template
		10401:      id.MustParseN("000v"), // Detail Template
		10402:      id.MustParseN("000w"), // Info Template
		10403:      id.MustParseN("000x"), // Form Template
		10405:      id.MustParseN("000y"), // Delete Template
		10700:      id.MustParseN("000z"), // Error Template
		19000:      id.MustParseN("000q"), // Sxn Start Code
		19990:      id.MustParseN("000r"), // Sxn Base Code
		20001:      id.MustParseN("0010"), // Base CSS
		25001:      id.MustParseN("0011"), // User CSS
		40001:      id.MustParseN("000o"), // Generic Emoji
		59900:      id.MustParseN("000p"), // Sxn Prelude
		60010:      id.MustParseN("0012"), // zettel
		60020:      id.MustParseN("0013"), // confguration
		60030:      id.MustParseN("0014"), // role
		60040:      id.MustParseN("0015"), // tag
		90000:      id.MustParseN("0016"), // New Menu
		90001:      id.MustParseN("0017"), // New Zettel
		90002:      id.MustParseN("0018"), // New User
		90003:      id.MustParseN("0019"), // New Tag
		90004:      id.MustParseN("001a"), // New Role
		// 100000000,   // Manual               -> 0020-00yz
		9999999997:  id.MustParseN("00zx"), // ZidSession
		9999999998:  id.MustParseN("00zy"), // ZidAppDirectory
		9999999999:  id.MustParseN("00zz"), // ZidMapping
		10000000000: id.MustParseN("0100"), // ZidDefaultHome
	}
	toNew := maps.Clone(defined)
	toOld := make(map[id.ZidN]id.Zid, len(toNew))
	for o, n := range toNew {
		if _, found := toOld[n]; found {
			panic("duplicate predefined zid")
		}
		toOld[n] = o
	}

	return &zidMapper{
		fetcher:   fetcher,
		defined:   defined,
		toNew:     toNew,
		toOld:     toOld,
		nextZidM:  id.MustParseN("0020"),
		hadManual: false,
		nextZidN:  id.MustParseN("0101"),
	}
}

// isWellDefined returns true, if the given zettel identifier is predefined
// (as stated in the manual), or is part of the manual itself, or is greater than
// 19699999999999.
func (zm *zidMapper) isWellDefined(zid id.Zid) bool {
	if _, found := zm.defined[zid]; found || (1000000000 <= zid && zid <= 1099999999) {
		return true
	}
	if _, err := time.Parse("20060102150405", zid.String()); err != nil {
		return false
	}
	return 19700000000000 <= zid
}

// Warnings returns all zettel identifier with warnings.
func (zm *zidMapper) Warnings(ctx context.Context) (*id.Set, error) {
	allZids, err := zm.fetcher.fetchZids(ctx)
	if err != nil {
		return nil, err
	}
	warnings := id.NewSet()
	allZids.ForEach(func(zid id.Zid) {
		if !zm.isWellDefined(zid) {
			warnings = warnings.Add(zid)
		}
	})
	return warnings, nil
}

func (zm *zidMapper) GetZidN(zidO id.Zid) id.ZidN {
	zm.mx.RLock()
	if zidN, found := zm.toNew[zidO]; found {
		zm.mx.RUnlock()
		return zidN
	}
	zm.mx.RUnlock()

	zm.mx.Lock()
	defer zm.mx.Unlock()
	// Double check to avoid races
	if zidN, found := zm.toNew[zidO]; found {
		return zidN
	}

	if 1000000000 <= zidO && zidO <= 1099999999 {
		if zidO == 1000000000 {
			zm.hadManual = true
		}
		if zm.hadManual {
			zidN := zm.nextZidM
			zm.nextZidM++
			zm.toNew[zidO] = zidN
			zm.toOld[zidN] = zidO
			return zidN
		}
	}

	zidN := zm.nextZidN
	zm.nextZidN++
	zm.toNew[zidO] = zidN
	zm.toOld[zidN] = zidO
	return zidN
}

// OldToNewMapping returns the mapping of old format identifier to new format identifier.
func (zm *zidMapper) OldToNewMapping(ctx context.Context) (map[id.Zid]id.ZidN, error) {
	allZids, err := zm.fetcher.fetchZids(ctx)
	if err != nil {
		return nil, err
	}

	result := make(map[id.Zid]id.ZidN, allZids.Length())
	allZids.ForEach(func(zidO id.Zid) {
		zidN := zm.GetZidN(zidO)
		result[zidO] = zidN
	})
	return result, nil
}

Changes to box/membox/membox.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package membox stores zettel volatile in main memory.
package membox

import (
	"context"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package membox stores zettel volatile in main memory.
package membox

import (
	"context"
108
109
110
111
112
113
114

115
116
117
118
119
120
121
	}
	meta := zettel.Meta.Clone()
	meta.Zid = zid
	zettel.Meta = meta
	mb.zettel[zid] = zettel
	mb.curBytes = newBytes
	mb.mx.Unlock()

	mb.notifyChanged(zid)
	mb.log.Trace().Zid(zid).Msg("CreateZettel")
	return zid, nil
}

func (mb *memBox) GetZettel(_ context.Context, zid id.Zid) (zettel.Zettel, error) {
	mb.mx.RLock()







>







111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
	}
	meta := zettel.Meta.Clone()
	meta.Zid = zid
	zettel.Meta = meta
	mb.zettel[zid] = zettel
	mb.curBytes = newBytes
	mb.mx.Unlock()

	mb.notifyChanged(zid)
	mb.log.Trace().Zid(zid).Msg("CreateZettel")
	return zid, nil
}

func (mb *memBox) GetZettel(_ context.Context, zid id.Zid) (zettel.Zettel, error) {
	mb.mx.RLock()
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
	mb.curBytes = newBytes
	mb.mx.Unlock()
	mb.notifyChanged(m.Zid)
	mb.log.Trace().Msg("UpdateZettel")
	return nil
}

func (*memBox) AllowRenameZettel(context.Context, id.Zid) bool { return true }

func (mb *memBox) RenameZettel(_ context.Context, curZid, newZid id.Zid) error {
	mb.mx.Lock()
	zettel, ok := mb.zettel[curZid]
	if !ok {
		mb.mx.Unlock()
		return box.ErrZettelNotFound{Zid: curZid}
	}

	// Check that there is no zettel with newZid
	if _, ok = mb.zettel[newZid]; ok {
		mb.mx.Unlock()
		return box.ErrInvalidZid{Zid: newZid.String()}
	}

	meta := zettel.Meta.Clone()
	meta.Zid = newZid
	zettel.Meta = meta
	mb.zettel[newZid] = zettel
	delete(mb.zettel, curZid)
	mb.mx.Unlock()
	mb.notifyChanged(curZid)
	mb.notifyChanged(newZid)
	mb.log.Trace().Msg("RenameZettel")
	return nil
}

func (mb *memBox) CanDeleteZettel(_ context.Context, zid id.Zid) bool {
	mb.mx.RLock()
	_, ok := mb.zettel[zid]
	mb.mx.RUnlock()
	return ok
}








<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







202
203
204
205
206
207
208




























209
210
211
212
213
214
215
	mb.curBytes = newBytes
	mb.mx.Unlock()
	mb.notifyChanged(m.Zid)
	mb.log.Trace().Msg("UpdateZettel")
	return nil
}





























func (mb *memBox) CanDeleteZettel(_ context.Context, zid id.Zid) bool {
	mb.mx.RLock()
	_, ok := mb.zettel[zid]
	mb.mx.RUnlock()
	return ok
}

Changes to box/notify/directory.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package notify

import (
	"errors"
	"fmt"
	"path/filepath"
	"regexp"
	"strings"
	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/query"








>
>
>









<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package notify

import (
	"errors"
	"fmt"
	"path/filepath"
	"regexp"

	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/query"
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
	if ds.entries == nil {
		return ds.logMissingEntry("update")
	}
	ds.entries[entry.Zid] = &entry
	return nil
}

// RenameDirEntry replaces an existing directory entry with a new one.
func (ds *DirService) RenameDirEntry(oldEntry *DirEntry, newZid id.Zid) (DirEntry, error) {
	ds.mx.Lock()
	defer ds.mx.Unlock()
	if ds.entries == nil {
		return DirEntry{}, ds.logMissingEntry("rename")
	}
	if _, found := ds.entries[newZid]; found {
		return DirEntry{}, box.ErrInvalidZid{Zid: newZid.String()}
	}
	oldZid := oldEntry.Zid
	newEntry := DirEntry{
		Zid:         newZid,
		MetaName:    renameFilename(oldEntry.MetaName, oldZid, newZid),
		ContentName: renameFilename(oldEntry.ContentName, oldZid, newZid),
		ContentExt:  oldEntry.ContentExt,
		// Duplicates must not be set, because duplicates will be deleted
	}
	delete(ds.entries, oldZid)
	ds.entries[newZid] = &newEntry
	return newEntry, nil
}

func renameFilename(name string, curID, newID id.Zid) string {
	if cur := curID.String(); strings.HasPrefix(name, cur) {
		name = newID.String() + name[len(cur):]
	}
	return name
}

// DeleteDirEntry removes a entry from the directory.
func (ds *DirService) DeleteDirEntry(zid id.Zid) error {
	ds.mx.Lock()
	defer ds.mx.Unlock()
	if ds.entries == nil {
		return ds.logMissingEntry("delete")
	}







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







181
182
183
184
185
186
187






























188
189
190
191
192
193
194
	if ds.entries == nil {
		return ds.logMissingEntry("update")
	}
	ds.entries[entry.Zid] = &entry
	return nil
}































// DeleteDirEntry removes a entry from the directory.
func (ds *DirService) DeleteDirEntry(zid id.Zid) error {
	ds.mx.Lock()
	defer ds.mx.Unlock()
	if ds.entries == nil {
		return ds.logMissingEntry("delete")
	}
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
		return nil, false
	}

	switch ev.Op {
	case Error:
		newEntries = nil
		if state != DsMissing {
			ds.log.Warn().Err(ev.Err).Msg("Notifier confused")
		}
	case Make:
		newEntries = make(entrySet)
	case List:
		if ev.Name == "" {
			zids := getNewZids(newEntries)
			ds.mx.Lock()







|







225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
		return nil, false
	}

	switch ev.Op {
	case Error:
		newEntries = nil
		if state != DsMissing {
			ds.log.Error().Err(ev.Err).Msg("Notifier confused")
		}
	case Make:
		newEntries = make(entrySet)
	case List:
		if ev.Name == "" {
			zids := getNewZids(newEntries)
			ds.mx.Lock()
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
		ds.mx.Lock()
		zid := ds.onDeleteFileEvent(ds.entries, ev.Name)
		ds.mx.Unlock()
		if zid != id.Invalid {
			ds.notifyChange(zid)
		}
	default:
		ds.log.Warn().Str("event", fmt.Sprintf("%v", ev)).Msg("Unknown zettel notification event")
	}
	return newEntries, true
}

func getNewZids(entries entrySet) id.Slice {
	zids := make(id.Slice, 0, len(entries))
	for zid := range entries {







|







266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
		ds.mx.Lock()
		zid := ds.onDeleteFileEvent(ds.entries, ev.Name)
		ds.mx.Unlock()
		if zid != id.Invalid {
			ds.notifyChange(zid)
		}
	default:
		ds.log.Error().Str("event", fmt.Sprintf("%v", ev)).Msg("Unknown zettel notification event")
	}
	return newEntries, true
}

func getNewZids(entries entrySet) id.Slice {
	zids := make(id.Slice, 0, len(entries))
	for zid := range entries {
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
	zid := seekZid(name)
	if zid == id.Invalid {
		return id.Invalid
	}
	entry := fetchdirEntry(entries, zid)
	dupName1, dupName2 := ds.updateEntry(entry, name)
	if dupName1 != "" {
		ds.log.Warn().Str("name", dupName1).Msg("Duplicate content (is ignored)")
		if dupName2 != "" {
			ds.log.Warn().Str("name", dupName2).Msg("Duplicate content (is ignored)")
		}
		return id.Invalid
	}
	return zid
}

func (ds *DirService) onDeleteFileEvent(entries entrySet, name string) id.Zid {







|

|







341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
	zid := seekZid(name)
	if zid == id.Invalid {
		return id.Invalid
	}
	entry := fetchdirEntry(entries, zid)
	dupName1, dupName2 := ds.updateEntry(entry, name)
	if dupName1 != "" {
		ds.log.Info().Str("name", dupName1).Msg("Duplicate content (is ignored)")
		if dupName2 != "" {
			ds.log.Info().Str("name", dupName2).Msg("Duplicate content (is ignored)")
		}
		return id.Invalid
	}
	return zid
}

func (ds *DirService) onDeleteFileEvent(entries entrySet, name string) id.Zid {

Changes to box/notify/directory_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package notify

import (
	"testing"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

package notify

import (
	"testing"

Changes to box/notify/entry.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package notify

import (
	"path/filepath"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

const (








>
>
>







|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package notify

import (
	"path/filepath"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

const (
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
	if contentName := e.ContentName; contentName != "" {
		if !extIsMetaAndContent(e.ContentExt) && e.MetaName == "" {
			e.MetaName = e.calcBaseName(contentName)
		}
		return
	}

	syntax := m.GetDefault(api.KeySyntax, "")
	ext := calcContentExt(syntax, m.YamlSep, getZettelFileSyntax)
	metaName := e.MetaName
	eimc := extIsMetaAndContent(ext)
	if eimc {
		if metaName != "" {
			ext = contentExtWithMeta(syntax, content)
		}







|







58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
	if contentName := e.ContentName; contentName != "" {
		if !extIsMetaAndContent(e.ContentExt) && e.MetaName == "" {
			e.MetaName = e.calcBaseName(contentName)
		}
		return
	}

	syntax := m.GetDefault(api.KeySyntax, meta.DefaultSyntax)
	ext := calcContentExt(syntax, m.YamlSep, getZettelFileSyntax)
	metaName := e.MetaName
	eimc := extIsMetaAndContent(ext)
	if eimc {
		if metaName != "" {
			ext = contentExtWithMeta(syntax, content)
		}

Changes to box/notify/fsdir.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package notify

import (
	"os"
	"path/filepath"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package notify

import (
	"os"
	"path/filepath"
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
			log.Error().
				Str("parentDir", absParentDir).Err(errParent).
				Str("path", absPath).Err(err).
				Msg("Unable to access Zettel directory and its parent directory")
			watcher.Close()
			return nil, err
		}
		log.Warn().
			Str("parentDir", absParentDir).Err(errParent).
			Msg("Parent of Zettel directory cannot be supervised")
		log.Warn().Str("path", absPath).
			Msg("Zettelstore might not detect a deletion or movement of the Zettel directory")
	} else if err != nil {
		// Not a problem, if container is not available. It might become available later.
		log.Warn().Err(err).Str("path", absPath).Msg("Zettel directory not available")
	}

	fsdn := &fsdirNotifier{
		log:     log,
		events:  make(chan Event),
		refresh: make(chan struct{}),
		done:    make(chan struct{}),







<
|

|



|







54
55
56
57
58
59
60

61
62
63
64
65
66
67
68
69
70
71
72
73
74
			log.Error().
				Str("parentDir", absParentDir).Err(errParent).
				Str("path", absPath).Err(err).
				Msg("Unable to access Zettel directory and its parent directory")
			watcher.Close()
			return nil, err
		}

		log.Info().Str("parentDir", absParentDir).Err(errParent).
			Msg("Parent of Zettel directory cannot be supervised")
		log.Info().Str("path", absPath).
			Msg("Zettelstore might not detect a deletion or movement of the Zettel directory")
	} else if err != nil {
		// Not a problem, if container is not available. It might become available later.
		log.Info().Err(err).Str("path", absPath).Msg("Zettel directory currently not available")
	}

	fsdn := &fsdirNotifier{
		log:     log,
		events:  make(chan Event),
		refresh: make(chan struct{}),
		done:    make(chan struct{}),

Changes to box/notify/helper.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package notify

import (
	"archive/zip"
	"os"

	"zettelstore.de/z/logger"
)

// MakeMetaFilename builds the name of the file containing metadata.
func MakeMetaFilename(basename string) string {
	return basename //+ ".meta"
}

// EntryFetcher return a list of (file) names of an directory.
type EntryFetcher interface {
	Fetch() ([]string, error)
}

type dirPathFetcher struct {
	dirPath string








>
>
>











<
<
<
<
<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22





23
24
25
26
27
28
29
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package notify

import (
	"archive/zip"
	"os"

	"zettelstore.de/z/logger"
)






// EntryFetcher return a list of (file) names of an directory.
type EntryFetcher interface {
	Fetch() ([]string, error)
}

type dirPathFetcher struct {
	dirPath string

Changes to box/notify/notify.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package notify provides some notification services to be used by box services.
package notify

import "fmt"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package notify provides some notification services to be used by box services.
package notify

import "fmt"

Changes to box/notify/simpledir.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package notify

import (
	"path/filepath"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package notify

import (
	"path/filepath"

Changes to cmd/cmd_file.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package cmd

import (
	"context"
	"flag"
	"fmt"
	"io"
	"os"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ---------- Subcommand: file -----------------------------------------------

func cmdFile(fs *flag.FlagSet) (int, error) {
	enc := fs.Lookup("t").Value.String()
	m, inp, err := getInput(fs.Args())
	if m == nil {
		return 2, err
	}
	z := parser.ParseZettel(
		context.Background(),
		zettel.Zettel{
			Meta:    m,
			Content: zettel.NewContent(inp.Src[inp.Pos:]),
		},
		m.GetDefault(api.KeySyntax, meta.SyntaxZmk),
		nil,
	)
	encdr := encoder.Create(api.Encoder(enc), &encoder.CreateParameter{Lang: m.GetDefault(api.KeyLang, api.ValueLangEN)})
	if encdr == nil {
		fmt.Fprintf(os.Stderr, "Unknown format %q\n", enc)
		return 2, nil
	}








>
>
>











|
|
|




















|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package cmd

import (
	"context"
	"flag"
	"fmt"
	"io"
	"os"

	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/input"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// ---------- Subcommand: file -----------------------------------------------

func cmdFile(fs *flag.FlagSet) (int, error) {
	enc := fs.Lookup("t").Value.String()
	m, inp, err := getInput(fs.Args())
	if m == nil {
		return 2, err
	}
	z := parser.ParseZettel(
		context.Background(),
		zettel.Zettel{
			Meta:    m,
			Content: zettel.NewContent(inp.Src[inp.Pos:]),
		},
		m.GetDefault(api.KeySyntax, meta.DefaultSyntax),
		nil,
	)
	encdr := encoder.Create(api.Encoder(enc), &encoder.CreateParameter{Lang: m.GetDefault(api.KeyLang, api.ValueLangEN)})
	if encdr == nil {
		fmt.Fprintf(os.Stderr, "Unknown format %q\n", enc)
		return 2, nil
	}

Changes to cmd/cmd_password.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"fmt"
	"os"

	"golang.org/x/term"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/zettel/id"
)

// ---------- Subcommand: password -------------------------------------------

func cmdPassword(fs *flag.FlagSet) (int, error) {








>
>
>











|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"fmt"
	"os"

	"golang.org/x/term"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/zettel/id"
)

// ---------- Subcommand: password -------------------------------------------

func cmdPassword(fs *flag.FlagSet) (int, error) {

Changes to cmd/cmd_run.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package cmd

import (
	"context"
	"flag"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package cmd

import (
	"context"
	"flag"
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
	ucQuery.SetEvaluate(&ucEvaluate)
	ucTagZettel := usecase.NewTagZettel(protectedBoxManager, &ucQuery)
	ucRoleZettel := usecase.NewRoleZettel(protectedBoxManager, &ucQuery)
	ucListSyntax := usecase.NewListSyntax(protectedBoxManager)
	ucListRoles := usecase.NewListRoles(protectedBoxManager)
	ucDelete := usecase.NewDeleteZettel(logUc, protectedBoxManager)
	ucUpdate := usecase.NewUpdateZettel(logUc, protectedBoxManager)
	ucRename := usecase.NewRenameZettel(logUc, protectedBoxManager)
	ucRefresh := usecase.NewRefresh(logUc, protectedBoxManager)
	ucReIndex := usecase.NewReIndex(logUc, protectedBoxManager)
	ucVersion := usecase.NewVersion(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string))

	a := api.New(
		webLog.Clone().Str("adapter", "api").Child(),
		webSrv, authManager, authManager, rtConfig, authPolicy)
	wui := webui.New(
		webLog.Clone().Str("adapter", "wui").Child(),
		webSrv, authManager, rtConfig, authManager, boxManager, authPolicy, &ucEvaluate)

	webSrv.Handle("/", wui.MakeGetRootHandler(protectedBoxManager))
	if assetDir := kern.GetConfig(kernel.WebService, kernel.WebAssetDir).(string); assetDir != "" {
		const assetPrefix = "/assets/"
		webSrv.Handle(assetPrefix, http.StripPrefix(assetPrefix, http.FileServer(http.Dir(assetDir))))
		webSrv.Handle("/favicon.ico", wui.MakeFaviconHandler(assetDir))
	}

	// Web user interface
	if !authManager.IsReadonly() {
		webSrv.AddZettelRoute('b', server.MethodGet, wui.MakeGetRenameZettelHandler(ucGetZettel))
		webSrv.AddZettelRoute('b', server.MethodPost, wui.MakePostRenameZettelHandler(&ucRename))
		webSrv.AddListRoute('c', server.MethodGet, wui.MakeGetZettelFromListHandler(&ucQuery, &ucEvaluate, ucListRoles, ucListSyntax))
		webSrv.AddListRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('c', server.MethodGet, wui.MakeGetCreateZettelHandler(
			ucGetZettel, &ucCreateZettel, ucListRoles, ucListSyntax))
		webSrv.AddZettelRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('d', server.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetZettel, ucGetAllZettel))
		webSrv.AddZettelRoute('d', server.MethodPost, wui.MakePostDeleteZettelHandler(&ucDelete))







<




















<
<







71
72
73
74
75
76
77

78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97


98
99
100
101
102
103
104
	ucQuery.SetEvaluate(&ucEvaluate)
	ucTagZettel := usecase.NewTagZettel(protectedBoxManager, &ucQuery)
	ucRoleZettel := usecase.NewRoleZettel(protectedBoxManager, &ucQuery)
	ucListSyntax := usecase.NewListSyntax(protectedBoxManager)
	ucListRoles := usecase.NewListRoles(protectedBoxManager)
	ucDelete := usecase.NewDeleteZettel(logUc, protectedBoxManager)
	ucUpdate := usecase.NewUpdateZettel(logUc, protectedBoxManager)

	ucRefresh := usecase.NewRefresh(logUc, protectedBoxManager)
	ucReIndex := usecase.NewReIndex(logUc, protectedBoxManager)
	ucVersion := usecase.NewVersion(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string))

	a := api.New(
		webLog.Clone().Str("adapter", "api").Child(),
		webSrv, authManager, authManager, rtConfig, authPolicy)
	wui := webui.New(
		webLog.Clone().Str("adapter", "wui").Child(),
		webSrv, authManager, rtConfig, authManager, boxManager, authPolicy, &ucEvaluate)

	webSrv.Handle("/", wui.MakeGetRootHandler(protectedBoxManager))
	if assetDir := kern.GetConfig(kernel.WebService, kernel.WebAssetDir).(string); assetDir != "" {
		const assetPrefix = "/assets/"
		webSrv.Handle(assetPrefix, http.StripPrefix(assetPrefix, http.FileServer(http.Dir(assetDir))))
		webSrv.Handle("/favicon.ico", wui.MakeFaviconHandler(assetDir))
	}

	// Web user interface
	if !authManager.IsReadonly() {


		webSrv.AddListRoute('c', server.MethodGet, wui.MakeGetZettelFromListHandler(&ucQuery, &ucEvaluate, ucListRoles, ucListSyntax))
		webSrv.AddListRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('c', server.MethodGet, wui.MakeGetCreateZettelHandler(
			ucGetZettel, &ucCreateZettel, ucListRoles, ucListSyntax))
		webSrv.AddZettelRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('d', server.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetZettel, ucGetAllZettel))
		webSrv.AddZettelRoute('d', server.MethodPost, wui.MakePostDeleteZettelHandler(&ucDelete))
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
	webSrv.AddListRoute('x', server.MethodPost, a.MakePostCommandHandler(&ucIsAuth, &ucRefresh))
	webSrv.AddListRoute('z', server.MethodGet, a.MakeQueryHandler(&ucQuery, &ucTagZettel, &ucRoleZettel, &ucReIndex))
	webSrv.AddZettelRoute('z', server.MethodGet, a.MakeGetZettelHandler(ucGetZettel, ucParseZettel, ucEvaluate))
	if !authManager.IsReadonly() {
		webSrv.AddListRoute('z', server.MethodPost, a.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('z', server.MethodPut, a.MakeUpdateZettelHandler(&ucUpdate))
		webSrv.AddZettelRoute('z', server.MethodDelete, a.MakeDeleteZettelHandler(&ucDelete))
		webSrv.AddZettelRoute('z', server.MethodMove, a.MakeRenameZettelHandler(&ucRename))
	}

	if authManager.WithAuth() {
		webSrv.SetUserRetriever(usecase.NewGetUserByZid(boxManager))
	}
}

type getUserImpl struct{}

func (*getUserImpl) GetUser(ctx context.Context) *meta.Meta { return server.GetUser(ctx) }







<










120
121
122
123
124
125
126

127
128
129
130
131
132
133
134
135
136
	webSrv.AddListRoute('x', server.MethodPost, a.MakePostCommandHandler(&ucIsAuth, &ucRefresh))
	webSrv.AddListRoute('z', server.MethodGet, a.MakeQueryHandler(&ucQuery, &ucTagZettel, &ucRoleZettel, &ucReIndex))
	webSrv.AddZettelRoute('z', server.MethodGet, a.MakeGetZettelHandler(ucGetZettel, ucParseZettel, ucEvaluate))
	if !authManager.IsReadonly() {
		webSrv.AddListRoute('z', server.MethodPost, a.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('z', server.MethodPut, a.MakeUpdateZettelHandler(&ucUpdate))
		webSrv.AddZettelRoute('z', server.MethodDelete, a.MakeDeleteZettelHandler(&ucDelete))

	}

	if authManager.WithAuth() {
		webSrv.SetUserRetriever(usecase.NewGetUserByZid(boxManager))
	}
}

type getUserImpl struct{}

func (*getUserImpl) GetUser(ctx context.Context) *meta.Meta { return server.GetUser(ctx) }

Changes to cmd/command.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package cmd

import (
	"flag"

	"zettelstore.de/client.fossil/maps"
	"zettelstore.de/z/logger"
)

// Command stores information about commands / sub-commands.
type Command struct {
	Name       string              // command name as it appears on the command line
	Func       CommandFunc         // function that executes a command








>
>
>







|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package cmd

import (
	"flag"

	"t73f.de/r/zsc/maps"
	"zettelstore.de/z/logger"
)

// Command stores information about commands / sub-commands.
type Command struct {
	Name       string              // command name as it appears on the command line
	Func       CommandFunc         // function that executes a command

Changes to cmd/main.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
33
34
35
36
37
38
39
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package cmd

import (
	"crypto/sha256"
	"flag"
	"fmt"
	"net"
	"net/url"
	"os"
	"runtime/debug"
	"strconv"
	"strings"
	"time"

	"zettelstore.de/client.fossil/api"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/impl"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/compbox"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/config"
	"zettelstore.de/z/input"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)









>
>
>
















|
>






<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

36
37
38
39
40
41
42
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package cmd

import (
	"crypto/sha256"
	"flag"
	"fmt"
	"net"
	"net/url"
	"os"
	"runtime/debug"
	"strconv"
	"strings"
	"time"

	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/input"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/impl"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/compbox"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/config"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
	return err == nil
}

func setConfigValue(err error, subsys kernel.Service, key string, val any) error {
	if err == nil {
		err = kernel.Main.SetConfig(subsys, key, fmt.Sprint(val))
		if err != nil {
			kernel.Main.GetKernelLogger().Fatal().Str("key", key).Str("value", fmt.Sprint(val)).Err(err).Msg("Unable to set configuration")
		}
	}
	return err
}

func executeCommand(name string, args ...string) int {
	command, ok := Get(name)







|







233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
	return err == nil
}

func setConfigValue(err error, subsys kernel.Service, key string, val any) error {
	if err == nil {
		err = kernel.Main.SetConfig(subsys, key, fmt.Sprint(val))
		if err != nil {
			kernel.Main.GetKernelLogger().Error().Str("key", key).Str("value", fmt.Sprint(val)).Err(err).Msg("Unable to set configuration")
		}
	}
	return err
}

func executeCommand(name string, args ...string) int {
	command, ok := Get(name)

Changes to cmd/register.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package cmd provides command generic functions.
package cmd

// Mention all needed encoders, parsers and stores to have them registered.
import (








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package cmd provides command generic functions.
package cmd

// Mention all needed encoders, parsers and stores to have them registered.
import (

Changes to cmd/zettelstore/main.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package main is the starting point for the zettelstore command.
package main

import (
	"os"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package main is the starting point for the zettelstore command.
package main

import (
	"os"

Changes to collect/collect.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"

Changes to collect/collect_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package collect_test provides some unit test for collectors.
package collect_test

import (
	"testing"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package collect_test provides some unit test for collectors.
package collect_test

import (
	"testing"

Changes to collect/order.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"









>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"

Deleted collect/split.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package collect provides functions to collect items from a syntax tree.
package collect

import (
	"zettelstore.de/z/ast"
	"zettelstore.de/z/strfun"
)

// DivideReferences divides the given list of rederences into zettel, local, and external References.
func DivideReferences(all []*ast.Reference) (zettel, local, external []*ast.Reference) {
	if len(all) == 0 {
		return nil, nil, nil
	}

	mapZettel := make(strfun.Set)
	mapLocal := make(strfun.Set)
	mapExternal := make(strfun.Set)
	for _, ref := range all {
		if ref.State == ast.RefStateSelf {
			continue
		}
		if ref.IsZettel() {
			zettel = appendRefToList(zettel, mapZettel, ref)
		} else if ref.IsExternal() {
			external = appendRefToList(external, mapExternal, ref)
		} else {
			local = appendRefToList(local, mapLocal, ref)
		}
	}
	return zettel, local, external
}

func appendRefToList(reflist []*ast.Reference, refSet strfun.Set, ref *ast.Reference) []*ast.Reference {
	s := ref.String()
	if !refSet.Has(s) {
		reflist = append(reflist, ref)
		refSet.Set(s)
	}
	return reflist
}
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<




































































































Changes to config/config.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package config provides functions to retrieve runtime configuration data.
package config

import (
	"context"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package config provides functions to retrieve runtime configuration data.
package config

import (
	"context"

Changes to docs/development/00010000000000.zettel.

1
2
3
4
5
6
7
8
9
10

id: 00010000000000
title: Developments Notes
role: zettel
syntax: zmk
created: 00010101000000
modified: 20221026184905

* [[Required Software|20210916193200]]
* [[Fuzzing tests|20221026184300]]
* [[Checklist for Release|20210916194900]]






|




>
1
2
3
4
5
6
7
8
9
10
11
id: 00010000000000
title: Developments Notes
role: zettel
syntax: zmk
created: 00010101000000
modified: 20231218182020

* [[Required Software|20210916193200]]
* [[Fuzzing tests|20221026184300]]
* [[Checklist for Release|20210916194900]]
* [[Development tools|20231218181900]]

Changes to docs/development/20210916193200.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
id: 20210916193200
title: Required Software
role: zettel
syntax: zmk
created: 20210916193200
modified: 20230405150541

The following software must be installed:

* A current, supported [[release of Go|https://go.dev/doc/devel/release]],
* [[Fossil|https://fossil-scm.org/]],
* [[Git|https://git-scm.org/]] (most dependencies are accessible via Git only).

Make sure that the software is in your path, e.g. via:
```sh
export PATH=$PATH:/usr/local/go/bin
export PATH=$PATH:$(go env GOPATH)/bin
```

The internal build tool need the following software.
It can be installed / updated via the build tool itself: ``go run tools/build.go tools``.

Otherwise you can install the software by hand:

* [[shadow|https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/shadow]] via ``go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest``,
* [[staticcheck|https://staticcheck.io/]] via ``go install honnef.co/go/tools/cmd/staticcheck@latest``,
* [[unparam|https://mvdan.cc/unparam]][^[[GitHub|https://github.com/mvdan/unparam]]] via ``go install mvdan.cc/unparam@latest``,
* [[govulncheck|https://golang.org/x/vuln/cmd/govulncheck]] via ``go install golang.org/x/vuln/cmd/govulncheck@latest``,





|














|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
id: 20210916193200
title: Required Software
role: zettel
syntax: zmk
created: 20210916193200
modified: 20231213194509

The following software must be installed:

* A current, supported [[release of Go|https://go.dev/doc/devel/release]],
* [[Fossil|https://fossil-scm.org/]],
* [[Git|https://git-scm.org/]] (most dependencies are accessible via Git only).

Make sure that the software is in your path, e.g. via:
```sh
export PATH=$PATH:/usr/local/go/bin
export PATH=$PATH:$(go env GOPATH)/bin
```

The internal build tool need the following software.
It can be installed / updated via the build tool itself: ``go run tools/devtools/devtools.go``.

Otherwise you can install the software by hand:

* [[shadow|https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/shadow]] via ``go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest``,
* [[staticcheck|https://staticcheck.io/]] via ``go install honnef.co/go/tools/cmd/staticcheck@latest``,
* [[unparam|https://mvdan.cc/unparam]][^[[GitHub|https://github.com/mvdan/unparam]]] via ``go install mvdan.cc/unparam@latest``,
* [[govulncheck|https://golang.org/x/vuln/cmd/govulncheck]] via ``go install golang.org/x/vuln/cmd/govulncheck@latest``,

Changes to docs/development/20210916194900.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
id: 20210916194900
title: Checklist for Release
role: zettel
syntax: zmk
created: 20210916194900
modified: 20230402181229

# Sync with the official repository
#* ``fossil sync -u``
# Make sure that there is no workspace defined.
#* ``ls ..`` must not have a file ''go.work'', in no parent folder.
# Make sure that all dependencies are up-to-date.
#* ``cat go.mod``
# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# All internal tests must succeed:
#* ``go run tools/build.go relcheck`` (alternatively: ``make relcheck``).
# The API tests must succeed on every development platform:
#* ``go run tools/build.go testapi`` (alternatively: ``make api``).
# Run [[linkchecker|https://linkchecker.github.io/linkchecker/]] with the manual:
#* ``go run -race cmd/zettelstore/main.go run -d docs/manual``
#* ``linkchecker http://127.0.0.1:23123 2>&1 | tee lc.txt``
#* Check all ""Error: 404 Not Found""
#* Check all ""Error: 403 Forbidden"": allowed for endpoint ''/p'' with encoding ''html'' for those zettel that are accessible only in ''expert-mode''.
#* Try to resolve other error messages and warnings
#* Warnings about empty content can be ignored





|








|

|

|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
id: 20210916194900
title: Checklist for Release
role: zettel
syntax: zmk
created: 20210916194900
modified: 20231213194631

# Sync with the official repository
#* ``fossil sync -u``
# Make sure that there is no workspace defined.
#* ``ls ..`` must not have a file ''go.work'', in no parent folder.
# Make sure that all dependencies are up-to-date.
#* ``cat go.mod``
# Clean up your Go workspace:
#* ``go run tools/clean/clean.go`` (alternatively: ``make clean``).
# All internal tests must succeed:
#* ``go run tools/check/check.go -r`` (alternatively: ``make relcheck``).
# The API tests must succeed on every development platform:
#* ``go run tools/testapi/testapi.go`` (alternatively: ``make api``).
# Run [[linkchecker|https://linkchecker.github.io/linkchecker/]] with the manual:
#* ``go run -race cmd/zettelstore/main.go run -d docs/manual``
#* ``linkchecker http://127.0.0.1:23123 2>&1 | tee lc.txt``
#* Check all ""Error: 404 Not Found""
#* Check all ""Error: 403 Forbidden"": allowed for endpoint ''/p'' with encoding ''html'' for those zettel that are accessible only in ''expert-mode''.
#* Try to resolve other error messages and warnings
#* Warnings about empty content can be ignored
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# Disable Fossil autosync mode:
#* ``fossil setting autosync off``
# Commit the new release version:
#* ``fossil commit --tag release --tag vVERSION -m "Version VERSION"``
#* **Important:** the tag must follow the given pattern, e.g. ''v0.0.15''.
   Otherwise client will not be able to import ''zettelkasten.de/z''.
# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# Create the release:
#* ``go run tools/build.go release`` (alternatively: ``make release``).
# Remove previous executables:
#* ``fossil uv remove --glob '*-PREVVERSION*'``
# Add executables for release:
#* ``cd releases``
#* ``fossil uv add *.zip``
#* ``cd ..``
#* Synchronize with main repository:
#* ``fossil sync -u``
# Enable autosync:
#* ``fossil setting autosync on``







|

|










40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# Disable Fossil autosync mode:
#* ``fossil setting autosync off``
# Commit the new release version:
#* ``fossil commit --tag release --tag vVERSION -m "Version VERSION"``
#* **Important:** the tag must follow the given pattern, e.g. ''v0.0.15''.
   Otherwise client will not be able to import ''zettelkasten.de/z''.
# Clean up your Go workspace:
#* ``go run tools/clean/clean.go`` (alternatively: ``make clean``).
# Create the release:
#* ``go run tools/build/build.go release`` (alternatively: ``make release``).
# Remove previous executables:
#* ``fossil uv remove --glob '*-PREVVERSION*'``
# Add executables for release:
#* ``cd releases``
#* ``fossil uv add *.zip``
#* ``cd ..``
#* Synchronize with main repository:
#* ``fossil sync -u``
# Enable autosync:
#* ``fossil setting autosync on``

Added docs/development/20231218181900.zettel.









































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
id: 20231218181900
title: Development tools
role: zettel
syntax: zmk
created: 20231218181956
modified: 20231218184500

The source code contains some tools to assist the development of Zettelstore.
These are located in the ''tools'' directory.

Most tool support the generic option ``-v``, which log internal activities.

Some of the tools can be called easier by using ``make``, that reads in a provided ''Makefile''.

=== Check
The ""check"" tool automates some testing activities.
It is called via the command line:
```
# go run tools/check/check.go
```
There is an additional option ``-r`` to check in advance of a release.

The following checks are executed:
* Execution of unit tests, like ``go test ./...``
* Analyze the source code for general problems, as in ``go vet ./...``
* Tries to find shadowed variable, via ``shadow ./...``
* Performs some additional checks on the source code, via ``staticcheck ./...``
* Checks the usage of function parameters and usage of return values, via ``unparam ./...``.
  In case the option ''-r'' is set, the check includes exported functions and internal tests.
* In case option ''-r'' is set, the source code is checked against the vulnerability database, via ``govulncheck ./...``

Please note, that most of the tools above are not automatically installed in a standard Go distribution.
Use the command ""devtools"" to install them.

=== Devtools
The following command installs all needed tools:
```
# go run tooles/devtools/devtools.go
```
It will also automatically update these tools.

=== TestAPI
The following command will perform some high-level tests:
```sh
# go run tools/testapi/testapi.go
```
Basically, a Zettelstore will be started and then API calls will be made to simulate some typical activities with the Zettelstore.

If a Zettelstore is already running on port 23123, this Zettelstore will be used instead.
Even if the API test should clean up later, some zettel might stay created if a test fails.
This feature is used, if you want to have more control on the running Zettelstore.
You should start it with the following command:
```sh
# go run -race cmd/zettelstore/main.go run -c testdata/testbox/19700101000000.zettel
```
This allows you to debug failing API tests.

=== HTMLlint
The following command will check the generated HTML code for validity:
```sh
# go run tools/htmllint/htmllint.go
```
In addition, you might specify the URL od a running Zettelstore.
Otherwise ''http://localhost:23123'' is used.

This command fetches first the list of all zettel.
This list is used to check the generated HTML code (''ZID'' is the paceholder for the zettel identification):

* Check all zettel HTML encodings, via the path ''/z/ZID?enc=html&part=zettel''
* Check all zettel web views, via the path ''/h/ZID''
* The info page of all zettel is checked, via path ''/i/ZID''
* A subset of max. 100 zettel will be checked for the validity of their edit page, via ''/e/ZID''
* 10 random zettel are checked for a valid create form, via ''/c/ZID''
* A maximum of 200 random zettel are checked for a valid delete dialog, via ''/d/ZID''

Depending on the selected Zettelstore, the command might take a long time.

You can shorten the time, if you disable any zettel query in the footer.

=== Build
The ""build"" tool allows to build the software, either for tests or for a release.

The following command will create a Zettelstore executable for the architecture of the current computer:
```sh
# go tools/build/build.go build
```
You will find the executable in the ''bin'' directory.

A full release will be build in the directory ''releases'', containing ZIP files for the computer architectures ""Linux/amd64"", ""Linux/arm"", ""MacOS/arm64"", ""MacOS/amd64"", and ""Windows/amd64"".
In addition, the manual is also build as a ZIP file:
```sh
# go run tools/build/build.go release
```

If you just want the ZIP file with the manual, please use:
```sh
# go run tools/build/build.go manual
```

In case you want to check the version of the Zettelstore to be build, use:
```sh
# go run tools/build/build.go version
```

=== Clean
To remove the directories ''bin'' and ''releases'', as well as all cached Go libraries used by Zettelstore, execute:
```sh
# go run tools/clean/clean.go
```

Internally, the following commands are executed
```sh
# rm -rf bin releases
# go clean ./...
# go clean -cache -modcache -testcache
```

Changes to docs/manual/00000000000100.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
id: 00000000000100
title: Zettelstore Runtime Configuration
role: configuration
syntax: none
created: 00010101000000
default-copyright: (c) 2020-present by Detlef Stern <ds@zettelstore.de>
default-license: EUPL-1.2-or-later
default-visibility: public
footer-zettel: 00001000000100
home-zettel: 00001000000000
modified: 20221205173642
site-name: Zettelstore Manual




|







1
2
3
4
5
6
7
8
9
10
11
12
id: 00000000000100
title: Zettelstore Runtime Configuration
role: configuration
syntax: none
created: 20210126175322
default-copyright: (c) 2020-present by Detlef Stern <ds@zettelstore.de>
default-license: EUPL-1.2-or-later
default-visibility: public
footer-zettel: 00001000000100
home-zettel: 00001000000000
modified: 20221205173642
site-name: Zettelstore Manual

Changes to docs/manual/00001000000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001000000000
title: Zettelstore Manual
role: manual
tags: #manual #zettelstore
syntax: zmk
created: 20210301190630
modified: 20231125185455
show-back-links: false

* [[Introduction|00001001000000]]
* [[Design goals|00001002000000]]
* [[Installation|00001003000000]]
* [[Configuration|00001004000000]]





|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001000000000
title: Zettelstore Manual
role: manual
tags: #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20231125185455
show-back-links: false

* [[Introduction|00001001000000]]
* [[Design goals|00001002000000]]
* [[Installation|00001003000000]]
* [[Configuration|00001004000000]]

Added docs/manual/00001000000002.zettel.















>
>
>
>
>
>
>
1
2
3
4
5
6
7
id: 00001000000002
title: manual
role: role
syntax: zmk
created: 20231128184200

Zettel with the role ""manual"" contain the manual of the zettelstore.

Changes to docs/manual/00001001000000.zettel.

1
2
3
4
5


6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
id: 00001001000000
title: Introduction to the Zettelstore
role: manual
tags: #introduction #manual #zettelstore
syntax: zmk



[[Personal knowledge
management|https://en.wikipedia.org/wiki/Personal_knowledge_management]] is
about collecting, classifying, storing, searching, retrieving, assessing,
evaluating, and sharing knowledge as a daily activity. Personal knowledge
management is done by most people, not necessarily as part of their main
business. It is essential for knowledge workers, like students, researchers,
lecturers, software developers, scientists, engineers, architects, to name
a few. Many hobbyists build up a significant amount of knowledge, even if the
do not need to think for a living. Personal knowledge management can be seen as
a prerequisite for many kinds of collaboration.

Zettelstore is a software that collects and relates your notes (""zettel"")
to represent and enhance your knowledge. It helps with many tasks of personal
knowledge management by explicitly supporting the ""[[Zettelkasten
method|https://en.wikipedia.org/wiki/Zettelkasten]]"". The method is based on
creating many individual notes, each with one idea or information, that are
related to each other. Since knowledge is typically build up gradually, one
major focus is a long-term store of these notes, hence the name
""Zettelstore"".





>
>

<
|
<
<
|
<
|
|
<
|

|
<
<
<
|
<
|
<
1
2
3
4
5
6
7
8

9


10

11
12

13
14
15



16

17

id: 00001001000000
title: Introduction to the Zettelstore
role: manual
tags: #introduction #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240710184612


[[Personal knowledge management|https://en.wikipedia.org/wiki/Personal_knowledge_management]] involves collecting, classifying, storing, searching, retrieving, assessing, evaluating, and sharing knowledge as a daily activity.


It's done by most individuals, not necessarily as part of their main business.

It's essential for knowledge workers, such as students, researchers, lecturers, software developers, scientists, engineers, architects, etc.
Many hobbyists build up a significant amount of knowledge, even if they do not need to think for a living.

Personal knowledge management can be seen as a prerequisite for many kinds of collaboration.

Zettelstore is software that collects and relates your notes (""zettel"") to represent and enhance your knowledge, supporting the ""[[Zettelkasten method|https://en.wikipedia.org/wiki/Zettelkasten]]"".



The method is based on creating many individual notes, each with one idea or piece of information, that is related to each other.

Since knowledge is typically built up gradually, one major focus is a long-term store of these notes, hence the name ""Zettelstore"".

Changes to docs/manual/00001003000000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001003000000
title: Installation of the Zettelstore software
role: manual
tags: #installation #manual #zettelstore
syntax: zmk

modified: 20220119145756

=== The curious user
You just want to check out the Zettelstore software

* Grab the appropriate executable and copy it into any directory
* Start the Zettelstore software, e.g. with a double click[^On Windows and macOS, the operating system tries to protect you from possible malicious software. If you encounter problem, please take a look on the [[Troubleshooting|00001018000000]] page.]





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001003000000
title: Installation of the Zettelstore software
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220119145756

=== The curious user
You just want to check out the Zettelstore software

* Grab the appropriate executable and copy it into any directory
* Start the Zettelstore software, e.g. with a double click[^On Windows and macOS, the operating system tries to protect you from possible malicious software. If you encounter problem, please take a look on the [[Troubleshooting|00001018000000]] page.]

Changes to docs/manual/00001003300000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001003300000
title: Zettelstore installation for the intermediate user
role: manual
tags: #installation #manual #zettelstore
syntax: zmk

modified: 20220114175754

You already tried the Zettelstore software and now you want to use it permanently.
Zettelstore should start automatically when you log into your computer.

* Grab the appropriate executable and copy it into the appropriate directory
* If you want to place your zettel into another directory, or if you want more than one [[Zettelstore box|00001004011200]], or if you want to [[enable authentication|00001010040100]], or if you want to tweak your Zettelstore in some other way, create an appropriate [[startup configuration file|00001004010000]].





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001003300000
title: Zettelstore installation for the intermediate user
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
created: 20211125191727
modified: 20220114175754

You already tried the Zettelstore software and now you want to use it permanently.
Zettelstore should start automatically when you log into your computer.

* Grab the appropriate executable and copy it into the appropriate directory
* If you want to place your zettel into another directory, or if you want more than one [[Zettelstore box|00001004011200]], or if you want to [[enable authentication|00001010040100]], or if you want to tweak your Zettelstore in some other way, create an appropriate [[startup configuration file|00001004010000]].

Changes to docs/manual/00001003305000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001003305000
title: Enable Zettelstore to start automatically on Windows
role: manual
tags: #installation #manual #zettelstore
syntax: zmk

modified: 20220218125541

Windows is a complicated beast. There are several ways to automatically start Zettelstore.

=== Startup folder

One way is to use the [[autostart folder|https://support.microsoft.com/en-us/windows/add-an-app-to-run-automatically-at-startup-in-windows-10-150da165-dcd9-7230-517b-cf3c295d89dd]].





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001003305000
title: Enable Zettelstore to start automatically on Windows
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
created: 20211125191727
modified: 20220218125541

Windows is a complicated beast. There are several ways to automatically start Zettelstore.

=== Startup folder

One way is to use the [[autostart folder|https://support.microsoft.com/en-us/windows/add-an-app-to-run-automatically-at-startup-in-windows-10-150da165-dcd9-7230-517b-cf3c295d89dd]].

Changes to docs/manual/00001003310000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001003310000
title: Enable Zettelstore to start automatically on macOS
role: manual
tags: #installation #manual #zettelstore
syntax: zmk

modified: 20220119124635

There are several ways to automatically start Zettelstore.

* [[Login Items|#login-items]]
* [[Launch Agent|#launch-agent]]






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001003310000
title: Enable Zettelstore to start automatically on macOS
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
created: 20220114181521
modified: 20220119124635

There are several ways to automatically start Zettelstore.

* [[Login Items|#login-items]]
* [[Launch Agent|#launch-agent]]

Changes to docs/manual/00001003315000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001003315000
title: Enable Zettelstore to start automatically on Linux
role: manual
tags: #installation #manual #zettelstore
syntax: zmk

modified: 20220307104944

Since there is no such thing as the one Linux, there are too many different ways to automatically start Zettelstore.

* One way is to interpret your Linux desktop system as a server and use the [[recipe to install Zettelstore on a server|00001003600000]].
** See below for a lighter alternative.
* If you are using the [[Gnome Desktop|https://www.gnome.org/]], you could use the tool [[Tweak|https://wiki.gnome.org/action/show/Apps/Tweaks]] (formerly known as ""GNOME Tweak Tool"" or just ""Tweak Tool"").





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001003315000
title: Enable Zettelstore to start automatically on Linux
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
created: 20220114181521
modified: 20220307104944

Since there is no such thing as the one Linux, there are too many different ways to automatically start Zettelstore.

* One way is to interpret your Linux desktop system as a server and use the [[recipe to install Zettelstore on a server|00001003600000]].
** See below for a lighter alternative.
* If you are using the [[Gnome Desktop|https://www.gnome.org/]], you could use the tool [[Tweak|https://wiki.gnome.org/action/show/Apps/Tweaks]] (formerly known as ""GNOME Tweak Tool"" or just ""Tweak Tool"").

Changes to docs/manual/00001003600000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001003600000
title: Installation of Zettelstore on a server
role: manual
tags: #installation #manual #zettelstore
syntax: zmk

modified: 20211125185833

You want to provide a shared Zettelstore that can be used from your various devices.
Installing Zettelstore as a Linux service is not that hard.

Grab the appropriate executable and copy it into the appropriate directory:
```sh





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001003600000
title: Installation of Zettelstore on a server
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
created: 20211125191727
modified: 20211125185833

You want to provide a shared Zettelstore that can be used from your various devices.
Installing Zettelstore as a Linux service is not that hard.

Grab the appropriate executable and copy it into the appropriate directory:
```sh

Changes to docs/manual/00001004000000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004000000
title: Configuration of Zettelstore
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk

modified: 20210510153233

There are some levels to change the behavior and/or the appearance of Zettelstore.

# The first level is the way to start Zettelstore services and to manage it via command line (and, in part, via a graphical user interface).
#* [[Command line parameters|00001004050000]]
# As an intermediate user, you usually want to have more control over how Zettelstore is started.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004000000
title: Configuration of Zettelstore
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20210510153233

There are some levels to change the behavior and/or the appearance of Zettelstore.

# The first level is the way to start Zettelstore services and to manage it via command line (and, in part, via a graphical user interface).
#* [[Command line parameters|00001004050000]]
# As an intermediate user, you usually want to have more control over how Zettelstore is started.

Changes to docs/manual/00001004010000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94

95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
id: 00001004010000
title: Zettelstore startup configuration
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20221128155143

The configuration file, as specified by the ''-c CONFIGFILE'' [[command line option|00001004051000]], allows you to specify some startup options.
These options cannot be stored in a [[configuration zettel|00001004020000]] because either they are needed before Zettelstore can start or because of security reasons.
For example, Zettelstore need to know in advance, on which network address is must listen or where zettel are stored.
An attacker that is able to change the owner can do anything.
Therefore only the owner of the computer on which Zettelstore runs can change this information.

The file for startup configuration must be created via a text editor in advance.

The syntax of the configuration file is the same as for any zettel metadata.
The following keys are supported:

; [!admin-port|''admin-port'']
: Specifies the TCP port through which you can reach the [[administrator console|00001004100000]].
  A value of ""0"" (the default) disables the administrator console.
  The administrator console will only be enabled if Zettelstore is started with the [[''run'' sub-command|00001004051000]].

  On most operating systems, the value must be greater than ""1024"" unless you start Zettelstore with the full privileges of a system administrator (which is not recommended).

  Default: ""0""
; [!asset-dir|''asset-dir'']
: Allows to specify a directory whose files are allowed be transferred directly with the help of the web server.
  The URL prefix for these files is ''/assets/''.
  You can use this if you want to transfer files that are too large for a note to users.
  Examples would be presentation files, PDF files, music files or video files.

  Files within the given directory will not be managed by Zettelstore.[^They will be managed by Zettelstore just in the case that the directory is one of the configured [[boxes|#box-uri-x]].]

  If you specify only the URL prefix, then the contents of the directory are listed to the user.
  To avoid this, create an empty file in the directory named ""index.html"".

  Default: """", no asset directory is set, the URL prefix ''/assets/'' is invalid.
; [!base-url|''base-url'']
: Sets the absolute base URL for the service.

  Note: [[''url-prefix''|#url-prefix]] must be the suffix of ''base-url'', otherwise the web service will not start.
  
  Default: ""http://127.0.0.1:23123/"".
; [!box-uri-x|''box-uri-X''], where __X__ is a number greater or equal to one
: Specifies a [[box|00001004011200]] where zettel are stored.
  During startup __X__ is counted up, starting with one, until no key is found.
  This allows to configure more than one box.

  If no ''box-uri-1'' key is given, the overall effect will be the same as if only ''box-uri-1'' was specified with the value ""dir://.zettel"".
  In this case, even a key ''box-uri-2'' will be ignored.
; [!debug-mode|''debug-mode'']
: Allows to debug the Zettelstore software (mostly used by the developers) if set to [[true|00001006030500]]
  Disables any timeout values of the internal web server and does not send some security-related data.
  Sets [[''log-level''|#log-level]] to ""debug"".

  Do not enable it for a production server.

  Default: ""false""
; [!default-dir-box-type|''default-dir-box-type'']
: Specifies the default value for the (sub-) type of [[directory boxes|00001004011400#type]].
  Zettel are typically stored in such boxes.

  Default: ""notify""
; [!insecure-cookie|''insecure-cookie'']
: Must be set to [[true|00001006030500]], if authentication is enabled and Zettelstore is not accessible not via HTTPS (but via HTTP).
  Otherwise web browser are free to ignore the authentication cookie.

  Default: ""false""
; [!insecure-html|''insecure-html'']
: Allows to use HTML, e.g. within supported markup languages, even if this might introduce security-related problems.
  However, HTML containing the ``<script>`` or the ``<iframe>`` tag is always ignored.
  But due to ""clever"" ways of combining HTML, CSS, JavaScript, there might be some negative security consequences.
  Please be aware of this!

  Allowed values: ""html"" (allow zettel with [[syntax ""html""|00001008000000#html]]), ""markdown"" (""html"", plus allow inline HTML for Markdown markup only), ""zettelmarkup"" (""markdown"", plus allow inline HTML for Zettelmarkup).
  Any other value is interpreted as ""secure"".

  Default: ""secure"".
; [!listen-addr|''listen-addr'']
: Configures the network address, where the Zettelstore service is listening for requests.
  Syntax is: ''[NETWORKIP]:PORT'', where ''NETWORKIP'' is the IP-address of the networking interface (or something like ""0.0.0.0"" if you want to listen on all network interfaces, and ''PORT'' is the TCP port.

  Default value: ""127.0.0.1:23123""
; [!log-level|''log-level'']
: Specify the [[logging level|00001004059700]] for the whole application or for a given (internal) service, overwriting the level ""debug"" set by configuration [[''debug-mode''|#debug-mode]].
  Can be changed at runtime, even for specific internal services, with the ''log-level'' command of the [[administrator console|00001004101000#log-level]].

  Several specifications are separated by the semicolon character (""'';''"", U+003B).
  Each specification consists of an optional service name, together with the colon character (""'':''"", U+003A), followed by the logging level.

  Default: ""info"".


  Examples: ""sense"" will produce sensing messages (e.g. a little more than ""info""); ""sense;web:debug"" will emit debugging messages for the web component of Zettelstore while still producing sensing messages for all other components.

  When you are familiar to operate the Zettelstore, you might set the level to ""warn"" or ""error"" to receive less noisy messages from the Zettelstore.
; [!max-request-size|''max-request-size'']
: Limits the maximum byte size of a web request body to prevent clients from accidentally or maliciously sending a large request and wasting server resources.
  The minimum value is 1024.

  Default: 16777216 (16 MiB). 
; [!owner|''owner'']
: [[Identifier|00001006050000]] of a zettel that contains data about the owner of the Zettelstore.
  The owner has full authorization for the Zettelstore.
  Only if owner is set to some value, user [[authentication|00001010000000]] is enabled.

  Ensure that key [[''secret''|#secret]] is set to a value of at least 16 bytes.
  Otherwise the Zettelstore will not start for security reasons.
; [!persistent-cookie|''persistent-cookie'']
: A [[boolean value|00001006030500]] to make the access cookie persistent.
  This is helpful if you access the Zettelstore via a mobile device.
  On these devices, the operating system is free to stop the web browser and to remove temporary cookies.
  Therefore, an authenticated user will be logged off.

  If ""true"", a persistent cookie is used.
  Its lifetime exceeds the lifetime of the authentication token (see option ''token-lifetime-html'') by 30 seconds.

  Default: ""false""
; [!read-only-mode|''read-only-mode'']
: Puts the Zettelstore service into a read-only mode, if set to a [[true value|00001006030500]].
  No changes are possible.

  Default: ""false"".
; [!secret|''secret'']
: A string value to make the communication with external clients strong enough so that sessions of the [[web user interface|00001014000000]] or [[API access token|00001010040700]] cannot be modified by some external unfriendly party.
  The string must have a length of at least 16 bytes.

  It is only needed to set this value, if [[authentication is enabled|00001010040100]] by setting key [[''owner''|#owner]] to some user identification.
; [!token-lifetime-api|''token-lifetime-api''], [!token-lifetime-html|''token-lifetime-html'']
: Define lifetime of access tokens in minutes.
  Values are only valid if authentication is enabled, i.e. key ''owner'' is set.

  ''token-lifetime-api'' is for accessing Zettelstore via its [[API|00001012000000]].
  Default: ""10"".

  ''token-lifetime-html'' specifies the lifetime for the HTML views.
  It is automatically extended, when a new HTML view is rendered.
  Default: ""60"".
; [!url-prefix|''url-prefix'']
: Add the given string as a prefix to the local part of a Zettelstore local URL/URI when rendering zettel representations.
  Must begin and end with a slash character (""''/''"", U+002F).

  Note: ''url-prefix'' must be the suffix of [[''base-url''|#base-url]], otherwise the web service will not start.

  Default: ""/"".

  This allows to use a forwarding proxy [[server|00001010090100]] in front of the Zettelstore.
; [!verbose-mode|''verbose-mode'']
: Be more verbose when logging data, if set to a [[true value|00001006030500]].

  Default: ""false""






|

|
|
|

|








|








|
<

|

|











|
|




|







|
<



|
|














|







|



>
|

|

|






|

|
<



|



|



|




|


|








|



|










1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
id: 00001004010000
title: Zettelstore startup configuration
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240710183532

The configuration file, specified by the ''-c CONFIGFILE'' [[command line option|00001004051000]], allows you to specify some startup options.
These cannot be stored in a [[configuration zettel|00001004020000]] because they are needed before Zettelstore can start or because of security reasons.
For example, Zettelstore needs to know in advance on which network address it must listen or where zettel are stored.
An attacker that is able to change the owner can do anything.
Therefore, only the owner of the computer on which Zettelstore runs can change this information.

The file for startup configuration must be created via a text editor in advance.

The syntax of the configuration file is the same as for any zettel metadata.
The following keys are supported:

; [!admin-port|''admin-port'']
: Specifies the TCP port through which you can reach the [[administrator console|00001004100000]].
  A value of ""0"" (the default) disables it.
  The administrator console will only be enabled if Zettelstore is started with the [[''run'' sub-command|00001004051000]].

  On most operating systems, the value must be greater than ""1024"" unless you start Zettelstore with the full privileges of a system administrator (which is not recommended).

  Default: ""0""
; [!asset-dir|''asset-dir'']
: Allows to specify a directory whose files are allowed be transferred directly with the help of the web server.
  The URL prefix for these files is ''/assets/''.
  You can use this if you want to transfer files that are too large for a zettel, such as presentation, PDF, music or video files.


  Files within the given directory will not be managed by Zettelstore.[^They will be managed by Zettelstore just in the very special case that the directory is one of the configured [[boxes|#box-uri-x]].]

  If you specify only the URL prefix in your web client, the contents of the directory are listed.
  To avoid this, create an empty file in the directory named ""index.html"".

  Default: """", no asset directory is set, the URL prefix ''/assets/'' is invalid.
; [!base-url|''base-url'']
: Sets the absolute base URL for the service.

  Note: [[''url-prefix''|#url-prefix]] must be the suffix of ''base-url'', otherwise the web service will not start.
  
  Default: ""http://127.0.0.1:23123/"".
; [!box-uri-x|''box-uri-X''], where __X__ is a number greater or equal to one
: Specifies a [[box|00001004011200]] where zettel are stored.
  During startup, __X__ is incremented, starting with one, until no key is found.
  This allows to configuring than one box.

  If no ''box-uri-1'' key is given, the overall effect will be the same as if only ''box-uri-1'' was specified with the value ""dir://.zettel"".
  In this case, even a key ''box-uri-2'' will be ignored.
; [!debug-mode|''debug-mode'']
: If set to [[true|00001006030500]], allows to debug the Zettelstore software (mostly used by the developers).
  Disables any timeout values of the internal web server and does not send some security-related data.
  Sets [[''log-level''|#log-level]] to ""debug"".

  Do not enable it for a production server.

  Default: ""false""
; [!default-dir-box-type|''default-dir-box-type'']
: Specifies the default value for the (sub-)type of [[directory boxes|00001004011400#type]], in which Zettel are typically stored.


  Default: ""notify""
; [!insecure-cookie|''insecure-cookie'']
: Must be set to [[true|00001006030500]] if authentication is enabled and Zettelstore is not accessible via HTTPS (but via HTTP).
  Otherwise web browsers are free to ignore the authentication cookie.

  Default: ""false""
; [!insecure-html|''insecure-html'']
: Allows to use HTML, e.g. within supported markup languages, even if this might introduce security-related problems.
  However, HTML containing the ``<script>`` or the ``<iframe>`` tag is always ignored.
  But due to ""clever"" ways of combining HTML, CSS, JavaScript, there might be some negative security consequences.
  Please be aware of this!

  Allowed values: ""html"" (allow zettel with [[syntax ""html""|00001008000000#html]]), ""markdown"" (""html"", plus allow inline HTML for Markdown markup only), ""zettelmarkup"" (""markdown"", plus allow inline HTML for Zettelmarkup).
  Any other value is interpreted as ""secure"".

  Default: ""secure"".
; [!listen-addr|''listen-addr'']
: Configures the network address, where the Zettelstore service is listening for requests.
  The syntax is: ''[NETWORKIP]:PORT'', where ''NETWORKIP'' is the IP address of the networking interface (or something like ""0.0.0.0"" if you want to listen on all network interfaces), and ''PORT'' is the TCP port.

  Default value: ""127.0.0.1:23123""
; [!log-level|''log-level'']
: Specify the [[logging level|00001004059700]] for the whole application or for a given (internal) service, overwriting the level ""debug"" set by configuration [[''debug-mode''|#debug-mode]].
  Can be changed at runtime, even for specific internal services, with the ''log-level'' command of the [[administrator console|00001004101000#log-level]].

  Several specifications are separated by the semicolon character (""'';''"", U+003B).
  Each consists of an optional service name, together with the colon character (""'':''"", U+003A), followed by the logging level.

  Default: ""info"".

  Examples: ""error"" will produce just error messages (e.g. no ""info"" messages).
  ""error;web:debug"" will emit debugging messages for the web component of Zettelstore while still producing error messages for all other components.

  When you are familiar with operating the Zettelstore, you might set the level to ""error"" to receive fewer noisy messages from it.
; [!max-request-size|''max-request-size'']
: It limits the maximum byte size of a web request body to prevent clients from accidentally or maliciously sending a large request and wasting server resources.
  The minimum value is 1024.

  Default: 16777216 (16 MiB). 
; [!owner|''owner'']
: [[Identifier|00001006050000]] of a zettel that contains data about the owner of the Zettelstore.
  The owner has full authorization for the Zettelstore.
  Only if set to some value, user [[authentication|00001010000000]] is enabled.

  Ensure that the key [[''secret''|#secret]] is set to a value of at least 16 bytes, otherwise the Zettelstore will not start for security reasons.

; [!persistent-cookie|''persistent-cookie'']
: A [[boolean value|00001006030500]] to make the access cookie persistent.
  This is helpful if you access the Zettelstore via a mobile device.
  On these, the operating system is free to stop the web browser and to remove temporary cookies.
  Therefore, an authenticated user will be logged off.

  If ""true"", a persistent cookie is used.
  Its lifetime exceeds the lifetime of the authentication token by 30 seconds (see option ''token-lifetime-html'').

  Default: ""false""
; [!read-only-mode|''read-only-mode'']
: If set to a [[true value|00001006030500]] the Zettelstore service puts into a read-only mode.
  No changes are possible.

  Default: ""false"".
; [!secret|''secret'']
: A string value to make the communication with external clients strong enough so that sessions of the [[web user interface|00001014000000]] or [[API access token|00001010040700]] cannot be altered by some external unfriendly party.
  The string must have a length of at least 16 bytes.

  This value is only needed to be set if [[authentication is enabled|00001010040100]] by setting the key [[''owner''|#owner]] to some user identification value.
; [!token-lifetime-api|''token-lifetime-api''], [!token-lifetime-html|''token-lifetime-html'']
: Define lifetime of access tokens in minutes.
  Values are only valid if authentication is enabled, i.e. key ''owner'' is set.

  ''token-lifetime-api'' is for accessing Zettelstore via its [[API|00001012000000]].
  Default: ""10"".

  ''token-lifetime-html'' specifies the lifetime for the HTML views.
  It is automatically extended when a new HTML view is rendered.
  Default: ""60"".
; [!url-prefix|''url-prefix'']
: Add the given string as a prefix to the local part of a Zettelstore local URL/URI when rendering zettel representations.
  It must begin and end with a slash character (""''/''"", U+002F).

  Note: ''url-prefix'' must be the suffix of [[''base-url''|#base-url]], otherwise the web service will not start.

  Default: ""/"".

  This allows to use a forwarding proxy [[server|00001010090100]] in front of the Zettelstore.
; [!verbose-mode|''verbose-mode'']
: Be more verbose when logging data, if set to a [[true value|00001006030500]].

  Default: ""false""

Changes to docs/manual/00001004011200.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004011200
title: Zettelstore boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk

modified: 20220307121547

A Zettelstore must store its zettel somehow and somewhere.
In most cases you want to store your zettel as files in a directory.
Under certain circumstances you may want to store your zettel elsewhere.

An example are the [[predefined zettel|00001005090000]] that come with a Zettelstore.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004011200
title: Zettelstore boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220307121547

A Zettelstore must store its zettel somehow and somewhere.
In most cases you want to store your zettel as files in a directory.
Under certain circumstances you may want to store your zettel elsewhere.

An example are the [[predefined zettel|00001005090000]] that come with a Zettelstore.

Changes to docs/manual/00001004011400.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
id: 00001004011400
title: Configure file directory boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk

modified: 20220724200512

Under certain circumstances, it is preferable to further configure a file directory box.
This is done by appending query parameters after the base box URI ''dir:\//DIR''.

The following parameters are supported:

|= Parameter:|Description|Default value:|





>
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001004011400
title: Configure file directory boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240710180215

Under certain circumstances, it is preferable to further configure a file directory box.
This is done by appending query parameters after the base box URI ''dir:\//DIR''.

The following parameters are supported:

|= Parameter:|Description|Default value:|
52
53
54
55
56
57
58
59

=== Readonly
Sometimes you may want to provide zettel from a file directory box, but you want to disallow any changes.
If you provide the query parameter ''readonly'' (with or without a corresponding value), the box will disallow any changes.
```
box-uri-1: dir:///home/zettel?readonly
```
If you put the whole Zettelstore in [[read-only|00001004010000]] [[mode|00001004051000]], all configured file directory boxes will be in read-only mode too, even if not explicitly configured.







|
53
54
55
56
57
58
59
60

=== Readonly
Sometimes you may want to provide zettel from a file directory box, but you want to disallow any changes.
If you provide the query parameter ''readonly'' (with or without a corresponding value), the box will disallow any changes.
```
box-uri-1: dir:///home/zettel?readonly
```
If you put the whole Zettelstore in [[read-only|00001004010000#read-only-mode]] [[mode|00001004051000]], all configured file directory boxes will be in read-only mode too, even if not explicitly configured.

Changes to docs/manual/00001004011600.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004011600
title: Configure memory boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk

modified: 20220307122554

Under most circumstances, it is preferable to further configure a memory box.
This is done by appending query parameters after the base box URI ''mem:''.

The following parameters are supported:






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004011600
title: Configure memory boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20220307112918
modified: 20220307122554

Under most circumstances, it is preferable to further configure a memory box.
This is done by appending query parameters after the base box URI ''mem:''.

The following parameters are supported:

Changes to docs/manual/00001004050200.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004050200
title: The ''help'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk

modified: 20210712233414

Lists all implemented sub-commands.

Example:
```
# zettelstore help





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004050200
title: The ''help'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20210712233414

Lists all implemented sub-commands.

Example:
```
# zettelstore help

Changes to docs/manual/00001004050400.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004050400
title: The ''version'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk

modified: 20211124182041

Emits some information about the Zettelstore's version.
This allows you to check, whether your installed Zettelstore is 

The name of the software (""Zettelstore"") and the build version information is given, as well as the compiler version, and an indication about the operating system and the processor architecture of that computer.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004050400
title: The ''version'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20211124182041

Emits some information about the Zettelstore's version.
This allows you to check, whether your installed Zettelstore is 

The name of the software (""Zettelstore"") and the build version information is given, as well as the compiler version, and an indication about the operating system and the processor architecture of that computer.

Changes to docs/manual/00001004051000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004051000
title: The ''run'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk

modified: 20220724162050

=== ``zettelstore run``
This starts the web service.

```
zettelstore run [-a PORT] [-c CONFIGFILE] [-d DIR] [-debug] [-p PORT] [-r] [-v]





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004051000
title: The ''run'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220724162050

=== ``zettelstore run``
This starts the web service.

```
zettelstore run [-a PORT] [-c CONFIGFILE] [-d DIR] [-debug] [-p PORT] [-r] [-v]

Changes to docs/manual/00001004051400.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004051400
title: The ''password'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk

modified: 20210712234305

This sub-command is used to create a hashed password for to be authenticated users.

It reads a password from standard input (two times, both must be equal) and writes the hashed password to standard output.

The general usage is:





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004051400
title: The ''password'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20210712234305

This sub-command is used to create a hashed password for to be authenticated users.

It reads a password from standard input (two times, both must be equal) and writes the hashed password to standard output.

The general usage is:

Changes to docs/manual/00001004059700.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
id: 00001004059700
title: List of supported logging levels
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk

modified: 20220113183606

Zettelstore supports various levels of logging output.
This allows you to see the inner workings of Zettelstore, or to avoid it.

Each level has an associated name and number.
A lower number signals more logging output.

|= Name | Number >| Description
| Trace | 1 | Show most of the inner workings
| Debug | 2 | Show many internal values that might be interesting for a [[Zettelstore developer|00000000000005]].
| Sense | 3 | Display sensing events, which are not essential information.
| Info  | 4 | Display information about an event. In most cases, there is no required action expected from you.
| Warn  | 5 | Show a warning, i.e. an event that might become an error or more. Mostly invalid data.
| Error | 6 | Notify about an error, which was handled automatically. Something is broken. User intervention is not required, in most cases. Monitor the application.
| Fatal | 7 | Notify about a significant error that cannot be handled automatically. At least some important functionality is disabled.
| Panic | 8 | The application is in an uncertain state and notifies you about its panic. At least some part of the application is possibly restarted.
| Mandatory | 9 | Important message will be shown, e.g. the Zettelstore version at startup time.
| Disabled | 10 | No messages will be shown

If you set the logging level to a certain value, only messages with the same or higher numerical value will be shown.
E.g. if you set the logging level to ""warn"", no ""trace"", ""debug"", ""sense", and ""info"" messages are shown, but ""warn"", ""error"", ""fatal"", ""panic"", and ""mandatory"" messages.





>
|







|


<
|
<
|
<
<
|
|


|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

18

19


20
21
22
23
24
id: 00001004059700
title: List of supported logging levels
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20211204182643
modified: 20240221134619

Zettelstore supports various levels of logging output.
This allows you to see the inner workings of Zettelstore, or to avoid it.

Each level has an associated name and number.
A lower number signals more logging output.

|= Name | Number :| Description
| Trace | 1 | Show most of the inner workings
| Debug | 2 | Show many internal values that might be interesting for a [[Zettelstore developer|00000000000005]].

| Info  | 3 | Display information about an event. In most cases, there is no required action expected from you.

| Error | 4 | Notify about an error, which was handled automatically. Something is broken. User intervention may be required, some important functionality may be disabled. Monitor the application.


| Mandatory | 5 | Important message will be shown, e.g. the Zettelstore version at startup time.
| Disabled | 6 | No messages will be shown

If you set the logging level to a certain value, only messages with the same or higher numerical value will be shown.
E.g. if you set the logging level to ""error"", no ""trace"", ""debug"", and ""info"" messages are shown, but ""error"" and ""mandatory"" messages.

Changes to docs/manual/00001004059900.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004059900
title: Command line flags for profiling the application
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk

modified: 20211122174951

If you want to measure potential bottlenecks within the software Zettelstore,
there are two [[command line|00001004050000]] flags for enabling the measurement (also called __profiling__):

; ''-cpuprofile FILE''
: Enables CPU profiling.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004059900
title: Command line flags for profiling the application
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
created: 20211122170506
modified: 20211122174951

If you want to measure potential bottlenecks within the software Zettelstore,
there are two [[command line|00001004050000]] flags for enabling the measurement (also called __profiling__):

; ''-cpuprofile FILE''
: Enables CPU profiling.

Changes to docs/manual/00001004100000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004100000
title: Zettelstore Administrator Console
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk

modified: 20211103162926

The administrator console is a service accessible only on the same computer on which Zettelstore is running.
It allows an experienced user to monitor and control some of the inner workings of Zettelstore.

You enable the administrator console by specifying a TCP port number greater than zero (better: greater than 1024) for it, either via the [[command-line parameter ''-a''|00001004051000#a]] or via the ''admin-port'' key of the [[startup configuration file|00001004010000#admin-port]].






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004100000
title: Zettelstore Administrator Console
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210510141304
modified: 20211103162926

The administrator console is a service accessible only on the same computer on which Zettelstore is running.
It allows an experienced user to monitor and control some of the inner workings of Zettelstore.

You enable the administrator console by specifying a TCP port number greater than zero (better: greater than 1024) for it, either via the [[command-line parameter ''-a''|00001004051000#a]] or via the ''admin-port'' key of the [[startup configuration file|00001004010000#admin-port]].

Changes to docs/manual/00001004101000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001004101000
title: List of supported commands of the administrator console
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk

modified: 20220823194553

; [!bye|''bye'']
: Closes the connection to the administrator console.
; [!config|''config SERVICE'']
: Displays all valid configuration keys for the given service.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004101000
title: List of supported commands of the administrator console
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
created: 20210510141304
modified: 20220823194553

; [!bye|''bye'']
: Closes the connection to the administrator console.
; [!config|''config SERVICE'']
: Displays all valid configuration keys for the given service.

Changes to docs/manual/00001005000000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
id: 00001005000000
title: Structure of Zettelstore
role: manual
tags: #design #manual #zettelstore
syntax: zmk

modified: 20220104213511

Zettelstore is a software that manages your zettel.
Since every zettel must be readable without any special tool, most zettel has to be stored as ordinary files within specific directories.
Typically, file names and file content must comply to specific rules so that Zettelstore can manage them.
If you add, delete, or change zettel files with other tools, e.g. a text editor, Zettelstore will monitor these actions.

Zettelstore provides additional services to the user.





>
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001005000000
title: Structure of Zettelstore
role: manual
tags: #design #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240711183257

Zettelstore is a software that manages your zettel.
Since every zettel must be readable without any special tool, most zettel has to be stored as ordinary files within specific directories.
Typically, file names and file content must comply to specific rules so that Zettelstore can manage them.
If you add, delete, or change zettel files with other tools, e.g. a text editor, Zettelstore will monitor these actions.

Zettelstore provides additional services to the user.
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Your zettel are stored typically as files in a specific directory.
If you have not explicitly specified the directory, a default directory will be used.
The directory has to be specified at [[startup time|00001004010000]].
Nested directories are not supported (yet).

Every file in this directory that should be monitored by Zettelstore must have a file name that begins with 14 digits (0-9), the [[zettel identifier|00001006050000]].
If you create a new zettel via the [[web user interface|00001014000000]] or via the [[API|00001012053200]], the zettel identifier will be the timestamp of the current date and time (format is ''YYYYMMDDhhmmss'').
This allows zettel to be sorted naturally by creation time.

Since the only restriction on zettel identifiers are the 14 digits, you are free to use other digit sequences.
The [[configuration zettel|00001004020000]] is one prominent example, as well as these manual zettel.
You can create these special zettel identifiers either with the __rename__ function of Zettelstore or by manually renaming the underlying zettel files.

It is allowed that the file name contains other characters after the 14 digits.
These are ignored by Zettelstore.

Two filename extensions are used by Zettelstore:
# ''.zettel'' is a format that stores metadata and content together in one file,
# the empty file extension is used, when the content must be stored in its own file, e.g. image data;







|

|

|







25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Your zettel are stored typically as files in a specific directory.
If you have not explicitly specified the directory, a default directory will be used.
The directory has to be specified at [[startup time|00001004010000]].
Nested directories are not supported (yet).

Every file in this directory that should be monitored by Zettelstore must have a file name that begins with 14 digits (0-9), the [[zettel identifier|00001006050000]].
If you create a new zettel via the [[web user interface|00001014000000]] or via the [[API|00001012053200]], the zettel identifier will be the timestamp of the current date and time (format is ''YYYYMMDDhhmmss'').
This allows zettel to be sorted naturally by creation time.[^Zettel identifier format will be migrated to a new format after version 0.19, without reference to the creation date. See [[Alphanumeric Zettel Identifier|00001006050200]] for some details.]

Since the only restriction on zettel identifiers are the 14 digits, you are free to use other digit sequences.[^Zettel identifier format will be migrated to a new format after version 0.19, without reference to the creation date.]
The [[configuration zettel|00001004020000]] is one prominent example, as well as these manual zettel.
You can create these special zettel by manually renaming the underlying zettel files.

It is allowed that the file name contains other characters after the 14 digits.
These are ignored by Zettelstore.

Two filename extensions are used by Zettelstore:
# ''.zettel'' is a format that stores metadata and content together in one file,
# the empty file extension is used, when the content must be stored in its own file, e.g. image data;
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

To allow changing predefined zettel, both the file store and the internal zettel store are internally chained together.
If you change a zettel, it will be always stored as a file.
If a zettel is requested, Zettelstore will first try to read that zettel from a file.
If such a file was not found, the internal zettel store is searched secondly.

Therefore, the file store ""shadows"" the internal zettel store.
If you want to read the original zettel, you either have to delete the zettel (which removes it from the file directory), or you have to rename it to another zettel identifier.
Now we have two places where zettel are stored: in the specific directory and within the Zettelstore software.

* [[List of predefined zettel|00001005090000]]

=== Boxes: alternative ways to store zettel
As described above, a zettel may be stored as a file inside a directory or inside the Zettelstore software itself.
Zettelstore allows other ways to store zettel by providing an abstraction called __box__.[^Formerly, zettel were stored physically in boxes, often made of wood.]







|







70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

To allow changing predefined zettel, both the file store and the internal zettel store are internally chained together.
If you change a zettel, it will be always stored as a file.
If a zettel is requested, Zettelstore will first try to read that zettel from a file.
If such a file was not found, the internal zettel store is searched secondly.

Therefore, the file store ""shadows"" the internal zettel store.
If you want to read the original zettel, you have to delete the zettel (which removes it from the file directory).
Now we have two places where zettel are stored: in the specific directory and within the Zettelstore software.

* [[List of predefined zettel|00001005090000]]

=== Boxes: alternative ways to store zettel
As described above, a zettel may be stored as a file inside a directory or inside the Zettelstore software itself.
Zettelstore allows other ways to store zettel by providing an abstraction called __box__.[^Formerly, zettel were stored physically in boxes, often made of wood.]

Changes to docs/manual/00001005090000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18


19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

47
48
49

50
51
id: 00001005090000
title: List of predefined zettel
role: manual
tags: #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20231129173425

The following table lists all predefined zettel with their purpose.

|= Identifier :|= Title | Purpose
| [[00000000000001]] | Zettelstore Version | Contains the version string of the running Zettelstore
| [[00000000000002]] | Zettelstore Host | Contains the name of the computer running the Zettelstore
| [[00000000000003]] | Zettelstore Operating System | Contains the operating system and CPU architecture of the computer running the Zettelstore
| [[00000000000004]] | Zettelstore License | Lists the license of Zettelstore
| [[00000000000005]] | Zettelstore Contributors | Lists all contributors of Zettelstore
| [[00000000000006]] | Zettelstore Dependencies | Lists all licensed content
| [[00000000000007]] | Zettelstore Log | Lists the last 8192 log messages


| [[00000000000020]] | Zettelstore Box Manager | Contains some statistics about zettel boxes and the the index process
| [[00000000000090]] | Zettelstore Supported Metadata Keys | Contains all supported metadata keys, their [[types|00001006030000]], and more
| [[00000000000092]] | Zettelstore Supported Parser | Lists all supported values for metadata [[syntax|00001006020000#syntax]] that are recognized by Zettelstore
| [[00000000000096]] | Zettelstore Startup Configuration | Contains the effective values of the [[startup configuration|00001004010000]]
| [[00000000000100]] | Zettelstore Runtime Configuration | Allows to [[configure Zettelstore at runtime|00001004020000]]

| [[00000000010100]] | Zettelstore Base HTML Template | Contains the general layout of the HTML view
| [[00000000010200]] | Zettelstore Login Form HTML Template | Layout of the login form, when authentication is [[enabled|00001010040100]]
| [[00000000010300]] | Zettelstore List Zettel HTML Template | Used when displaying a list of zettel
| [[00000000010401]] | Zettelstore Detail HTML Template | Layout for the HTML detail view of one zettel
| [[00000000010402]] | Zettelstore Info HTML Template | Layout for the information view of a specific zettel
| [[00000000010403]] | Zettelstore Form HTML Template | Form that is used to create a new or to change an existing zettel that contains text
| [[00000000010404]] | Zettelstore Rename Form HTML Template | View that is displayed to change the [[zettel identifier|00001006050000]]
| [[00000000010405]] | Zettelstore Delete HTML Template | View to confirm the deletion of a zettel
| [[00000000010700]] | Zettelstore Error HTML Template | View to show an error message
| [[00000000019000]] | Zettelstore Sxn Start Code | Starting point of sxn functions to build the templates
| [[00000000019990]] | Zettelstore Sxn Base Code | Base sxn functions to build the templates
| [[00000000020001]] | Zettelstore Base CSS | System-defined CSS file that is included by the [[Base HTML Template|00000000010100]]
| [[00000000025001]] | Zettelstore User CSS | User-defined CSS file that is included by the [[Base HTML Template|00000000010100]]
| [[00000000040001]] | Generic Emoji | Image that is shown if [[original image reference|00001007040322]] is invalid
| [[00000000060010]] | zettel | [[Role zettel|00001012051800]] for the role ""[[zettel|00001006020100#zettel]]""
| [[00000000060020]] | confguration | [[Role zettel|00001012051800]] for the role ""[[confguration|00001006020100#confguration]]""
| [[00000000060030]] | role | [[Role zettel|00001012051800]] for the role ""[[role|00001006020100#role]]""
| [[00000000060040]] | tag | [[Role zettel|00001012051800]] for the role ""[[tag|00001006020100#tag]]""
| [[00000000090000]] | New Menu | Contains items that should be in the zettel template menu
| [[00000000090001]] | New Zettel | Template for a new zettel with role ""[[zettel|00001006020100#zettel]]""
| [[00000000090002]] | New User | Template for a new [[user zettel|00001010040200]]
| [[00000000090003]] | New Tag | Template for a new [[tag zettel|00001006020100#tag]]
| [[00000000090004]] | New Role | Template for a new [[role zettel|00001006020100#role]]

| [[00010000000000]] | Home | Default home zettel, contains some welcome information

If a zettel is not linked, it is not accessible for the current user.


**Important:** All identifier may change until a stable version of the software is released.






|

|









>
>





>






<
















>



>


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
id: 00001005090000
title: List of predefined zettel
role: manual
tags: #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240711183318

The following table lists all predefined zettel with their purpose.[^Zettel identifier format will be migrated to a new format after version 0.19.]

|= Identifier :|= Title | Purpose
| [[00000000000001]] | Zettelstore Version | Contains the version string of the running Zettelstore
| [[00000000000002]] | Zettelstore Host | Contains the name of the computer running the Zettelstore
| [[00000000000003]] | Zettelstore Operating System | Contains the operating system and CPU architecture of the computer running the Zettelstore
| [[00000000000004]] | Zettelstore License | Lists the license of Zettelstore
| [[00000000000005]] | Zettelstore Contributors | Lists all contributors of Zettelstore
| [[00000000000006]] | Zettelstore Dependencies | Lists all licensed content
| [[00000000000007]] | Zettelstore Log | Lists the last 8192 log messages
| [[00000000000008]] | Zettelstore Memory | Some statistics about main memory usage
| [[00000000000009]] | Zettelstore Sx Engine | Statistics about the [[Sx|https://t73f.de/r/sx]] engine, which interprets symbolic expressions
| [[00000000000020]] | Zettelstore Box Manager | Contains some statistics about zettel boxes and the the index process
| [[00000000000090]] | Zettelstore Supported Metadata Keys | Contains all supported metadata keys, their [[types|00001006030000]], and more
| [[00000000000092]] | Zettelstore Supported Parser | Lists all supported values for metadata [[syntax|00001006020000#syntax]] that are recognized by Zettelstore
| [[00000000000096]] | Zettelstore Startup Configuration | Contains the effective values of the [[startup configuration|00001004010000]]
| [[00000000000100]] | Zettelstore Runtime Configuration | Allows to [[configure Zettelstore at runtime|00001004020000]]
| [[00000000000102]] | Zettelstore Warnings | Warnings about potential problematic zettel identifier
| [[00000000010100]] | Zettelstore Base HTML Template | Contains the general layout of the HTML view
| [[00000000010200]] | Zettelstore Login Form HTML Template | Layout of the login form, when authentication is [[enabled|00001010040100]]
| [[00000000010300]] | Zettelstore List Zettel HTML Template | Used when displaying a list of zettel
| [[00000000010401]] | Zettelstore Detail HTML Template | Layout for the HTML detail view of one zettel
| [[00000000010402]] | Zettelstore Info HTML Template | Layout for the information view of a specific zettel
| [[00000000010403]] | Zettelstore Form HTML Template | Form that is used to create a new or to change an existing zettel that contains text

| [[00000000010405]] | Zettelstore Delete HTML Template | View to confirm the deletion of a zettel
| [[00000000010700]] | Zettelstore Error HTML Template | View to show an error message
| [[00000000019000]] | Zettelstore Sxn Start Code | Starting point of sxn functions to build the templates
| [[00000000019990]] | Zettelstore Sxn Base Code | Base sxn functions to build the templates
| [[00000000020001]] | Zettelstore Base CSS | System-defined CSS file that is included by the [[Base HTML Template|00000000010100]]
| [[00000000025001]] | Zettelstore User CSS | User-defined CSS file that is included by the [[Base HTML Template|00000000010100]]
| [[00000000040001]] | Generic Emoji | Image that is shown if [[original image reference|00001007040322]] is invalid
| [[00000000060010]] | zettel | [[Role zettel|00001012051800]] for the role ""[[zettel|00001006020100#zettel]]""
| [[00000000060020]] | confguration | [[Role zettel|00001012051800]] for the role ""[[confguration|00001006020100#confguration]]""
| [[00000000060030]] | role | [[Role zettel|00001012051800]] for the role ""[[role|00001006020100#role]]""
| [[00000000060040]] | tag | [[Role zettel|00001012051800]] for the role ""[[tag|00001006020100#tag]]""
| [[00000000090000]] | New Menu | Contains items that should be in the zettel template menu
| [[00000000090001]] | New Zettel | Template for a new zettel with role ""[[zettel|00001006020100#zettel]]""
| [[00000000090002]] | New User | Template for a new [[user zettel|00001010040200]]
| [[00000000090003]] | New Tag | Template for a new [[tag zettel|00001006020100#tag]]
| [[00000000090004]] | New Role | Template for a new [[role zettel|00001006020100#role]]
| [[00009999999998]] | Zettelstore Application Directory | Maps application name to application specific zettel
| [[00010000000000]] | Home | Default home zettel, contains some welcome information

If a zettel is not linked, it is not accessible for the current user.
In most cases, you must at least enable [[''expert-mode''|00001004020000#expert-mode]].

**Important:** All identifier may change until a stable version of the software is released.

Changes to docs/manual/00001006010000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12

13
14
15
16
17
18
19
20
21
22
23
24
25
26
id: 00001006010000
title: Syntax of Metadata
role: manual
tags: #manual #syntax #zettelstore
syntax: zmk

modified: 20220218131923

The metadata of a zettel is a collection of key-value pairs.
The syntax roughly resembles the internal header of an email ([[RFC5322|https://tools.ietf.org/html/rfc5322]]).

The key is a sequence of alphanumeric characters, a hyphen-minus character (""''-''"", U+002D) is also allowed.
It begins at the first position of a new line.


A key is separated from its value either by
* a colon character (""'':''""),
* a non-empty sequence of space characters,
* a sequence of space characters, followed by a colon, followed by a sequence of space characters.

A Value is a sequence of printable characters.
If the value should be continued in the following line, that following line (""continuation line"") must begin with a non-empty sequence of space characters.
The rest of the following line will be interpreted as the next part of the value.
There can be more than one continuation line for a value.

A non-continuation line that contains a possibly empty sequence of characters, followed by the percent sign character (""''%''"") is treated as a comment line.
It will be ignored.






>
|






>






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
id: 00001006010000
title: Syntax of Metadata
role: manual
tags: #manual #syntax #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240219193158

The metadata of a zettel is a collection of key-value pairs.
The syntax roughly resembles the internal header of an email ([[RFC5322|https://tools.ietf.org/html/rfc5322]]).

The key is a sequence of alphanumeric characters, a hyphen-minus character (""''-''"", U+002D) is also allowed.
It begins at the first position of a new line.
Uppercase letters of a key are translated to their lowercase equivalence.

A key is separated from its value either by
* a colon character (""'':''""),
* a non-empty sequence of space characters,
* a sequence of space characters, followed by a colon, followed by a sequence of space characters.

A value is a sequence of printable characters.
If the value should be continued in the following line, that following line (""continuation line"") must begin with a non-empty sequence of space characters.
The rest of the following line will be interpreted as the next part of the value.
There can be more than one continuation line for a value.

A non-continuation line that contains a possibly empty sequence of characters, followed by the percent sign character (""''%''"") is treated as a comment line.
It will be ignored.

Changes to docs/manual/00001006020000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001006020000
title: Supported Metadata Keys
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230704161159

Although you are free to define your own metadata, by using any key (according to the [[syntax|00001006010000]]), some keys have a special meaning that is enforced by Zettelstore.
See the [[computed list of supported metadata keys|00000000000090]] for details.

Most keys conform to a [[type|00001006030000]].

; [!author|''author'']






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001006020000
title: Supported Metadata Keys
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240711183409

Although you are free to define your own metadata, by using any key (according to the [[syntax|00001006010000]]), some keys have a special meaning that is enforced by Zettelstore.
See the [[computed list of supported metadata keys|00000000000090]] for details.

Most keys conform to a [[type|00001006030000]].

; [!author|''author'']
33
34
35
36
37
38
39






40
41
42
43
44
45
46
  This is a computed value.
  There is no need to set it via Zettelstore.
  
  If it is not stored within a zettel, it will be computed based on the value of the [[Zettel Identifier|00001006050000]]: if it contains a value >= 19700101000000, it will be coerced to da date/time; otherwise the version time of the running software will be used.

  Please note that the value von ''created'' will be different (in most cases) to the value of [[''id''|#id]] / the zettel identifier, because it is exact up to the second.
  When calculating a zettel identifier, Zettelstore tries to set the second value to zero, if possible.






; [!credential|''credential'']
: Contains the hashed password, as it was emitted by [[``zettelstore password``|00001004051400]].
  It is internally created by hashing the password, the [[zettel identifier|00001006050000]], and the value of the ''ident'' key.

  It is only used for zettel with a ''role'' value of ""user"".
; [!dead|''dead'']
: Property that contains all references that does __not__ identify a zettel.







>
>
>
>
>
>







33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
  This is a computed value.
  There is no need to set it via Zettelstore.
  
  If it is not stored within a zettel, it will be computed based on the value of the [[Zettel Identifier|00001006050000]]: if it contains a value >= 19700101000000, it will be coerced to da date/time; otherwise the version time of the running software will be used.

  Please note that the value von ''created'' will be different (in most cases) to the value of [[''id''|#id]] / the zettel identifier, because it is exact up to the second.
  When calculating a zettel identifier, Zettelstore tries to set the second value to zero, if possible.
; [!created-missing|''created-missing'']
: If set to ""true"", the value of [[''created''|#created]] was not stored within a zettel.
  To allow the migration of [[zettel identifier|00001006050000]] to a new scheme, you should update the value of ''created'' to a reasonable value.
  Otherwise you might lose that information in future releases.

  This key will be removed when the migration to a new zettel identifier format has been completed.
; [!credential|''credential'']
: Contains the hashed password, as it was emitted by [[``zettelstore password``|00001004051400]].
  It is internally created by hashing the password, the [[zettel identifier|00001006050000]], and the value of the ''ident'' key.

  It is only used for zettel with a ''role'' value of ""user"".
; [!dead|''dead'']
: Property that contains all references that does __not__ identify a zettel.
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
; [!url|''url'']
: Defines an URL / URI for this zettel that possibly references external material.
  One use case is to specify the document that the current zettel comments on.
  The URL will be rendered special in the [[web user interface|00001014000000]] if you use the default template.
; [!useless-files|''useless-files'']
: Contains the file names that are rejected to serve the content of a zettel.
  Is used for [[directory boxes|00001004011400]] and [[file boxes|00001004011200#file]].
  If a zettel is renamed or deleted, these files will be deleted.
; [!user-id|''user-id'']
: Provides some unique user identification for an [[user zettel|00001010040200]].
  It is used as a user name for authentication.

  It is only used for zettel with a ''role'' value of ""user"".
; [!user-role|''user-role'']
: Defines the basic privileges of an authenticated user, e.g. reading / changing zettel.







|







138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
; [!url|''url'']
: Defines an URL / URI for this zettel that possibly references external material.
  One use case is to specify the document that the current zettel comments on.
  The URL will be rendered special in the [[web user interface|00001014000000]] if you use the default template.
; [!useless-files|''useless-files'']
: Contains the file names that are rejected to serve the content of a zettel.
  Is used for [[directory boxes|00001004011400]] and [[file boxes|00001004011200#file]].
  If a zettel is deleted, these files will also be deleted.
; [!user-id|''user-id'']
: Provides some unique user identification for an [[user zettel|00001010040200]].
  It is used as a user name for authentication.

  It is only used for zettel with a ''role'' value of ""user"".
; [!user-role|''user-role'']
: Defines the basic privileges of an authenticated user, e.g. reading / changing zettel.

Changes to docs/manual/00001006020400.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001006020400
title: Supported values for metadata key ''read-only''
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk

modified: 20211124132040

A zettel can be marked as read-only, if it contains a metadata value for key
[[''read-only''|00001006020000#read-only]].
If user authentication is [[enabled|00001010040100]], it is possible to allow some users to change the zettel,
depending on their [[user role|00001010070300]].
Otherwise, the read-only mark is just a binary value.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001006020400
title: Supported values for metadata key ''read-only''
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210126175322
modified: 20211124132040

A zettel can be marked as read-only, if it contains a metadata value for key
[[''read-only''|00001006020000#read-only]].
If user authentication is [[enabled|00001010040100]], it is possible to allow some users to change the zettel,
depending on their [[user role|00001010070300]].
Otherwise, the read-only mark is just a binary value.

Changes to docs/manual/00001006030000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
id: 00001006030000
title: Supported Key Types
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230612183742

All [[supported metadata keys|00001006020000]] conform to a type.

User-defined metadata keys conform also to a type, based on the suffix of the key.

|=Suffix|Type
| ''-date'' | [[Timestamp|00001006034500]]
| ''-number'' | [[Number|00001006033000]]
| ''-role'' | [[Word|00001006035500]]
| ''-set'' | [[WordSet|00001006036000]]
| ''-time'' | [[Timestamp|00001006034500]]
| ''-title'' | [[Zettelmarkup|00001006036500]]
| ''-url'' | [[URL|00001006035000]]
| ''-zettel''  | [[Identifier|00001006032000]]
| ''-zid''  | [[Identifier|00001006032000]]
| ''-zids''  | [[IdentifierSet|00001006032500]]
| any other suffix | [[EString|00001006031500]]






|









<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

17
18
19
20
21
22
23
id: 00001006030000
title: Supported Key Types
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240219161909

All [[supported metadata keys|00001006020000]] conform to a type.

User-defined metadata keys conform also to a type, based on the suffix of the key.

|=Suffix|Type
| ''-date'' | [[Timestamp|00001006034500]]
| ''-number'' | [[Number|00001006033000]]
| ''-role'' | [[Word|00001006035500]]

| ''-time'' | [[Timestamp|00001006034500]]
| ''-title'' | [[Zettelmarkup|00001006036500]]
| ''-url'' | [[URL|00001006035000]]
| ''-zettel''  | [[Identifier|00001006032000]]
| ''-zid''  | [[Identifier|00001006032000]]
| ''-zids''  | [[IdentifierSet|00001006032500]]
| any other suffix | [[EString|00001006031500]]
35
36
37
38
39
40
41
42
43
* [[IdentifierSet|00001006032500]]
* [[Number|00001006033000]]
* [[String|00001006033500]]
* [[TagSet|00001006034000]]
* [[Timestamp|00001006034500]]
* [[URL|00001006035000]]
* [[Word|00001006035500]]
* [[WordSet|00001006036000]]
* [[Zettelmarkup|00001006036500]]







<

34
35
36
37
38
39
40

41
* [[IdentifierSet|00001006032500]]
* [[Number|00001006033000]]
* [[String|00001006033500]]
* [[TagSet|00001006034000]]
* [[Timestamp|00001006034500]]
* [[URL|00001006035000]]
* [[Word|00001006035500]]

* [[Zettelmarkup|00001006036500]]

Changes to docs/manual/00001006030500.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001006030500
title: Boolean Value
role: manual
tags: #manual #reference #zettel #zettelstore
syntax: zmk

modified: 20220304114040

On some places, metadata values are interpreted as a truth value.

Every character sequence that begins with a ""0"", ""F"", ""N"", ""f"", or a ""n"" is interpreted as the boolean ""false"" value.
All values are interpreted as the boolean ""true"" value.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001006030500
title: Boolean Value
role: manual
tags: #manual #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20220304114040

On some places, metadata values are interpreted as a truth value.

Every character sequence that begins with a ""0"", ""F"", ""N"", ""f"", or a ""n"" is interpreted as the boolean ""false"" value.
All values are interpreted as the boolean ""true"" value.

Deleted docs/manual/00001006036000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
id: 00001006036000
title: WordSet Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
created: 20210212135017
modified: 20230419175745

Values of this type denote a (sorted) set of [[words|00001006035500]].

A set is different to a list, as no duplicate values are allowed.

=== Allowed values
Must be a sequence of at least one word, separated by space characters.

=== Query comparison
All comparisons are done case-insensitive, i.e. ""hell"" will be the prefix of ""World, Hello"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<








































Changes to docs/manual/00001006050000.zettel.

1
2
3
4
5

6
7
8
9
10


11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29











id: 00001006050000
title: Zettel identifier
role: manual
tags: #design #manual #zettelstore
syntax: zmk

modified: 20210721123222

Each zettel is given a unique identifier.
To some degree, the zettel identifier is part of the metadata.
Basically, the identifier is given by the [[Zettelstore|00001005000000]] software.



Every zettel identifier consists of 14 digits.
They resemble a timestamp: the first four digits could represent the year, the
next two represent the month, following by day, hour, minute, and second.

This allows to order zettel chronologically in a canonical way.

In most cases the zettel identifier is the timestamp when the zettel was created.

However, the Zettelstore software just checks for exactly 14 digits.
Anybody is free to assign a ""non-timestamp"" identifier to a zettel, e.g. with
a month part of ""35"" or with ""99"" as the last two digits.

Some zettel identifier are [[reserved|00001006055000]] and should not be used otherwise.
All identifiers of zettel initially provided by an empty Zettelstore begin with ""000000"", except the home zettel ''00010000000000''.
Zettel identifier of this manual have be chosen to begin with ""000010"".

A zettel can have any identifier that contains 14 digits and that is not in use
by another zettel managed by the same Zettelstore.
















>
|




>
>



|















>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
id: 00001006050000
title: Zettel identifier
role: manual
tags: #design #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240708154551

Each zettel is given a unique identifier.
To some degree, the zettel identifier is part of the metadata.
Basically, the identifier is given by the [[Zettelstore|00001005000000]] software.

=== Timestamp-based identifier

Every zettel identifier consists of 14 digits.
They resemble a timestamp: the first four digits could represent the year, the
next two represent the month, following by day, hour, minute, and second.[^Zettel identifier format will be migrated to a new format after version 0.19, without reference to the creation date.]

This allows to order zettel chronologically in a canonical way.

In most cases the zettel identifier is the timestamp when the zettel was created.

However, the Zettelstore software just checks for exactly 14 digits.
Anybody is free to assign a ""non-timestamp"" identifier to a zettel, e.g. with
a month part of ""35"" or with ""99"" as the last two digits.

Some zettel identifier are [[reserved|00001006055000]] and should not be used otherwise.
All identifiers of zettel initially provided by an empty Zettelstore begin with ""000000"", except the home zettel ''00010000000000''.
Zettel identifier of this manual have be chosen to begin with ""000010"".

A zettel can have any identifier that contains 14 digits and that is not in use
by another zettel managed by the same Zettelstore.

=== Identifiers with four alphanumeric characters
In the future, above identifier format will change.
The migration to the new format starts with Zettelstore version 0.18 and will last approximately until version 0.22.

Above described format of 14 digits will be changed to four alphanumeric characters, i.e. the digits ''0'' to ''9'', and the letters ''a'' to ''z''.
You might note that using 14 digits you are allowed a little less than 10^^14^^ Zettel, i.e. more than 999 trillion zettel, while the new scheme only allows you to create 36^^4^^-1 zettel (1679615 zettel, to be exact).
Since Zettelstore is a single-user system, more than a million zettel should be enough.
However, there must be a way to replace an identifier with 14 digits by an identifier with four characters.

As a first step, the list of [[reserved zettel identifier|00001006055000]] is updated, as well as ways of client software to use predefined identifier.

Added docs/manual/00001006050200.zettel.



































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
id: 00001006050200
title: Alphanumeric Zettel Identifier
role: manual
tags: #design #manual #zettelstore
syntax: zmk
created: 20240705200557
modified: 20240711183512
precursor: 00001006050000

Timestamp-based zettel identifier (14 digits) will be migrated to a new format.
Instead of using the current date and time of zettel creation, the new format is based in incrementing zettel identifier.
When creating a new zettel, its identifier is calculated by adding one to the current maximum zettel identifier.
The external representation if the new format identifier is a sequence of four alphanumeric characters, i.e. the 36
characters ''0'' &hellip; ''9'', and ''a'' &hellip; ''z''.
The external representation is basically a ""base-36"" encoding of the number.

The characters ''A'' &hellip; ''Z'' are mapped to the lower-case ''a'' &hellip; ''z''.

=== Migration process
Please note: the following is just a plan.
Plans tend to be revised if they get in contact with reality.

; Version 0.18 (current)
: Provides some tools to check your own zettelstore for problematic zettel identifier.
  For example, zettel without metadata key ''created'' should be updated by the user, especially if the zettel identifier is below ''19700101000000''.
  Most likely, this is the case for zettel created before version 0.7 (2022-08-17).

  Zettel [[Zettelstore Warnings|00000000000102]] (''00000000000102'') lists these problematic zettel identifier.[^Only visible in [[expert mode|00001004020000#expert-mode]].]
  You should update your zettel to remove these warnings to ensure a smooth migration.

  If you have developed an application, that defines a specific zettel identifier to be used as application configuration, you should must the new zettel [[Zettelstore Application Directory|00009999999998]] (''00009999999998'').

  There is an explicit, but preliminary mapping of the old format to the new one, and vice versa.
  This mapping will be calculated with the order of the identifier in the old format.
  The zettel [[Zettelstore Identifier Mapping|00009999999999]] (''00009999999999'') will show this mapping.[^Only visible in [[expert mode|00001004020000#expert-mode]].]

; Version 0.19
: The new identifier format will be used initially internal.
  Operation to rename a zettel, i.e. assigning a new identifier to a zettel, is removed permanently.

  The old format with 14 digits is still used to create URIs and to link zettel.

  You will have some time to update your zettel data if you detect some issues.
; Version 0.20
: The internal search index is based on the new format identifier.
; Version 0.21
: The new format is used to calculate URIs and to form links.
; Version 0.22
: Old format identifier are full legacy.

Changes to docs/manual/00001006055000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
id: 00001006055000
title: Reserved zettel identifier
role: manual
tags: #design #manual #zettelstore
syntax: zmk

modified: 20220311111751

[[Zettel identifier|00001006050000]] are typically created by examine the current date and time.
By renaming a zettel, you are able to provide any sequence of 14 digits.
If no other zettel has the same identifier, you are allowed to rename a zettel.

To make things easier, you normally should not use zettel identifier that begin with four zeroes (''0000'').

All zettel provided by an empty zettelstore begin with six zeroes[^Exception: the predefined home zettel ''00010000000000''. But you can [[configure|00001004020000#home-zettel]] another zettel with another identifier as the new home zettel.].
Zettel identifier of this manual have be chosen to begin with ''000010''.

However, some external applications may need a range of specific zettel identifier to work properly.
Identifier that begin with ''00009'' can be used for such purpose.
To request a reservation, please send an email to the maintainer of Zettelstore.
The request must include the following data:
; Title
: Title of you application
; Description
: A brief description what the application is used for and why you need to reserve some zettel identifier
; Number
: Specify the amount of zettel identifier you are planning to use.
  Minimum size is 100.
  If you need more than 10.000, your justification will contain more words.

=== Reserved Zettel Identifier

|= From | To | Description
| 00000000000000 | 0000000000000 | This is an invalid zettel identifier
| 00000000000001 | 0000009999999 | [[Predefined zettel|00001005090000]]
| 00000100000000 | 0000019999999 | This [[Zettelstore manual|00001000000000]]
| 00000200000000 | 0000899999999 | Reserved for future use
| 00009000000000 | 0000999999999 | Reserved for applications

This list may change in the future.

==== External Applications
|= From | To | Description
| 00009000001000 | 00009000001999 | [[Zettel Presenter|https://zettelstore.de/contrib]], an application to display zettel as a HTML-based slideshow
| 00009000002000 | 00009000002999 | [[Zettel Blog|https://zettelstore.de/contrib]], an application to collect and transform zettel into a blog





>
|


|
<

|




|
<
<
<
<
|
<
|
<
|
<
<





|
|
|
|

|

|


<
1
2
3
4
5
6
7
8
9
10

11
12
13
14
15
16
17




18

19

20


21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

id: 00001006055000
title: Reserved zettel identifier
role: manual
tags: #design #manual #zettelstore
syntax: zmk
created: 20210721105704
modified: 20240711183638

[[Zettel identifier|00001006050000]] are typically created by examine the current date and time.
By renaming the name of the underlying zettel file, you are able to provide any sequence of 14 digits[^Zettel identifier format will be migrated to a new format after version 0.19.].


To make things easier, you must not use zettel identifier that begin with four zeroes (''0000'').

All zettel provided by an empty zettelstore begin with six zeroes[^Exception: the predefined home zettel ''00010000000000''. But you can [[configure|00001004020000#home-zettel]] another zettel with another identifier as the new home zettel.].
Zettel identifier of this manual have be chosen to begin with ''000010''.

However, some external applications may need at least one defined zettel identifier to work properly.




Zettel [[Zettelstore Application Directory|00009999999998]] (''00009999999998'') can be used to associate a name to a zettel identifier.

For example, if your application is named ""app"", you create a metadata key ''app-zid''.

Its value is the zettel identifier of the zettel that configures your application.



=== Reserved Zettel Identifier

|= From | To | Description
| 00000000000000 | 0000000000000 | This is an invalid zettel identifier
| 00000000000001 | 0000099999999 | [[Predefined zettel|00001005090000]]
| 00001000000000 | 0000109999999 | This [[Zettelstore manual|00001000000000]]
| 00001100000000 | 0000899999999 | Reserved, do not use.
| 00009000000000 | 0000999999999 | Reserved for applications (legacy)

Since the format of zettel identifier will change in the near future, no external application is allowed to use the range ''00000000000001'' &hellip; ''0000999999999''.

==== External Applications (Legacy)
|= From | To | Description
| 00009000001000 | 00009000001999 | [[Zettel Presenter|https://zettelstore.de/contrib]], an application to display zettel as a HTML-based slideshow

Changes to docs/manual/00001007010000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007010000
title: Zettelmarkup: General Principles
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20211124175047

Any document can be thought as a sequence of paragraphs and other [[block-structured elements|00001007030000]] (""blocks""), such as [[headings|00001007030300]], [[lists|00001007030200]], quotations, and code blocks.
Some of these blocks can contain other blocks, for example lists may contain other lists or paragraphs.
Other blocks contain [[inline-structured elements|00001007040000]] (""inlines""), such as text, [[links|00001007040310]], emphasized text, and images.

With the exception of lists and tables, the markup for blocks always begins at the first position of a line with three or more identical characters.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007010000
title: Zettelmarkup: General Principles
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20211124175047

Any document can be thought as a sequence of paragraphs and other [[block-structured elements|00001007030000]] (""blocks""), such as [[headings|00001007030300]], [[lists|00001007030200]], quotations, and code blocks.
Some of these blocks can contain other blocks, for example lists may contain other lists or paragraphs.
Other blocks contain [[inline-structured elements|00001007040000]] (""inlines""), such as text, [[links|00001007040310]], emphasized text, and images.

With the exception of lists and tables, the markup for blocks always begins at the first position of a line with three or more identical characters.

Changes to docs/manual/00001007020000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007020000
title: Zettelmarkup: Basic Definitions
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218130713

Every Zettelmarkup content consists of a sequence of Unicode code-points.
Unicode code-points are called in the following as **character**s.

Characters are encoded with UTF-8.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007020000
title: Zettelmarkup: Basic Definitions
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218130713

Every Zettelmarkup content consists of a sequence of Unicode code-points.
Unicode code-points are called in the following as **character**s.

Characters are encoded with UTF-8.

Changes to docs/manual/00001007030000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030000
title: Zettelmarkup: Block-Structured Elements
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220311181036

Every markup for blocks-structured elements (""blocks"") begins at the very first position of a line.

There are five kinds of blocks: lists, one-line blocks, line-range blocks, tables, and paragraphs.

=== Lists





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030000
title: Zettelmarkup: Block-Structured Elements
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220311181036

Every markup for blocks-structured elements (""blocks"") begins at the very first position of a line.

There are five kinds of blocks: lists, one-line blocks, line-range blocks, tables, and paragraphs.

=== Lists

Changes to docs/manual/00001007030100.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030100
title: Zettelmarkup: Description Lists
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218131155

A description list is a sequence of terms to be described together with the descriptions of each term.
Every term can described in multiple ways.

A description term (short: __term__) is specified with one semicolon (""'';''"", U+003B) at the first position, followed by a space character and the described term, specified as a sequence of line elements.
If the following lines should also be part of the term, exactly two spaces must be given at the beginning of each following line.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030100
title: Zettelmarkup: Description Lists
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218131155

A description list is a sequence of terms to be described together with the descriptions of each term.
Every term can described in multiple ways.

A description term (short: __term__) is specified with one semicolon (""'';''"", U+003B) at the first position, followed by a space character and the described term, specified as a sequence of line elements.
If the following lines should also be part of the term, exactly two spaces must be given at the beginning of each following line.

Changes to docs/manual/00001007030200.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030200
title: Zettelmarkup: Nested Lists
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218133902

There are thee kinds of lists that can be nested: ordered lists, unordered lists, and quotation lists.

Ordered lists are specified with the number sign (""''#''"", U+0023), unordered lists use the asterisk (""''*''"", U+002A), and quotation lists are specified with the greater-than sing (""''>''"", U+003E).
Let's call these three characters __list characters__.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030200
title: Zettelmarkup: Nested Lists
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218133902

There are thee kinds of lists that can be nested: ordered lists, unordered lists, and quotation lists.

Ordered lists are specified with the number sign (""''#''"", U+0023), unordered lists use the asterisk (""''*''"", U+002A), and quotation lists are specified with the greater-than sing (""''>''"", U+003E).
Let's call these three characters __list characters__.

Changes to docs/manual/00001007030300.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030300
title: Zettelmarkup: Headings
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218133755

To specify a (sub-) section of a zettel, you should use the headings syntax: at
the beginning of a new line type at least three equal signs (""''=''"", U+003D), plus at least one
space and enter the text of the heading as [[inline elements|00001007040000]].

```zmk





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030300
title: Zettelmarkup: Headings
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218133755

To specify a (sub-) section of a zettel, you should use the headings syntax: at
the beginning of a new line type at least three equal signs (""''=''"", U+003D), plus at least one
space and enter the text of the heading as [[inline elements|00001007040000]].

```zmk

Changes to docs/manual/00001007030500.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030500
title: Zettelmarkup: Verbatim Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218131500

Verbatim blocks are used to enter text that should not be interpreted.
They begin with at least three grave accent characters (""''`''"", U+0060) at the first position of a line.
Alternatively, a modifier letter grave accent (""''ˋ''"", U+02CB) is also allowed[^On some devices, such as an iPhone / iPad, a grave accent character is harder to enter and is often confused with a modifier letter grave accent.].

You can add some [[attributes|00001007050000]] on the beginning line of a verbatim block, following the initiating characters.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030500
title: Zettelmarkup: Verbatim Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218131500

Verbatim blocks are used to enter text that should not be interpreted.
They begin with at least three grave accent characters (""''`''"", U+0060) at the first position of a line.
Alternatively, a modifier letter grave accent (""''ˋ''"", U+02CB) is also allowed[^On some devices, such as an iPhone / iPad, a grave accent character is harder to enter and is often confused with a modifier letter grave accent.].

You can add some [[attributes|00001007050000]] on the beginning line of a verbatim block, following the initiating characters.

Changes to docs/manual/00001007030600.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030600
title: Zettelmarkup: Quotation Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218131806

A simple way to enter a quotation is to use the [[quotation list|00001007030200]].
A quotation list loosely follows the convention of quoting text within emails.
However, if you want to attribute the quotation to someone, a quotation block is more appropriately.

This kind of line-range block begins with at least three less-than characters (""''<''"", U+003C) at the first position of a line.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030600
title: Zettelmarkup: Quotation Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218131806

A simple way to enter a quotation is to use the [[quotation list|00001007030200]].
A quotation list loosely follows the convention of quoting text within emails.
However, if you want to attribute the quotation to someone, a quotation block is more appropriately.

This kind of line-range block begins with at least three less-than characters (""''<''"", U+003C) at the first position of a line.

Changes to docs/manual/00001007030700.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030700
title: Zettelmarkup: Verse Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218132432

Sometimes, you want to enter text with significant space characters at the beginning of each line and with significant line endings.
Poetry is one typical example.
Of course, you could help yourself with hard space characters and hard line breaks, by entering a backslash character before a space character and at the end of each line.
Using a verse block might be easier.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030700
title: Zettelmarkup: Verse Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218132432

Sometimes, you want to enter text with significant space characters at the beginning of each line and with significant line endings.
Poetry is one typical example.
Of course, you could help yourself with hard space characters and hard line breaks, by entering a backslash character before a space character and at the end of each line.
Using a verse block might be easier.

Changes to docs/manual/00001007030800.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007030800
title: Zettelmarkup: Region Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220323190829

Region blocks does not directly have a visual representation.
They just group a range of lines.
You can use region blocks to enter [[attributes|00001007050000]] that apply only to this range of lines.
One example is to enter a multi-line warning that should be visible.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007030800
title: Zettelmarkup: Region Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220323190829

Region blocks does not directly have a visual representation.
They just group a range of lines.
You can use region blocks to enter [[attributes|00001007050000]] that apply only to this range of lines.
One example is to enter a multi-line warning that should be visible.

Changes to docs/manual/00001007031000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007031000
title: Zettelmarkup: Tables
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218131107

Tables are used to show some data in a two-dimensional fashion.
In zettelmarkup, table are not specified explicitly, but by entering __table rows__.
Therefore, a table can be seen as a sequence of table rows.
A table row is nothing as a sequence of __table cells__.
The length of a table is the number of table rows, the width of a table is the maximum length of its rows.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007031000
title: Zettelmarkup: Tables
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218131107

Tables are used to show some data in a two-dimensional fashion.
In zettelmarkup, table are not specified explicitly, but by entering __table rows__.
Therefore, a table can be seen as a sequence of table rows.
A table row is nothing as a sequence of __table cells__.
The length of a table is the number of table rows, the width of a table is the maximum length of its rows.

Changes to docs/manual/00001007031140.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001007031140
title: Zettelmarkup: Query Transclusion
role: manual
tags: #manual #search #zettelmarkup #zettelstore
syntax: zmk
created: 20220809132350
modified: 20231023163751

A query transclusion is specified by the following sequence, starting at the first position in a line: ''{{{query:query-expression}}}''.
The line must literally start with the sequence ''{{{query:''.
Everything after this prefix is interpreted as a [[query expression|00001007700000]].

When evaluated, the query expression is evaluated, often resulting in a list of [[links|00001007040310]] to zettel, matching the query expression.
The result replaces the query transclusion element.






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001007031140
title: Zettelmarkup: Query Transclusion
role: manual
tags: #manual #search #zettelmarkup #zettelstore
syntax: zmk
created: 20220809132350
modified: 20240219161800

A query transclusion is specified by the following sequence, starting at the first position in a line: ''{{{query:query-expression}}}''.
The line must literally start with the sequence ''{{{query:''.
Everything after this prefix is interpreted as a [[query expression|00001007700000]].

When evaluated, the query expression is evaluated, often resulting in a list of [[links|00001007040310]] to zettel, matching the query expression.
The result replaces the query transclusion element.
45
46
47
48
49
50
51
52
53
54
55
56
57



58
59
60
61
62
63
64
65
66
: Transform the zettel list into an [[Atom 1.0|https://www.rfc-editor.org/rfc/rfc4287]]-conformant document / feed.
  The document is embedded into the referencing zettel.
; ''RSS'' (aggregate)
: Transform the zettel list into a [[RSS 2.0|https://www.rssboard.org/rss-specification]]-conformant document / feed.
  The document is embedded into the referencing zettel.
; ''KEYS'' (aggregate)
: Emit a list of all metadata keys, together with the number of zettel having the key.
; ''REINDEX'' (aggregate)
: Will be ignored.
  This action may have been copied from an existing [[API query call|00001012051400]] (or from a WebUI query), but is here superfluous (and possibly harmful).
; Any [[metadata key|00001006020000]] of type [[Word|00001006035500]], [[WordSet|00001006036000]], or [[TagSet|00001006034000]] (aggregates)
: Emit an aggregate of the given metadata key.
  The key can be given in any letter case[^Except if the key name collides with one of the above names. In this case use at least one lower case letter.].




Example:
```zmk
{{{query:tags:#search | tags}}}
```
This is a tag cloud of all tags that are used together with the tag #search:
:::example
{{{query:tags:#search | tags}}}
:::







|

|
|


>
>
>









45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
: Transform the zettel list into an [[Atom 1.0|https://www.rfc-editor.org/rfc/rfc4287]]-conformant document / feed.
  The document is embedded into the referencing zettel.
; ''RSS'' (aggregate)
: Transform the zettel list into a [[RSS 2.0|https://www.rssboard.org/rss-specification]]-conformant document / feed.
  The document is embedded into the referencing zettel.
; ''KEYS'' (aggregate)
: Emit a list of all metadata keys, together with the number of zettel having the key.
; ''REDIRECT'', ''REINDEX'' (aggregate)
: Will be ignored.
  These actions may have been copied from an existing [[API query call|00001012051400]] (or from a WebUI query), but are here superfluous (and possibly harmful).
; Any [[metadata key|00001006020000]] of type [[Word|00001006035500]] or of type [[TagSet|00001006034000]] (aggregates)
: Emit an aggregate of the given metadata key.
  The key can be given in any letter case[^Except if the key name collides with one of the above names. In this case use at least one lower case letter.].

To allow some kind of backward compatibility, an action written in uppercase letters that leads to an empty result list, will be ignored.
In this case the list of selected zettel is returned.

Example:
```zmk
{{{query:tags:#search | tags}}}
```
This is a tag cloud of all tags that are used together with the tag #search:
:::example
{{{query:tags:#search | tags}}}
:::

Changes to docs/manual/00001007040200.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007040200
title: Zettelmarkup: Literal-like formatting
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220311185110

There are some reasons to mark text that should be rendered as uninterpreted:
* Mark text as literal, sometimes as part of a program.
* Mark text as input you give into a computer via a keyboard.
* Mark text as output from some computer, e.g. shown at the command line.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007040200
title: Zettelmarkup: Literal-like formatting
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220311185110

There are some reasons to mark text that should be rendered as uninterpreted:
* Mark text as literal, sometimes as part of a program.
* Mark text as input you give into a computer via a keyboard.
* Mark text as output from some computer, e.g. shown at the command line.

Changes to docs/manual/00001007040300.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007040300
title: Zettelmarkup: Reference-like text
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20210810172531

An important aspect of knowledge work is to interconnect your zettel as well as provide links to (external) material.

There are several kinds of references that are allowed in Zettelmarkup:
* [[Links to other zettel or to (external) material|00001007040310]]
* [[Embedded zettel or (external) material|00001007040320]] (""inline transclusion"")





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007040300
title: Zettelmarkup: Reference-like text
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20210810172531

An important aspect of knowledge work is to interconnect your zettel as well as provide links to (external) material.

There are several kinds of references that are allowed in Zettelmarkup:
* [[Links to other zettel or to (external) material|00001007040310]]
* [[Embedded zettel or (external) material|00001007040320]] (""inline transclusion"")

Changes to docs/manual/00001007040324.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001007040324
title: Zettelmarkup: Inline-mode Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210811154251
modified: 20221116165428

Inline-mode transclusion applies to all zettel that are parsed in a non-trivial way, e.g. as structured textual content.
For example, textual content is assumed if the [[syntax|00001006020000#syntax]] of a zettel is ""zmk"" ([[Zettelmarkup|00001007000000]]), or ""markdown"" / ""md"" ([[Markdown|00001008010000]]).

Since this type of transclusion is at the level of [[inline-structured elements|00001007040000]], the transclude specification must be replaced with some inline-structured elements.

First, the referenced zettel is read.






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001007040324
title: Zettelmarkup: Inline-mode Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210811154251
modified: 20231222164501

Inline-mode transclusion applies to all zettel that are parsed in a non-trivial way, e.g. as structured textual content.
For example, textual content is assumed if the [[syntax|00001006020000#syntax]] of a zettel is ""zmk"" ([[Zettelmarkup|00001007000000]]), or ""markdown"" / ""md"" ([[Markdown|00001008010000]]).

Since this type of transclusion is at the level of [[inline-structured elements|00001007040000]], the transclude specification must be replaced with some inline-structured elements.

First, the referenced zettel is read.
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
   Example: ``{{00001007040322#spin}}`` is rendered as ::{{00001007040322#spin}}::{=example}.

** Just specifying the fragment identifier will reference something in the current page.
   This is not allowed, to prevent a possible endless recursion.

* If the reference is a [[hosted or based|00001007040310#link-specifications]] link / URL to an image, that image will be rendered.

  Example: ``{{//z/00000000040001}}`` is rendered as ::{{//z/00000000040001}}::{=example}

If no inline-structured elements are found, the transclude specification is replaced by an error message.

To avoid an exploding ""transclusion bomb"", a form of a [[billion laughs attack|https://en.wikipedia.org/wiki/Billion_laughs_attack]] (also known as ""XML bomb""), the total number of transclusions / expansions is limited.
The limit can be controlled by setting the value [[''max-transclusions''|00001004020000#max-transclusions]] of the runtime configuration zettel.

=== See also
[[Full transclusion|00001007031100]] does not work inside some text, but is used for [[block-structured elements|00001007030000]].







|








34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
   Example: ``{{00001007040322#spin}}`` is rendered as ::{{00001007040322#spin}}::{=example}.

** Just specifying the fragment identifier will reference something in the current page.
   This is not allowed, to prevent a possible endless recursion.

* If the reference is a [[hosted or based|00001007040310#link-specifications]] link / URL to an image, that image will be rendered.

  Example: ``{{//z/00000000040001}}{alt=Emoji}`` is rendered as ::{{//z/00000000040001}}{alt=Emoji}::{=example}

If no inline-structured elements are found, the transclude specification is replaced by an error message.

To avoid an exploding ""transclusion bomb"", a form of a [[billion laughs attack|https://en.wikipedia.org/wiki/Billion_laughs_attack]] (also known as ""XML bomb""), the total number of transclusions / expansions is limited.
The limit can be controlled by setting the value [[''max-transclusions''|00001004020000#max-transclusions]] of the runtime configuration zettel.

=== See also
[[Full transclusion|00001007031100]] does not work inside some text, but is used for [[block-structured elements|00001007030000]].

Changes to docs/manual/00001007040330.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007040330
title: Zettelmarkup: Footnotes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218130100

A footnote begins with a left square bracket, followed by a circumflex accent (""''^''"", U+005E), followed by some text, and ends with a right square bracket.

Example:

``Main text[^Footnote text.].`` is rendered in HTML as: ::Main text[^Footnote text.].::{=example}.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007040330
title: Zettelmarkup: Footnotes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210810155955
modified: 20220218130100

A footnote begins with a left square bracket, followed by a circumflex accent (""''^''"", U+005E), followed by some text, and ends with a right square bracket.

Example:

``Main text[^Footnote text.].`` is rendered in HTML as: ::Main text[^Footnote text.].::{=example}.

Changes to docs/manual/00001007040340.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007040340
title: Zettelmarkup: Citation Key
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218133447

A citation key references some external material that is part of a bibliographical collection.

Currently, Zettelstore implements this only partially, it is ""work in progress"".

However, the syntax is: beginning with a left square bracket and followed by an at sign character (""''@''"", U+0040), a the citation key is given.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007040340
title: Zettelmarkup: Citation Key
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210810155955
modified: 20220218133447

A citation key references some external material that is part of a bibliographical collection.

Currently, Zettelstore implements this only partially, it is ""work in progress"".

However, the syntax is: beginning with a left square bracket and followed by an at sign character (""''@''"", U+0040), a the citation key is given.

Changes to docs/manual/00001007040350.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007040350
title: Zettelmarkup: Mark
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220218133206

A mark allows to name a point within a zettel.
This is useful if you want to reference some content in a zettel, either with a [[link|00001007040310]] or with an [[inline-mode transclusion|00001007040324]].

A mark begins with a left square bracket, followed by an exclamation mark character (""''!''"", U+0021).
Now the optional mark name follows.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007040350
title: Zettelmarkup: Mark
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210810155955
modified: 20220218133206

A mark allows to name a point within a zettel.
This is useful if you want to reference some content in a zettel, either with a [[link|00001007040310]] or with an [[inline-mode transclusion|00001007040324]].

A mark begins with a left square bracket, followed by an exclamation mark character (""''!''"", U+0021).
Now the optional mark name follows.

Changes to docs/manual/00001007050000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001007050000
title: Zettelmarkup: Attributes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

modified: 20220630194106

Attributes allows to modify the way how material is presented.
Alternatively, they provide additional information to markup elements.
To some degree, attributes are similar to [[HTML attributes|https://html.spec.whatwg.org/multipage/dom.html#global-attributes]].

Typical use cases for attributes are to specify the (natural) [[language|00001007050100]] for a text region, to specify the [[programming language|00001007050200]] for highlighting program code, or to make white space visible in plain text.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007050000
title: Zettelmarkup: Attributes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220630194106

Attributes allows to modify the way how material is presented.
Alternatively, they provide additional information to markup elements.
To some degree, attributes are similar to [[HTML attributes|https://html.spec.whatwg.org/multipage/dom.html#global-attributes]].

Typical use cases for attributes are to specify the (natural) [[language|00001007050100]] for a text region, to specify the [[programming language|00001007050200]] for highlighting program code, or to make white space visible in plain text.

Changes to docs/manual/00001007050200.zettel.

1
2
3
4
5

6
7
id: 00001007050200
title: Zettelmarkup: Supported Attribute Values for Programming Languages
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
role: manual


TBD





>


1
2
3
4
5
6
7
8
id: 00001007050200
title: Zettelmarkup: Supported Attribute Values for Programming Languages
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
role: manual
created: 20210126175322

TBD

Changes to docs/manual/00001007706000.zettel.

1
2
3
4
5

6
7
8
9
10
id: 00001007706000
title: Search value
role: manual
tags: #manual #search #zettelstore
syntax: zmk

modified: 20220807162031

A search value specifies a value to be searched for, depending on the [[search operator|00001007705000]].

A search value should be lower case, because all comparisons are done in a case-insensitive way and there are some upper case keywords planned.





>





1
2
3
4
5
6
7
8
9
10
11
id: 00001007706000
title: Search value
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20220805150154
modified: 20220807162031

A search value specifies a value to be searched for, depending on the [[search operator|00001007705000]].

A search value should be lower case, because all comparisons are done in a case-insensitive way and there are some upper case keywords planned.

Changes to docs/manual/00001007720300.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23

24
25
26
27


28
29
30
31
32
33
34
35
36
37
id: 00001007720300
title: Query: Context Directive
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20230707204706
modified: 20230724153832

A context directive calculates the __context__ of a list of zettel identifier.
It starts with the keyword ''CONTEXT''.

Optionally you may specify some context details, after the keyword ''CONTEXT'', separated by space characters.
These are:

* ''BACKWARD'': search for context only though backward links,
* ''FORWARD'': search for context only through forward links,
* ''COST'', one or more space characters, and a positive integer: set the maximum __cost__ (default: 17),
* ''MAX'', one or more space characters, and a positive integer: set the maximum number of context zettel (default: 200).

If no ''BACKWARD'' and ''FORWARD'' is specified, a search for context zettel will be done though backward and forward links.

The cost of a context zettel is calculated iteratively:
* Each of the specified zettel hast a cost of one.
* A zettel found as a single folge zettel or single precursor zettel has the cost of the originating zettel, plus one.

* A zettel found as a single successor zettel or single predecessor zettel has the cost of the originating zettel, plus two.
* A zettel found via another link without being part of a [[set of zettel identifier|00001006032500]], has the cost of the originating zettel, plus three.
* A zettel which is part of a set of zettel identifier, has the cost of the originating zettel, plus one of the three choices above and multiplied with roughly a logarithmic value based on the size of the set.
* A zettel with the same tag, has the cost of the originating zettel, plus the number of zettel with the same tag (if it is less than eight), or the cost of the originating zettel plus two, multiplied by number of zettel with the same tag divided by four.



The maximum cost is only checked for all zettel that are not directly reachable from the initial, specified list of zettel.
This ensures that initial zettel that have only a highly used tag, will also produce some context zettel.

Despite its possibly complicated structure, this algorithm ensures in practice that the zettel context is a list of zettel, where the first elements are ""near"" to the specified zettel and the last elements are more ""distant"" to the specified zettel.
It also penalties zettel that acts as a ""hub"" to other zettel, to make it more likely that only relevant zettel appear on the context list.

This directive may be specified only once as a query directive.
A second occurence of ''CONTEXT'' is interpreted as a [[search expression|00001007701000]].
In most cases it is easier to adjust the maximum cost than to perform another context search, which is relatively expensive in terms of retrieving power.






|






>


|
|






>
|
|
|
|
>
>









|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
id: 00001007720300
title: Query: Context Directive
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20230707204706
modified: 20240209191045

A context directive calculates the __context__ of a list of zettel identifier.
It starts with the keyword ''CONTEXT''.

Optionally you may specify some context details, after the keyword ''CONTEXT'', separated by space characters.
These are:
* ''FULL'': additionally search for zettel with the same tags,
* ''BACKWARD'': search for context only though backward links,
* ''FORWARD'': search for context only through forward links,
* ''COST'': one or more space characters, and a positive integer: set the maximum __cost__ (default: 17),
* ''MAX'': one or more space characters, and a positive integer: set the maximum number of context zettel (default: 200).

If no ''BACKWARD'' and ''FORWARD'' is specified, a search for context zettel will be done though backward and forward links.

The cost of a context zettel is calculated iteratively:
* Each of the specified zettel hast a cost of one.
* A zettel found as a single folge zettel or single precursor zettel has the cost of the originating zettel, plus one.
* A zettel found as a single subordinate zettel or single superior zettel has the cost of the originating zettel, plus 1.2.
* A zettel found as a single successor zettel or single predecessor zettel has the cost of the originating zettel, plus seven.
* A zettel found via another link without being part of a [[set of zettel identifier|00001006032500]], has the cost of the originating zettel, plus two.
* A zettel which is part of a set of zettel identifier, has the cost of the originating zettel, plus one of the four choices above and multiplied with roughly a linear-logarithmic value based on the size of the set.
* A zettel with the same tag, has the cost of the originating zettel, plus a linear-logarithmic number based on the number of zettel with this tag.
  If a zettel belongs to more than one tag compared with the current zettel, there is a discount of 90% per additional tag.
  This only applies if the ''FULL'' directive was specified.

The maximum cost is only checked for all zettel that are not directly reachable from the initial, specified list of zettel.
This ensures that initial zettel that have only a highly used tag, will also produce some context zettel.

Despite its possibly complicated structure, this algorithm ensures in practice that the zettel context is a list of zettel, where the first elements are ""near"" to the specified zettel and the last elements are more ""distant"" to the specified zettel.
It also penalties zettel that acts as a ""hub"" to other zettel, to make it more likely that only relevant zettel appear on the context list.

This directive may be specified only once as a query directive.
A second occurence of ''CONTEXT'' is interpreted as a [[search expression|00001007701000]].
In most cases it is easier to adjust the maximum cost than to perform another context search, which is relatively expensive in terms of retrieving effort.

Changes to docs/manual/00001007720900.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007720900
title: Query: Items Directive
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 00010101000000
modified: 20230729120755

The items directive works on zettel that act as a ""table of contents"" for other zettel.
The [[initial zettel|00001000000000]] of this manual is one example, the [[general API description|00001012000000]] is another.
Every zettel with a certain internal structure can act as the ""table of contents"" for others.

What is a ""table of contents""?





|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007720900
title: Query: Items Directive
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20230729102142
modified: 20230729120755

The items directive works on zettel that act as a ""table of contents"" for other zettel.
The [[initial zettel|00001000000000]] of this manual is one example, the [[general API description|00001012000000]] is another.
Every zettel with a certain internal structure can act as the ""table of contents"" for others.

What is a ""table of contents""?

Changes to docs/manual/00001007770000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18



id: 00001007770000
title: Query: Action List
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20230707205246
modified: 20230707205532

With a [[list of zettel identifier|00001007710000]], a [[query directives|00001007720000]], or a [[search expression|00001007701000]], a list of zettel is selected.
__Actions__ allow to modify this list to a certain degree.

Which actions are allowed depends on the context.
However, actions are further separated into __parameter action__ and __aggregate actions__.
A parameter action just sets a parameter for an aggregate action.
An aggregate action transforms the list of selected zettel into a different, aggregate form.
Only the first aggregate form is executed, following aggregate actions are ignored.

In most contexts, valid actions include the name of metadata keys, at least of type [[Word|00001006035500]], [[WordSet|00001006036000]], or [[TagSet|00001006034000]].









|










|
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
id: 00001007770000
title: Query: Action List
role: manual
tags: #manual #search #zettelstore
syntax: zmk
created: 20230707205246
modified: 20240219161813

With a [[list of zettel identifier|00001007710000]], a [[query directives|00001007720000]], or a [[search expression|00001007701000]], a list of zettel is selected.
__Actions__ allow to modify this list to a certain degree.

Which actions are allowed depends on the context.
However, actions are further separated into __parameter action__ and __aggregate actions__.
A parameter action just sets a parameter for an aggregate action.
An aggregate action transforms the list of selected zettel into a different, aggregate form.
Only the first aggregate form is executed, following aggregate actions are ignored.

In most contexts, valid actions include the name of metadata keys, at least of type [[Word|00001006035500]] or [[TagSet|00001006034000]].

To allow some kind of backward compatibility, an action written in uppercase letters that leads to an empty result list, will be ignored.
In this case the list of selected zettel is returned.

Changes to docs/manual/00001007780000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
id: 00001007780000
title: Formal syntax of query expressions
role: manual
tags: #manual #reference #search #zettelstore
syntax: zmk
created: 20220810144539
modified: 20230731160413

```
QueryExpression   := ZettelList? QueryDirective* SearchExpression ActionExpression?
ZettelList        := (ZID (SPACE+ ZID)*).
ZID               := '0'+ ('1' .. '9'') DIGIT*
                   | ('1' .. '9') DIGIT*.
QueryDirective    := ContextDirective
                   | IdentDirective
                   | ItemsDirective
                   | UnlinkedDirective.
ContextDirective  := "CONTEXT" (SPACE+ ContextDetail)*.
ContextDetail     := "BACKWARD"

                   | "FORWARD"
                   | "COST" SPACE+ PosInt
                   | "MAX" SPACE+ PosInt.
IdentDirective    := IDENT.
ItemsDirective    := ITEMS.
UnlinkedDirective := UNLINKED (SPACE+ PHRASE SPACE+ Word)*.
SearchExpression  := SearchTerm (SPACE+ SearchTerm)*.






|











|
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
id: 00001007780000
title: Formal syntax of query expressions
role: manual
tags: #manual #reference #search #zettelstore
syntax: zmk
created: 20220810144539
modified: 20240219155949

```
QueryExpression   := ZettelList? QueryDirective* SearchExpression ActionExpression?
ZettelList        := (ZID (SPACE+ ZID)*).
ZID               := '0'+ ('1' .. '9'') DIGIT*
                   | ('1' .. '9') DIGIT*.
QueryDirective    := ContextDirective
                   | IdentDirective
                   | ItemsDirective
                   | UnlinkedDirective.
ContextDirective  := "CONTEXT" (SPACE+ ContextDetail)*.
ContextDetail     := "FULL"
                   | "BACKWARD"
                   | "FORWARD"
                   | "COST" SPACE+ PosInt
                   | "MAX" SPACE+ PosInt.
IdentDirective    := IDENT.
ItemsDirective    := ITEMS.
UnlinkedDirective := UNLINKED (SPACE+ PHRASE SPACE+ Word)*.
SearchExpression  := SearchTerm (SPACE+ SearchTerm)*.
38
39
40
41
42
43
44










45
46
SearchOperator    := '!'
                   | ('!')? ('~' | ':' | '[' | '}').
ExistOperator     := '?'
                   | '!' '?'.
PosInt            := '0'
                   | ('1' .. '9') DIGIT*.
ActionExpression  := '|' (Word (SPACE+ Word)*)?










Word              := NO-SPACE NO-SPACE*
```







>
>
>
>
>
>
>
>
>
>


39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
SearchOperator    := '!'
                   | ('!')? ('~' | ':' | '[' | '}').
ExistOperator     := '?'
                   | '!' '?'.
PosInt            := '0'
                   | ('1' .. '9') DIGIT*.
ActionExpression  := '|' (Word (SPACE+ Word)*)?
Action            := Word
                   | 'ATOM'
                   | 'KEYS'
                   | 'N' NO-SPACE*
                   | 'MAX' PosInt
                   | 'MIN' PosInt
                   | 'REDIRECT'
                   | 'REINDEX'
                   | 'RSS'
                   | 'TITLE' (SPACE Word)* .
Word              := NO-SPACE NO-SPACE*
```

Changes to docs/manual/00001007790000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

id: 00001007790000
title: Useful query expressions
role: manual
tags: #example #manual #search #zettelstore
syntax: zmk
created: 20220810144539
modified: 20230706155134

|= Query Expression |= Meaning
| [[query:role:configuration]] | Zettel that contains some configuration data for the Zettelstore
| [[query:ORDER REVERSE created LIMIT 40]] | 40 recently created zettel
| [[query:ORDER REVERSE published LIMIT 40]] | 40 recently updated zettel
| [[query:PICK 40]] | 40 random zettel, ordered by zettel identifier
| [[query:dead?]] | Zettel with invalid / dead links
| [[query:backward!? precursor!?]] | Zettel that are not referenced by other zettel
| [[query:tags!?]] | Zettel without tags
| [[query:expire? ORDER expire]] | Zettel with an expire date, ordered from the nearest to the latest
| [[query:00001007700000 CONTEXT]] | Zettel within the context of the [[given zettel|00001007700000]]







|











>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
id: 00001007790000
title: Useful query expressions
role: manual
tags: #example #manual #search #zettelstore
syntax: zmk
created: 20220810144539
modified: 20240216003702

|= Query Expression |= Meaning
| [[query:role:configuration]] | Zettel that contains some configuration data for the Zettelstore
| [[query:ORDER REVERSE created LIMIT 40]] | 40 recently created zettel
| [[query:ORDER REVERSE published LIMIT 40]] | 40 recently updated zettel
| [[query:PICK 40]] | 40 random zettel, ordered by zettel identifier
| [[query:dead?]] | Zettel with invalid / dead links
| [[query:backward!? precursor!?]] | Zettel that are not referenced by other zettel
| [[query:tags!?]] | Zettel without tags
| [[query:expire? ORDER expire]] | Zettel with an expire date, ordered from the nearest to the latest
| [[query:00001007700000 CONTEXT]] | Zettel within the context of the [[given zettel|00001007700000]]
| [[query:PICK 1 | REDIRECT]] | Redirect to a random zettel

Changes to docs/manual/00001007800000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007800000
title: Zettelmarkup: Summary of Formatting Characters
role: manual
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
created: 20220805150154
modified: 20231113191330

The following table gives an overview about the use of all characters that begin a markup element.

|= Character :|= [[Blocks|00001007030000]] <|= [[Inlines|00001007040000]] <
| ''!''  | (free) | (free)
| ''"''  | [[Verse block|00001007030700]] | [[Short inline quote|00001007040100]]





|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007800000
title: Zettelmarkup: Summary of Formatting Characters
role: manual
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
created: 20210126175322
modified: 20231113191330

The following table gives an overview about the use of all characters that begin a markup element.

|= Character :|= [[Blocks|00001007030000]] <|= [[Inlines|00001007040000]] <
| ''!''  | (free) | (free)
| ''"''  | [[Verse block|00001007030700]] | [[Short inline quote|00001007040100]]

Changes to docs/manual/00001007903000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001007903000
title: Zettelmarkup: First Steps
role: manual
tags: #manual #tutorial #zettelmarkup #zettelstore
syntax: zmk
created: 20220810182917
modified: 20220926183359

[[Zettelmarkup|00001007000000]] allows you to leave your text as it is, at least in many situations.
Some characters have a special meaning, but you have to enter them is a defined way to see a visible change.
Zettelmarkup is designed to be used for zettel, which are relatively short.
It allows to produce longer texts, but you should probably use a different tool, if you want to produce an scientific paper, to name an example.

=== Paragraphs






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001007903000
title: Zettelmarkup: First Steps
role: manual
tags: #manual #tutorial #zettelmarkup #zettelstore
syntax: zmk
created: 20220810182917
modified: 20231201135849

[[Zettelmarkup|00001007000000]] allows you to leave your text as it is, at least in many situations.
Some characters have a special meaning, but you have to enter them is a defined way to see a visible change.
Zettelmarkup is designed to be used for zettel, which are relatively short.
It allows to produce longer texts, but you should probably use a different tool, if you want to produce an scientific paper, to name an example.

=== Paragraphs
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
| ''An __emphasized__ word'' | An __emphasized__ word | Put two underscore characters before and after the text you want to emphasize
| ''Someone uses **bold** text'' | Someone uses **bold** text | Put two asterisks before and after the text you want to see bold
| ''He says: ""I love you!""'' | Her says: ""I love you!"" | Put two quotation mark characters before and after the text you want to quote.

You probably see a principle.

One nice thing about the quotation mark characters: they are rendered according to the current language.
Examples: ""english""{lang=en}, ""french""{lang=fr}, ""german""{lang=de}, ""finnish""{lang=fi}.
You will see later, how to change the current language.

=== Lists
Quite often, text consists of lists.
Zettelmarkup supports different types of lists.
The most important lists are:
* Unnumbered lists,







|







27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
| ''An __emphasized__ word'' | An __emphasized__ word | Put two underscore characters before and after the text you want to emphasize
| ''Someone uses **bold** text'' | Someone uses **bold** text | Put two asterisks before and after the text you want to see bold
| ''He says: ""I love you!""'' | Her says: ""I love you!"" | Put two quotation mark characters before and after the text you want to quote.

You probably see a principle.

One nice thing about the quotation mark characters: they are rendered according to the current language.
Examples: ""english""{lang=en}, ""french""{lang=fr}, ""german""{lang=de}.
You will see later, how to change the current language.

=== Lists
Quite often, text consists of lists.
Zettelmarkup supports different types of lists.
The most important lists are:
* Unnumbered lists,

Changes to docs/manual/00001007990000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
id: 00001007990000
title: Zettelmarkup: Cheat Sheet
role: manual
tags: #manual #reference #zettelmarkup
syntax: zmk
created: 20221209191905
modified: 20221209193310

=== Overview
This Zettelmarkup cheat sheet provides a quick overview of many Zettelmarkup elements.
It can not cover any special case.
If you need more information about any of these elements, please refer to the detailed description.

=== Basic Syntax
|[[Text formatting|00001007040100]]|''__italic text__'' &rarr; __italic text__, ''**bold text**'' &rarr; **bold text**, ''""quoted text""'' &rarr; ""quoted text""
|[[Text editing|00001007040100]]|''>>inserted text>>'' &rarr; >>inserted text>>, ''~~deleted text~~'' &rarr; ~~deleted text~~
|[[Text literal formatting|00001007040200]]|''\'\'entered text\'\''' &rarr; ''entered text'', ''``source code``'' &rarr; ``source code``, ''==text output=='' &rarr; ==text output==
|[[Superscript, subscript|00001007040100]]|''m^^2^^'' &rarr; m^^2^^, ''H,,2,,O'' &rarr; H,,2,,O
|[[Links to other zettel|00001007040310]]|''[[Link text|00001007990000]]'' &rarr; [[Link text|00001007990000]]
|[[Links to external resources|00001007040310]]|''[[Zettelstore|https://zettelstore.de]]'' &rarr; [[Zettelstore|https://zettelstore.de]]
|[[Embed an image|00001007040322]]|''{{Image text|00000000040001}}'' &rarr; {{Image text|00000000040001}}
|[[Embed content of first paragraph|00001007040324]]|''{{00001007990000}}'' &rarr; {{00001007990000}}






|







|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
id: 00001007990000
title: Zettelmarkup: Cheat Sheet
role: manual
tags: #manual #reference #zettelmarkup
syntax: zmk
created: 20221209191905
modified: 20231201140000

=== Overview
This Zettelmarkup cheat sheet provides a quick overview of many Zettelmarkup elements.
It can not cover any special case.
If you need more information about any of these elements, please refer to the detailed description.

=== Basic Syntax
|[[Text formatting|00001007040100]]|''__italic text__'' &rarr; __italic text__, ''**bold text**'' &rarr; **bold text**, ''""quoted text""'' &rarr; ""quoted text"", ''##marked text##'' &rarr; ##marked text##
|[[Text editing|00001007040100]]|''>>inserted text>>'' &rarr; >>inserted text>>, ''~~deleted text~~'' &rarr; ~~deleted text~~
|[[Text literal formatting|00001007040200]]|''\'\'entered text\'\''' &rarr; ''entered text'', ''``source code``'' &rarr; ``source code``, ''==text output=='' &rarr; ==text output==
|[[Superscript, subscript|00001007040100]]|''m^^2^^'' &rarr; m^^2^^, ''H,,2,,O'' &rarr; H,,2,,O
|[[Links to other zettel|00001007040310]]|''[[Link text|00001007990000]]'' &rarr; [[Link text|00001007990000]]
|[[Links to external resources|00001007040310]]|''[[Zettelstore|https://zettelstore.de]]'' &rarr; [[Zettelstore|https://zettelstore.de]]
|[[Embed an image|00001007040322]]|''{{Image text|00000000040001}}'' &rarr; {{Image text|00000000040001}}
|[[Embed content of first paragraph|00001007040324]]|''{{00001007990000}}'' &rarr; {{00001007990000}}

Changes to docs/manual/00001008000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001008000000
title: Other Markup Languages
role: manual
tags: #manual #zettelstore
syntax: zmk
created: 20210126175300
modified: 20230529223634

[[Zettelmarkup|00001007000000]] is not the only markup language you can use to define your content.
Zettelstore is quite agnostic with respect to markup languages.
Of course, Zettelmarkup plays an important role.
However, with the exception of zettel titles, you can use any (markup) language that is supported:

* CSS






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001008000000
title: Other Markup Languages
role: manual
tags: #manual #zettelstore
syntax: zmk
created: 20210126175300
modified: 20240413160242

[[Zettelmarkup|00001007000000]] is not the only markup language you can use to define your content.
Zettelstore is quite agnostic with respect to markup languages.
Of course, Zettelmarkup plays an important role.
However, with the exception of zettel titles, you can use any (markup) language that is supported:

* CSS
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
: Only the metadata of a zettel is ""parsed"".
  Useful for displaying the full metadata.
  The [[runtime configuration zettel|00000000000100]] uses this syntax.
  The zettel content is ignored.
; [!svg|''svg'']
: [[Scalable Vector Graphics|https://www.w3.org/TR/SVG2/]].
; [!sxn|''sxn'']
: S-Expressions, as implemented by [[sx|https://zettelstore.de/sx]].
  Often used to specify templates when rendering a zettel as HTML for the [[web user interface|00001014000000]] (with the help of sxhtml]).
; [!text|''text''], [!plain|''plain''], [!txt|''txt'']
: Plain text that must not be interpreted further.
; [!zmk|''zmk'']
: [[Zettelmarkup|00001007000000]].

The actual values are also listed in a zettel named [[Zettelstore Supported Parser|00000000000092]].

If you specify something else, your content will be interpreted as plain text.







|









45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
: Only the metadata of a zettel is ""parsed"".
  Useful for displaying the full metadata.
  The [[runtime configuration zettel|00000000000100]] uses this syntax.
  The zettel content is ignored.
; [!svg|''svg'']
: [[Scalable Vector Graphics|https://www.w3.org/TR/SVG2/]].
; [!sxn|''sxn'']
: S-Expressions, as implemented by [[Sx|https://t73f.de/r/sx]].
  Often used to specify templates when rendering a zettel as HTML for the [[web user interface|00001014000000]] (with the help of sxhtml]).
; [!text|''text''], [!plain|''plain''], [!txt|''txt'']
: Plain text that must not be interpreted further.
; [!zmk|''zmk'']
: [[Zettelmarkup|00001007000000]].

The actual values are also listed in a zettel named [[Zettelstore Supported Parser|00000000000092]].

If you specify something else, your content will be interpreted as plain text.

Changes to docs/manual/00001010040100.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001010040100
title: Enable authentication
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk

modified: 20220419192817

To enable authentication, you must create a zettel that stores [[authentication data|00001010040200]] for the owner.
Then you must reference this zettel within the [[startup configuration|00001004010000#owner]] under the key ''owner''.
Once the startup configuration contains a valid [[zettel identifier|00001006050000]] under that key, authentication is enabled.

Please note that you must also set key ''secret'' of the [[startup configuration|00001004010000#secret]] to some random string data (minimum length is 16 bytes) to secure the data exchanged with a client system.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010040100
title: Enable authentication
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220419192817

To enable authentication, you must create a zettel that stores [[authentication data|00001010040200]] for the owner.
Then you must reference this zettel within the [[startup configuration|00001004010000#owner]] under the key ''owner''.
Once the startup configuration contains a valid [[zettel identifier|00001006050000]] under that key, authentication is enabled.

Please note that you must also set key ''secret'' of the [[startup configuration|00001004010000#secret]] to some random string data (minimum length is 16 bytes) to secure the data exchanged with a client system.

Changes to docs/manual/00001010040400.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001010040400
title: Authentication process
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk

modified: 20211127174943

When someone tries to authenticate itself with an user identifier / ""user name"" and a password, the following process is executed:

# If meta key ''owner'' of the configuration zettel does not have a valid [[zettel identifier|00001006050000]] as value, authentication fails.
# Retrieve all zettel, where the meta key ''user-id'' has the same value as the given user identification. If the list is empty, authentication fails.
# From above list, the zettel with the numerically smallest identifier is selected.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010040400
title: Authentication process
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk
created: 20210126175322
modified: 20211127174943

When someone tries to authenticate itself with an user identifier / ""user name"" and a password, the following process is executed:

# If meta key ''owner'' of the configuration zettel does not have a valid [[zettel identifier|00001006050000]] as value, authentication fails.
# Retrieve all zettel, where the meta key ''user-id'' has the same value as the given user identification. If the list is empty, authentication fails.
# From above list, the zettel with the numerically smallest identifier is selected.

Changes to docs/manual/00001010040700.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001010040700
title: Access token
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk

modified: 20211202120950

If an user is authenticated, an ""access token"" is created that must be sent with every request to prove the identity of the caller.
Otherwise the user will not be recognized by Zettelstore.

If the user was authenticated via the [[web user interface|00001014000000]], the access token is stored in a [[""session cookie""|https://en.wikipedia.org/wiki/HTTP_cookie#Session_cookie]].
When the web browser is closed, theses cookies are not saved.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010040700
title: Access token
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk
created: 20210126175322
modified: 20211202120950

If an user is authenticated, an ""access token"" is created that must be sent with every request to prove the identity of the caller.
Otherwise the user will not be recognized by Zettelstore.

If the user was authenticated via the [[web user interface|00001014000000]], the access token is stored in a [[""session cookie""|https://en.wikipedia.org/wiki/HTTP_cookie#Session_cookie]].
When the web browser is closed, theses cookies are not saved.

Changes to docs/manual/00001010070300.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001010070300
title: User roles
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk

modified: 20220214175212

Every user is associated with some basic privileges.
These are specified in the [[user zettel|00001010040200]] with the key ''user-role''.
The following values are supported:

; [!reader|""reader""]





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010070300
title: User roles
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220214175212

Every user is associated with some basic privileges.
These are specified in the [[user zettel|00001010040200]] with the key ''user-role''.
The following values are supported:

; [!reader|""reader""]

Changes to docs/manual/00001010070400.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001010070400
title: Authorization and read-only mode
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk

modified: 20211103164251

It is possible to enable both the read-only mode of the Zettelstore __and__ authentication/authorization.
Both modes are independent from each other.
This gives four use cases:

; Not read-only, no authorization





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010070400
title: Authorization and read-only mode
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
created: 20210126175322
modified: 20211103164251

It is possible to enable both the read-only mode of the Zettelstore __and__ authentication/authorization.
Both modes are independent from each other.
This gives four use cases:

; Not read-only, no authorization

Changes to docs/manual/00001010070600.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
id: 00001010070600
title: Access rules
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk

modified: 20211124142456

Whether an operation of the Zettelstore is allowed or rejected, depends on various factors.

The following rules are checked first, in this order:

# In read-only mode, every operation except the ""Read"" operation is rejected.
# If there is no owner, authentication is disabled and every operation is allowed for everybody.





>
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001010070600
title: Access rules
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240711183714

Whether an operation of the Zettelstore is allowed or rejected, depends on various factors.

The following rules are checked first, in this order:

# In read-only mode, every operation except the ""Read"" operation is rejected.
# If there is no owner, authentication is disabled and every operation is allowed for everybody.
38
39
40
41
42
43
44
45
46
47
48
49
50
51
** If the zettel is the [[user zettel|00001010040200]] of the authenticated user, proceed as follows:
*** If some sensitive meta values are changed (e.g. user identifier, zettel role, user role, but not hashed password), reject the access
*** Since the user just updates some uncritical values, grant the access
   In other words: a user is allowed to change its user zettel, even if s/he has no writer privilege and if only uncritical data is changed.
** If the ''user-role'' of the user is ""reader"", reject the access.
** If the user is not allowed to create a new zettel, reject the access.
** Otherwise grant the access.
* Rename a zettel
** Reject the access.
   Only the owner of the Zettelstore is currently allowed to give a new identifier for a zettel.
* Delete a zettel
** Reject the access.
   Only the owner of the Zettelstore is allowed to delete a zettel.
   This may change in the future.







<
<
<




39
40
41
42
43
44
45



46
47
48
49
** If the zettel is the [[user zettel|00001010040200]] of the authenticated user, proceed as follows:
*** If some sensitive meta values are changed (e.g. user identifier, zettel role, user role, but not hashed password), reject the access
*** Since the user just updates some uncritical values, grant the access
   In other words: a user is allowed to change its user zettel, even if s/he has no writer privilege and if only uncritical data is changed.
** If the ''user-role'' of the user is ""reader"", reject the access.
** If the user is not allowed to create a new zettel, reject the access.
** Otherwise grant the access.



* Delete a zettel
** Reject the access.
   Only the owner of the Zettelstore is allowed to delete a zettel.
   This may change in the future.

Changes to docs/manual/00001010090100.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001010090100
title: External server to encrypt message transport
role: manual
tags: #configuration #encryption #manual #security #zettelstore
syntax: zmk

modified: 20220217180826

Since Zettelstore does not encrypt the messages it exchanges with its clients, you may need some additional software to enable encryption.

=== Public-key encryption
To enable encryption, you probably use some kind of encryption keys.
In most cases, you need to deploy a ""public-key encryption"" process, where your side publish a public encryption key that only works with a corresponding private decryption key.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010090100
title: External server to encrypt message transport
role: manual
tags: #configuration #encryption #manual #security #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220217180826

Since Zettelstore does not encrypt the messages it exchanges with its clients, you may need some additional software to enable encryption.

=== Public-key encryption
To enable encryption, you probably use some kind of encryption keys.
In most cases, you need to deploy a ""public-key encryption"" process, where your side publish a public encryption key that only works with a corresponding private decryption key.

Changes to docs/manual/00001012000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012000000
title: API
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20231128183617

The API (short for ""**A**pplication **P**rogramming **I**nterface"") is the primary way to communicate with a running Zettelstore.
Most integration with other systems and services is done through the API.
The [[web user interface|00001014000000]] is just an alternative, secondary way of interacting with a Zettelstore.

=== Background
The API is HTTP-based and uses plain text and [[symbolic expressions|00001012930000]] as its main encoding formats for exchanging messages between a Zettelstore and its client software.






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012000000
title: API
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240711183736

The API (short for ""**A**pplication **P**rogramming **I**nterface"") is the primary way to communicate with a running Zettelstore.
Most integration with other systems and services is done through the API.
The [[web user interface|00001014000000]] is just an alternative, secondary way of interacting with a Zettelstore.

=== Background
The API is HTTP-based and uses plain text and [[symbolic expressions|00001012930000]] as its main encoding formats for exchanging messages between a Zettelstore and its client software.
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
=== Working with zettel
* [[Create a new zettel|00001012053200]]
* [[Retrieve metadata and content of an existing zettel|00001012053300]]
* [[Retrieve metadata of an existing zettel|00001012053400]]
* [[Retrieve evaluated metadata and content of an existing zettel in various encodings|00001012053500]]
* [[Retrieve parsed metadata and content of an existing zettel in various encodings|00001012053600]]
* [[Update metadata and content of a zettel|00001012054200]]
* [[Rename a zettel|00001012054400]]
* [[Delete a zettel|00001012054600]]

=== Various helper methods
* [[Retrieve administrative data|00001012070500]]
* [[Execute some commands|00001012080100]]
** [[Check for authentication|00001012080200]]
** [[Refresh internal data|00001012080500]]







<







30
31
32
33
34
35
36

37
38
39
40
41
42
43
=== Working with zettel
* [[Create a new zettel|00001012053200]]
* [[Retrieve metadata and content of an existing zettel|00001012053300]]
* [[Retrieve metadata of an existing zettel|00001012053400]]
* [[Retrieve evaluated metadata and content of an existing zettel in various encodings|00001012053500]]
* [[Retrieve parsed metadata and content of an existing zettel in various encodings|00001012053600]]
* [[Update metadata and content of a zettel|00001012054200]]

* [[Delete a zettel|00001012054600]]

=== Various helper methods
* [[Retrieve administrative data|00001012070500]]
* [[Execute some commands|00001012080100]]
** [[Check for authentication|00001012080200]]
** [[Refresh internal data|00001012080500]]

Changes to docs/manual/00001012050600.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001012050600
title: API: Provide an access token
role: manual
tags: #api #manual #zettelstore
syntax: zmk

modified: 20220218130020

The [[authentication process|00001012050200]] provides you with an [[access token|00001012921000]].
Most API calls need such an access token, so that they know the identity of the caller.

You send the access token in the ""Authorization"" request header field, as described in [[RFC 6750, section 2.1|https://tools.ietf.org/html/rfc6750#section-2.1]].
You need to use the ""Bearer"" authentication scheme to transmit the access token.





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012050600
title: API: Provide an access token
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220218130020

The [[authentication process|00001012050200]] provides you with an [[access token|00001012921000]].
Most API calls need such an access token, so that they know the identity of the caller.

You send the access token in the ""Authorization"" request header field, as described in [[RFC 6750, section 2.1|https://tools.ietf.org/html/rfc6750#section-2.1]].
You need to use the ""Bearer"" authentication scheme to transmit the access token.

Changes to docs/manual/00001012051400.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012051400
title: API: Query the list of all zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20220912111111
modified: 20231023162927
precursor: 00001012051200

The [[endpoint|00001012920000]] ''/z'' also allows you to filter the list of all zettel[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] and optionally to provide some actions.

A [[query|00001007700000]] is an optional [[search expression|00001007700000#search-expression]], together with an optional [[list of actions|00001007700000#action-list]] (described below).
An empty search expression will select all zettel.
An empty list of action, or no valid action, returns the list of all selected zettel metadata.






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012051400
title: API: Query the list of all zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20220912111111
modified: 20240711161320
precursor: 00001012051200

The [[endpoint|00001012920000]] ''/z'' also allows you to filter the list of all zettel[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] and optionally to provide some actions.

A [[query|00001007700000]] is an optional [[search expression|00001007700000#search-expression]], together with an optional [[list of actions|00001007700000#action-list]] (described below).
An empty search expression will select all zettel.
An empty list of action, or no valid action, returns the list of all selected zettel metadata.
105
106
107
108
109
110
111





112
113
114
115
116
117
118
119

120
121


122
123
124
125
126
127
128


129
130
131
132
The following actions are supported:
; ''MINn'' (parameter)
: Emit only those values with at least __n__ aggregated values.
  __n__ must be a positive integer, ''MIN'' must be given in upper-case letters.
; ''MAXn'' (parameter)
: Emit only those values with at most __n__ aggregated values.
  __n__ must be a positive integer, ''MAX'' must be given in upper-case letters.





; ''REINDEX'' (aggregate)
: Updates the internal search index for the selected zettel, roughly similar to the [[refresh|00001012080500]] API call.
  It is not really an aggregate, since it is used only for its side effect.
  It is allowed to specify another aggregate.
; Any [[metadata key|00001006020000]] of type [[Word|00001006035500]], [[WordSet|00001006036000]], or [[TagSet|00001006034000]] (aggregates)
: Emit an aggregate of the given metadata key.
  The key can be given in any letter case.


Only the first aggregate action will be executed.




=== HTTP Status codes
; ''200''
: Query was successful.
; ''204''
: Query was successful, but results in no content.
  Most likely, you specified no appropriate aggregator.


; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the access bearer token was not valid, or you forgot to specify a valid query.







>
>
>
>
>




|



>
|

>
>







>
>




105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
The following actions are supported:
; ''MINn'' (parameter)
: Emit only those values with at least __n__ aggregated values.
  __n__ must be a positive integer, ''MIN'' must be given in upper-case letters.
; ''MAXn'' (parameter)
: Emit only those values with at most __n__ aggregated values.
  __n__ must be a positive integer, ''MAX'' must be given in upper-case letters.
; ''KEYS'' (aggregate)
: Emit a list of all metadata keys, together with the number of zettel having the key.
; ''REDIRECT'' (aggregate)
: Performs a HTTP redirect to the first selected zettel, using HTTP status code 302.
  The zettel identifier is in the body.
; ''REINDEX'' (aggregate)
: Updates the internal search index for the selected zettel, roughly similar to the [[refresh|00001012080500]] API call.
  It is not really an aggregate, since it is used only for its side effect.
  It is allowed to specify another aggregate.
; Any [[metadata key|00001006020000]] of type [[Word|00001006035500]] or [[TagSet|00001006034000]] (aggregates)
: Emit an aggregate of the given metadata key.
  The key can be given in any letter case.

First, ''REINDEX'' actions are executed, then ''REDIRECT''.
If no ''REDIRECT'' was found the first other aggregate action will be executed.

To allow some kind of backward compatibility, an action written in uppercase letters that leads to an empty result list, will be ignored.
In this case the list of selected zettel is returned.

=== HTTP Status codes
; ''200''
: Query was successful.
; ''204''
: Query was successful, but results in no content.
  Most likely, you specified no appropriate aggregator.
; ''302''
: Query was successful, redirect to first zettel in list.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the access bearer token was not valid, or you forgot to specify a valid query.

Changes to docs/manual/00001012053500.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
id: 00001012053500
title: API: Retrieve evaluated metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210726174524
modified: 20230807170112

The [[endpoint|00001012920000]] to work with evaluated metadata and content of a specific zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the [[zettel identifier|00001006050000]].

For example, to retrieve some evaluated data about this zettel you are currently viewing in [[Sz encoding|00001012920516]], just send a HTTP GET request to the endpoint ''/z/00001012053500''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] with the query parameter ''enc=sz''.
If successful, the output is a symbolic expression value:
```sh
# curl 'http://127.0.0.1:23123/z/00001012053500?enc=sz'
((PARA (TEXT "The") (SPACE) (LINK-ZETTEL () "00001012920000" (TEXT "endpoint")) (SPACE) (TEXT "to") (SPACE) (TEXT "work") (SPACE) (TEXT "with") (SPACE) (TEXT "evaluated") (SPACE) (TEXT "metadata") (SPACE) (TEXT "and") (SPACE) (TEXT "content") (SPACE) (TEXT "of") (SPACE) (TEXT "a") (SPACE) (TEXT "specific") (SPACE) (TEXT "zettel") (SPACE) (TEXT "is") (SPACE) (LITERAL-INPUT () "/z/{ID}") (TEXT ",") (SPACE) (TEXT "where") (SPACE) (LITERAL-INPUT () "{ID}") ...
```

To select another encoding, you must provide the query parameter ''enc=ENCODING''.
Others are ""[[html|00001012920510]]"", ""[[text|00001012920519]]"", and some [[more|00001012920500]].
In addition, you may provide a query parameter ''part=PART'' to select the relevant [[part|00001012920800]] of a zettel.
```sh
# curl 'http://127.0.0.1:23123/z/00001012053500?enc=html&part=zettel'






|







|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
id: 00001012053500
title: API: Retrieve evaluated metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210726174524
modified: 20240620171057

The [[endpoint|00001012920000]] to work with evaluated metadata and content of a specific zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the [[zettel identifier|00001006050000]].

For example, to retrieve some evaluated data about this zettel you are currently viewing in [[Sz encoding|00001012920516]], just send a HTTP GET request to the endpoint ''/z/00001012053500''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] with the query parameter ''enc=sz''.
If successful, the output is a symbolic expression value:
```sh
# curl 'http://127.0.0.1:23123/z/00001012053500?enc=sz'
(BLOCK (PARA (TEXT "The ") (LINK-ZETTEL () "00001012920000" (TEXT "endpoint")) (TEXT " to work with parsed metadata and content of a specific zettel is ") (LITERAL-INPUT () "/z/{ID}") (TEXT ", where ") (LITERAL-INPUT () "{ID}") (TEXT " is a placeholder for the ") ...
```

To select another encoding, you must provide the query parameter ''enc=ENCODING''.
Others are ""[[html|00001012920510]]"", ""[[text|00001012920519]]"", and some [[more|00001012920500]].
In addition, you may provide a query parameter ''part=PART'' to select the relevant [[part|00001012920800]] of a zettel.
```sh
# curl 'http://127.0.0.1:23123/z/00001012053500?enc=html&part=zettel'

Changes to docs/manual/00001012053600.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
id: 00001012053600
title: API: Retrieve parsed metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230807170019

The [[endpoint|00001012920000]] to work with parsed metadata and content of a specific zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the [[zettel identifier|00001006050000]].

A __parsed__ zettel is basically an [[unevaluated|00001012053500]] zettel: the zettel is read and analyzed, but its content is not __evaluated__.
By using this endpoint, you are able to retrieve the structure of a zettel before it is evaluated.

For example, to retrieve some data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/z/00001012053600''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] with the query parameter ''parseonly'' (and other appropriate query parameter).
For example:
```sh
# curl 'http://127.0.0.1:23123/z/00001012053600?enc=sz&parseonly'
((PARA (TEXT "The") (SPACE) (LINK-ZETTEL () "00001012920000" (TEXT "endpoint")) (SPACE) (TEXT "to") (SPACE) (TEXT "work") (SPACE) (TEXT "with") (SPACE) ...
```

Similar to [[retrieving an encoded zettel|00001012053500]], you can specify an [[encoding|00001012920500]] and state which [[part|00001012920800]] of a zettel you are interested in.
The same default values applies to this endpoint.

=== HTTP Status codes
; ''200''






|










|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
id: 00001012053600
title: API: Retrieve parsed metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240620170909

The [[endpoint|00001012920000]] to work with parsed metadata and content of a specific zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the [[zettel identifier|00001006050000]].

A __parsed__ zettel is basically an [[unevaluated|00001012053500]] zettel: the zettel is read and analyzed, but its content is not __evaluated__.
By using this endpoint, you are able to retrieve the structure of a zettel before it is evaluated.

For example, to retrieve some data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/z/00001012053600''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header] with the query parameter ''parseonly'' (and other appropriate query parameter).
For example:
```sh
# curl 'http://127.0.0.1:23123/z/00001012053600?enc=sz&parseonly'
(BLOCK (PARA (TEXT "The ") (LINK-ZETTEL () "00001012920000" (TEXT "endpoint")) (TEXT " to work with parsed metadata and content of a specific zettel is ") (LITERAL-INPUT () "/z/{ID}") (TEXT ", where ") ...
```

Similar to [[retrieving an encoded zettel|00001012053500]], you can specify an [[encoding|00001012920500]] and state which [[part|00001012920800]] of a zettel you are interested in.
The same default values applies to this endpoint.

=== HTTP Status codes
; ''200''

Deleted docs/manual/00001012054400.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
id: 00001012054400
title: API: Rename a zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
created: 20210713150005
modified: 20221219154659

Renaming a zettel is effectively just specifying a new identifier for the zettel.
Since more than one [[box|00001004011200]] might contain a zettel with the old identifier, the rename operation must success in every relevant box to be overall successful.
If the rename operation fails in one box, Zettelstore tries to rollback previous successful operations.

As a consequence, you cannot rename a zettel when its identifier is used in a read-only box.
This applies to all [[predefined zettel|00001005090000]], for example.

The [[endpoint|00001012920000]] to rename a zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the [[zettel identifier|00001006050000]].
You must send a HTTP MOVE request to this endpoint, and you must specify the new zettel identifier as an URL, placed under the HTTP request header key ''Destination''.
```
# curl -X MOVE -H "Destination: 10000000000001" http://127.0.0.1:23123/z/00001000000000
```

Only the last 14 characters of the value of ''Destination'' are taken into account and those must form an unused [[zettel identifier|00001006050000]].
If the value contains less than 14 characters that do not form an unused zettel identifier, the response will contain a HTTP status code ''400''.
All other characters, besides those 14 digits, are effectively ignored.
However, the value should form a valid URL that could be used later to [[read the content|00001012053300]] of the freshly renamed zettel.

=== HTTP Status codes
; ''204''
: Rename was successful, there is no body in the response.
; ''400''
: Request was not valid.
  For example, the HTTP header did not contain a valid ''Destination'' key, or the new identifier is already in use.
; ''403''
: You are not allowed to delete the given zettel.
  In most cases you have either not enough [[access rights|00001010070600]] or at least one box containing the given identifier operates in read-only mode.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

=== Rationale for the MOVE method
HTTP [[standardizes|https://www.rfc-editor.org/rfc/rfc7231.txt]] eight methods.
None of them is conceptually close to a rename operation.

Everyone is free to ""invent"" some new method to be used in HTTP.
To avoid a divergency, there is a [[methods registry|https://www.iana.org/assignments/http-methods/]] that tracks those extensions.
The [[HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV)|https://www.rfc-editor.org/rfc/rfc4918.txt]] defines the method MOVE that is quite close to the desired rename operation.
In fact, some command line tools use a ""move"" method for renaming files.

Therefore, Zettelstore adopts somehow WebDAV's MOVE method and its use of the ''Destination'' HTTP header key.
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<


































































































Changes to docs/manual/00001012920000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
id: 00001012920000
title: Endpoints used by the API
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20230731162343

All API endpoints conform to the pattern ''[PREFIX]LETTER[/ZETTEL-ID]'', where:
; ''PREFIX''
: is the URL prefix (default: ""/""), configured via the ''url-prefix'' [[startup configuration|00001004010000]],
; ''LETTER''
: is a single letter that specifies the resource type,
; ''ZETTEL-ID''
: is an optional 14 digits string that uniquely [[identify a zettel|00001006050000]].

The following letters are currently in use:

|= Letter:| Without zettel identifier | With [[zettel identifier|00001006050000]] | Mnemonic
| ''a'' | POST: [[client authentication|00001012050200]] | | **A**uthenticate
|       | PUT: [[renew access token|00001012050400]] |
| ''x'' | GET: [[retrieve administrative data|00001012070500]] | | E**x**ecute
|       | POST: [[execute command|00001012080100]]
| ''z'' | GET: [[list zettel|00001012051200]]/[[query zettel|00001012051400]] | GET: [[retrieve zettel|00001012053300]] | **Z**ettel
|       | POST: [[create new zettel|00001012053200]] | PUT: [[update zettel|00001012054200]]
|       |  | DELETE: [[delete zettel|00001012054600]]
|       |  | MOVE: [[rename zettel|00001012054400]]

The full URL will contain either the ""http"" oder ""https"" scheme, a host name, and an optional port number.

The API examples will assume the ""http"" schema, the local host ""127.0.0.1"", the default port ""23123"", and the default empty ''PREFIX'' ""/"".
Therefore, all URLs in the API documentation will begin with ""http://127.0.0.1:23123/"".






|



















<





1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
id: 00001012920000
title: Endpoints used by the API
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20240711183819

All API endpoints conform to the pattern ''[PREFIX]LETTER[/ZETTEL-ID]'', where:
; ''PREFIX''
: is the URL prefix (default: ""/""), configured via the ''url-prefix'' [[startup configuration|00001004010000]],
; ''LETTER''
: is a single letter that specifies the resource type,
; ''ZETTEL-ID''
: is an optional 14 digits string that uniquely [[identify a zettel|00001006050000]].

The following letters are currently in use:

|= Letter:| Without zettel identifier | With [[zettel identifier|00001006050000]] | Mnemonic
| ''a'' | POST: [[client authentication|00001012050200]] | | **A**uthenticate
|       | PUT: [[renew access token|00001012050400]] |
| ''x'' | GET: [[retrieve administrative data|00001012070500]] | | E**x**ecute
|       | POST: [[execute command|00001012080100]]
| ''z'' | GET: [[list zettel|00001012051200]]/[[query zettel|00001012051400]] | GET: [[retrieve zettel|00001012053300]] | **Z**ettel
|       | POST: [[create new zettel|00001012053200]] | PUT: [[update zettel|00001012054200]]
|       |  | DELETE: [[delete zettel|00001012054600]]


The full URL will contain either the ""http"" oder ""https"" scheme, a host name, and an optional port number.

The API examples will assume the ""http"" schema, the local host ""127.0.0.1"", the default port ""23123"", and the default empty ''PREFIX'' ""/"".
Therefore, all URLs in the API documentation will begin with ""http://127.0.0.1:23123/"".

Changes to docs/manual/00001012920510.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001012920510
title: HTML Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk

modified: 20210726193034

A zettel representation in HTML.
This representation is different form the [[web user interface|00001014000000]] as it contains the zettel representation only and no additional data such as the menu bar.

It is intended to be used by external clients.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012920510
title: HTML Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20210726193034

A zettel representation in HTML.
This representation is different form the [[web user interface|00001014000000]] as it contains the zettel representation only and no additional data such as the menu bar.

It is intended to be used by external clients.

Changes to docs/manual/00001012920519.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001012920519
title: Text Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk

modified: 20210726193119

A zettel representation contains just all textual data of a zettel.
Could be used for creating a search index.

Every line may contain zero, one, or more words, separated by space character.






>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012920519
title: Text Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20210726193119

A zettel representation contains just all textual data of a zettel.
Could be used for creating a search index.

Every line may contain zero, one, or more words, separated by space character.

Changes to docs/manual/00001012920522.zettel.

1
2
3
4
5

6
7
8
9
10
11
id: 00001012920522
title: Zmk Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk

modified: 20211124140857

A zettel representation that tries to recreate a [[Zettelmarkup|00001007000000]] representation of the zettel.
Useful if you want to convert [[other markup languages|00001008000000]] to Zettelmarkup (e.g. [[Markdown|00001008010000]]).

If transferred via HTTP, the content type will be ''text/plain''.





>






1
2
3
4
5
6
7
8
9
10
11
12
id: 00001012920522
title: Zmk Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20211124140857

A zettel representation that tries to recreate a [[Zettelmarkup|00001007000000]] representation of the zettel.
Useful if you want to convert [[other markup languages|00001008000000]] to Zettelmarkup (e.g. [[Markdown|00001008010000]]).

If transferred via HTTP, the content type will be ''text/plain''.

Changes to docs/manual/00001012920800.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
id: 00001012920800
title: Values to specify zettel parts
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk

modified: 20220214175335

When working with [[zettel|00001006000000]], you could work with the whole zettel, with its metadata, or with its content:
; [!zettel|''zettel'']
: Specifies that you work with a zettel as a whole.
  Contains identifier, metadata, and content of a zettel.
; [!meta|''meta'']





>







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012920800
title: Values to specify zettel parts
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20210126175322
modified: 20220214175335

When working with [[zettel|00001006000000]], you could work with the whole zettel, with its metadata, or with its content:
; [!zettel|''zettel'']
: Specifies that you work with a zettel as a whole.
  Contains identifier, metadata, and content of a zettel.
; [!meta|''meta'']

Changes to docs/manual/00001012921200.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
id: 00001012921200
title: API: Encoding of Zettel Access Rights
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20220201173115
modified: 20230807164817

Various API calls return a symbolic expression list ''(rights N)'', with ''N'' as a number, that encodes the access rights the user currently has.
''N'' is an integer number between 0 and 62.[^Not all values in this range are used.]

The value ""0"" signals that something went wrong internally while determining the access rights.

A value of ""1"" says, that the current user has no access right for the given zettel.
In most cases, this value will not occur, because only zettel are presented, which are at least readable by the current user.

Values ""2"" to ""62"" are binary encoded values, where each bit signals a special right.

|=Bit number:|Bit value:|Meaning
| 1 |  2 | User is allowed to create a new zettel
| 2 |  4 | User is allowed to read the zettel
| 3 |  8 | User is allowed to update the zettel
| 4 | 16 | User is allowed to rename the zettel
| 5 | 32 | User is allowed to delete the zettel

The algorithm to calculate the actual access rights from the value is relatively simple:
# Search for the biggest bit value that is less than the rights value.
  This is an access right for the current user.
# Subtract the bit value from the rights value.
  Remember the difference as the new rights value.






|















|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
id: 00001012921200
title: API: Encoding of Zettel Access Rights
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20220201173115
modified: 20240711183931

Various API calls return a symbolic expression list ''(rights N)'', with ''N'' as a number, that encodes the access rights the user currently has.
''N'' is an integer number between 0 and 62.[^Not all values in this range are used.]

The value ""0"" signals that something went wrong internally while determining the access rights.

A value of ""1"" says, that the current user has no access right for the given zettel.
In most cases, this value will not occur, because only zettel are presented, which are at least readable by the current user.

Values ""2"" to ""62"" are binary encoded values, where each bit signals a special right.

|=Bit number:|Bit value:|Meaning
| 1 |  2 | User is allowed to create a new zettel
| 2 |  4 | User is allowed to read the zettel
| 3 |  8 | User is allowed to update the zettel
| 4 | 16 | (not in use; was assigned to an operation)
| 5 | 32 | User is allowed to delete the zettel

The algorithm to calculate the actual access rights from the value is relatively simple:
# Search for the biggest bit value that is less than the rights value.
  This is an access right for the current user.
# Subtract the bit value from the rights value.
  Remember the difference as the new rights value.

Changes to docs/manual/00001012930500.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012930500
title: Syntax of Symbolic Expressions
role: manual
tags: #manual #reference #zettelstore
syntax: zmk
created: 20230403151127
modified: 20230703174218

=== Syntax of lists
A list always starts with the left parenthesis (""''(''"", U+0028) and ends with a right parenthesis (""'')''"", U+0029).
A list may contain a possibly empty sequence of elements, i.e. lists and / or atoms.

Internally, lists are composed of __cells__.
A cell allows to store two values.






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012930500
title: Syntax of Symbolic Expressions
role: manual
tags: #manual #reference #zettelstore
syntax: zmk
created: 20230403151127
modified: 20240413160345

=== Syntax of lists
A list always starts with the left parenthesis (""''(''"", U+0028) and ends with a right parenthesis (""'')''"", U+0029).
A list may contain a possibly empty sequence of elements, i.e. lists and / or atoms.

Internally, lists are composed of __cells__.
A cell allows to store two values.
67
68
69
70
71
72
73
74
75
76
77
To allow a string to contain a backslash, it also must be prefixed by one backslash.
Unicode characters with a code less than U+FF are encoded by by the sequence ""''\\xNM''"", where ''NM'' is the hex encoding of the character.
Unicode characters with a code less than U+FFFF are encoded by by the sequence ""''\\uNMOP''"", where ''NMOP'' is the hex encoding of the character.
Unicode characters with a code less than U+FFFFFF are encoded by by the sequence ""''\\UNMOPQR''"", where ''NMOPQR'' is the hex encoding of the character.
In addition, the sequence ""''\\t''"" encodes a horizontal tab (U+0009), the sequence ""''\\n''"" encodes a line feed (U+000A).

=== See also
* Currently, Zettelstore uses [[sx|https://zettelstore.de/sx]] (""Symbolic eXPression Framework"") to implement symbolic expression.
  The project page might contain additional information about the full syntax.

  Zettelstore only uses lists, numbers, string, and symbols to represent zettel.







|



67
68
69
70
71
72
73
74
75
76
77
To allow a string to contain a backslash, it also must be prefixed by one backslash.
Unicode characters with a code less than U+FF are encoded by by the sequence ""''\\xNM''"", where ''NM'' is the hex encoding of the character.
Unicode characters with a code less than U+FFFF are encoded by by the sequence ""''\\uNMOP''"", where ''NMOP'' is the hex encoding of the character.
Unicode characters with a code less than U+FFFFFF are encoded by by the sequence ""''\\UNMOPQR''"", where ''NMOPQR'' is the hex encoding of the character.
In addition, the sequence ""''\\t''"" encodes a horizontal tab (U+0009), the sequence ""''\\n''"" encodes a line feed (U+000A).

=== See also
* Currently, Zettelstore uses [[Sx|https://t73f.de/r/sx]] (""Symbolic eXpression framework"") to implement symbolic expressions.
  The project page might contain additional information about the full syntax.

  Zettelstore only uses lists, numbers, string, and symbols to represent zettel.

Changes to docs/manual/00001012931000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012931000
title: Encoding of Sz
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403153903
modified: 20230405133249

Zettel in a [[Sz encoding|00001012920516]] are represented as a [[symbolic expression|00001012930000]].
To process these symbolic expressions, you need to know, how a specific part of a zettel is represented by a symbolic expression.

Basically, each part of a zettel is represented as a list, often a nested list.
The first element of that list is always an unique symbol, which denotes that part.
The meaning / semantic of all other elements depend on that symbol.






|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012931000
title: Encoding of Sz
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403153903
modified: 20240123120319

Zettel in a [[Sz encoding|00001012920516]] are represented as a [[symbolic expression|00001012930000]].
To process these symbolic expressions, you need to know, how a specific part of a zettel is represented by a symbolic expression.

Basically, each part of a zettel is represented as a list, often a nested list.
The first element of that list is always an unique symbol, which denotes that part.
The meaning / semantic of all other elements depend on that symbol.
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

Metadata is represented by a list, where the first element is the symbol ''META''.
Following elements represent each metadatum[^""Metadatum"" is used as the singular form of metadata.] of a zettel in standard order.

Standard order is: [[Title|00001006020000#title]], [[Role|00001006020000#role]], [[Tags|00001006020000#tags]], [[Syntax|00001006020000#syntax]], all other [[keys|00001006020000]] in alphabetic order.

:::syntax
__Metadata__ **=** ''(META'' [[__Metadatum__|00001012931200]] __Metadatum__ &hellip; __Metadatum__ '')''.
:::
=== Content

Zettel content is represented by a block.
:::syntax
__Content__ **=** [[__Block__|#block]].
:::

==== Block
A block is represented by a list with the symbol ''BLOCK'' as the first element.
All following elements represent a nested [[block-structured element|00001007030000]].

:::syntax
[!block|__Block__] **=** ''(BLOCK'' [[__BlockElement__|00001012931400]] __BlockElement__ &hellip; __BlockElement__ '')''.
:::

==== Inline
Both block-structured elements and some metadata values may contain [[inline-structured elements|00001007040000]].
Similar, inline-structured elements are represented as follows:

:::syntax
__Inline__ **=** ''(INLINE'' [[__InlineElement__|00001012931600]] __InlineElement__ &hellip; __InlineElement__ '')''.
:::

==== Attribute
[[Attributes|00001007050000]] may be specified for both block- and inline- structured elements.
Attributes are represented by the following schema.
Please note, the the symbol ''quote'' is lower-case by intention.

:::syntax
__Attribute__ **=** ''('' **[** ''quote'' ''('' [[__AttributeKeyValue__|00001012931800]] __AttributeKeyValue__ &hellip; __AttributeKeyValue__ '')'' **]** ')'.
:::

Either, there are no attributes.
These are specified by the empty list ''()''.
Or there are attributes.
In this case, the first element of the list must be the symbol ''quote'': ''(quote'' ''('' A,,1,, A,,2,, &hellip; A,,n,, '')'''')''.








|













|







|





<


|







25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

60
61
62
63
64
65
66
67
68
69

Metadata is represented by a list, where the first element is the symbol ''META''.
Following elements represent each metadatum[^""Metadatum"" is used as the singular form of metadata.] of a zettel in standard order.

Standard order is: [[Title|00001006020000#title]], [[Role|00001006020000#role]], [[Tags|00001006020000#tags]], [[Syntax|00001006020000#syntax]], all other [[keys|00001006020000]] in alphabetic order.

:::syntax
__Metadata__ **=** ''(META'' [[__Metadatum__|00001012931200]] &hellip; '')''.
:::
=== Content

Zettel content is represented by a block.
:::syntax
__Content__ **=** [[__Block__|#block]].
:::

==== Block
A block is represented by a list with the symbol ''BLOCK'' as the first element.
All following elements represent a nested [[block-structured element|00001007030000]].

:::syntax
[!block|__Block__] **=** ''(BLOCK'' [[__BlockElement__|00001012931400]] &hellip; '')''.
:::

==== Inline
Both block-structured elements and some metadata values may contain [[inline-structured elements|00001007040000]].
Similar, inline-structured elements are represented as follows:

:::syntax
__Inline__ **=** ''(INLINE'' [[__InlineElement__|00001012931600]] &hellip; '')''.
:::

==== Attribute
[[Attributes|00001007050000]] may be specified for both block- and inline- structured elements.
Attributes are represented by the following schema.


:::syntax
__Attribute__ **=** ''('' **[** [[__AttributeKeyValue__|00001012931800]] &hellip; **]** ')'.
:::

Either, there are no attributes.
These are specified by the empty list ''()''.
Or there are attributes.
In this case, the first element of the list must be the symbol ''quote'': ''(quote'' ''('' A,,1,, A,,2,, &hellip; A,,n,, '')'''')''.

Changes to docs/manual/00001012931200.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
id: 00001012931200
title: Encoding of Sz Metadata
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161618
modified: 20230405121932

A single metadata (""metadatum"") is represented by a triple: a symbol representing the type, a symbol representing the key, and either a string or a list that represent the value.

The key symbol must be ""quoted"", i.e. for the key ""title"": ''(quote title)''.
This property may be relaxed in future versions of the Zettelstore.

The symbol depends on the [[metadata key type|00001006030000]].
The value also depends somehow on the key type: a set of values is represented as a list, all other values are represented by a string, even if it is a number.

The following table maps key types to symbols and to the type of the value representation.

|=Key Type<| Symbol<| Value<
| [[Credential|00001006031000]] | ''CREDENTIAL'' | string
| [[EString|00001006031500]] | ''EMPTY-STRING'' | string
| [[Identifier|00001006032000]] | ''ZID'' | string
| [[IdentifierSet|00001006032500]] | ''ZID-SET'' | list
| [[Number|00001006033000]] | ''NUMBER'' | string
| [[String|00001006033500]] | ''STRING'' | string
| [[TagSet|00001006034000]] | ''TAG-SET'' | list
| [[Timestamp|00001006034500]] | ''TIMESTAMP'' | string
| [[URL|00001006035000]] | ''URL'' | string
| [[Word|00001006035500]] | ''WORD'' | string
| [[WordSet|00001006036000]] | ''WORD-SET'' | list
| [[Zettelmarkup|00001006036500]] | ''ZETTELMARKUP'' | string

If the value is represented as a list, its first element is the symbol ''list'', and all other elements are strings with the appropriate values.

:::syntax
__ListValue__ **=** ''(list'' String,,1,, String,,2,, &hellip; String,,n,, '')''.
:::

Examples:
* The title of this zettel is represented as: ''(EMPTY-STRING (quote title) "Encoding of Sz Metadata")''
* The tags of this zettel are represented as: ''(TAG-SET (quote tags) (list "#api" "#manual" "#reference" "#zettelstore"))''






|



<
<
<









|


|



<


<
<

|



|
|
1
2
3
4
5
6
7
8
9
10



11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28


29
30
31
32
33
34
35
id: 00001012931200
title: Encoding of Sz Metadata
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161618
modified: 20240219161848

A single metadata (""metadatum"") is represented by a triple: a symbol representing the type, a symbol representing the key, and either a string or a list that represent the value.




The symbol depends on the [[metadata key type|00001006030000]].
The value also depends somehow on the key type: a set of values is represented as a list, all other values are represented by a string, even if it is a number.

The following table maps key types to symbols and to the type of the value representation.

|=Key Type<| Symbol<| Value<
| [[Credential|00001006031000]] | ''CREDENTIAL'' | string
| [[EString|00001006031500]] | ''EMPTY-STRING'' | string
| [[Identifier|00001006032000]] | ''ZID'' | string
| [[IdentifierSet|00001006032500]] | ''ZID-SET'' | ListValue
| [[Number|00001006033000]] | ''NUMBER'' | string
| [[String|00001006033500]] | ''STRING'' | string
| [[TagSet|00001006034000]] | ''TAG-SET'' | ListValue
| [[Timestamp|00001006034500]] | ''TIMESTAMP'' | string
| [[URL|00001006035000]] | ''URL'' | string
| [[Word|00001006035500]] | ''WORD'' | string

| [[Zettelmarkup|00001006036500]] | ''ZETTELMARKUP'' | string



:::syntax
__ListValue__ **=** ''('' String,,1,, String,,2,, &hellip; String,,n,, '')''.
:::

Examples:
* The title of this zettel is represented as: ''(EMPTY-STRING title "Encoding of Sz Metadata")''
* The tags of this zettel are represented as: ''(TAG-SET tags ("#api" "#manual" "#reference" "#zettelstore"))''

Changes to docs/manual/00001012931400.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012931400
title: Encoding of Sz Block Elements
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161803
modified: 20230405132916

=== ''PARA''
:::syntax
__Paragraph__ **=** ''(PARA'' [[__InlineElement__|00001012931600]] &hellip; '')''.
:::
A paragraph is just a list of inline elements.







|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001012931400
title: Encoding of Sz Block Elements
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161803
modified: 20240123120132

=== ''PARA''
:::syntax
__Paragraph__ **=** ''(PARA'' [[__InlineElement__|00001012931600]] &hellip; '')''.
:::
A paragraph is just a list of inline elements.

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
=== ''DESCRIPTION''
:::syntax
__Description__ **=** ''(DESCRIPTION'' __DescriptionTerm__ __DescriptionValues__ __DescriptionTerm__ __DescriptionValues__ &hellip; '')''.
:::
A description is a sequence of one ore more terms and values.

:::syntax
__DescriptionTerm__ **=** [[__Inline__|00001012931000#inline]].
:::
A description term is just an inline-structured value.

:::syntax
__DescriptionValues__ **=** ''(BLOCK'' [[__Block__|00001012931000#block]] &hellip; '')''.
:::
Description values are sequences of blocks.

=== ''TABLE''
:::syntax
__Table__ **=** ''(TABLE'' __TableHeader__ __TableRow__ &hellip; '')''.
:::
A table is a table header and a sequence of table rows.

:::syntax
__TableHeader__ **=** ''()'' **|** ''(list'' __TableCell__ &hellip; '')''.
:::
A table header is either the empty list or a list of table cells stating with the ''list'' symbol.

:::syntax
__TableRow__ **=** ''(list'' __TableCell__ &hellip; '')''.
:::
A table row is a list with the initial symbol ''list'', followed by table cells.

=== ''CELL'', ''CELL-*''
There are four kinds of table cells, one for each possible cell alignment.
The structure is the same for all kind.

:::syntax
__TableCell__ **=** __DefaultCell__ **|** __CenterCell__ **|** __LeftCell__ **|** __RightCell__.







|















|

|


|

|







46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
=== ''DESCRIPTION''
:::syntax
__Description__ **=** ''(DESCRIPTION'' __DescriptionTerm__ __DescriptionValues__ __DescriptionTerm__ __DescriptionValues__ &hellip; '')''.
:::
A description is a sequence of one ore more terms and values.

:::syntax
__DescriptionTerm__ **=** ''('' [[__InlineElement__|00001012931600]] &hellip; '')''.
:::
A description term is just an inline-structured value.

:::syntax
__DescriptionValues__ **=** ''(BLOCK'' [[__Block__|00001012931000#block]] &hellip; '')''.
:::
Description values are sequences of blocks.

=== ''TABLE''
:::syntax
__Table__ **=** ''(TABLE'' __TableHeader__ __TableRow__ &hellip; '')''.
:::
A table is a table header and a sequence of table rows.

:::syntax
__TableHeader__ **=** ''()'' **|** ''('' __TableCell__ &hellip; '')''.
:::
A table header is either the empty list or a list of table cells.

:::syntax
__TableRow__ **=** ''('' __TableCell__ &hellip; '')''.
:::
A table row is a list of table cells.

=== ''CELL'', ''CELL-*''
There are four kinds of table cells, one for each possible cell alignment.
The structure is the same for all kind.

:::syntax
__TableCell__ **=** __DefaultCell__ **|** __CenterCell__ **|** __LeftCell__ **|** __RightCell__.
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133

=== ''REGION-*''
The following lists specifies different kinds of regions.
A region treat a sequence of block elements to be belonging together in certain ways.
The have a similar structure.

:::syntax
__BlockRegion__ **=** ''(REGION-BLOCK'' [[__Attributes__|00001012931000#attribute]] [[__Block__|00001012931000#block]] [[__Inline__|00001012931000#inline]] '')''.
:::
A block region just treats the block to belong in an unspecified way.
Typically, the reason is given in the attributes.
The inline describes the block.

:::syntax
__QuoteRegion__ **=** ''(REGION-QUOTE'' [[__Attributes__|00001012931000#attribute]] [[__Block__|00001012931000#block]] [[__Inline__|00001012931000#inline]] '')''.
:::
A block region just treats the block to contain a longer quotation.
Attributes may further specify the quotation.
The inline typically describes author / source of the quotation.

:::syntax
__VerseRegion__ **=** ''(REGION-VERSE'' [[__Attributes__|00001012931000#attribute]] [[__Block__|00001012931000#block]] [[__Inline__|00001012931000#inline]] '')''.
:::
A block region just treats the block to contain a verse.
Soft line break are transformed into hard line breaks to save the structure of the verse / poem.
Attributes may further specify something.
The inline typically describes author / source of the verse.

=== ''VERBATIM-*''







|






|






|







105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133

=== ''REGION-*''
The following lists specifies different kinds of regions.
A region treat a sequence of block elements to be belonging together in certain ways.
The have a similar structure.

:::syntax
__BlockRegion__ **=** ''(REGION-BLOCK'' [[__Attributes__|00001012931000#attribute]] ''('' [[__BlockElement__|00001012931400]] &hellip; '')'' [[__InlineElement__|00001012931600]] &hellip; '')''.
:::
A block region just treats the block to belong in an unspecified way.
Typically, the reason is given in the attributes.
The inline describes the block.

:::syntax
__QuoteRegion__ **=** ''(REGION-QUOTE'' [[__Attributes__|00001012931000#attribute]] ''('' [[__BlockElement__|00001012931400]] &hellip; '')'' [[__InlineElement__|00001012931600]] &hellip; '')''.
:::
A block region just treats the block to contain a longer quotation.
Attributes may further specify the quotation.
The inline typically describes author / source of the quotation.

:::syntax
__VerseRegion__ **=** ''(REGION-VERSE'' [[__Attributes__|00001012931000#attribute]] ''('' [[__BlockElement__|00001012931400]] &hellip; '')'' [[__InlineElement__|00001012931600]] &hellip; '')''.
:::
A block region just treats the block to contain a verse.
Soft line break are transformed into hard line breaks to save the structure of the verse / poem.
Attributes may further specify something.
The inline typically describes author / source of the verse.

=== ''VERBATIM-*''
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
:::syntax
__ZettelVerbatim__ **=** ''(VERBATIM-ZETTEL'' [[__Attributes__|00001012931000#attribute]] String '')''.
:::
The string contains text that should be treated as (nested) zettel content.

=== ''BLOB''
:::syntax
__BLOB__ **=** ''(BLOB'' [[__Inline__|00001012931000#inline]] String,,1,, String,,2,, '')''.
:::
A BLOB contains an image in block mode.
The inline states some description.
The first string contains the syntax of the image.
The second string contains the actual image.
If the syntax is ""SVG"", then the second string contains the SVG code.
Otherwise the (binary) image data is encoded with base64.

=== ''TRANSCLUDE''
:::syntax
__Transclude__ **=** ''(TRANSCLUDE'' [[__Attributes__|00001012931000#attribute]] [[__Reference__|00001012931900]] '')''.
:::
A transclude list only occurs for a parsed zettel, but not for a evaluated zettel.
Evaluating a zettel also means that all transclusions are resolved.

__Reference__ denotes the zettel to be transcluded.







|


|













163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
:::syntax
__ZettelVerbatim__ **=** ''(VERBATIM-ZETTEL'' [[__Attributes__|00001012931000#attribute]] String '')''.
:::
The string contains text that should be treated as (nested) zettel content.

=== ''BLOB''
:::syntax
__BLOB__ **=** ''(BLOB'' ''('' [[__InlineElement__|00001012931600]] &hellip; '')'' String,,1,, String,,2,, '')''.
:::
A BLOB contains an image in block mode.
The inline elements states some description.
The first string contains the syntax of the image.
The second string contains the actual image.
If the syntax is ""SVG"", then the second string contains the SVG code.
Otherwise the (binary) image data is encoded with base64.

=== ''TRANSCLUDE''
:::syntax
__Transclude__ **=** ''(TRANSCLUDE'' [[__Attributes__|00001012931000#attribute]] [[__Reference__|00001012931900]] '')''.
:::
A transclude list only occurs for a parsed zettel, but not for a evaluated zettel.
Evaluating a zettel also means that all transclusions are resolved.

__Reference__ denotes the zettel to be transcluded.

Changes to docs/manual/00001012931600.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
id: 00001012931600
title: Encoding of Sz Inline Elements
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161845
modified: 20231113191203

=== ''TEXT''
:::syntax
__Text__ **=** ''(TEXT'' String '')''.
:::
Specifies the string as some text content, typically a word.

=== ''SPACE''
:::syntax
__Space__ **=** ''(SPACE'' **[** String **]** '')''.
:::
Specifies some space, typically white space.
If the string is not given it is assumed to be ''" "'' (one space character).
Otherwise it contains the space characters.

=== ''SOFT''
:::syntax
__Soft__ **=** ''(SOFT)''.
:::
Denotes a soft line break.
It is typically translated into a space character, but signals the point in the textual content, where a line break occurred.

=== ''HARD''
:::syntax
__Hard__ **=** ''(HARD)''.
:::
Specifies a hard line break, i.e. the user wants to have a line break here.







|





|
<
<
<
<
<
<
<
<






|







1
2
3
4
5
6
7
8
9
10
11
12
13








14
15
16
17
18
19
20
21
22
23
24
25
26
27
id: 00001012931600
title: Encoding of Sz Inline Elements
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161845
modified: 20240620170546

=== ''TEXT''
:::syntax
__Text__ **=** ''(TEXT'' String '')''.
:::
Specifies the string as some text content, including white space characters.









=== ''SOFT''
:::syntax
__Soft__ **=** ''(SOFT)''.
:::
Denotes a soft line break.
It will be often encoded as a space character, but signals the point in the textual content, where a line break occurred.

=== ''HARD''
:::syntax
__Hard__ **=** ''(HARD)''.
:::
Specifies a hard line break, i.e. the user wants to have a line break here.

120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
The first string is the mark string used in the content.
The second string is the mark string transformed to a slug, i.e. transformed to lower case, remove non-ASCII characters.
The third string is the slug string, but made unique for the whole zettel.
Then follows the marked text as a sequence of __InlineElement__s.

=== ''ENDNOTE''
:::syntax
__Endnote__ **=** ''(ENDNOTE'' [[__Attributes__|00001012931000#attribute]] ''(quote'' [[__InlineElement__|00001012931600]] &hellip; '')'''')''.
:::
Specifies endnote / footnote text.

=== ''FORMAT-*''
The following lists specifies some inline text formatting.
The structure is always the same, the initial symbol denotes the actual formatting.








|







112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
The first string is the mark string used in the content.
The second string is the mark string transformed to a slug, i.e. transformed to lower case, remove non-ASCII characters.
The third string is the slug string, but made unique for the whole zettel.
Then follows the marked text as a sequence of __InlineElement__s.

=== ''ENDNOTE''
:::syntax
__Endnote__ **=** ''(ENDNOTE'' [[__Attributes__|00001012931000#attribute]] [[__InlineElement__|00001012931600]] &hellip; '')''.
:::
Specifies endnote / footnote text.

=== ''FORMAT-*''
The following lists specifies some inline text formatting.
The structure is always the same, the initial symbol denotes the actual formatting.

Changes to docs/manual/00001012931800.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
id: 00001012931800
title: Encoding of Sz Attribute Values
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161923
modified: 20230403163701

An attribute is represented by a single cell.
The first element of the cell references the attribute key, the second value the corresponding value.

:::syntax
__AttributeKeyValue__ **=** ''('' __AttributeKey__ ''.'' __AttributeValue__ '')''.
:::

__AttributeKey__ and __AttributeValue__ are [[string values|00001012930500]].

An empty key denotes the generic attribute.

A key with the value ''"-"'' specifies the default attribute.
In this case, the attribute value is not interpreted.

Some examples:
* ''()'' represents the absence of attributes,
* ''(quote (("-" . "")))'' represent the default attribute,
* ''(quote (("-" . "") ("" . "syntax")))'' adds the generic attrribute with the value ""syntax"",
* ''(quote ())'' will also represent the absence of attribute (in a more complicated way),
* ''(quote (("lang" . "en")))'' denote the attribute key ""lang"" with a value ""en"".






|

















|
|
<
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
id: 00001012931800
title: Encoding of Sz Attribute Values
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230403161923
modified: 20240122115245

An attribute is represented by a single cell.
The first element of the cell references the attribute key, the second value the corresponding value.

:::syntax
__AttributeKeyValue__ **=** ''('' __AttributeKey__ ''.'' __AttributeValue__ '')''.
:::

__AttributeKey__ and __AttributeValue__ are [[string values|00001012930500]].

An empty key denotes the generic attribute.

A key with the value ''"-"'' specifies the default attribute.
In this case, the attribute value is not interpreted.

Some examples:
* ''()'' represents the absence of attributes,
* ''(("-" . ""))'' represent the default attribute,
* ''(("-" . "") ("" . "syntax"))'' adds the generic attribute with the value ""syntax"",

* ''(("lang" . "en"))'' denotes the attribute key ""lang"" with a value ""en"".

Changes to docs/manual/00001012931900.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
id: 00001012931900
title: Encoding of Sz Reference Values
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230405123046
modified: 20230405124516

A reference is encoded as the actual reference value, and a symbol describing the state of that actual reference value.

:::syntax
__Reference__ **=** ''(quote'' __ReferenceState__ String '')''.
:::
The ''quote'' is needed for internal reasons, the string contains the actual reference value.

:::syntax
__ReferenceState__ **=** ''INVALID'' **|** ''ZETTEL'' **|** ''SELF'' **|** ''FOUND'' **|** ''BROKEN'' **|** ''HOSTED'' **|** ''BASED'' **|** ''QUERY'' **|** ''EXTERNAL''.
:::

The meaning of the state symbols corresponds to that of the symbols used for the description of [[link references|00001012931600#link]].







|




|

|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
id: 00001012931900
title: Encoding of Sz Reference Values
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
created: 20230405123046
modified: 20240122094720

A reference is encoded as the actual reference value, and a symbol describing the state of that actual reference value.

:::syntax
__Reference__ **=** ''('' __ReferenceState__ String '')''.
:::
The string contains the actual reference value.

:::syntax
__ReferenceState__ **=** ''INVALID'' **|** ''ZETTEL'' **|** ''SELF'' **|** ''FOUND'' **|** ''BROKEN'' **|** ''HOSTED'' **|** ''BASED'' **|** ''QUERY'' **|** ''EXTERNAL''.
:::

The meaning of the state symbols corresponds to that of the symbols used for the description of [[link references|00001012931600#link]].

29
30
31
32
33
34
35

36

37

38

; ''FOUND''
: The reference value is a valid reference to an existing zettel.
  This value is only possible after evaluating the zettel.
; ''BROKEN''
: The reference value is a valid reference to an missing zettel.
  This value is only possible after evaluating the zettel.
; ''HOSTED''

; ''BASED''

; ''QUERY''

; ''EXTERNAL''








>

>

>

>
29
30
31
32
33
34
35
36
37
38
39
40
41
42
; ''FOUND''
: The reference value is a valid reference to an existing zettel.
  This value is only possible after evaluating the zettel.
; ''BROKEN''
: The reference value is a valid reference to an missing zettel.
  This value is only possible after evaluating the zettel.
; ''HOSTED''
: The reference value starts with one slash character, denoting an absolute local reference.
; ''BASED''
: The reference value starts with two slash characters, denoting a local reference interpreted relative to the Zettelstore base URL.
; ''QUERY''
: The reference value contains a query expression.
; ''EXTERNAL''
: The reference value contains a full URL, referencing a resource outside of the Zettelstore server.

Changes to docs/manual/00001014000000.zettel.

1
2
3
4
5

6
7
8
id: 00001014000000
title: Web user interface
tags: #manual #webui #zettelstore
syntax: zmk
role: manual


The Web user interface is just a secondary way to interact with a Zettelstore.
Using external software that interacts via the [[API|00001012000000]] is the recommended way.





>



1
2
3
4
5
6
7
8
9
id: 00001014000000
title: Web user interface
tags: #manual #webui #zettelstore
syntax: zmk
role: manual
created: 20210126175322

The Web user interface is just a secondary way to interact with a Zettelstore.
Using external software that interacts via the [[API|00001012000000]] is the recommended way.

Changes to docs/manual/00001018000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001018000000
title: Troubleshooting
role: manual
tags: #manual #zettelstore
syntax: zmk
created: 00010101000000
modified: 20221020132617

This page lists some problems and their solutions that may occur when using your Zettelstore.

=== Installation
* **Problem:** When you double-click on the Zettelstore executable icon, macOS complains that Zettelstore is an application from an unknown developer.
  Therefore, it will not start Zettelstore.
** **Solution:** Press the ''Ctrl'' key while opening the context menu of the Zettelstore executable with a right-click.





|
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
id: 00001018000000
title: Troubleshooting
role: manual
tags: #manual #zettelstore
syntax: zmk
created: 20211027105921
modified: 20240221134749

This page lists some problems and their solutions that may occur when using your Zettelstore.

=== Installation
* **Problem:** When you double-click on the Zettelstore executable icon, macOS complains that Zettelstore is an application from an unknown developer.
  Therefore, it will not start Zettelstore.
** **Solution:** Press the ''Ctrl'' key while opening the context menu of the Zettelstore executable with a right-click.
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
   The difference between these two is the missing encryption of user name / password and for the answer of the Zettelstore if you use the ''http://'' schema.
   To be secure by default, the Zettelstore will not work in an insecure environment.
** **Solution 1:** If you are sure that your communication medium is safe, even if you use the ''http:/\/'' schema (for example, you are running the Zettelstore on the same computer you are working on, or if the Zettelstore is running on a computer in your protected local network), then you could add the entry ''insecure-cookie: true'' in you [[startup configuration|00001004010000#insecure-cookie]] file.
** **Solution 2:** If you are not sure about the security of your communication medium (for example, if unknown persons might use your local network), then you should run an [[external server|00001010090100]] in front of your Zettelstore to enable the use of the ''https://'' schema.

=== Working with Zettel Files
* **Problem:** When you delete a zettel file by removing it from the ""disk"", e.g. by dropping it into the trash folder, by dragging into another folder, or by removing it from the command line, Zettelstore sometimes did not detect that change.
  If you access the zettel via Zettelstore, a fatal error is reported.
** **Explanation:** Sometimes, the operating system does not tell Zettelstore about the removed zettel.
   This occurs mostly under MacOS.
** **Solution 1:** If you are running Zettelstore in [[""simple-mode""|00001004051100]] or if you have enabled [[''expert-mode''|00001004020000#expert-mode]], you are allowed to refresh the internal data by selecting ""Refresh"" in the Web User Interface (you find it in the menu ""Lists"").
** **Solution 2:** There is an [[API|00001012080500]] call to make Zettelstore aware of this change.
** **Solution 3:** If you have an enabled [[Administrator Console|00001004100000]] you can use the command [[''refresh''|00001004101000#refresh]] to make your changes visible.
** **Solution 4:** You configure the zettel box as [[""simple""|00001004011400]].








|







24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
   The difference between these two is the missing encryption of user name / password and for the answer of the Zettelstore if you use the ''http://'' schema.
   To be secure by default, the Zettelstore will not work in an insecure environment.
** **Solution 1:** If you are sure that your communication medium is safe, even if you use the ''http:/\/'' schema (for example, you are running the Zettelstore on the same computer you are working on, or if the Zettelstore is running on a computer in your protected local network), then you could add the entry ''insecure-cookie: true'' in you [[startup configuration|00001004010000#insecure-cookie]] file.
** **Solution 2:** If you are not sure about the security of your communication medium (for example, if unknown persons might use your local network), then you should run an [[external server|00001010090100]] in front of your Zettelstore to enable the use of the ''https://'' schema.

=== Working with Zettel Files
* **Problem:** When you delete a zettel file by removing it from the ""disk"", e.g. by dropping it into the trash folder, by dragging into another folder, or by removing it from the command line, Zettelstore sometimes did not detect that change.
  If you access the zettel via Zettelstore, an error is reported.
** **Explanation:** Sometimes, the operating system does not tell Zettelstore about the removed zettel.
   This occurs mostly under MacOS.
** **Solution 1:** If you are running Zettelstore in [[""simple-mode""|00001004051100]] or if you have enabled [[''expert-mode''|00001004020000#expert-mode]], you are allowed to refresh the internal data by selecting ""Refresh"" in the Web User Interface (you find it in the menu ""Lists"").
** **Solution 2:** There is an [[API|00001012080500]] call to make Zettelstore aware of this change.
** **Solution 3:** If you have an enabled [[Administrator Console|00001004100000]] you can use the command [[''refresh''|00001004101000#refresh]] to make your changes visible.
** **Solution 4:** You configure the zettel box as [[""simple""|00001004011400]].

Deleted docs/manual/20231128184200.zettel.

1
2
3
4
5
6
7
id: 20231128184200
title: manual
role: role
syntax: zmk
created: 20231128184200

Zettel with the role ""manual"" contain the manual of the zettelstore.
<
<
<
<
<
<
<














Changes to encoder/encoder.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package encoder provides a generic interface to encode the abstract syntax
// tree into some text form.
package encoder

import (
	"errors"
	"fmt"
	"io"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/zettel/meta"
)

// Encoder is an interface that allows to encode different parts of a zettel.
type Encoder interface {
	WriteZettel(io.Writer, *ast.ZettelNode, EvalMetaFunc) (int, error)








>
>
>











|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package encoder provides a generic interface to encode the abstract syntax
// tree into some text form.
package encoder

import (
	"errors"
	"fmt"
	"io"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/zettel/meta"
)

// Encoder is an interface that allows to encode different parts of a zettel.
type Encoder interface {
	WriteZettel(io.Writer, *ast.ZettelNode, EvalMetaFunc) (int, error)

Changes to encoder/encoder_blob_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package encoder_test

import (
	"testing"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/config"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"

	_ "zettelstore.de/z/parser/blob" // Allow to use BLOB parser.
)









>
>
>







|
|
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package encoder_test

import (
	"testing"

	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/input"
	"zettelstore.de/z/config"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"

	_ "zettelstore.de/z/parser/blob" // Allow to use BLOB parser.
)

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
			0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x00, 0x00, 0x00, 0x00, 0x3a, 0x7e, 0x9b,
			0x55, 0x00, 0x00, 0x00, 0x0a, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0x62, 0x00, 0x00, 0x00,
			0x06, 0x00, 0x03, 0x36, 0x37, 0x7c, 0xa8, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae,
			0x42, 0x60, 0x82,
		},
		expect: expectMap{
			encoderHTML:  `<p><img alt="PNG" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="></p>`,
			encoderSz:    `(BLOCK (BLOB (INLINE (TEXT "PNG")) "png" "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="))`,
			encoderSHTML: `((p (img (@ (alt . "PNG") (src . "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg==")))))`,
			encoderText:  "",
			encoderZmk:   `%% Unable to display BLOB with description 'PNG' and syntax 'png'.`,
		},
	},
}








|







40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
			0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x00, 0x00, 0x00, 0x00, 0x3a, 0x7e, 0x9b,
			0x55, 0x00, 0x00, 0x00, 0x0a, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0x62, 0x00, 0x00, 0x00,
			0x06, 0x00, 0x03, 0x36, 0x37, 0x7c, 0xa8, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae,
			0x42, 0x60, 0x82,
		},
		expect: expectMap{
			encoderHTML:  `<p><img alt="PNG" src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="></p>`,
			encoderSz:    `(BLOCK (BLOB ((TEXT "PNG")) "png" "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="))`,
			encoderSHTML: `((p (img (@ (alt . "PNG") (src . "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg==")))))`,
			encoderText:  "",
			encoderZmk:   `%% Unable to display BLOB with description 'PNG' and syntax 'png'.`,
		},
	},
}

Changes to encoder/encoder_block_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package encoder_test

var tcsBlock = []zmkTestCase{
	{
		descr: "Empty Zettelmarkup should produce near nothing",








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package encoder_test

var tcsBlock = []zmkTestCase{
	{
		descr: "Empty Zettelmarkup should produce near nothing",
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
	},
	{
		descr: "Simple text: Hello, world",
		zmk:   "Hello, world",
		expect: expectMap{
			encoderHTML:  "<p>Hello, world</p>",
			encoderMD:    "Hello, world",
			encoderSz:    `(BLOCK (PARA (TEXT "Hello,") (SPACE) (TEXT "world")))`,
			encoderSHTML: `((p "Hello," " " "world"))`,
			encoderText:  "Hello, world",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple block comment",
		zmk:   "%%%\nNo\nrender\n%%%",







|
|







28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
	},
	{
		descr: "Simple text: Hello, world",
		zmk:   "Hello, world",
		expect: expectMap{
			encoderHTML:  "<p>Hello, world</p>",
			encoderMD:    "Hello, world",
			encoderSz:    `(BLOCK (PARA (TEXT "Hello, world")))`,
			encoderSHTML: `((p "Hello, world"))`,
			encoderText:  "Hello, world",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple block comment",
		zmk:   "%%%\nNo\nrender\n%%%",
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
	},
	{
		descr: "Rendered block comment",
		zmk:   "%%%{-}\nRender\n%%%",
		expect: expectMap{
			encoderHTML:  "<!--\nRender\n-->\n",
			encoderMD:    "",
			encoderSz:    `(BLOCK (VERBATIM-COMMENT (quote (("-" . ""))) "Render"))`,
			encoderSHTML: "((@@@ \"Render\"))",
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Heading",
		zmk:   `=== Top`,
		expect: expectMap{
			encoderHTML:  "<h2 id=\"top\">Top</h2>",
			encoderMD:    "# Top",
			encoderSz:    `(BLOCK (HEADING 1 () "top" "top" (INLINE (TEXT "Top"))))`,
			encoderSHTML: `((h2 (@ (id . "top")) "Top"))`,
			encoderText:  `Top`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple List",
		zmk:   "* A\n* B\n* C",
		expect: expectMap{







|







|

|
|
|
|
|







52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
	},
	{
		descr: "Rendered block comment",
		zmk:   "%%%{-}\nRender\n%%%",
		expect: expectMap{
			encoderHTML:  "<!--\nRender\n-->\n",
			encoderMD:    "",
			encoderSz:    `(BLOCK (VERBATIM-COMMENT (("-" . "")) "Render"))`,
			encoderSHTML: "((@@@ \"Render\"))",
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Heading",
		zmk:   `=== Top Job`,
		expect: expectMap{
			encoderHTML:  "<h2 id=\"top-job\">Top Job</h2>",
			encoderMD:    "# Top Job",
			encoderSz:    `(BLOCK (HEADING 1 () "top-job" "top-job" (TEXT "Top Job")))`,
			encoderSHTML: `((h2 (@ (id . "top-job")) "Top Job"))`,
			encoderText:  `Top Job`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple List",
		zmk:   "* A\n* B\n* C",
		expect: expectMap{
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
	},
	{
		descr: "Thematic break with attribute",
		zmk:   `---{lang="zmk"}`,
		expect: expectMap{
			encoderHTML:  `<hr lang="zmk">`,
			encoderMD:    "---",
			encoderSz:    `(BLOCK (THEMATIC (quote (("lang" . "zmk")))))`,
			encoderSHTML: `((hr (@ (lang . "zmk"))))`,
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "No list after paragraph",







|







124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
	},
	{
		descr: "Thematic break with attribute",
		zmk:   `---{lang="zmk"}`,
		expect: expectMap{
			encoderHTML:  `<hr lang="zmk">`,
			encoderMD:    "---",
			encoderSz:    `(BLOCK (THEMATIC (("lang" . "zmk"))))`,
			encoderSHTML: `((hr (@ (lang . "zmk"))))`,
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "No list after paragraph",
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple List Quote",
		zmk:   "> ToBeOrNotToBe",
		expect: expectMap{
			encoderHTML:  "<blockquote><p>ToBeOrNotToBe</p></blockquote>",
			encoderMD:    "> ToBeOrNotToBe",
			encoderSz:    `(BLOCK (QUOTATION (INLINE (TEXT "ToBeOrNotToBe"))))`,
			encoderSHTML: `((blockquote (p "ToBeOrNotToBe")))`,
			encoderText:  "ToBeOrNotToBe",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Quote Block",
		zmk:   "<<<\nToBeOrNotToBe\n<<< Romeo",
		expect: expectMap{
			encoderHTML:  "<blockquote><p>ToBeOrNotToBe</p><cite>Romeo</cite></blockquote>",
			encoderMD:    "> ToBeOrNotToBe",
			encoderSz:    `(BLOCK (REGION-QUOTE () (BLOCK (PARA (TEXT "ToBeOrNotToBe"))) (INLINE (TEXT "Romeo"))))`,
			encoderSHTML: `((blockquote (p "ToBeOrNotToBe") (cite "Romeo")))`,
			encoderText:  "ToBeOrNotToBe\nRomeo",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Quote Block with multiple paragraphs",
		zmk:   "<<<\nToBeOr\n\nNotToBe\n<<< Romeo",
		expect: expectMap{
			encoderHTML:  "<blockquote><p>ToBeOr</p><p>NotToBe</p><cite>Romeo</cite></blockquote>",
			encoderMD:    "> ToBeOr\n\n> NotToBe",
			encoderSz:    `(BLOCK (REGION-QUOTE () (BLOCK (PARA (TEXT "ToBeOr")) (PARA (TEXT "NotToBe"))) (INLINE (TEXT "Romeo"))))`,
			encoderSHTML: `((blockquote (p "ToBeOr") (p "NotToBe") (cite "Romeo")))`,
			encoderText:  "ToBeOr\nNotToBe\nRomeo",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Verse block",
		zmk: `"""
A line
  another line
Back

Paragraph

    Spacy  Para
""" Author`,
		expect: expectMap{
			encoderHTML:  "<div><p>A\u00a0line<br>\u00a0\u00a0another\u00a0line<br>Back</p><p>Paragraph</p><p>\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para</p><cite>Author</cite></div>",
			encoderMD:    "",
			encoderSz:    "(BLOCK (REGION-VERSE () (BLOCK (PARA (TEXT \"A\") (SPACE \"\u00a0\") (TEXT \"line\") (HARD) (SPACE \"\u00a0\u00a0\") (TEXT \"another\") (SPACE \"\u00a0\") (TEXT \"line\") (HARD) (TEXT \"Back\")) (PARA (TEXT \"Paragraph\")) (PARA (SPACE \"\u00a0\u00a0\u00a0\u00a0\") (TEXT \"Spacy\") (SPACE \"\u00a0\u00a0\") (TEXT \"Para\"))) (INLINE (TEXT \"Author\"))))",
			encoderSHTML: "((div (p \"A\" \"\u00a0\" \"line\" (br) \"\u00a0\u00a0\" \"another\" \"\u00a0\" \"line\" (br) \"Back\") (p \"Paragraph\") (p \"\u00a0\u00a0\u00a0\u00a0\" \"Spacy\" \"\u00a0\u00a0\" \"Para\") (cite \"Author\")))",
			encoderText:  "A line\n another line\nBack\nParagraph\n Spacy Para\nAuthor",
			encoderZmk:   "\"\"\"\nA\u00a0line\\\n\u00a0\u00a0another\u00a0line\\\nBack\nParagraph\n\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\n\"\"\" Author",
		},
	},
	{
		descr: "Span Block",
		zmk: `:::
A simple
   span
and much more
:::`,
		expect: expectMap{
			encoderHTML:  "<div><p>A simple  span and much more</p></div>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (REGION-BLOCK () (BLOCK (PARA (TEXT "A") (SPACE) (TEXT "simple") (SOFT) (SPACE) (TEXT "span") (SOFT) (TEXT "and") (SPACE) (TEXT "much") (SPACE) (TEXT "more"))) (INLINE)))`,
			encoderSHTML: `((div (p "A" " " "simple" " " " " "span" " " "and" " " "much" " " "more")))`,
			encoderText:  `A simple  span and much more`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Verbatim Code",
		zmk:   "```\nHello\nWorld\n```",







|


|






|

|

|
|
|









|



















|
|












|

|
|







158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple List Quote",
		zmk:   "> ToBeOrNotToBe",
		expect: expectMap{
			encoderHTML:  "<blockquote>ToBeOrNotToBe</blockquote>",
			encoderMD:    "> ToBeOrNotToBe",
			encoderSz:    `(BLOCK (QUOTATION (INLINE (TEXT "ToBeOrNotToBe"))))`,
			encoderSHTML: `((blockquote (@L "ToBeOrNotToBe")))`,
			encoderText:  "ToBeOrNotToBe",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Quote Block",
		zmk:   "<<<\nToBeOrNotToBe\n<<< Romeo Julia",
		expect: expectMap{
			encoderHTML:  "<blockquote><p>ToBeOrNotToBe</p><cite>Romeo Julia</cite></blockquote>",
			encoderMD:    "> ToBeOrNotToBe",
			encoderSz:    `(BLOCK (REGION-QUOTE () ((PARA (TEXT "ToBeOrNotToBe"))) (TEXT "Romeo Julia")))`,
			encoderSHTML: `((blockquote (p "ToBeOrNotToBe") (cite "Romeo Julia")))`,
			encoderText:  "ToBeOrNotToBe\nRomeo Julia",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Quote Block with multiple paragraphs",
		zmk:   "<<<\nToBeOr\n\nNotToBe\n<<< Romeo",
		expect: expectMap{
			encoderHTML:  "<blockquote><p>ToBeOr</p><p>NotToBe</p><cite>Romeo</cite></blockquote>",
			encoderMD:    "> ToBeOr\n\n> NotToBe",
			encoderSz:    `(BLOCK (REGION-QUOTE () ((PARA (TEXT "ToBeOr")) (PARA (TEXT "NotToBe"))) (TEXT "Romeo")))`,
			encoderSHTML: `((blockquote (p "ToBeOr") (p "NotToBe") (cite "Romeo")))`,
			encoderText:  "ToBeOr\nNotToBe\nRomeo",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Verse block",
		zmk: `"""
A line
  another line
Back

Paragraph

    Spacy  Para
""" Author`,
		expect: expectMap{
			encoderHTML:  "<div><p>A\u00a0line<br>\u00a0\u00a0another\u00a0line<br>Back</p><p>Paragraph</p><p>\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para</p><cite>Author</cite></div>",
			encoderMD:    "",
			encoderSz:    "(BLOCK (REGION-VERSE () ((PARA (TEXT \"A\u00a0line\") (HARD) (TEXT \"\u00a0\u00a0another\u00a0line\") (HARD) (TEXT \"Back\")) (PARA (TEXT \"Paragraph\")) (PARA (TEXT \"\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\"))) (TEXT \"Author\")))",
			encoderSHTML: "((div (p \"A\u00a0line\" (br) \"\u00a0\u00a0another\u00a0line\" (br) \"Back\") (p \"Paragraph\") (p \"\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\") (cite \"Author\")))",
			encoderText:  "A line\n another line\nBack\nParagraph\n Spacy Para\nAuthor",
			encoderZmk:   "\"\"\"\nA\u00a0line\\\n\u00a0\u00a0another\u00a0line\\\nBack\nParagraph\n\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\n\"\"\" Author",
		},
	},
	{
		descr: "Span Block",
		zmk: `:::
A simple
   span
and much more
:::`,
		expect: expectMap{
			encoderHTML:  "<div><p>A simple    span and much more</p></div>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (REGION-BLOCK () ((PARA (TEXT "A simple") (SOFT) (TEXT "   span") (SOFT) (TEXT "and much more")))))`,
			encoderSHTML: `((div (p "A simple" " " "   span" " " "and much more")))`,
			encoderText:  `A simple  span and much more`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Verbatim Code",
		zmk:   "```\nHello\nWorld\n```",
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
	},
	{
		descr: "Simple Verbatim Code with visible spaces",
		zmk:   "```{-}\nHello World\n```",
		expect: expectMap{
			encoderHTML:  "<pre><code>Hello\u2423World</code></pre>",
			encoderMD:    "    Hello World",
			encoderSz:    `(BLOCK (VERBATIM-CODE (quote (("-" . ""))) "Hello World"))`,
			encoderSHTML: "((pre (code \"Hello\u2423World\")))",
			encoderText:  "Hello World",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Verbatim Eval",







|







244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
	},
	{
		descr: "Simple Verbatim Code with visible spaces",
		zmk:   "```{-}\nHello World\n```",
		expect: expectMap{
			encoderHTML:  "<pre><code>Hello\u2423World</code></pre>",
			encoderMD:    "    Hello World",
			encoderSz:    `(BLOCK (VERBATIM-CODE (("-" . "")) "Hello World"))`,
			encoderSHTML: "((pre (code \"Hello\u2423World\")))",
			encoderText:  "Hello World",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Verbatim Eval",
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
	},
	{
		descr: "Simple Description List",
		zmk:   "; Zettel\n: Paper\n: Note\n; Zettelkasten\n: Slip box",
		expect: expectMap{
			encoderHTML:  "<dl><dt>Zettel</dt><dd><p>Paper</p></dd><dd><p>Note</p></dd><dt>Zettelkasten</dt><dd><p>Slip box</p></dd></dl>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (DESCRIPTION (INLINE (TEXT "Zettel")) (BLOCK (BLOCK (PARA (TEXT "Paper"))) (BLOCK (PARA (TEXT "Note")))) (INLINE (TEXT "Zettelkasten")) (BLOCK (BLOCK (PARA (TEXT "Slip") (SPACE) (TEXT "box"))))))`,
			encoderSHTML: `((dl (dt "Zettel") (dd (p "Paper")) (dd (p "Note")) (dt "Zettelkasten") (dd (p "Slip" " " "box"))))`,
			encoderText:  "Zettel\nPaper\nNote\nZettelkasten\nSlip box",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Description List with paragraphs as item",
		zmk:   "; Zettel\n: Paper\n\n  Note\n; Zettelkasten\n: Slip box",
		expect: expectMap{
			encoderHTML:  "<dl><dt>Zettel</dt><dd><p>Paper</p><p>Note</p></dd><dt>Zettelkasten</dt><dd><p>Slip box</p></dd></dl>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (DESCRIPTION (INLINE (TEXT "Zettel")) (BLOCK (BLOCK (PARA (TEXT "Paper")) (PARA (TEXT "Note")))) (INLINE (TEXT "Zettelkasten")) (BLOCK (BLOCK (PARA (TEXT "Slip") (SPACE) (TEXT "box"))))))`,
			encoderSHTML: `((dl (dt "Zettel") (dd (p "Paper") (p "Note")) (dt "Zettelkasten") (dd (p "Slip" " " "box"))))`,
			encoderText:  "Zettel\nPaper\nNote\nZettelkasten\nSlip box",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Description List with keys, but no descriptions",
		zmk:   "; K1\n: D11\n: D12\n; K2\n; K3\n: D31",
		expect: expectMap{
			encoderHTML:  "<dl><dt>K1</dt><dd><p>D11</p></dd><dd><p>D12</p></dd><dt>K2</dt><dt>K3</dt><dd><p>D31</p></dd></dl>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (DESCRIPTION (INLINE (TEXT "K1")) (BLOCK (BLOCK (PARA (TEXT "D11"))) (BLOCK (PARA (TEXT "D12")))) (INLINE (TEXT "K2")) (BLOCK) (INLINE (TEXT "K3")) (BLOCK (BLOCK (PARA (TEXT "D31"))))))`,
			encoderSHTML: `((dl (dt "K1") (dd (p "D11")) (dd (p "D12")) (dt "K2") (dt "K3") (dd (p "D31"))))`,
			encoderText:  "K1\nD11\nD12\nK2\nK3\nD31",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Table",
		zmk:   "|c1|c2|c3\n|d1||d3",
		expect: expectMap{
			encoderHTML:  `<table><tbody><tr><td>c1</td><td>c2</td><td>c3</td></tr><tr><td>d1</td><td></td><td>d3</td></tr></tbody></table>`,
			encoderMD:    "",
			encoderSz:    `(BLOCK (TABLE () (list (CELL (TEXT "c1")) (CELL (TEXT "c2")) (CELL (TEXT "c3"))) (list (CELL (TEXT "d1")) (CELL) (CELL (TEXT "d3")))))`,
			encoderSHTML: `((table (tbody (tr (td "c1") (td "c2") (td "c3")) (tr (td "d1") (td) (td "d3")))))`,
			encoderText:  "c1 c2 c3\nd1  d3",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Table with alignment and comment",
		zmk: `|h1>|=h2|h3:|
|%--+---+---+
|<c1|c2|:c3|
|f1|f2|=f3`,
		expect: expectMap{
			encoderHTML:  `<table><thead><tr><td class="right">h1</td><td>h2</td><td class="center">h3</td></tr></thead><tbody><tr><td class="left">c1</td><td>c2</td><td class="center">c3</td></tr><tr><td class="right">f1</td><td>f2</td><td class="center">=f3</td></tr></tbody></table>`,
			encoderMD:    "",
			encoderSz:    `(BLOCK (TABLE (list (CELL-RIGHT (TEXT "h1")) (CELL (TEXT "h2")) (CELL-CENTER (TEXT "h3"))) (list (CELL-LEFT (TEXT "c1")) (CELL (TEXT "c2")) (CELL-CENTER (TEXT "c3"))) (list (CELL-RIGHT (TEXT "f1")) (CELL (TEXT "f2")) (CELL-CENTER (TEXT "=f3")))))`,
			encoderSHTML: `((table (thead (tr (td (@ (class . "right")) "h1") (td "h2") (td (@ (class . "center")) "h3"))) (tbody (tr (td (@ (class . "left")) "c1") (td "c2") (td (@ (class . "center")) "c3")) (tr (td (@ (class . "right")) "f1") (td "f2") (td (@ (class . "center")) "=f3")))))`,
			encoderText:  "h1 h2 h3\nc1 c2 c3\nf1 f2 =f3",
			encoderZmk: `|=h1>|=h2|=h3:
|<c1|c2|c3
|f1|f2|=f3`,
		},
	},
	{
		descr: "Simple Endnote",
		zmk:   `Text[^Endnote]`,
		expect: expectMap{
			encoderHTML:  "<p>Text<sup id=\"fnref:1\"><a class=\"zs-noteref\" href=\"#fn:1\" role=\"doc-noteref\">1</a></sup></p><ol class=\"zs-endnotes\"><li class=\"zs-endnote\" id=\"fn:1\" role=\"doc-endnote\" value=\"1\">Endnote <a class=\"zs-endnote-backref\" href=\"#fnref:1\" role=\"doc-backlink\">\u21a9\ufe0e</a></li></ol>",
			encoderMD:    "Text",
			encoderSz:    `(BLOCK (PARA (TEXT "Text") (ENDNOTE () (quote (INLINE (TEXT "Endnote"))))))`,
			encoderSHTML: "((p \"Text\" (sup (@ (id . \"fnref:1\")) (a (@ (class . \"zs-noteref\") (href . \"#fn:1\") (role . \"doc-noteref\")) \"1\"))))",
			encoderText:  "Text Endnote",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Nested Endnotes",
		zmk:   `Text[^Endnote[^Nested]]`,
		expect: expectMap{
			encoderHTML:  "<p>Text<sup id=\"fnref:1\"><a class=\"zs-noteref\" href=\"#fn:1\" role=\"doc-noteref\">1</a></sup></p><ol class=\"zs-endnotes\"><li class=\"zs-endnote\" id=\"fn:1\" role=\"doc-endnote\" value=\"1\">Endnote<sup id=\"fnref:2\"><a class=\"zs-noteref\" href=\"#fn:2\" role=\"doc-noteref\">2</a></sup> <a class=\"zs-endnote-backref\" href=\"#fnref:1\" role=\"doc-backlink\">\u21a9\ufe0e</a></li><li class=\"zs-endnote\" id=\"fn:2\" role=\"doc-endnote\" value=\"2\">Nested <a class=\"zs-endnote-backref\" href=\"#fnref:2\" role=\"doc-backlink\">\u21a9\ufe0e</a></li></ol>",
			encoderMD:    "Text",
			encoderSz:    `(BLOCK (PARA (TEXT "Text") (ENDNOTE () (quote (INLINE (TEXT "Endnote") (ENDNOTE () (quote (INLINE (TEXT "Nested")))))))))`,
			encoderSHTML: "((p \"Text\" (sup (@ (id . \"fnref:1\")) (a (@ (class . \"zs-noteref\") (href . \"#fn:1\") (role . \"doc-noteref\")) \"1\"))))",
			encoderText:  "Text Endnote Nested",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Transclusion",
		zmk:   `{{{http://example.com/image}}}{width="100px"}`,
		expect: expectMap{
			encoderHTML:  `<p><img class="external" src="http://example.com/image" width="100px"></p>`,
			encoderMD:    "",
			encoderSz:    `(BLOCK (TRANSCLUDE (quote (("width" . "100px"))) (quote (EXTERNAL "http://example.com/image"))))`,
			encoderSHTML: `((p (img (@ (class . "external") (src . "http://example.com/image") (width . "100px")))))`,
			encoderText:  "",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "A paragraph with a inline comment only should be empty in HTML",







|
|










|
|










|











|














|













|











|











|







280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
	},
	{
		descr: "Simple Description List",
		zmk:   "; Zettel\n: Paper\n: Note\n; Zettelkasten\n: Slip box",
		expect: expectMap{
			encoderHTML:  "<dl><dt>Zettel</dt><dd><p>Paper</p></dd><dd><p>Note</p></dd><dt>Zettelkasten</dt><dd><p>Slip box</p></dd></dl>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (DESCRIPTION ((TEXT "Zettel")) (BLOCK (BLOCK (PARA (TEXT "Paper"))) (BLOCK (PARA (TEXT "Note")))) ((TEXT "Zettelkasten")) (BLOCK (BLOCK (PARA (TEXT "Slip box"))))))`,
			encoderSHTML: `((dl (dt "Zettel") (dd (p "Paper")) (dd (p "Note")) (dt "Zettelkasten") (dd (p "Slip box"))))`,
			encoderText:  "Zettel\nPaper\nNote\nZettelkasten\nSlip box",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Description List with paragraphs as item",
		zmk:   "; Zettel\n: Paper\n\n  Note\n; Zettelkasten\n: Slip box",
		expect: expectMap{
			encoderHTML:  "<dl><dt>Zettel</dt><dd><p>Paper</p><p>Note</p></dd><dt>Zettelkasten</dt><dd><p>Slip box</p></dd></dl>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (DESCRIPTION ((TEXT "Zettel")) (BLOCK (BLOCK (PARA (TEXT "Paper")) (PARA (TEXT "Note")))) ((TEXT "Zettelkasten")) (BLOCK (BLOCK (PARA (TEXT "Slip box"))))))`,
			encoderSHTML: `((dl (dt "Zettel") (dd (p "Paper") (p "Note")) (dt "Zettelkasten") (dd (p "Slip box"))))`,
			encoderText:  "Zettel\nPaper\nNote\nZettelkasten\nSlip box",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Description List with keys, but no descriptions",
		zmk:   "; K1\n: D11\n: D12\n; K2\n; K3\n: D31",
		expect: expectMap{
			encoderHTML:  "<dl><dt>K1</dt><dd><p>D11</p></dd><dd><p>D12</p></dd><dt>K2</dt><dt>K3</dt><dd><p>D31</p></dd></dl>",
			encoderMD:    "",
			encoderSz:    `(BLOCK (DESCRIPTION ((TEXT "K1")) (BLOCK (BLOCK (PARA (TEXT "D11"))) (BLOCK (PARA (TEXT "D12")))) ((TEXT "K2")) (BLOCK) ((TEXT "K3")) (BLOCK (BLOCK (PARA (TEXT "D31"))))))`,
			encoderSHTML: `((dl (dt "K1") (dd (p "D11")) (dd (p "D12")) (dt "K2") (dt "K3") (dd (p "D31"))))`,
			encoderText:  "K1\nD11\nD12\nK2\nK3\nD31",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple Table",
		zmk:   "|c1|c2|c3\n|d1||d3",
		expect: expectMap{
			encoderHTML:  `<table><tbody><tr><td>c1</td><td>c2</td><td>c3</td></tr><tr><td>d1</td><td></td><td>d3</td></tr></tbody></table>`,
			encoderMD:    "",
			encoderSz:    `(BLOCK (TABLE () ((CELL (TEXT "c1")) (CELL (TEXT "c2")) (CELL (TEXT "c3"))) ((CELL (TEXT "d1")) (CELL) (CELL (TEXT "d3")))))`,
			encoderSHTML: `((table (tbody (tr (td "c1") (td "c2") (td "c3")) (tr (td "d1") (td) (td "d3")))))`,
			encoderText:  "c1 c2 c3\nd1  d3",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Table with alignment and comment",
		zmk: `|h1>|=h2|h3:|
|%--+---+---+
|<c1|c2|:c3|
|f1|f2|=f3`,
		expect: expectMap{
			encoderHTML:  `<table><thead><tr><td class="right">h1</td><td>h2</td><td class="center">h3</td></tr></thead><tbody><tr><td class="left">c1</td><td>c2</td><td class="center">c3</td></tr><tr><td class="right">f1</td><td>f2</td><td class="center">=f3</td></tr></tbody></table>`,
			encoderMD:    "",
			encoderSz:    `(BLOCK (TABLE ((CELL-RIGHT (TEXT "h1")) (CELL (TEXT "h2")) (CELL-CENTER (TEXT "h3"))) ((CELL-LEFT (TEXT "c1")) (CELL (TEXT "c2")) (CELL-CENTER (TEXT "c3"))) ((CELL-RIGHT (TEXT "f1")) (CELL (TEXT "f2")) (CELL-CENTER (TEXT "=f3")))))`,
			encoderSHTML: `((table (thead (tr (td (@ (class . "right")) "h1") (td "h2") (td (@ (class . "center")) "h3"))) (tbody (tr (td (@ (class . "left")) "c1") (td "c2") (td (@ (class . "center")) "c3")) (tr (td (@ (class . "right")) "f1") (td "f2") (td (@ (class . "center")) "=f3")))))`,
			encoderText:  "h1 h2 h3\nc1 c2 c3\nf1 f2 =f3",
			encoderZmk: `|=h1>|=h2|=h3:
|<c1|c2|c3
|f1|f2|=f3`,
		},
	},
	{
		descr: "Simple Endnote",
		zmk:   `Text[^Endnote]`,
		expect: expectMap{
			encoderHTML:  "<p>Text<sup id=\"fnref:1\"><a class=\"zs-noteref\" href=\"#fn:1\" role=\"doc-noteref\">1</a></sup></p><ol class=\"zs-endnotes\"><li class=\"zs-endnote\" id=\"fn:1\" role=\"doc-endnote\" value=\"1\">Endnote <a class=\"zs-endnote-backref\" href=\"#fnref:1\" role=\"doc-backlink\">\u21a9\ufe0e</a></li></ol>",
			encoderMD:    "Text",
			encoderSz:    `(BLOCK (PARA (TEXT "Text") (ENDNOTE () (TEXT "Endnote"))))`,
			encoderSHTML: "((p \"Text\" (sup (@ (id . \"fnref:1\")) (a (@ (class . \"zs-noteref\") (href . \"#fn:1\") (role . \"doc-noteref\")) \"1\"))))",
			encoderText:  "Text Endnote",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Nested Endnotes",
		zmk:   `Text[^Endnote[^Nested]]`,
		expect: expectMap{
			encoderHTML:  "<p>Text<sup id=\"fnref:1\"><a class=\"zs-noteref\" href=\"#fn:1\" role=\"doc-noteref\">1</a></sup></p><ol class=\"zs-endnotes\"><li class=\"zs-endnote\" id=\"fn:1\" role=\"doc-endnote\" value=\"1\">Endnote<sup id=\"fnref:2\"><a class=\"zs-noteref\" href=\"#fn:2\" role=\"doc-noteref\">2</a></sup> <a class=\"zs-endnote-backref\" href=\"#fnref:1\" role=\"doc-backlink\">\u21a9\ufe0e</a></li><li class=\"zs-endnote\" id=\"fn:2\" role=\"doc-endnote\" value=\"2\">Nested <a class=\"zs-endnote-backref\" href=\"#fnref:2\" role=\"doc-backlink\">\u21a9\ufe0e</a></li></ol>",
			encoderMD:    "Text",
			encoderSz:    `(BLOCK (PARA (TEXT "Text") (ENDNOTE () (TEXT "Endnote") (ENDNOTE () (TEXT "Nested")))))`,
			encoderSHTML: "((p \"Text\" (sup (@ (id . \"fnref:1\")) (a (@ (class . \"zs-noteref\") (href . \"#fn:1\") (role . \"doc-noteref\")) \"1\"))))",
			encoderText:  "Text Endnote Nested",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Transclusion",
		zmk:   `{{{http://example.com/image}}}{width="100px"}`,
		expect: expectMap{
			encoderHTML:  `<p><img class="external" src="http://example.com/image" width="100px"></p>`,
			encoderMD:    "",
			encoderSz:    `(BLOCK (TRANSCLUDE (("width" . "100px")) (EXTERNAL "http://example.com/image")))`,
			encoderSHTML: `((p (img (@ (class . "external") (src . "http://example.com/image") (width . "100px")))))`,
			encoderText:  "",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "A paragraph with a inline comment only should be empty in HTML",

Changes to encoder/encoder_inline_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package encoder_test

var tcsInline = []zmkTestCase{
	{
		descr: "Empty Zettelmarkup should produce near nothing (inline)",








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package encoder_test

var tcsInline = []zmkTestCase{
	{
		descr: "Empty Zettelmarkup should produce near nothing (inline)",
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
	},
	{
		descr: "Simple text: Hello, world (inline)",
		zmk:   `Hello, world`,
		expect: expectMap{
			encoderHTML:  "Hello, world",
			encoderMD:    "Hello, world",
			encoderSz:    `(INLINE (TEXT "Hello,") (SPACE) (TEXT "world"))`,
			encoderSHTML: `("Hello," " " "world")`,
			encoderText:  "Hello, world",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Soft Break",
		zmk:   "soft\nbreak",







|
|







28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
	},
	{
		descr: "Simple text: Hello, world (inline)",
		zmk:   `Hello, world`,
		expect: expectMap{
			encoderHTML:  "Hello, world",
			encoderMD:    "Hello, world",
			encoderSz:    `(INLINE (TEXT "Hello, world"))`,
			encoderSHTML: `("Hello, world")`,
			encoderText:  "Hello, world",
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Soft Break",
		zmk:   "soft\nbreak",
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
	},
	{
		descr: "Quotes formatting (german)",
		zmk:   `""quotes""{lang=de}`,
		expect: expectMap{
			encoderHTML:  `<span lang="de">&bdquo;quotes&ldquo;</span>`,
			encoderMD:    "<q>quotes</q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE (quote (("lang" . "de"))) (TEXT "quotes")))`,
			encoderSHTML: `((span (@ (lang . "de")) (@H "&bdquo;") "quotes" (@H "&ldquo;")))`,
			encoderText:  `quotes`,
			encoderZmk:   `""quotes""{lang="de"}`,
		},
	},
	{
		descr: "Empty quotes (default)",







|







160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
	},
	{
		descr: "Quotes formatting (german)",
		zmk:   `""quotes""{lang=de}`,
		expect: expectMap{
			encoderHTML:  `<span lang="de">&bdquo;quotes&ldquo;</span>`,
			encoderMD:    "<q>quotes</q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE (("lang" . "de")) (TEXT "quotes")))`,
			encoderSHTML: `((span (@ (lang . "de")) (@H "&bdquo;") "quotes" (@H "&ldquo;")))`,
			encoderText:  `quotes`,
			encoderZmk:   `""quotes""{lang="de"}`,
		},
	},
	{
		descr: "Empty quotes (default)",
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
	},
	{
		descr: "Empty quotes (unknown)",
		zmk:   `""""{lang=unknown}`,
		expect: expectMap{
			encoderHTML:  `<span lang="unknown">&quot;&quot;</span>`,
			encoderMD:    "<q></q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE (quote (("lang" . "unknown")))))`,
			encoderSHTML: `((span (@ (lang . "unknown")) (@H "&quot;" "&quot;")))`,
			encoderText:  ``,
			encoderZmk:   `""""{lang="unknown"}`,
		},
	},
	{
		descr: "Nested quotes (default)",
		zmk:   `""say: ::""yes, ::""or?""::""::""`,
		expect: expectMap{
			encoderHTML:  `&ldquo;say: <span>&lsquo;yes, <span>&ldquo;or?&rdquo;</span>&rsquo;</span>&rdquo;`,
			encoderMD:    "<q>say: <q>yes, <q>or?</q></q></q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE () (TEXT "say:") (SPACE) (FORMAT-SPAN () (FORMAT-QUOTE () (TEXT "yes,") (SPACE) (FORMAT-SPAN () (FORMAT-QUOTE () (TEXT "or?")))))))`,
			encoderSHTML: `((@L (@H "&ldquo;") "say:" " " (span (@L (@H "&lsquo;") "yes," " " (span (@L (@H "&ldquo;") "or?" (@H "&rdquo;"))) (@H "&rsquo;"))) (@H "&rdquo;")))`,
			encoderText:  `say: yes, or?`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Two quotes",
		zmk:   `""yes"" or ""no""`,
		expect: expectMap{
			encoderHTML:  `&ldquo;yes&rdquo; or &ldquo;no&rdquo;`,
			encoderMD:    "<q>yes</q> or <q>no</q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE () (TEXT "yes")) (SPACE) (TEXT "or") (SPACE) (FORMAT-QUOTE () (TEXT "no")))`,
			encoderSHTML: `((@L (@H "&ldquo;") "yes" (@H "&rdquo;")) " " "or" " " (@L (@H "&ldquo;") "no" (@H "&rdquo;")))`,
			encoderText:  `yes or no`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Mark formatting",
		zmk:   `##marked##`,







|











|
|










|
|







184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
	},
	{
		descr: "Empty quotes (unknown)",
		zmk:   `""""{lang=unknown}`,
		expect: expectMap{
			encoderHTML:  `<span lang="unknown">&quot;&quot;</span>`,
			encoderMD:    "<q></q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE (("lang" . "unknown"))))`,
			encoderSHTML: `((span (@ (lang . "unknown")) (@H "&quot;" "&quot;")))`,
			encoderText:  ``,
			encoderZmk:   `""""{lang="unknown"}`,
		},
	},
	{
		descr: "Nested quotes (default)",
		zmk:   `""say: ::""yes, ::""or?""::""::""`,
		expect: expectMap{
			encoderHTML:  `&ldquo;say: <span>&lsquo;yes, <span>&ldquo;or?&rdquo;</span>&rsquo;</span>&rdquo;`,
			encoderMD:    "<q>say: <q>yes, <q>or?</q></q></q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE () (TEXT "say: ") (FORMAT-SPAN () (FORMAT-QUOTE () (TEXT "yes, ") (FORMAT-SPAN () (FORMAT-QUOTE () (TEXT "or?")))))))`,
			encoderSHTML: `((@L (@H "&ldquo;") "say: " (span (@L (@H "&lsquo;") "yes, " (span (@L (@H "&ldquo;") "or?" (@H "&rdquo;"))) (@H "&rsquo;"))) (@H "&rdquo;")))`,
			encoderText:  `say: yes, or?`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Two quotes",
		zmk:   `""yes"" or ""no""`,
		expect: expectMap{
			encoderHTML:  `&ldquo;yes&rdquo; or &ldquo;no&rdquo;`,
			encoderMD:    "<q>yes</q> or <q>no</q>",
			encoderSz:    `(INLINE (FORMAT-QUOTE () (TEXT "yes")) (TEXT " or ") (FORMAT-QUOTE () (TEXT "no")))`,
			encoderSHTML: `((@L (@H "&ldquo;") "yes" (@H "&rdquo;")) " or " (@L (@H "&ldquo;") "no" (@H "&rdquo;")))`,
			encoderText:  `yes or no`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Mark formatting",
		zmk:   `##marked##`,
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
	},
	{
		descr: "Code formatting with visible space",
		zmk:   "``x y``{-}",
		expect: expectMap{
			encoderHTML:  "<code>x\u2423y</code>",
			encoderMD:    "`x y`",
			encoderSz:    `(INLINE (LITERAL-CODE (quote (("-" . ""))) "x y"))`,
			encoderSHTML: "((code \"x\u2423y\"))",
			encoderText:  `x y`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "HTML in Code formatting",
		zmk:   "``<script `` abc",
		expect: expectMap{
			encoderHTML:  "<code>&lt;script </code> abc",
			encoderMD:    "`<script ` abc",
			encoderSz:    `(INLINE (LITERAL-CODE () "<script ") (SPACE) (TEXT "abc"))`,
			encoderSHTML: `((code "<script ") " " "abc")`,
			encoderText:  `<script  abc`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Input formatting",
		zmk:   `''input''`,







|











|
|







256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
	},
	{
		descr: "Code formatting with visible space",
		zmk:   "``x y``{-}",
		expect: expectMap{
			encoderHTML:  "<code>x\u2423y</code>",
			encoderMD:    "`x y`",
			encoderSz:    `(INLINE (LITERAL-CODE (("-" . "")) "x y"))`,
			encoderSHTML: "((code \"x\u2423y\"))",
			encoderText:  `x y`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "HTML in Code formatting",
		zmk:   "``<script `` abc",
		expect: expectMap{
			encoderHTML:  "<code>&lt;script </code> abc",
			encoderMD:    "`<script ` abc",
			encoderSz:    `(INLINE (LITERAL-CODE () "<script ") (TEXT " abc"))`,
			encoderSHTML: `((code "<script ") " abc")`,
			encoderText:  `<script  abc`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Input formatting",
		zmk:   `''input''`,
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
	},
	{
		descr: "Nested Span Quote formatting",
		zmk:   `::""abc""::{lang=fr}`,
		expect: expectMap{
			encoderHTML:  `<span lang="fr">&laquo;&nbsp;abc&nbsp;&raquo;</span>`,
			encoderMD:    "<q>abc</q>",
			encoderSz:    `(INLINE (FORMAT-SPAN (quote (("lang" . "fr"))) (FORMAT-QUOTE () (TEXT "abc"))))`,
			encoderSHTML: `((span (@ (lang . "fr")) (@L (@H "&laquo;" "&nbsp;") "abc" (@H "&nbsp;" "&raquo;"))))`,
			encoderText:  `abc`,
			encoderZmk:   `::""abc""::{lang="fr"}`,
		},
	},
	{
		descr: "Simple Citation",







|







316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
	},
	{
		descr: "Nested Span Quote formatting",
		zmk:   `::""abc""::{lang=fr}`,
		expect: expectMap{
			encoderHTML:  `<span lang="fr">&laquo;&nbsp;abc&nbsp;&raquo;</span>`,
			encoderMD:    "<q>abc</q>",
			encoderSz:    `(INLINE (FORMAT-SPAN (("lang" . "fr")) (FORMAT-QUOTE () (TEXT "abc"))))`,
			encoderSHTML: `((span (@ (lang . "fr")) (@L (@H "&laquo;" "&nbsp;") "abc" (@H "&nbsp;" "&raquo;"))))`,
			encoderText:  `abc`,
			encoderZmk:   `::""abc""::{lang="fr"}`,
		},
	},
	{
		descr: "Simple Citation",
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
		},
	}, {
		descr: "No comment",
		zmk:   `% comment`,
		expect: expectMap{
			encoderHTML:  `% comment`,
			encoderMD:    "% comment",
			encoderSz:    `(INLINE (TEXT "%") (SPACE) (TEXT "comment"))`,
			encoderSHTML: `("%" " " "comment")`,
			encoderText:  `% comment`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Line comment (nogen HTML)",
		zmk:   `%% line comment`,







|
|







351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
		},
	}, {
		descr: "No comment",
		zmk:   `% comment`,
		expect: expectMap{
			encoderHTML:  `% comment`,
			encoderMD:    "% comment",
			encoderSz:    `(INLINE (TEXT "% comment"))`,
			encoderSHTML: `("% comment")`,
			encoderText:  `% comment`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Line comment (nogen HTML)",
		zmk:   `%% line comment`,
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
	},
	{
		descr: "Line comment",
		zmk:   `%%{-} line comment`,
		expect: expectMap{
			encoderHTML:  `<!-- line comment -->`,
			encoderMD:    "",
			encoderSz:    `(INLINE (LITERAL-COMMENT (quote (("-" . ""))) "line comment"))`,
			encoderSHTML: `((@@ "line comment"))`,
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Comment after text",
		zmk:   `Text %%{-} comment`,
		expect: expectMap{
			encoderHTML:  `Text<!-- comment -->`,
			encoderMD:    "Text",
			encoderSz:    `(INLINE (TEXT "Text") (LITERAL-COMMENT (quote (("-" . ""))) "comment"))`,
			encoderSHTML: `("Text" (@@ "comment"))`,
			encoderText:  `Text`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Comment after text and with -->",
		zmk:   `Text %%{-} comment --> end`,
		expect: expectMap{
			encoderHTML:  `Text<!-- comment -&#45;> end -->`,
			encoderMD:    "Text",
			encoderSz:    `(INLINE (TEXT "Text") (LITERAL-COMMENT (quote (("-" . ""))) "comment --> end"))`,
			encoderSHTML: `("Text" (@@ "comment --> end"))`,
			encoderText:  `Text`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple inline endnote",
		zmk:   `[^endnote]`,
		expect: expectMap{
			encoderHTML:  `<sup id="fnref:1"><a class="zs-noteref" href="#fn:1" role="doc-noteref">1</a></sup>`,
			encoderMD:    "",
			encoderSz:    `(INLINE (ENDNOTE () (quote (INLINE (TEXT "endnote")))))`,
			encoderSHTML: `((sup (@ (id . "fnref:1")) (a (@ (class . "zs-noteref") (href . "#fn:1") (role . "doc-noteref")) "1")))`,
			encoderText:  `endnote`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple mark",







|











|











|











|







375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
	},
	{
		descr: "Line comment",
		zmk:   `%%{-} line comment`,
		expect: expectMap{
			encoderHTML:  `<!-- line comment -->`,
			encoderMD:    "",
			encoderSz:    `(INLINE (LITERAL-COMMENT (("-" . "")) "line comment"))`,
			encoderSHTML: `((@@ "line comment"))`,
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Comment after text",
		zmk:   `Text %%{-} comment`,
		expect: expectMap{
			encoderHTML:  `Text<!-- comment -->`,
			encoderMD:    "Text",
			encoderSz:    `(INLINE (TEXT "Text") (LITERAL-COMMENT (("-" . "")) "comment"))`,
			encoderSHTML: `("Text" (@@ "comment"))`,
			encoderText:  `Text`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Comment after text and with -->",
		zmk:   `Text %%{-} comment --> end`,
		expect: expectMap{
			encoderHTML:  `Text<!-- comment -&#45;> end -->`,
			encoderMD:    "Text",
			encoderSz:    `(INLINE (TEXT "Text") (LITERAL-COMMENT (("-" . "")) "comment --> end"))`,
			encoderSHTML: `("Text" (@@ "comment --> end"))`,
			encoderText:  `Text`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple inline endnote",
		zmk:   `[^endnote]`,
		expect: expectMap{
			encoderHTML:  `<sup id="fnref:1"><a class="zs-noteref" href="#fn:1" role="doc-noteref">1</a></sup>`,
			encoderMD:    "",
			encoderSz:    `(INLINE (ENDNOTE () (TEXT "endnote")))`,
			encoderSHTML: `((sup (@ (id . "fnref:1")) (a (@ (class . "zs-noteref") (href . "#fn:1") (role . "doc-noteref")) "1")))`,
			encoderText:  `endnote`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Simple mark",
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
	},
	{
		descr: "Mark with text",
		zmk:   `[!mark|with text]`,
		expect: expectMap{
			encoderHTML:  `<a id="mark">with text</a>`,
			encoderMD:    "with text",
			encoderSz:    `(INLINE (MARK "mark" "mark" "mark" (TEXT "with") (SPACE) (TEXT "text")))`,
			encoderSHTML: `((a (@ (id . "mark")) "with" " " "text"))`,
			encoderText:  `with text`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Invalid Link",
		zmk:   `[[link|00000000000000]]`,







|
|







435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
	},
	{
		descr: "Mark with text",
		zmk:   `[!mark|with text]`,
		expect: expectMap{
			encoderHTML:  `<a id="mark">with text</a>`,
			encoderMD:    "with text",
			encoderSz:    `(INLINE (MARK "mark" "mark" "mark" (TEXT "with text")))`,
			encoderSHTML: `((a (@ (id . "mark")) "with text"))`,
			encoderText:  `with text`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Invalid Link",
		zmk:   `[[link|00000000000000]]`,
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
	},
	{
		descr: "Dummy Embed",
		zmk:   `{{abc}}`,
		expect: expectMap{
			encoderHTML:  `<img src="abc">`,
			encoderMD:    "![abc](abc)",
			encoderSz:    `(INLINE (EMBED () (quote (EXTERNAL "abc")) ""))`,
			encoderSHTML: `((img (@ (src . "abc"))))`,
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Inline HTML Zettel",







|







627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
	},
	{
		descr: "Dummy Embed",
		zmk:   `{{abc}}`,
		expect: expectMap{
			encoderHTML:  `<img src="abc">`,
			encoderMD:    "![abc](abc)",
			encoderSz:    `(INLINE (EMBED () (EXTERNAL "abc") ""))`,
			encoderSHTML: `((img (@ (src . "abc"))))`,
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Inline HTML Zettel",
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
	},
	{
		descr: "Inline Text Zettel",
		zmk:   `@@<hr>@@{="text"}`,
		expect: expectMap{
			encoderHTML:  ``,
			encoderMD:    "<hr>",
			encoderSz:    `(INLINE (LITERAL-ZETTEL (quote (("" . "text"))) "<hr>"))`,
			encoderSHTML: `(())`,
			encoderText:  `<hr>`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "",







|







651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
	},
	{
		descr: "Inline Text Zettel",
		zmk:   `@@<hr>@@{="text"}`,
		expect: expectMap{
			encoderHTML:  ``,
			encoderMD:    "<hr>",
			encoderSz:    `(INLINE (LITERAL-ZETTEL (("" . "text")) "<hr>"))`,
			encoderSHTML: `(())`,
			encoderText:  `<hr>`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "",

Changes to encoder/encoder_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
27
28
29
30
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package encoder_test

import (
	"fmt"
	"strings"
	"testing"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/sx.fossil/sxreader"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/meta"

	_ "zettelstore.de/z/encoder/htmlenc"  // Allow to use HTML encoder.
	_ "zettelstore.de/z/encoder/mdenc"    // Allow to use markdown encoder.
	_ "zettelstore.de/z/encoder/shtmlenc" // Allow to use SHTML encoder.
	_ "zettelstore.de/z/encoder/szenc"    // Allow to use sz encoder.








>
>
>









|
|
>



<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package encoder_test

import (
	"fmt"
	"strings"
	"testing"

	"t73f.de/r/sx/sxreader"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/input"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/encoder"

	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/meta"

	_ "zettelstore.de/z/encoder/htmlenc"  // Allow to use HTML encoder.
	_ "zettelstore.de/z/encoder/mdenc"    // Allow to use markdown encoder.
	_ "zettelstore.de/z/encoder/shtmlenc" // Allow to use SHTML encoder.
	_ "zettelstore.de/z/encoder/szenc"    // Allow to use sz encoder.
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
		return
	}
	val, err := sxreader.MakeReader(strings.NewReader(exp)).Read()
	if err != nil {
		t.Error(err)
		return
	}
	got := val.Repr()
	if exp != got {
		prefix := fmt.Sprintf("Test #%d", testNum)
		if d := descr; d != "" {
			prefix += "\nReason:   " + d
		}
		prefix += "\nMode:     " + pe.mode()
		t.Errorf("%s\n\nExpected: %q\nGot:      %q", prefix, exp, got)







|







115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
		return
	}
	val, err := sxreader.MakeReader(strings.NewReader(exp)).Read()
	if err != nil {
		t.Error(err)
		return
	}
	got := val.String()
	if exp != got {
		prefix := fmt.Sprintf("Test #%d", testNum)
		if d := descr; d != "" {
			prefix += "\nReason:   " + d
		}
		prefix += "\nMode:     " + pe.mode()
		t.Errorf("%s\n\nExpected: %q\nGot:      %q", prefix, exp, got)

Changes to encoder/htmlenc/htmlenc.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package htmlenc encodes the abstract syntax tree into HTML5 via zettelstore-client.
package htmlenc

import (
	"io"
	"strings"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/client.fossil/shtml"
	"zettelstore.de/sx.fossil"
	"zettelstore.de/sx.fossil/sxhtml"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/szenc"
	"zettelstore.de/z/encoder/textenc"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderHTML, func(params *encoder.CreateParameter) encoder.Encoder { return Create(params) })
}

// Create an encoder.
func Create(params *encoder.CreateParameter) *Encoder {
	// We need a new transformer every time, because tx.inVerse must be unique.
	// If we can refactor it out, the transformer can be created only once.
	return &Encoder{
		tx:      szenc.NewTransformer(),
		th:      shtml.NewEvaluator(1, nil),
		lang:    params.Lang,
		textEnc: textenc.Create(),
	}
}

type Encoder struct {
	tx      *szenc.Transformer








>
>
>









|
|
|
|


















|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package htmlenc encodes the abstract syntax tree into HTML5 via zettelstore-client.
package htmlenc

import (
	"io"
	"strings"

	"t73f.de/r/sx"
	"t73f.de/r/sxwebs/sxhtml"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/shtml"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/szenc"
	"zettelstore.de/z/encoder/textenc"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderHTML, func(params *encoder.CreateParameter) encoder.Encoder { return Create(params) })
}

// Create an encoder.
func Create(params *encoder.CreateParameter) *Encoder {
	// We need a new transformer every time, because tx.inVerse must be unique.
	// If we can refactor it out, the transformer can be created only once.
	return &Encoder{
		tx:      szenc.NewTransformer(),
		th:      shtml.NewEvaluator(1),
		lang:    params.Lang,
		textEnc: textenc.Create(),
	}
}

type Encoder struct {
	tx      *szenc.Transformer
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
	xast := he.tx.GetSz(&zn.Ast)
	hast, err := he.th.Evaluate(xast, &env)
	if err != nil {
		return 0, err
	}
	hen := he.th.Endnotes(&env)

	sf := he.th.SymbolFactory()
	symAttr := sf.MustMake(sxhtml.NameSymAttr)

	head := sx.MakeList(sf.MustMake("head"))
	curr := head
	curr = curr.AppendBang(sx.Nil().Cons(sx.Nil().Cons(sx.Cons(sf.MustMake("charset"), sx.String("utf-8"))).Cons(symAttr)).Cons(sf.MustMake("meta")))
	for elem := hm; elem != nil; elem = elem.Tail() {
		curr = curr.AppendBang(elem.Car())
	}
	var sb strings.Builder
	if hasTitle {
		he.textEnc.WriteInlines(&sb, &isTitle)
	} else {
		sb.Write(zn.Meta.Zid.Bytes())
	}
	_ = curr.AppendBang(sx.Nil().Cons(sx.String(sb.String())).Cons(sf.MustMake("title")))

	body := sx.MakeList(sf.MustMake("body"))
	curr = body
	if hasTitle {
		curr = curr.AppendBang(htitle.Cons(sf.MustMake("h1")))
	}
	for elem := hast; elem != nil; elem = elem.Tail() {
		curr = curr.AppendBang(elem.Car())
	}
	if hen != nil {
		curr = curr.AppendBang(sx.Nil().Cons(sf.MustMake("hr")))
		_ = curr.AppendBang(hen)
	}

	doc := sx.MakeList(
		sf.MustMake(sxhtml.NameSymDoctype),
		sx.MakeList(sf.MustMake("html"), head, body),
	)

	gen := sxhtml.NewGenerator(sf, sxhtml.WithNewline)
	return gen.WriteHTML(w, doc)
}

// WriteMeta encodes meta data as HTML5.
func (he *Encoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	env := shtml.MakeEnvironment(he.lang)
	hm, err := he.th.Evaluate(he.tx.GetMeta(m, evalMeta), &env)
	if err != nil {
		return 0, err
	}
	gen := sxhtml.NewGenerator(he.th.SymbolFactory(), sxhtml.WithNewline)
	return gen.WriteListHTML(w, hm)
}

func (he *Encoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return he.WriteBlocks(w, &zn.Ast)
}

// WriteBlocks encodes a block slice.
func (he *Encoder) WriteBlocks(w io.Writer, bs *ast.BlockSlice) (int, error) {
	env := shtml.MakeEnvironment(he.lang)
	hobj, err := he.th.Evaluate(he.tx.GetSz(bs), &env)
	if err == nil {
		gen := sxhtml.NewGenerator(he.th.SymbolFactory())
		length, err2 := gen.WriteListHTML(w, hobj)
		if err2 != nil {
			return length, err2
		}

		l, err2 := gen.WriteHTML(w, he.th.Endnotes(&env))
		length += l
		return length, err2
	}
	return 0, err
}

// WriteInlines writes an inline slice to the writer
func (he *Encoder) WriteInlines(w io.Writer, is *ast.InlineSlice) (int, error) {
	env := shtml.MakeEnvironment(he.lang)
	hobj, err := he.th.Evaluate(he.tx.GetSz(is), &env)
	if err == nil {
		gen := sxhtml.NewGenerator(sx.FindSymbolFactory(hobj))
		length, err2 := gen.WriteListHTML(w, hobj)
		if err2 != nil {
			return length, err2
		}
		return length, nil
	}
	return 0, err
}







<
<
|
<
|
|
<
|
<






|

|
|

|

<
|
<

|
|



|
|


|










|












|

















|








76
77
78
79
80
81
82


83

84
85

86

87
88
89
90
91
92
93
94
95
96
97
98
99

100

101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
	xast := he.tx.GetSz(&zn.Ast)
	hast, err := he.th.Evaluate(xast, &env)
	if err != nil {
		return 0, err
	}
	hen := he.th.Endnotes(&env)



	var head sx.ListBuilder

	head.Add(shtml.SymHead)
	head.Add(sx.Nil().Cons(sx.Nil().Cons(sx.Cons(sx.MakeSymbol("charset"), sx.MakeString("utf-8"))).Cons(sxhtml.SymAttr)).Cons(shtml.SymMeta))

	head.ExtendBang(hm)

	var sb strings.Builder
	if hasTitle {
		he.textEnc.WriteInlines(&sb, &isTitle)
	} else {
		sb.Write(zn.Meta.Zid.Bytes())
	}
	head.Add(sx.MakeList(shtml.SymAttrTitle, sx.MakeString(sb.String())))

	var body sx.ListBuilder
	body.Add(shtml.SymBody)
	if hasTitle {
		body.Add(htitle.Cons(shtml.SymH1))
	}

	body.ExtendBang(hast)

	if hen != nil {
		body.Add(sx.Cons(shtml.SymHR, nil))
		body.Add(hen)
	}

	doc := sx.MakeList(
		sxhtml.SymDoctype,
		sx.MakeList(shtml.SymHtml, head.List(), body.List()),
	)

	gen := sxhtml.NewGenerator().SetNewline()
	return gen.WriteHTML(w, doc)
}

// WriteMeta encodes meta data as HTML5.
func (he *Encoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	env := shtml.MakeEnvironment(he.lang)
	hm, err := he.th.Evaluate(he.tx.GetMeta(m, evalMeta), &env)
	if err != nil {
		return 0, err
	}
	gen := sxhtml.NewGenerator().SetNewline()
	return gen.WriteListHTML(w, hm)
}

func (he *Encoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return he.WriteBlocks(w, &zn.Ast)
}

// WriteBlocks encodes a block slice.
func (he *Encoder) WriteBlocks(w io.Writer, bs *ast.BlockSlice) (int, error) {
	env := shtml.MakeEnvironment(he.lang)
	hobj, err := he.th.Evaluate(he.tx.GetSz(bs), &env)
	if err == nil {
		gen := sxhtml.NewGenerator()
		length, err2 := gen.WriteListHTML(w, hobj)
		if err2 != nil {
			return length, err2
		}

		l, err2 := gen.WriteHTML(w, he.th.Endnotes(&env))
		length += l
		return length, err2
	}
	return 0, err
}

// WriteInlines writes an inline slice to the writer
func (he *Encoder) WriteInlines(w io.Writer, is *ast.InlineSlice) (int, error) {
	env := shtml.MakeEnvironment(he.lang)
	hobj, err := he.th.Evaluate(he.tx.GetSz(is), &env)
	if err == nil {
		gen := sxhtml.NewGenerator()
		length, err2 := gen.WriteListHTML(w, hobj)
		if err2 != nil {
			return length, err2
		}
		return length, nil
	}
	return 0, err
}

Changes to encoder/mdenc/mdenc.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package mdenc encodes the abstract syntax tree back into Markdown.
package mdenc

import (
	"io"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderMD, func(*encoder.CreateParameter) encoder.Encoder { return Create() })








>
>
>








|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package mdenc encodes the abstract syntax tree back into Markdown.
package mdenc

import (
	"io"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderMD, func(*encoder.CreateParameter) encoder.Encoder { return Create() })
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
	case *ast.NestedListNode:
		v.visitNestedList(n)
	case *ast.DescriptionListNode:
		return nil // Should write no content
	case *ast.TableNode:
		return nil // Should write no content
	case *ast.TextNode:
		v.visitText(n)
	case *ast.SpaceNode:
		v.b.WriteString(n.Lexeme)
	case *ast.BreakNode:
		v.visitBreak(n)
	case *ast.LinkNode:
		v.visitLink(n)
	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.FootnoteNode:







<
<
|







116
117
118
119
120
121
122


123
124
125
126
127
128
129
130
	case *ast.NestedListNode:
		v.visitNestedList(n)
	case *ast.DescriptionListNode:
		return nil // Should write no content
	case *ast.TableNode:
		return nil // Should write no content
	case *ast.TextNode:


		v.b.WriteString(n.Text)
	case *ast.BreakNode:
		v.visitBreak(n)
	case *ast.LinkNode:
		v.visitLink(n)
	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.FootnoteNode:
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
			ast.Walk(v, in)
		}
	}

	v.listPrefix = prefix
}

func (v *visitor) visitText(tn *ast.TextNode) {
	v.b.WriteString(tn.Text)
}

func (v *visitor) visitBreak(bn *ast.BreakNode) {
	if bn.Hard {
		v.b.WriteString("\\\n")
	} else {
		v.b.WriteByte('\n')
	}
	if l := len(v.listInfo); l > 0 {







<
<
<
<







260
261
262
263
264
265
266




267
268
269
270
271
272
273
			ast.Walk(v, in)
		}
	}

	v.listPrefix = prefix
}





func (v *visitor) visitBreak(bn *ast.BreakNode) {
	if bn.Hard {
		v.b.WriteString("\\\n")
	} else {
		v.b.WriteByte('\n')
	}
	if l := len(v.listInfo); l > 0 {
349
350
351
352
353
354
355
356
357
358
359
	case ast.LiteralComment, ast.LiteralHTML: // ignore everything
	default:
		v.b.Write(ln.Content)
	}
}

func (v *visitor) writeSpaces(n int) {
	for i := 0; i < n; i++ {
		v.b.WriteByte(' ')
	}
}







|



346
347
348
349
350
351
352
353
354
355
356
	case ast.LiteralComment, ast.LiteralHTML: // ignore everything
	default:
		v.b.Write(ln.Content)
	}
}

func (v *visitor) writeSpaces(n int) {
	for range n {
		v.b.WriteByte(' ')
	}
}

Changes to encoder/shtmlenc/shtmlenc.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package shtmlenc encodes the abstract syntax tree into a s-expr which represents HTML.
package shtmlenc

import (
	"io"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/client.fossil/shtml"
	"zettelstore.de/sx.fossil"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/szenc"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderSHTML, func(params *encoder.CreateParameter) encoder.Encoder { return Create(params) })
}

// Create a SHTML encoder
func Create(params *encoder.CreateParameter) *Encoder {
	// We need a new transformer every time, because tx.inVerse must be unique.
	// If we can refactor it out, the transformer can be created only once.
	return &Encoder{
		tx:   szenc.NewTransformer(),
		th:   shtml.NewEvaluator(1, nil),
		lang: params.Lang,
	}
}

type Encoder struct {
	tx   *szenc.Transformer
	th   *shtml.Evaluator








>
>
>








|
|
|
















|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2023-present Detlef Stern
//-----------------------------------------------------------------------------

// Package shtmlenc encodes the abstract syntax tree into a s-expr which represents HTML.
package shtmlenc

import (
	"io"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/shtml"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/szenc"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderSHTML, func(params *encoder.CreateParameter) encoder.Encoder { return Create(params) })
}

// Create a SHTML encoder
func Create(params *encoder.CreateParameter) *Encoder {
	// We need a new transformer every time, because tx.inVerse must be unique.
	// If we can refactor it out, the transformer can be created only once.
	return &Encoder{
		tx:   szenc.NewTransformer(),
		th:   shtml.NewEvaluator(1),
		lang: params.Lang,
	}
}

type Encoder struct {
	tx   *szenc.Transformer
	th   *shtml.Evaluator

Changes to encoder/szenc/szenc.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package szenc encodes the abstract syntax tree into a s-expr for zettel.
package szenc

import (
	"io"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/sx.fossil"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderSz, func(*encoder.CreateParameter) encoder.Encoder { return Create() })








>
>
>








|
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package szenc encodes the abstract syntax tree into a s-expr for zettel.
package szenc

import (
	"io"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderSz, func(*encoder.CreateParameter) encoder.Encoder { return Create() })

Changes to encoder/szenc/transform.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215





































216
217
218
219
220
221
222
223
224
225
226
227
228
229

230




231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307







308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328












329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372












373
374
375
376
377
378
379

380












381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package szenc

import (
	"encoding/base64"
	"fmt"
	"strings"

	"zettelstore.de/client.fossil/attrs"
	"zettelstore.de/client.fossil/sz"
	"zettelstore.de/sx.fossil"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

// NewTransformer returns a new transformer to create s-expressions from AST nodes.
func NewTransformer() *Transformer {
	sf := sx.MakeMappedFactory(1024)
	t := Transformer{sf: sf}
	t.zetSyms.InitializeZettelSymbols(sf)

	t.mapVerbatimKindS = map[ast.VerbatimKind]*sx.Symbol{
		ast.VerbatimZettel:  t.zetSyms.SymVerbatimZettel,
		ast.VerbatimProg:    t.zetSyms.SymVerbatimProg,
		ast.VerbatimEval:    t.zetSyms.SymVerbatimEval,
		ast.VerbatimMath:    t.zetSyms.SymVerbatimMath,
		ast.VerbatimComment: t.zetSyms.SymVerbatimComment,
		ast.VerbatimHTML:    t.zetSyms.SymVerbatimHTML,
	}

	t.mapRegionKindS = map[ast.RegionKind]*sx.Symbol{
		ast.RegionSpan:  t.zetSyms.SymRegionBlock,
		ast.RegionQuote: t.zetSyms.SymRegionQuote,
		ast.RegionVerse: t.zetSyms.SymRegionVerse,
	}
	t.mapNestedListKindS = map[ast.NestedListKind]*sx.Symbol{
		ast.NestedListOrdered:   t.zetSyms.SymListOrdered,
		ast.NestedListUnordered: t.zetSyms.SymListUnordered,
		ast.NestedListQuote:     t.zetSyms.SymListQuote,
	}
	t.alignmentSymbolS = map[ast.Alignment]*sx.Symbol{
		ast.AlignDefault: t.zetSyms.SymCell,
		ast.AlignLeft:    t.zetSyms.SymCellLeft,
		ast.AlignCenter:  t.zetSyms.SymCellCenter,
		ast.AlignRight:   t.zetSyms.SymCellRight,
	}
	t.mapRefStateLink = map[ast.RefState]*sx.Symbol{
		ast.RefStateInvalid:  t.zetSyms.SymLinkInvalid,
		ast.RefStateZettel:   t.zetSyms.SymLinkZettel,
		ast.RefStateSelf:     t.zetSyms.SymLinkSelf,
		ast.RefStateFound:    t.zetSyms.SymLinkFound,
		ast.RefStateBroken:   t.zetSyms.SymLinkBroken,
		ast.RefStateHosted:   t.zetSyms.SymLinkHosted,
		ast.RefStateBased:    t.zetSyms.SymLinkBased,
		ast.RefStateQuery:    t.zetSyms.SymLinkQuery,
		ast.RefStateExternal: t.zetSyms.SymLinkExternal,
	}
	t.mapFormatKindS = map[ast.FormatKind]*sx.Symbol{
		ast.FormatEmph:   t.zetSyms.SymFormatEmph,
		ast.FormatStrong: t.zetSyms.SymFormatStrong,
		ast.FormatDelete: t.zetSyms.SymFormatDelete,
		ast.FormatInsert: t.zetSyms.SymFormatInsert,
		ast.FormatSuper:  t.zetSyms.SymFormatSuper,
		ast.FormatSub:    t.zetSyms.SymFormatSub,
		ast.FormatQuote:  t.zetSyms.SymFormatQuote,
		ast.FormatMark:   t.zetSyms.SymFormatMark,
		ast.FormatSpan:   t.zetSyms.SymFormatSpan,
	}
	t.mapLiteralKindS = map[ast.LiteralKind]*sx.Symbol{
		ast.LiteralZettel:  t.zetSyms.SymLiteralZettel,
		ast.LiteralProg:    t.zetSyms.SymLiteralProg,
		ast.LiteralInput:   t.zetSyms.SymLiteralInput,
		ast.LiteralOutput:  t.zetSyms.SymLiteralOutput,
		ast.LiteralComment: t.zetSyms.SymLiteralComment,
		ast.LiteralHTML:    t.zetSyms.SymLiteralHTML,
		ast.LiteralMath:    t.zetSyms.SymLiteralMath,
	}
	t.mapRefStateS = map[ast.RefState]*sx.Symbol{
		ast.RefStateInvalid:  t.zetSyms.SymRefStateInvalid,
		ast.RefStateZettel:   t.zetSyms.SymRefStateZettel,
		ast.RefStateSelf:     t.zetSyms.SymRefStateSelf,
		ast.RefStateFound:    t.zetSyms.SymRefStateFound,
		ast.RefStateBroken:   t.zetSyms.SymRefStateBroken,
		ast.RefStateHosted:   t.zetSyms.SymRefStateHosted,
		ast.RefStateBased:    t.zetSyms.SymRefStateBased,
		ast.RefStateQuery:    t.zetSyms.SymRefStateQuery,
		ast.RefStateExternal: t.zetSyms.SymRefStateExternal,
	}
	t.mapMetaTypeS = map[*meta.DescriptionType]*sx.Symbol{
		meta.TypeCredential:   t.zetSyms.SymTypeCredential,
		meta.TypeEmpty:        t.zetSyms.SymTypeEmpty,
		meta.TypeID:           t.zetSyms.SymTypeID,
		meta.TypeIDSet:        t.zetSyms.SymTypeIDSet,
		meta.TypeNumber:       t.zetSyms.SymTypeNumber,
		meta.TypeString:       t.zetSyms.SymTypeString,
		meta.TypeTagSet:       t.zetSyms.SymTypeTagSet,
		meta.TypeTimestamp:    t.zetSyms.SymTypeTimestamp,
		meta.TypeURL:          t.zetSyms.SymTypeURL,
		meta.TypeWord:         t.zetSyms.SymTypeWord,
		meta.TypeWordSet:      t.zetSyms.SymTypeWordSet,
		meta.TypeZettelmarkup: t.zetSyms.SymTypeZettelmarkup,
	}
	return &t
}

type Transformer struct {
	sf                 sx.SymbolFactory
	zetSyms            sz.ZettelSymbols
	mapVerbatimKindS   map[ast.VerbatimKind]*sx.Symbol
	mapRegionKindS     map[ast.RegionKind]*sx.Symbol
	mapNestedListKindS map[ast.NestedListKind]*sx.Symbol
	alignmentSymbolS   map[ast.Alignment]*sx.Symbol
	mapRefStateLink    map[ast.RefState]*sx.Symbol
	mapFormatKindS     map[ast.FormatKind]*sx.Symbol
	mapLiteralKindS    map[ast.LiteralKind]*sx.Symbol
	mapRefStateS       map[ast.RefState]*sx.Symbol
	mapMetaTypeS       map[*meta.DescriptionType]*sx.Symbol
	inVerse            bool
}

func (t *Transformer) GetSz(node ast.Node) *sx.Pair {
	switch n := node.(type) {
	case *ast.BlockSlice:
		return t.getBlockSlice(n)
	case *ast.InlineSlice:
		return t.getInlineSlice(*n)
	case *ast.ParaNode:
		return t.getInlineSlice(n.Inlines).Tail().Cons(t.zetSyms.SymPara)
	case *ast.VerbatimNode:
		return sx.MakeList(
			mapGetS(t, t.mapVerbatimKindS, n.Kind),
			t.getAttributes(n.Attrs),
			sx.String(string(n.Content)),
		)
	case *ast.RegionNode:
		return t.getRegion(n)
	case *ast.HeadingNode:
		return sx.MakeList(
			t.zetSyms.SymHeading,
			sx.Int64(int64(n.Level)),
			t.getAttributes(n.Attrs),
			sx.String(n.Slug),
			sx.String(n.Fragment),
			t.getInlineSlice(n.Inlines),
		)
	case *ast.HRuleNode:
		return sx.MakeList(t.zetSyms.SymThematic, t.getAttributes(n.Attrs))
	case *ast.NestedListNode:
		return t.getNestedList(n)
	case *ast.DescriptionListNode:
		return t.getDescriptionList(n)
	case *ast.TableNode:
		return t.getTable(n)
	case *ast.TranscludeNode:
		return sx.MakeList(t.zetSyms.SymTransclude, t.getAttributes(n.Attrs), t.getReference(n.Ref))
	case *ast.BLOBNode:
		return t.getBLOB(n)
	case *ast.TextNode:
		return sx.MakeList(t.zetSyms.SymText, sx.String(n.Text))
	case *ast.SpaceNode:
		if t.inVerse {
			return sx.MakeList(t.zetSyms.SymSpace, sx.String(n.Lexeme))
		}
		return sx.MakeList(t.zetSyms.SymSpace)
	case *ast.BreakNode:
		if n.Hard {
			return sx.MakeList(t.zetSyms.SymHard)
		}
		return sx.MakeList(t.zetSyms.SymSoft)
	case *ast.LinkNode:
		return t.getLink(n)
	case *ast.EmbedRefNode:
		return t.getInlineSlice(n.Inlines).Tail().
			Cons(sx.String(n.Syntax)).
			Cons(t.getReference(n.Ref)).
			Cons(t.getAttributes(n.Attrs)).
			Cons(t.zetSyms.SymEmbed)
	case *ast.EmbedBLOBNode:
		return t.getEmbedBLOB(n)
	case *ast.CiteNode:
		return t.getInlineSlice(n.Inlines).Tail().
			Cons(sx.String(n.Key)).
			Cons(t.getAttributes(n.Attrs)).
			Cons(t.zetSyms.SymCite)
	case *ast.FootnoteNode:
		text := sx.Nil().Cons(sx.Nil().Cons(t.getInlineSlice(n.Inlines)).Cons(t.zetSyms.SymQuote))
		return text.Cons(t.getAttributes(n.Attrs)).Cons(t.zetSyms.SymEndnote)
	case *ast.MarkNode:
		return t.getInlineSlice(n.Inlines).Tail().
			Cons(sx.String(n.Fragment)).
			Cons(sx.String(n.Slug)).
			Cons(sx.String(n.Mark)).
			Cons(t.zetSyms.SymMark)
	case *ast.FormatNode:
		return t.getInlineSlice(n.Inlines).Tail().
			Cons(t.getAttributes(n.Attrs)).
			Cons(mapGetS(t, t.mapFormatKindS, n.Kind))
	case *ast.LiteralNode:
		return sx.MakeList(
			mapGetS(t, t.mapLiteralKindS, n.Kind),
			t.getAttributes(n.Attrs),
			sx.String(string(n.Content)),
		)
	}
	return sx.MakeList(t.zetSyms.SymUnknown, sx.String(fmt.Sprintf("%T %v", node, node)))





































}

func (t *Transformer) getRegion(rn *ast.RegionNode) *sx.Pair {
	saveInVerse := t.inVerse
	if rn.Kind == ast.RegionVerse {
		t.inVerse = true
	}
	symBlocks := t.GetSz(&rn.Blocks)
	t.inVerse = saveInVerse
	return sx.MakeList(
		mapGetS(t, t.mapRegionKindS, rn.Kind),
		t.getAttributes(rn.Attrs),
		symBlocks,
		t.GetSz(&rn.Inlines),

	)




}

func (t *Transformer) getNestedList(ln *ast.NestedListNode) *sx.Pair {
	nlistObjs := make([]sx.Object, len(ln.Items)+1)
	nlistObjs[0] = mapGetS(t, t.mapNestedListKindS, ln.Kind)
	isCompact := isCompactList(ln.Items)
	for i, item := range ln.Items {
		if isCompact && len(item) > 0 {
			paragraph := t.GetSz(item[0])
			nlistObjs[i+1] = paragraph.Tail().Cons(t.zetSyms.SymInline)
			continue
		}
		itemObjs := make([]sx.Object, len(item))
		for j, in := range item {
			itemObjs[j] = t.GetSz(in)
		}
		if isCompact {
			nlistObjs[i+1] = sx.MakeList(itemObjs...).Cons(t.zetSyms.SymInline)
		} else {
			nlistObjs[i+1] = sx.MakeList(itemObjs...).Cons(t.zetSyms.SymBlock)
		}
	}
	return sx.MakeList(nlistObjs...)
}
func isCompactList(itemSlice []ast.ItemSlice) bool {
	for _, items := range itemSlice {
		if len(items) > 1 {
			return false
		}
		if len(items) == 1 {
			if _, ok := items[0].(*ast.ParaNode); !ok {
				return false
			}
		}
	}
	return true
}

func (t *Transformer) getDescriptionList(dn *ast.DescriptionListNode) *sx.Pair {
	dlObjs := make([]sx.Object, 2*len(dn.Descriptions)+1)
	dlObjs[0] = t.zetSyms.SymDescription
	for i, def := range dn.Descriptions {
		dlObjs[2*i+1] = t.getInlineSlice(def.Term)
		descObjs := make([]sx.Object, len(def.Descriptions))
		for j, b := range def.Descriptions {
			dVal := make([]sx.Object, len(b))
			for k, dn := range b {
				dVal[k] = t.GetSz(dn)
			}
			descObjs[j] = sx.MakeList(dVal...).Cons(t.zetSyms.SymBlock)
		}
		dlObjs[2*i+2] = sx.MakeList(descObjs...).Cons(t.zetSyms.SymBlock)
	}
	return sx.MakeList(dlObjs...)
}

func (t *Transformer) getTable(tn *ast.TableNode) *sx.Pair {
	tObjs := make([]sx.Object, len(tn.Rows)+2)
	tObjs[0] = t.zetSyms.SymTable
	tObjs[1] = t.getHeader(tn.Header)
	for i, row := range tn.Rows {
		tObjs[i+2] = t.getRow(row)
	}
	return sx.MakeList(tObjs...)
}
func (t *Transformer) getHeader(header ast.TableRow) *sx.Pair {
	if len(header) == 0 {
		return nil
	}
	return t.getRow(header)
}
func (t *Transformer) getRow(row ast.TableRow) *sx.Pair {
	rObjs := make([]sx.Object, len(row))
	for i, cell := range row {
		rObjs[i] = t.getCell(cell)
	}
	return sx.MakeList(rObjs...).Cons(t.zetSyms.SymList)







}

func (t *Transformer) getCell(cell *ast.TableCell) *sx.Pair {
	return t.getInlineSlice(cell.Inlines).Tail().Cons(mapGetS(t, t.alignmentSymbolS, cell.Align))
}

func (t *Transformer) getBLOB(bn *ast.BLOBNode) *sx.Pair {
	var lastObj sx.Object
	if bn.Syntax == meta.SyntaxSVG {
		lastObj = sx.String(string(bn.Blob))
	} else {
		lastObj = getBase64String(bn.Blob)
	}
	return sx.MakeList(
		t.zetSyms.SymBLOB,
		t.getInlineSlice(bn.Description),
		sx.String(bn.Syntax),
		lastObj,
	)
}













func (t *Transformer) getLink(ln *ast.LinkNode) *sx.Pair {
	return t.getInlineSlice(ln.Inlines).Tail().
		Cons(sx.String(ln.Ref.Value)).
		Cons(t.getAttributes(ln.Attrs)).
		Cons(mapGetS(t, t.mapRefStateLink, ln.Ref.State))
}

func (t *Transformer) getEmbedBLOB(en *ast.EmbedBLOBNode) *sx.Pair {
	tail := t.getInlineSlice(en.Inlines).Tail()
	if en.Syntax == meta.SyntaxSVG {
		tail = tail.Cons(sx.String(string(en.Blob)))
	} else {
		tail = tail.Cons(getBase64String(en.Blob))
	}
	return tail.Cons(sx.String(en.Syntax)).Cons(t.getAttributes(en.Attrs)).Cons(t.zetSyms.SymEmbedBLOB)
}

func (t *Transformer) getBlockSlice(bs *ast.BlockSlice) *sx.Pair {
	objs := make([]sx.Object, len(*bs))
	for i, n := range *bs {
		objs[i] = t.GetSz(n)
	}
	return sx.MakeList(objs...).Cons(t.zetSyms.SymBlock)
}
func (t *Transformer) getInlineSlice(is ast.InlineSlice) *sx.Pair {
	objs := make([]sx.Object, len(is))
	for i, n := range is {
		objs[i] = t.GetSz(n)
	}
	return sx.MakeList(objs...).Cons(t.zetSyms.SymInline)
}

func (t *Transformer) getAttributes(a attrs.Attributes) sx.Object {
	if a.IsEmpty() {
		return sx.Nil()
	}
	keys := a.Keys()
	objs := make([]sx.Object, 0, len(keys))
	for _, k := range keys {
		objs = append(objs, sx.Cons(sx.String(k), sx.String(a[k])))
	}
	return sx.Nil().Cons(sx.MakeList(objs...)).Cons(t.zetSyms.SymQuote)
}













func (t *Transformer) getReference(ref *ast.Reference) *sx.Pair {
	return sx.MakeList(
		t.zetSyms.SymQuote,
		sx.MakeList(
			mapGetS(t, t.mapRefStateS, ref.State),
			sx.String(ref.Value),
		),

	)












}

func (t *Transformer) GetMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) *sx.Pair {
	pairs := m.ComputedPairs()
	objs := make([]sx.Object, 0, len(pairs))
	for _, p := range pairs {
		key := p.Key
		ty := m.Type(key)
		symType := mapGetS(t, t.mapMetaTypeS, ty)
		var obj sx.Object
		if ty.IsSet {
			setList := meta.ListFromValue(p.Value)
			setObjs := make([]sx.Object, len(setList))
			for i, val := range setList {
				setObjs[i] = sx.String(val)
			}
			obj = sx.MakeList(setObjs...).Cons(t.zetSyms.SymList)
		} else if ty == meta.TypeZettelmarkup {
			is := evalMeta(p.Value)
			obj = t.GetSz(&is)
		} else {
			obj = sx.String(p.Value)
		}
		symKey := sx.MakeList(t.zetSyms.SymQuote, t.sf.MustMake(key))
		objs = append(objs, sx.Nil().Cons(obj).Cons(symKey).Cons(symType))
	}
	return sx.MakeList(objs...).Cons(t.zetSyms.SymMeta)
}

func mapGetS[T comparable](t *Transformer, m map[T]*sx.Symbol, k T) *sx.Symbol {
	if result, found := m[k]; found {
		return result
	}
	return t.sf.MustMake(fmt.Sprintf("**%v:NOT-FOUND**", k))
}

func getBase64String(data []byte) sx.String {
	var sb strings.Builder
	encoder := base64.NewEncoder(base64.StdEncoding, &sb)
	_, err := encoder.Write(data)
	if err == nil {
		err = encoder.Close()
	}
	if err == nil {
		return sx.String(sb.String())
	}
	return sx.String("")
}








>
>
>









|
|
|







<
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<




<
<
<
<
<
<
<
<
<
<
<
|





|

|

|


|
|
|




|
|
|
|
|
|
<
<

|







|



<
<
<
<
<
|


|

|



|
|
|
|
|



|
|
|
|

|
|

|
|
|
|
|

|
|
|


|
|
|


|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







|

|
|
|
<
|
>
|
>
>
>
>



|
|




|


|




|

|



















|
|

|
|

|



|

|





|
|













|



|
>
>
>
>
>
>
>



|





|




|
|
|




>
>
>
>
>
>
>
>
>
>
>
>

|
|
|
|



|

|



|


|
|



|

|
|



|


|




|

|

|


>
>
>
>
>
>
>
>
>
>
>
>
|
|
<
<
<
<
<
>
|
>
>
>
>
>
>
>
>
>
>
>
>




|



|



|

|

|


|

|

<
|

|


|



|










|

|

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

31



















































































32
33
34
35











36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61


62
63
64
65
66
67
68
69
70
71
72
73
74





75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165

166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347





348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384

385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

package szenc

import (
	"encoding/base64"
	"fmt"
	"strings"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/attrs"
	"t73f.de/r/zsc/sz"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

// NewTransformer returns a new transformer to create s-expressions from AST nodes.
func NewTransformer() *Transformer {

	t := Transformer{}



















































































	return &t
}

type Transformer struct {











	inVerse bool
}

func (t *Transformer) GetSz(node ast.Node) *sx.Pair {
	switch n := node.(type) {
	case *ast.BlockSlice:
		return t.getBlockList(n).Cons(sz.SymBlock)
	case *ast.InlineSlice:
		return t.getInlineList(*n).Cons(sz.SymInline)
	case *ast.ParaNode:
		return t.getInlineList(n.Inlines).Cons(sz.SymPara)
	case *ast.VerbatimNode:
		return sx.MakeList(
			mapGetS(mapVerbatimKindS, n.Kind),
			getAttributes(n.Attrs),
			sx.MakeString(string(n.Content)),
		)
	case *ast.RegionNode:
		return t.getRegion(n)
	case *ast.HeadingNode:
		return t.getInlineList(n.Inlines).
			Cons(sx.MakeString(n.Fragment)).
			Cons(sx.MakeString(n.Slug)).
			Cons(getAttributes(n.Attrs)).
			Cons(sx.Int64(int64(n.Level))).
			Cons(sz.SymHeading)


	case *ast.HRuleNode:
		return sx.MakeList(sz.SymThematic, getAttributes(n.Attrs))
	case *ast.NestedListNode:
		return t.getNestedList(n)
	case *ast.DescriptionListNode:
		return t.getDescriptionList(n)
	case *ast.TableNode:
		return t.getTable(n)
	case *ast.TranscludeNode:
		return sx.MakeList(sz.SymTransclude, getAttributes(n.Attrs), getReference(n.Ref))
	case *ast.BLOBNode:
		return t.getBLOB(n)
	case *ast.TextNode:





		return sx.MakeList(sz.SymText, sx.MakeString(n.Text))
	case *ast.BreakNode:
		if n.Hard {
			return sx.MakeList(sz.SymHard)
		}
		return sx.MakeList(sz.SymSoft)
	case *ast.LinkNode:
		return t.getLink(n)
	case *ast.EmbedRefNode:
		return t.getInlineList(n.Inlines).
			Cons(sx.MakeString(n.Syntax)).
			Cons(getReference(n.Ref)).
			Cons(getAttributes(n.Attrs)).
			Cons(sz.SymEmbed)
	case *ast.EmbedBLOBNode:
		return t.getEmbedBLOB(n)
	case *ast.CiteNode:
		return t.getInlineList(n.Inlines).
			Cons(sx.MakeString(n.Key)).
			Cons(getAttributes(n.Attrs)).
			Cons(sz.SymCite)
	case *ast.FootnoteNode:
		// (ENDNODE attrs InlineElement ...)
		return t.getInlineList(n.Inlines).Cons(getAttributes(n.Attrs)).Cons(sz.SymEndnote)
	case *ast.MarkNode:
		return t.getInlineList(n.Inlines).
			Cons(sx.MakeString(n.Fragment)).
			Cons(sx.MakeString(n.Slug)).
			Cons(sx.MakeString(n.Mark)).
			Cons(sz.SymMark)
	case *ast.FormatNode:
		return t.getInlineList(n.Inlines).
			Cons(getAttributes(n.Attrs)).
			Cons(mapGetS(mapFormatKindS, n.Kind))
	case *ast.LiteralNode:
		return sx.MakeList(
			mapGetS(mapLiteralKindS, n.Kind),
			getAttributes(n.Attrs),
			sx.MakeString(string(n.Content)),
		)
	}
	return sx.MakeList(sz.SymUnknown, sx.MakeString(fmt.Sprintf("%T %v", node, node)))
}

var mapVerbatimKindS = map[ast.VerbatimKind]*sx.Symbol{
	ast.VerbatimZettel:  sz.SymVerbatimZettel,
	ast.VerbatimProg:    sz.SymVerbatimProg,
	ast.VerbatimEval:    sz.SymVerbatimEval,
	ast.VerbatimMath:    sz.SymVerbatimMath,
	ast.VerbatimComment: sz.SymVerbatimComment,
	ast.VerbatimHTML:    sz.SymVerbatimHTML,
}

var mapFormatKindS = map[ast.FormatKind]*sx.Symbol{
	ast.FormatEmph:   sz.SymFormatEmph,
	ast.FormatStrong: sz.SymFormatStrong,
	ast.FormatDelete: sz.SymFormatDelete,
	ast.FormatInsert: sz.SymFormatInsert,
	ast.FormatSuper:  sz.SymFormatSuper,
	ast.FormatSub:    sz.SymFormatSub,
	ast.FormatQuote:  sz.SymFormatQuote,
	ast.FormatMark:   sz.SymFormatMark,
	ast.FormatSpan:   sz.SymFormatSpan,
}

var mapLiteralKindS = map[ast.LiteralKind]*sx.Symbol{
	ast.LiteralZettel:  sz.SymLiteralZettel,
	ast.LiteralProg:    sz.SymLiteralProg,
	ast.LiteralInput:   sz.SymLiteralInput,
	ast.LiteralOutput:  sz.SymLiteralOutput,
	ast.LiteralComment: sz.SymLiteralComment,
	ast.LiteralHTML:    sz.SymLiteralHTML,
	ast.LiteralMath:    sz.SymLiteralMath,
}

var mapRegionKindS = map[ast.RegionKind]*sx.Symbol{
	ast.RegionSpan:  sz.SymRegionBlock,
	ast.RegionQuote: sz.SymRegionQuote,
	ast.RegionVerse: sz.SymRegionVerse,
}

func (t *Transformer) getRegion(rn *ast.RegionNode) *sx.Pair {
	saveInVerse := t.inVerse
	if rn.Kind == ast.RegionVerse {
		t.inVerse = true
	}
	symBlocks := t.getBlockList(&rn.Blocks)
	t.inVerse = saveInVerse
	return t.getInlineList(rn.Inlines).
		Cons(symBlocks).
		Cons(getAttributes(rn.Attrs)).

		Cons(mapGetS(mapRegionKindS, rn.Kind))
}

var mapNestedListKindS = map[ast.NestedListKind]*sx.Symbol{
	ast.NestedListOrdered:   sz.SymListOrdered,
	ast.NestedListUnordered: sz.SymListUnordered,
	ast.NestedListQuote:     sz.SymListQuote,
}

func (t *Transformer) getNestedList(ln *ast.NestedListNode) *sx.Pair {
	nlistObjs := make(sx.Vector, len(ln.Items)+1)
	nlistObjs[0] = mapGetS(mapNestedListKindS, ln.Kind)
	isCompact := isCompactList(ln.Items)
	for i, item := range ln.Items {
		if isCompact && len(item) > 0 {
			paragraph := t.GetSz(item[0])
			nlistObjs[i+1] = paragraph.Tail().Cons(sz.SymInline)
			continue
		}
		itemObjs := make(sx.Vector, len(item))
		for j, in := range item {
			itemObjs[j] = t.GetSz(in)
		}
		if isCompact {
			nlistObjs[i+1] = sx.MakeList(itemObjs...).Cons(sz.SymInline)
		} else {
			nlistObjs[i+1] = sx.MakeList(itemObjs...).Cons(sz.SymBlock)
		}
	}
	return sx.MakeList(nlistObjs...)
}
func isCompactList(itemSlice []ast.ItemSlice) bool {
	for _, items := range itemSlice {
		if len(items) > 1 {
			return false
		}
		if len(items) == 1 {
			if _, ok := items[0].(*ast.ParaNode); !ok {
				return false
			}
		}
	}
	return true
}

func (t *Transformer) getDescriptionList(dn *ast.DescriptionListNode) *sx.Pair {
	dlObjs := make(sx.Vector, 2*len(dn.Descriptions)+1)
	dlObjs[0] = sz.SymDescription
	for i, def := range dn.Descriptions {
		dlObjs[2*i+1] = t.getInlineList(def.Term)
		descObjs := make(sx.Vector, len(def.Descriptions))
		for j, b := range def.Descriptions {
			dVal := make(sx.Vector, len(b))
			for k, dn := range b {
				dVal[k] = t.GetSz(dn)
			}
			descObjs[j] = sx.MakeList(dVal...).Cons(sz.SymBlock)
		}
		dlObjs[2*i+2] = sx.MakeList(descObjs...).Cons(sz.SymBlock)
	}
	return sx.MakeList(dlObjs...)
}

func (t *Transformer) getTable(tn *ast.TableNode) *sx.Pair {
	tObjs := make(sx.Vector, len(tn.Rows)+2)
	tObjs[0] = sz.SymTable
	tObjs[1] = t.getHeader(tn.Header)
	for i, row := range tn.Rows {
		tObjs[i+2] = t.getRow(row)
	}
	return sx.MakeList(tObjs...)
}
func (t *Transformer) getHeader(header ast.TableRow) *sx.Pair {
	if len(header) == 0 {
		return nil
	}
	return t.getRow(header)
}
func (t *Transformer) getRow(row ast.TableRow) *sx.Pair {
	rObjs := make(sx.Vector, len(row))
	for i, cell := range row {
		rObjs[i] = t.getCell(cell)
	}
	return sx.MakeList(rObjs...)
}

var alignmentSymbolS = map[ast.Alignment]*sx.Symbol{
	ast.AlignDefault: sz.SymCell,
	ast.AlignLeft:    sz.SymCellLeft,
	ast.AlignCenter:  sz.SymCellCenter,
	ast.AlignRight:   sz.SymCellRight,
}

func (t *Transformer) getCell(cell *ast.TableCell) *sx.Pair {
	return t.getInlineList(cell.Inlines).Cons(mapGetS(alignmentSymbolS, cell.Align))
}

func (t *Transformer) getBLOB(bn *ast.BLOBNode) *sx.Pair {
	var lastObj sx.Object
	if bn.Syntax == meta.SyntaxSVG {
		lastObj = sx.MakeString(string(bn.Blob))
	} else {
		lastObj = getBase64String(bn.Blob)
	}
	return sx.MakeList(
		sz.SymBLOB,
		t.getInlineList(bn.Description),
		sx.MakeString(bn.Syntax),
		lastObj,
	)
}

var mapRefStateLink = map[ast.RefState]*sx.Symbol{
	ast.RefStateInvalid:  sz.SymLinkInvalid,
	ast.RefStateZettel:   sz.SymLinkZettel,
	ast.RefStateSelf:     sz.SymLinkSelf,
	ast.RefStateFound:    sz.SymLinkFound,
	ast.RefStateBroken:   sz.SymLinkBroken,
	ast.RefStateHosted:   sz.SymLinkHosted,
	ast.RefStateBased:    sz.SymLinkBased,
	ast.RefStateQuery:    sz.SymLinkQuery,
	ast.RefStateExternal: sz.SymLinkExternal,
}

func (t *Transformer) getLink(ln *ast.LinkNode) *sx.Pair {
	return t.getInlineList(ln.Inlines).
		Cons(sx.MakeString(ln.Ref.Value)).
		Cons(getAttributes(ln.Attrs)).
		Cons(mapGetS(mapRefStateLink, ln.Ref.State))
}

func (t *Transformer) getEmbedBLOB(en *ast.EmbedBLOBNode) *sx.Pair {
	tail := t.getInlineList(en.Inlines)
	if en.Syntax == meta.SyntaxSVG {
		tail = tail.Cons(sx.MakeString(string(en.Blob)))
	} else {
		tail = tail.Cons(getBase64String(en.Blob))
	}
	return tail.Cons(sx.MakeString(en.Syntax)).Cons(getAttributes(en.Attrs)).Cons(sz.SymEmbedBLOB)
}

func (t *Transformer) getBlockList(bs *ast.BlockSlice) *sx.Pair {
	objs := make(sx.Vector, len(*bs))
	for i, n := range *bs {
		objs[i] = t.GetSz(n)
	}
	return sx.MakeList(objs...)
}
func (t *Transformer) getInlineList(is ast.InlineSlice) *sx.Pair {
	objs := make(sx.Vector, len(is))
	for i, n := range is {
		objs[i] = t.GetSz(n)
	}
	return sx.MakeList(objs...)
}

func getAttributes(a attrs.Attributes) sx.Object {
	if a.IsEmpty() {
		return sx.Nil()
	}
	keys := a.Keys()
	objs := make(sx.Vector, 0, len(keys))
	for _, k := range keys {
		objs = append(objs, sx.Cons(sx.MakeString(k), sx.MakeString(a[k])))
	}
	return sx.MakeList(objs...)
}

var mapRefStateS = map[ast.RefState]*sx.Symbol{
	ast.RefStateInvalid:  sz.SymRefStateInvalid,
	ast.RefStateZettel:   sz.SymRefStateZettel,
	ast.RefStateSelf:     sz.SymRefStateSelf,
	ast.RefStateFound:    sz.SymRefStateFound,
	ast.RefStateBroken:   sz.SymRefStateBroken,
	ast.RefStateHosted:   sz.SymRefStateHosted,
	ast.RefStateBased:    sz.SymRefStateBased,
	ast.RefStateQuery:    sz.SymRefStateQuery,
	ast.RefStateExternal: sz.SymRefStateExternal,
}

func getReference(ref *ast.Reference) *sx.Pair {
	return sx.MakeList(mapGetS(mapRefStateS, ref.State), sx.MakeString(ref.Value))





}

var mapMetaTypeS = map[*meta.DescriptionType]*sx.Symbol{
	meta.TypeCredential:   sz.SymTypeCredential,
	meta.TypeEmpty:        sz.SymTypeEmpty,
	meta.TypeID:           sz.SymTypeID,
	meta.TypeIDSet:        sz.SymTypeIDSet,
	meta.TypeNumber:       sz.SymTypeNumber,
	meta.TypeString:       sz.SymTypeString,
	meta.TypeTagSet:       sz.SymTypeTagSet,
	meta.TypeTimestamp:    sz.SymTypeTimestamp,
	meta.TypeURL:          sz.SymTypeURL,
	meta.TypeWord:         sz.SymTypeWord,
	meta.TypeZettelmarkup: sz.SymTypeZettelmarkup,
}

func (t *Transformer) GetMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) *sx.Pair {
	pairs := m.ComputedPairs()
	objs := make(sx.Vector, 0, len(pairs))
	for _, p := range pairs {
		key := p.Key
		ty := m.Type(key)
		symType := mapGetS(mapMetaTypeS, ty)
		var obj sx.Object
		if ty.IsSet {
			setList := meta.ListFromValue(p.Value)
			setObjs := make(sx.Vector, len(setList))
			for i, val := range setList {
				setObjs[i] = sx.MakeString(val)
			}
			obj = sx.MakeList(setObjs...)
		} else if ty == meta.TypeZettelmarkup {
			is := evalMeta(p.Value)
			obj = t.getInlineList(is)
		} else {
			obj = sx.MakeString(p.Value)
		}

		objs = append(objs, sx.Nil().Cons(obj).Cons(sx.MakeSymbol(key)).Cons(symType))
	}
	return sx.MakeList(objs...).Cons(sz.SymMeta)
}

func mapGetS[T comparable](m map[T]*sx.Symbol, k T) *sx.Symbol {
	if result, found := m[k]; found {
		return result
	}
	return sx.MakeSymbol(fmt.Sprintf("**%v:NOT-FOUND**", k))
}

func getBase64String(data []byte) sx.String {
	var sb strings.Builder
	encoder := base64.NewEncoder(base64.StdEncoding, &sb)
	_, err := encoder.Write(data)
	if err == nil {
		err = encoder.Close()
	}
	if err == nil {
		return sx.MakeString(sb.String())
	}
	return sx.MakeString("")
}

Changes to encoder/textenc/textenc.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package textenc encodes the abstract syntax tree into its text.
package textenc

import (
	"io"

	"zettelstore.de/client.fossil/api"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderText, func(*encoder.CreateParameter) encoder.Encoder { return Create() })








>
>
>








|
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package textenc encodes the abstract syntax tree into its text.
package textenc

import (
	"io"

	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/input"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/zettel/meta"
)

func init() {
	encoder.Register(api.EncoderText, func(*encoder.CreateParameter) encoder.Encoder { return Create() })
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
		return nil
	case *ast.TableNode:
		v.visitTable(n)
		return nil
	case *ast.TranscludeNode, *ast.BLOBNode:
		return nil
	case *ast.TextNode:
		v.b.WriteString(n.Text)
		return nil
	case *ast.SpaceNode:
		v.b.WriteByte(' ')
		return nil
	case *ast.BreakNode:
		if n.Hard {
			v.b.WriteByte('\n')
		} else {
			v.b.WriteByte(' ')
		}







|
<
<
<







129
130
131
132
133
134
135
136



137
138
139
140
141
142
143
		return nil
	case *ast.TableNode:
		v.visitTable(n)
		return nil
	case *ast.TranscludeNode, *ast.BLOBNode:
		return nil
	case *ast.TextNode:
		v.visitText(n.Text)



		return nil
	case *ast.BreakNode:
		if n.Hard {
			v.b.WriteByte('\n')
		} else {
			v.b.WriteByte(' ')
		}
223
224
225
226
227
228
229















230
231
232
233
234
235
func (v *visitor) visitInlineSlice(is *ast.InlineSlice) {
	for i, in := range *is {
		v.inlinePos = i
		ast.Walk(v, in)
	}
	v.inlinePos = 0
}
















func (v *visitor) writePosChar(pos int, ch byte) {
	if pos > 0 {
		v.b.WriteByte(ch)
	}
}







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>






224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
func (v *visitor) visitInlineSlice(is *ast.InlineSlice) {
	for i, in := range *is {
		v.inlinePos = i
		ast.Walk(v, in)
	}
	v.inlinePos = 0
}

func (v *visitor) visitText(s string) {
	spaceFound := false
	for _, ch := range s {
		if input.IsSpace(ch) {
			if !spaceFound {
				v.b.WriteByte(' ')
				spaceFound = true
			}
			continue
		}
		spaceFound = false
		v.b.WriteString(string(ch))
	}
}

func (v *visitor) writePosChar(pos int, ch byte) {
	if pos > 0 {
		v.b.WriteByte(ch)
	}
}

Changes to encoder/write.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

package encoder

import (
	"encoding/base64"
	"io"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package encoder

import (
	"encoding/base64"
	"io"

Changes to encoder/zmkenc/zmkenc.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package zmkenc encodes the abstract syntax tree back into Zettelmarkup.
package zmkenc

import (
	"fmt"
	"io"
	"strings"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/client.fossil/attrs"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/textenc"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/meta"
)









>
>
>










|
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package zmkenc encodes the abstract syntax tree back into Zettelmarkup.
package zmkenc

import (
	"fmt"
	"io"
	"strings"

	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/attrs"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/textenc"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/meta"
)

133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
	case *ast.TranscludeNode:
		v.b.WriteStrings("{{{", n.Ref.String(), "}}}")
		v.visitAttributes(n.Attrs)
	case *ast.BLOBNode:
		v.visitBLOB(n)
	case *ast.TextNode:
		v.visitText(n)
	case *ast.SpaceNode:
		v.b.WriteString(n.Lexeme)
	case *ast.BreakNode:
		v.visitBreak(n)
	case *ast.LinkNode:
		v.visitLink(n)
	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.EmbedBLOBNode:







<
<







136
137
138
139
140
141
142


143
144
145
146
147
148
149
	case *ast.TranscludeNode:
		v.b.WriteStrings("{{{", n.Ref.String(), "}}}")
		v.visitAttributes(n.Attrs)
	case *ast.BLOBNode:
		v.visitBLOB(n)
	case *ast.TextNode:
		v.visitText(n)


	case *ast.BreakNode:
		v.visitBreak(n)
	case *ast.LinkNode:
		v.visitLink(n)
	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.EmbedBLOBNode:
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
			last = i + 1
			continue
		}
		if i < len(tn.Text)-1 {
			s := tn.Text[i : i+2]
			if escapeSeqs.Has(s) {
				v.b.WriteString(tn.Text[last:i])
				for j := 0; j < len(s); j++ {
					v.b.WriteBytes('\\', s[j])
				}
				i++
				last = i + 1
				continue
			}
		}







|







369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
			last = i + 1
			continue
		}
		if i < len(tn.Text)-1 {
			s := tn.Text[i : i+2]
			if escapeSeqs.Has(s) {
				v.b.WriteString(tn.Text[last:i])
				for j := range len(s) {
					v.b.WriteBytes('\\', s[j])
				}
				i++
				last = i + 1
				continue
			}
		}
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
		}
	}
	v.b.WriteByte('}')
}

func (v *visitor) writeEscaped(s string, toEscape byte) {
	last := 0
	for i := 0; i < len(s); i++ {
		if b := s[i]; b == toEscape || b == '\\' {
			v.b.WriteString(s[last:i])
			v.b.WriteBytes('\\', b)
			last = i + 1
		}
	}
	v.b.WriteString(s[last:])
}

func syntaxToHTML(a attrs.Attributes) attrs.Attributes {
	return a.Clone().Set("", meta.SyntaxHTML).Remove(api.KeySyntax)
}







|












524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
		}
	}
	v.b.WriteByte('}')
}

func (v *visitor) writeEscaped(s string, toEscape byte) {
	last := 0
	for i := range len(s) {
		if b := s[i]; b == toEscape || b == '\\' {
			v.b.WriteString(s[last:i])
			v.b.WriteBytes('\\', b)
			last = i + 1
		}
	}
	v.b.WriteString(s[last:])
}

func syntaxToHTML(a attrs.Attributes) attrs.Attributes {
	return a.Clone().Set("", meta.SyntaxHTML).Remove(api.KeySyntax)
}

Changes to encoding/atom/atom.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package atom provides an Atom encoding.
package atom

import (
	"bytes"
	"time"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/config"
	"zettelstore.de/z/encoding"
	"zettelstore.de/z/encoding/xml"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/query"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"








>
>
>









|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package atom provides an Atom encoding.
package atom

import (
	"bytes"
	"time"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/config"
	"zettelstore.de/z/encoding"
	"zettelstore.de/z/encoding/xml"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/query"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
	entryUpdated := ""
	if val, found := m.Get(api.KeyPublished); found {
		if published, err := time.ParseInLocation(id.TimestampLayout, val, time.Local); err == nil {
			entryUpdated = published.UTC().Format(time.RFC3339)
		}
	}

	link := c.NewURLBuilderAbs().SetZid(api.ZettelID(m.Zid.String())).String()

	buf.WriteString("  <entry>\n")
	xml.WriteTag(buf, "    ", "title", encoding.TitleAsText(m))
	xml.WriteTag(buf, "    ", "id", link)
	buf.WriteString(`    <link rel="self" href="`)
	strfun.XMLEscape(buf, link)
	buf.WriteString(`"/>` + "\n")







|







80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
	entryUpdated := ""
	if val, found := m.Get(api.KeyPublished); found {
		if published, err := time.ParseInLocation(id.TimestampLayout, val, time.Local); err == nil {
			entryUpdated = published.UTC().Format(time.RFC3339)
		}
	}

	link := c.NewURLBuilderAbs().SetZid(m.Zid.ZettelID()).String()

	buf.WriteString("  <entry>\n")
	xml.WriteTag(buf, "    ", "title", encoding.TitleAsText(m))
	xml.WriteTag(buf, "    ", "id", link)
	buf.WriteString(`    <link rel="self" href="`)
	strfun.XMLEscape(buf, link)
	buf.WriteString(`"/>` + "\n")

Changes to encoding/encoding.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package encoding provides helper functions for encodings.
package encoding

import (
	"time"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// LastUpdated returns the formated time of the zettel which was updated at the latest time.
func LastUpdated(ml []*meta.Meta, timeFormat string) string {








>
>
>








|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package encoding provides helper functions for encodings.
package encoding

import (
	"time"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// LastUpdated returns the formated time of the zettel which was updated at the latest time.
func LastUpdated(ml []*meta.Meta, timeFormat string) string {

Changes to encoding/rss/rss.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package rss provides a RSS encoding.
package rss

import (
	"bytes"
	"context"
	"time"

	"zettelstore.de/client.fossil/api"
	"zettelstore.de/z/config"
	"zettelstore.de/z/encoding"
	"zettelstore.de/z/encoding/xml"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/query"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"








>
>
>










|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package rss provides a RSS encoding.
package rss

import (
	"bytes"
	"context"
	"time"

	"t73f.de/r/zsc/api"
	"zettelstore.de/z/config"
	"zettelstore.de/z/encoding"
	"zettelstore.de/z/encoding/xml"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/query"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/zettel/id"
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
	itemPublished := ""
	if val, found := m.Get(api.KeyPublished); found {
		if published, err := time.ParseInLocation(id.TimestampLayout, val, time.Local); err == nil {
			itemPublished = published.UTC().Format(time.RFC1123Z)
		}
	}

	link := c.NewURLBuilderAbs().SetZid(api.ZettelID(m.Zid.String())).String()

	buf.WriteString("  <item>\n")
	xml.WriteTag(buf, "    ", "title", encoding.TitleAsText(m))
	xml.WriteTag(buf, "    ", "link", link)
	xml.WriteTag(buf, "    ", "guid", link)
	if itemPublished != "" {
		xml.WriteTag(buf, "    ", "pubDate", itemPublished)







|







90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
	itemPublished := ""
	if val, found := m.Get(api.KeyPublished); found {
		if published, err := time.ParseInLocation(id.TimestampLayout, val, time.Local); err == nil {
			itemPublished = published.UTC().Format(time.RFC1123Z)
		}
	}

	link := c.NewURLBuilderAbs().SetZid(m.Zid.ZettelID()).String()

	buf.WriteString("  <item>\n")
	xml.WriteTag(buf, "    ", "title", encoding.TitleAsText(m))
	xml.WriteTag(buf, "    ", "link", link)
	xml.WriteTag(buf, "    ", "guid", link)
	if itemPublished != "" {
		xml.WriteTag(buf, "    ", "pubDate", itemPublished)

Changes to encoding/xml/xml.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package xml provides helper for a XML-based encoding.
package xml

import (
	"bytes"








>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package xml provides helper for a XML-based encoding.
package xml

import (
	"bytes"

Changes to evaluator/evaluator.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14

15
16
17
18
19
20
21


22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

46
47
48

























49
50
51




52
53
54
55
56
57
58
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.



//-----------------------------------------------------------------------------

// Package evaluator interprets and evaluates the AST.
package evaluator

import (

	"context"
	"errors"
	"fmt"
	"path"
	"strconv"
	"strings"



	"zettelstore.de/client.fossil/api"
	"zettelstore.de/client.fossil/attrs"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/parser/cleaner"
	"zettelstore.de/z/parser/draw"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// Port contains all methods to retrieve zettel (or part of it) to evaluate a zettel.
type Port interface {
	GetZettel(context.Context, id.Zid) (zettel.Zettel, error)
	QueryMeta(ctx context.Context, q *query.Query) ([]*meta.Meta, error)
}

// EvaluateZettel evaluates the given zettel in the given context, with the
// given ports, and the given environment.
func EvaluateZettel(ctx context.Context, port Port, rtConfig config.Config, zn *ast.ZettelNode) {

	if zn.Syntax == meta.SyntaxNone {
		// AST is empty, evaluate to a description list of metadata.
		zn.Ast = evaluateMetadata(zn.Meta)

























		return
	}
	EvaluateBlock(ctx, port, rtConfig, &zn.Ast)




}

// EvaluateBlock evaluates the given block list in the given context, with
// the given ports, and the given environment.
func EvaluateBlock(ctx context.Context, port Port, rtConfig config.Config, bns *ast.BlockSlice) {
	evaluateNode(ctx, port, rtConfig, bns)
	cleaner.CleanBlockSlice(bns, true)








>
>
>






>







>
>
|
|
>



<


















>
|


>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
<
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

83
84
85
86
87
88
89
90
91
92
93
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package evaluator interprets and evaluates the AST.
package evaluator

import (
	"bytes"
	"context"
	"errors"
	"fmt"
	"path"
	"strconv"
	"strings"

	"t73f.de/r/sx/sxbuiltins"
	"t73f.de/r/sx/sxreader"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/attrs"
	"t73f.de/r/zsc/input"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"

	"zettelstore.de/z/parser"
	"zettelstore.de/z/parser/cleaner"
	"zettelstore.de/z/parser/draw"
	"zettelstore.de/z/query"
	"zettelstore.de/z/zettel"
	"zettelstore.de/z/zettel/id"
	"zettelstore.de/z/zettel/meta"
)

// Port contains all methods to retrieve zettel (or part of it) to evaluate a zettel.
type Port interface {
	GetZettel(context.Context, id.Zid) (zettel.Zettel, error)
	QueryMeta(ctx context.Context, q *query.Query) ([]*meta.Meta, error)
}

// EvaluateZettel evaluates the given zettel in the given context, with the
// given ports, and the given environment.
func EvaluateZettel(ctx context.Context, port Port, rtConfig config.Config, zn *ast.ZettelNode) {
	switch zn.Syntax {
	case meta.SyntaxNone:
		// AST is empty, evaluate to a description list of metadata.
		zn.Ast = evaluateMetadata(zn.Meta)
	case meta.SyntaxSxn:
		zn.Ast = evaluateSxn(zn.Ast)
	default:
		EvaluateBlock(ctx, port, rtConfig, &zn.Ast)
	}
}

func evaluateSxn(bs ast.BlockSlice) ast.BlockSlice {
	// Check for structure made in parser/plain/plain.go:parseSxnBlocks
	if len(bs) == 1 {
		// If len(bs) > 1 --> an error was found during parsing
		if vn, isVerbatim := bs[0].(*ast.VerbatimNode); isVerbatim && vn.Kind == ast.VerbatimProg {
			if classAttr, hasClass := vn.Attrs.Get(""); hasClass && classAttr == meta.SyntaxSxn {
				rd := sxreader.MakeReader(bytes.NewReader(vn.Content))
				if objs, err := rd.ReadAll(); err == nil {
					result := make(ast.BlockSlice, len(objs))
					for i, obj := range objs {
						var buf bytes.Buffer
						sxbuiltins.Print(&buf, obj)
						res