Zettelstore

Check-in Differences
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Difference From v0.4 To v0.3

2022-03-10
13:20
Increase version to 0.5-dev to begin next development cycle ... (check-in: 73f6c7eb13 user: stern tags: trunk)
2022-03-09
14:06
Version 0.4 ... (check-in: 54ed47f372 user: stern tags: trunk, release, v0.4)
13:54
Update release checklist ... (check-in: 711a670d7a user: stern tags: trunk)
2022-02-09
10:59
Increase version to 0.4-dev to begin next development cycle ... (check-in: 0677d15ffc user: stern tags: trunk)
09:43
Version 0.3 ... (check-in: 870abfeef3 user: stern tags: trunk, release, v0.3)
2022-02-08
19:02
Cleanup generated DJSON to make it easier to parse and to reduce size ... (check-in: 06878e3dd0 user: stern tags: trunk)

Changes to VERSION.

1


1
-
+
0.4
0.3

Changes to ast/ast.go.

22
23
24
25
26
27
28
29

30
31
32
33
34
35
36
37
38
39
40
41
42





















43
44
45
46
47
48
49
50
51












52
53
54
55
56
57
58
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91







-
+













+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+









+
+
+
+
+
+
+
+
+
+
+
+







// ZettelNode is the root node of the abstract syntax tree.
// It is *not* part of the visitor pattern.
type ZettelNode struct {
	Meta    *meta.Meta     // Original metadata
	Content domain.Content // Original content
	Zid     id.Zid         // Zettel identification.
	InhMeta *meta.Meta     // Metadata of the zettel, with inherited values.
	Ast     BlockSlice     // Zettel abstract syntax tree is a sequence of block nodes.
	Ast     *BlockListNode // Zettel abstract syntax tree is a sequence of block nodes.
	Syntax  string         // Syntax / parser that produced the Ast
}

// Node is the interface, all nodes must implement.
type Node interface {
	WalkChildren(v Visitor)
}

// BlockNode is the interface that all block nodes must implement.
type BlockNode interface {
	Node
	blockNode()
}

// BlockSlice is a slice of BlockNodes.
type BlockSlice []BlockNode

// FirstParagraphInlines returns the inline list of the first paragraph that
// contains a inline list.
func (bns BlockSlice) FirstParagraphInlines() *InlineListNode {
	if len(bns) > 0 {
		for _, bn := range bns {
			pn, ok := bn.(*ParaNode)
			if !ok {
				continue
			}
			inl := pn.Inlines
			if inl != nil && len(inl.List) > 0 {
				return inl
			}
		}
	}
	return nil
}

// ItemNode is a node that can occur as a list item.
type ItemNode interface {
	BlockNode
	itemNode()
}

// ItemSlice is a slice of ItemNodes.
type ItemSlice []ItemNode

// ItemListNode is a list of BlockNodes.
type ItemListNode struct {
	List []ItemNode
}

// WalkChildren walks down to the descriptions.
func (iln *ItemListNode) WalkChildren(v Visitor) {
	for _, bn := range iln.List {
		Walk(v, bn)
	}
}

// DescriptionNode is a node that contains just textual description.
type DescriptionNode interface {
	ItemNode
	descriptionNode()
}

Added ast/attr.go.












































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package ast

import "strings"

// Attributes store additional information about some node types.
type Attributes struct {
	Attrs map[string]string
}

// IsEmpty returns true if there are no attributes.
func (a *Attributes) IsEmpty() bool { return a == nil || len(a.Attrs) == 0 }

// HasDefault returns true, if the default attribute "-" has been set.
func (a *Attributes) HasDefault() bool {
	if a != nil {
		_, ok := a.Attrs["-"]
		return ok
	}
	return false
}

// RemoveDefault removes the default attribute
func (a *Attributes) RemoveDefault() {
	a.Remove("-")
}

// Get returns the attribute value of the given key and a succes value.
func (a *Attributes) Get(key string) (string, bool) {
	if a != nil {
		value, ok := a.Attrs[key]
		return value, ok
	}
	return "", false
}

// Clone returns a duplicate of the attribute.
func (a *Attributes) Clone() *Attributes {
	if a == nil {
		return nil
	}
	attrs := make(map[string]string, len(a.Attrs))
	for k, v := range a.Attrs {
		attrs[k] = v
	}
	return &Attributes{attrs}
}

// Set changes the attribute that a given key has now a given value.
func (a *Attributes) Set(key, value string) *Attributes {
	if a == nil {
		return &Attributes{map[string]string{key: value}}
	}
	if a.Attrs == nil {
		a.Attrs = make(map[string]string)
	}
	a.Attrs[key] = value
	return a
}

// Remove the key from the attributes.
func (a *Attributes) Remove(key string) *Attributes {
	if a != nil {
		delete(a.Attrs, key)
	}
	return a
}

// AddClass adds a value to the class attribute.
func (a *Attributes) AddClass(class string) *Attributes {
	if a == nil {
		return &Attributes{map[string]string{"class": class}}
	}
	if a.Attrs == nil {
		a.Attrs = make(map[string]string)
	}
	classes := a.GetClasses()
	for _, cls := range classes {
		if cls == class {
			return a
		}
	}
	classes = append(classes, class)
	a.Attrs["class"] = strings.Join(classes, " ")
	return a
}

// GetClasses returns the class values as a string slice
func (a *Attributes) GetClasses() []string {
	if a == nil {
		return nil
	}
	classes, ok := a.Attrs["class"]
	if !ok {
		return nil
	}
	return strings.Fields(classes)
}

Added ast/attr_test.go.


















































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package ast_test

import (
	"testing"

	"zettelstore.de/z/ast"
)

func TestHasDefault(t *testing.T) {
	t.Parallel()
	attr := &ast.Attributes{}
	if attr.HasDefault() {
		t.Error("Should not have default attr")
	}
	attr = &ast.Attributes{Attrs: map[string]string{"-": "value"}}
	if !attr.HasDefault() {
		t.Error("Should have default attr")
	}
}

func TestAttrClone(t *testing.T) {
	t.Parallel()
	orig := &ast.Attributes{}
	clone := orig.Clone()
	if len(clone.Attrs) > 0 {
		t.Error("Attrs must be empty")
	}

	orig = &ast.Attributes{Attrs: map[string]string{"": "0", "-": "1", "a": "b"}}
	clone = orig.Clone()
	m := clone.Attrs
	if m[""] != "0" || m["-"] != "1" || m["a"] != "b" || len(m) != len(orig.Attrs) {
		t.Error("Wrong cloned map")
	}
	m["a"] = "c"
	if orig.Attrs["a"] != "b" {
		t.Error("Aliased map")
	}
}

Changes to ast/block.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20






21





22
23
24
25



26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

52
53
54
55
56
57
58
59

60
61
62
63

64
65
66
67

68


69
70
71
72
73
74
75
76

77
78
79
80
81
82
83
1
2
3
4
5
6
7
8
9
10
11
12


13
14




15
16
17
18
19
20
21
22
23
24
25
26
27



28
29
30
31
32
33
34
35















36
37
38
39
40

41
42
43
44
45
46
47
48

49
50
51
52

53
54
55
56
57
58

59
60
61
62
63
64
65
66
67

68
69
70
71
72
73
74
75












-
-


-
-
-
-
+
+
+
+
+
+

+
+
+
+
+

-
-
-
+
+
+





-
-
-
-
-
-
-
-
-
-
-
-
-
-
-





-
+







-
+



-
+




+
-
+
+







-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package ast

import "zettelstore.de/c/zjson"

// Definition of Block nodes.

// BlockSlice is a slice of BlockNodes.
type BlockSlice []BlockNode

func (*BlockSlice) blockNode() { /* Just a marker */ }
// BlockListNode is a list of BlockNodes.
type BlockListNode struct {
	List BlockSlice
}

func (*BlockListNode) blockNode() { /* Just a marker */ }

// CreateBlockListNode make a new block list node from nodes
func CreateBlockListNode(nodes ...BlockNode) *BlockListNode {
	return &BlockListNode{List: nodes}
}

// WalkChildren walks down to the descriptions.
func (bs *BlockSlice) WalkChildren(v Visitor) {
	if bs != nil {
		for _, bn := range *bs {
func (bln *BlockListNode) WalkChildren(v Visitor) {
	if bns := bln.List; bns != nil {
		for _, bn := range bns {
			Walk(v, bn)
		}
	}
}

// FirstParagraphInlines returns the inline list of the first paragraph that
// contains a inline list.
func (bs BlockSlice) FirstParagraphInlines() InlineSlice {
	for _, bn := range bs {
		pn, ok := bn.(*ParaNode)
		if !ok {
			continue
		}
		if inl := pn.Inlines; len(inl) > 0 {
			return inl
		}
	}
	return nil
}

//--------------------------------------------------------------------------

// ParaNode contains just a sequence of inline elements.
// Another name is "paragraph".
type ParaNode struct {
	Inlines InlineSlice
	Inlines *InlineListNode
}

func (*ParaNode) blockNode()       { /* Just a marker */ }
func (*ParaNode) itemNode()        { /* Just a marker */ }
func (*ParaNode) descriptionNode() { /* Just a marker */ }

// NewParaNode creates an empty ParaNode.
func NewParaNode() *ParaNode { return &ParaNode{} }
func NewParaNode() *ParaNode { return &ParaNode{Inlines: &InlineListNode{}} }

// CreateParaNode creates a parameter block from inline nodes.
func CreateParaNode(nodes ...InlineNode) *ParaNode {
	return &ParaNode{Inlines: nodes}
	return &ParaNode{Inlines: CreateInlineListNode(nodes...)}
}

// WalkChildren walks down the inline elements.
func (pn *ParaNode) WalkChildren(v Visitor) {
	if iln := pn.Inlines; iln != nil {
	Walk(v, &pn.Inlines)
		Walk(v, iln)
	}
}

//--------------------------------------------------------------------------

// VerbatimNode contains uninterpreted text
type VerbatimNode struct {
	Kind    VerbatimKind
	Attrs   zjson.Attributes
	Attrs   *Attributes
	Content []byte
}

// VerbatimKind specifies the format that is applied to code inline nodes.
type VerbatimKind uint8

// Constants for VerbatimCode
96
97
98
99
100
101
102
103
104
105



106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125




126
127
128
129
130
131
132
133
134
135
136




137
138
139
140
141
142
143

144


145
146
147
148
149
150
151

152
153
154
155
156
157
158
159
160
161
162
163
164
165
166

167
168
169
170
171
172
173
88
89
90
91
92
93
94



95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115


116
117
118
119
120
121
122
123
124
125
126




127
128
129
130
131
132
133
134
135
136
137
138

139
140
141
142
143
144
145
146

147
148
149
150
151
152
153
154
155
156
157
158
159
160
161

162
163
164
165
166
167
168
169







-
-
-
+
+
+


















-
-
+
+
+
+







-
-
-
-
+
+
+
+







+
-
+
+






-
+














-
+







func (*VerbatimNode) WalkChildren(Visitor) { /* No children*/ }

//--------------------------------------------------------------------------

// RegionNode encapsulates a region of block nodes.
type RegionNode struct {
	Kind    RegionKind
	Attrs   zjson.Attributes
	Blocks  BlockSlice
	Inlines InlineSlice // Optional text at the end of the region
	Attrs   *Attributes
	Blocks  *BlockListNode
	Inlines *InlineListNode // Optional text at the end of the region
}

// RegionKind specifies the actual region type.
type RegionKind uint8

// Values for RegionCode
const (
	_           RegionKind = iota
	RegionSpan             // Just a span of blocks
	RegionQuote            // A longer quotation
	RegionVerse            // Line breaks matter
)

func (*RegionNode) blockNode() { /* Just a marker */ }
func (*RegionNode) itemNode()  { /* Just a marker */ }

// WalkChildren walks down the blocks and the text.
func (rn *RegionNode) WalkChildren(v Visitor) {
	Walk(v, &rn.Blocks)
	Walk(v, &rn.Inlines)
	Walk(v, rn.Blocks)
	if iln := rn.Inlines; iln != nil {
		Walk(v, iln)
	}
}

//--------------------------------------------------------------------------

// HeadingNode stores the heading text and level.
type HeadingNode struct {
	Level    int
	Inlines  InlineSlice // Heading text, possibly formatted
	Slug     string      // Heading text, normalized
	Fragment string      // Heading text, suitable to be used as an unique URL fragment
	Attrs    zjson.Attributes
	Inlines  *InlineListNode // Heading text, possibly formatted
	Slug     string          // Heading text, normalized
	Fragment string          // Heading text, suitable to be used as an unique URL fragment
	Attrs    *Attributes
}

func (*HeadingNode) blockNode() { /* Just a marker */ }
func (*HeadingNode) itemNode()  { /* Just a marker */ }

// WalkChildren walks the heading text.
func (hn *HeadingNode) WalkChildren(v Visitor) {
	if iln := hn.Inlines; iln != nil {
	Walk(v, &hn.Inlines)
		Walk(v, iln)
	}
}

//--------------------------------------------------------------------------

// HRuleNode specifies a horizontal rule.
type HRuleNode struct {
	Attrs zjson.Attributes
	Attrs *Attributes
}

func (*HRuleNode) blockNode() { /* Just a marker */ }
func (*HRuleNode) itemNode()  { /* Just a marker */ }

// WalkChildren does nothing.
func (*HRuleNode) WalkChildren(Visitor) { /* No children*/ }

//--------------------------------------------------------------------------

// NestedListNode specifies a nestable list, either ordered or unordered.
type NestedListNode struct {
	Kind  NestedListKind
	Items []ItemSlice
	Attrs zjson.Attributes
	Attrs *Attributes
}

// NestedListKind specifies the actual list type.
type NestedListKind uint8

// Values for ListCode
const (
194
195
196
197
198
199
200
201

202
203
204
205
206
207
208
209
210
211
212



213
214
215
216
217
218
219
190
191
192
193
194
195
196

197
198
199
200
201
202
203
204
205



206
207
208
209
210
211
212
213
214
215







-
+








-
-
-
+
+
+







// DescriptionListNode specifies a description list.
type DescriptionListNode struct {
	Descriptions []Description
}

// Description is one element of a description list.
type Description struct {
	Term         InlineSlice
	Term         *InlineListNode
	Descriptions []DescriptionSlice
}

func (*DescriptionListNode) blockNode() { /* Just a marker */ }

// WalkChildren walks down to the descriptions.
func (dn *DescriptionListNode) WalkChildren(v Visitor) {
	if descrs := dn.Descriptions; descrs != nil {
		for i, desc := range descrs {
			if len(desc.Term) > 0 {
				Walk(v, &descrs[i].Term) // Otherwise, changes in desc.Term will not go back into AST
		for _, desc := range descrs {
			if term := desc.Term; term != nil {
				Walk(v, term)
			}
			if dss := desc.Descriptions; dss != nil {
				for _, dns := range dss {
					WalkDescriptionSlice(v, dns)
				}
			}
		}
227
228
229
230
231
232
233
234
235


236
237
238
239
240
241
242
223
224
225
226
227
228
229


230
231
232
233
234
235
236
237
238







-
-
+
+







	Header TableRow    // The header row
	Align  []Alignment // Default column alignment
	Rows   []TableRow  // The slice of cell rows
}

// TableCell contains the data for one table cell
type TableCell struct {
	Align   Alignment   // Cell alignment
	Inlines InlineSlice // Cell content
	Align   Alignment       // Cell alignment
	Inlines *InlineListNode // Cell content
}

// TableRow is a slice of cells.
type TableRow []*TableCell

// Alignment specifies text alignment.
// Currently only for tables.
252
253
254
255
256
257
258
259
260




261
262
263
264
265
266




267
268
269
270
271
272
273
248
249
250
251
252
253
254


255
256
257
258
259
260
261
262


263
264
265
266
267
268
269
270
271
272
273







-
-
+
+
+
+




-
-
+
+
+
+







)

func (*TableNode) blockNode() { /* Just a marker */ }

// WalkChildren walks down to the cells.
func (tn *TableNode) WalkChildren(v Visitor) {
	if header := tn.Header; header != nil {
		for i := range header {
			Walk(v, &header[i].Inlines) // Otherwise changes will not go back
		for _, cell := range header {
			if iln := cell.Inlines; iln != nil {
				Walk(v, iln)
			}
		}
	}
	if rows := tn.Rows; rows != nil {
		for _, row := range rows {
			for i := range row {
				Walk(v, &row[i].Inlines) // Otherwise changes will not go back
			for _, cell := range row {
				if iln := cell.Inlines; iln != nil {
					Walk(v, iln)
				}
			}
		}
	}
}

//--------------------------------------------------------------------------

Changes to ast/inline.go.

1
2
3
4
5
6
7
8
9
10
11
12

13
14
15





16
17
18


19
20
21
22
23




24
25
26

27
28
29


30
31
32
33
34
35
36

37
38
39
40
41
42
43
44















45
46
47
48
49
50
51
1
2
3
4
5
6
7
8
9
10
11
12
13



14
15
16
17
18



19
20

21



22
23
24
25

26

27
28


29
30
31
32
33
34
35
36

37
38
39
40





41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62












+
-
-
-
+
+
+
+
+
-
-
-
+
+
-

-
-
-
+
+
+
+
-

-
+

-
-
+
+






-
+



-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package ast

// Definitions of inline nodes.
import (
	"unicode/utf8"


// InlineListNode is a list of BlockNodes.
type InlineListNode struct {
	List []InlineNode
}
	"zettelstore.de/c/zjson"
)


func (*InlineListNode) inlineNode() { /* Just a marker */ }
// Definitions of inline nodes.

// InlineSlice is a list of BlockNodes.
type InlineSlice []InlineNode

// CreateInlineListNode make a new inline list node from nodes
func CreateInlineListNode(nodes ...InlineNode) *InlineListNode {
	return &InlineListNode{List: nodes}
}
func (*InlineSlice) inlineNode() { /* Just a marker */ }

// CreateInlineSliceFromWords makes a new inline list from words,
// CreateInlineListNodeFromWords makes a new inline list from words,
// that will be space-separated.
func CreateInlineSliceFromWords(words ...string) InlineSlice {
	inl := make(InlineSlice, 0, 2*len(words)-1)
func CreateInlineListNodeFromWords(words ...string) *InlineListNode {
	inl := make([]InlineNode, 0, 2*len(words)-1)
	for i, word := range words {
		if i > 0 {
			inl = append(inl, &SpaceNode{Lexeme: " "})
		}
		inl = append(inl, &TextNode{Text: word})
	}
	return inl
	return &InlineListNode{List: inl}
}

// WalkChildren walks down to the list.
func (is *InlineSlice) WalkChildren(v Visitor) {
	for _, in := range *is {
		Walk(v, in)
	}
}
func (iln *InlineListNode) WalkChildren(v Visitor) {
	if ins := iln.List; ins != nil {
		for _, bn := range ins {
			Walk(v, bn)
		}
	}
}

// IsEmpty returns true if the list has no elements.
func (iln *InlineListNode) IsEmpty() bool { return iln == nil || len(iln.List) == 0 }

// Append inline node(s) to the list.
func (iln *InlineListNode) Append(in ...InlineNode) {
	iln.List = append(iln.List, in...)
}

// --------------------------------------------------------------------------

// TextNode just contains some text.
type TextNode struct {
	Text string // The text itself.
}
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105



106
107
108
109
110
111
112
113


114
115
116
117
118
119
120
121
122
123



124
125
126
127
128
129
130
131

132


133
134
135
136
137
138
139
140
141
142




143
144
145
146
147
148
149

150


151
152
153
154
155
156
157
158
159



160
161
162
163
164
165

166


167
168
169
170
171
172
173
174
175
176
177



178
179
180
181
182
183
184

185
186
187
188
189
190
191
192
193
194
195


196
197
198
199
200
201

202


203
204
205
206
207
208
209
210
211


212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227











228
229
230
231
232
233

234


235
236
237
238
239
240
241
242

243
244
245
246
247
248
249
250
251
252
253
254
255


256
257
258
259
260
261
262
263
86
87
88
89
90
91
92





93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109


110
111
112
113
114
115
116
117
118


119
120
121
122
123
124
125
126
127



128
129
130

131
132
133
134
135
136
137
138

139
140
141
142
143
144
145
146




147
148
149
150
151
152
153
154
155
156
157
158

159
160
161
162
163
164
165
166



167
168
169
170
171
172
173
174
175
176

177
178
179
180
181
182
183
184
185
186



187
188
189

190
191
192
193
194

195




196
197
198
199
200


201
202
203
204
205
206
207
208
209

210
211
212
213
214
215
216
217
218


219
220
221
222
223
224
225
226
227









228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245

246
247
248
249
250
251
252
253
254

255
256
257
258
259
260
261
262
263
264
265
266


267
268
269
270
271
272
273
274
275
276







-
-
-
-
-

















-
-
+
+
+






-
-
+
+







-
-
-
+
+
+
-







+
-
+
+






-
-
-
-
+
+
+
+







+
-
+
+






-
-
-
+
+
+






+
-
+
+








-
-
-
+
+
+
-





-
+
-
-
-
-





-
-
+
+






+
-
+
+







-
-
+
+







-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+






+
-
+
+







-
+











-
-
+
+








}

func (*SpaceNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*SpaceNode) WalkChildren(Visitor) { /* No children*/ }

// Count returns the number of space runes.
func (sn *SpaceNode) Count() int {
	return utf8.RuneCountInString(sn.Lexeme)
}

// --------------------------------------------------------------------------

// BreakNode signals a new line that must / should be interpreted as a new line break.
type BreakNode struct {
	Hard bool // Hard line break?
}

func (*BreakNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*BreakNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// LinkNode contains the specified link.
type LinkNode struct {
	Ref     *Reference
	Inlines InlineSlice      // The text associated with the link.
	Attrs   zjson.Attributes // Optional attributes
	Inlines *InlineListNode // The text associated with the link.
	OnlyRef bool            // True if no text was specified.
	Attrs   *Attributes     // Optional attributes
}

func (*LinkNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the link text.
func (ln *LinkNode) WalkChildren(v Visitor) {
	if len(ln.Inlines) > 0 {
		Walk(v, &ln.Inlines)
	if iln := ln.Inlines; iln != nil {
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// EmbedRefNode contains the specified embedded reference material.
type EmbedRefNode struct {
	Ref     *Reference       // The reference to be embedded.
	Inlines InlineSlice      // Optional text associated with the image.
	Attrs   zjson.Attributes // Optional attributes
	Ref     *Reference      // The reference to be embedded.
	Inlines *InlineListNode // Optional text associated with the image.
	Attrs   *Attributes     // Optional attributes
	Syntax  string           // Syntax of referenced material, if known
}

func (*EmbedRefNode) inlineNode()      { /* Just a marker */ }
func (*EmbedRefNode) inlineEmbedNode() { /* Just a marker */ }

// WalkChildren walks to the text that describes the embedded material.
func (en *EmbedRefNode) WalkChildren(v Visitor) {
	if iln := en.Inlines; iln != nil {
	Walk(v, &en.Inlines)
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// EmbedBLOBNode contains the specified embedded BLOB material.
type EmbedBLOBNode struct {
	Blob    []byte           // BLOB data itself.
	Syntax  string           // Syntax of Blob
	Inlines InlineSlice      // Optional text associated with the image.
	Attrs   zjson.Attributes // Optional attributes
	Blob    []byte          // BLOB data itself.
	Syntax  string          // Syntax of Blob
	Inlines *InlineListNode // Optional text associated with the image.
	Attrs   *Attributes     // Optional attributes
}

func (*EmbedBLOBNode) inlineNode()      { /* Just a marker */ }
func (*EmbedBLOBNode) inlineEmbedNode() { /* Just a marker */ }

// WalkChildren walks to the text that describes the embedded material.
func (en *EmbedBLOBNode) WalkChildren(v Visitor) {
	if iln := en.Inlines; iln != nil {
	Walk(v, &en.Inlines)
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// CiteNode contains the specified citation.
type CiteNode struct {
	Key     string           // The citation key
	Inlines InlineSlice      // Optional text associated with the citation.
	Attrs   zjson.Attributes // Optional attributes
	Key     string          // The citation key
	Inlines *InlineListNode // Optional text associated with the citation.
	Attrs   *Attributes     // Optional attributes
}

func (*CiteNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the cite text.
func (cn *CiteNode) WalkChildren(v Visitor) {
	if iln := cn.Inlines; iln != nil {
	Walk(v, &cn.Inlines)
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// MarkNode contains the specified merked position.
// It is a BlockNode too, because although it is typically parsed during inline
// mode, it is moved into block mode afterwards.
type MarkNode struct {
	Mark     string      // The mark text itself
	Slug     string      // Slugified form of Mark
	Fragment string      // Unique form of Slug
	Text     string
	Slug     string // Slugified form of Text
	Fragment string // Unique form of Slug
	Inlines  InlineSlice // Marked inline content
}

func (*MarkNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (mn *MarkNode) WalkChildren(v Visitor) {
func (*MarkNode) WalkChildren(Visitor) { /* No children*/ }
	if len(mn.Inlines) > 0 {
		Walk(v, &mn.Inlines)
	}
}

// --------------------------------------------------------------------------

// FootnoteNode contains the specified footnote.
type FootnoteNode struct {
	Inlines InlineSlice      // The footnote text.
	Attrs   zjson.Attributes // Optional attributes
	Inlines *InlineListNode // The footnote text.
	Attrs   *Attributes     // Optional attributes
}

func (*FootnoteNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the footnote text.
func (fn *FootnoteNode) WalkChildren(v Visitor) {
	if iln := fn.Inlines; iln != nil {
	Walk(v, &fn.Inlines)
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// FormatNode specifies some inline formatting.
type FormatNode struct {
	Kind    FormatKind
	Attrs   zjson.Attributes // Optional attributes.
	Inlines InlineSlice
	Attrs   *Attributes // Optional attributes.
	Inlines *InlineListNode
}

// FormatKind specifies the format that is applied to the inline nodes.
type FormatKind uint8

// Constants for FormatCode
const (
	_            FormatKind = iota
	FormatEmph              // Emphasized text.
	FormatStrong            // Strongly emphasized text.
	FormatInsert            // Inserted text.
	FormatDelete            // Deleted text.
	FormatSuper             // Superscripted text.
	FormatSub               // SubscriptedText.
	FormatQuote             // Quoted text.
	FormatSpan              // Generic inline container.
	_               FormatKind = iota
	FormatEmph                 // Emphasized text.
	FormatStrong               // Strongly emphasized text.
	FormatInsert               // Inserted text.
	FormatDelete               // Deleted text.
	FormatSuper                // Superscripted text.
	FormatSub                  // SubscriptedText.
	FormatQuote                // Quoted text.
	FormatQuotation            // Quotation text.
	FormatSpan                 // Generic inline container.
	FormatMonospace            // Monospaced text.
)

func (*FormatNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the formatted text.
func (fn *FormatNode) WalkChildren(v Visitor) {
	if iln := fn.Inlines; iln != nil {
	Walk(v, &fn.Inlines)
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// LiteralNode specifies some uninterpreted text.
type LiteralNode struct {
	Kind    LiteralKind
	Attrs   zjson.Attributes // Optional attributes.
	Attrs   *Attributes // Optional attributes.
	Content []byte
}

// LiteralKind specifies the format that is applied to code inline nodes.
type LiteralKind uint8

// Constants for LiteralCode
const (
	_              LiteralKind = iota
	LiteralZettel              // Zettel content
	LiteralProg                // Inline program code
	LiteralInput               // Computer input, e.g. Keyboard strokes
	LiteralOutput              // Computer output
	LiteralKeyb                // Keyboard strokes
	LiteralOutput              // Sample output.
	LiteralComment             // Inline comment
	LiteralHTML                // Inline HTML, e.g. for Markdown
)

func (*LiteralNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*LiteralNode) WalkChildren(Visitor) { /* No children*/ }

Changes to ast/walk_test.go.

9
10
11
12
13
14
15
16
17
18
19
20
21

22
23

24
25
26

27
28
29
30
31
32
33

34
35
36
37
38

39
40
41
42
43
44

45
46
47
48

49
50
51
52
53






54
55
56

57
58




59
60
61

62
63
64
65

66
67
68
69
70
71
9
10
11
12
13
14
15

16
17
18
19

20
21

22
23
24

25
26
27
28
29
30
31

32
33
34
35
36

37
38
39
40
41
42

43
44
45
46
47
48





49
50
51
52
53
54
55
56
57
58


59
60
61
62
63
64

65
66
67
68

69
70
71
72
73
74
75







-




-
+

-
+


-
+






-
+




-
+





-
+




+
-
-
-
-
-
+
+
+
+
+
+



+
-
-
+
+
+
+


-
+



-
+






//-----------------------------------------------------------------------------

package ast_test

import (
	"testing"

	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
)

func BenchmarkWalk(b *testing.B) {
	root := ast.BlockSlice{
	root := ast.CreateBlockListNode(
		&ast.HeadingNode{
			Inlines: ast.CreateInlineSliceFromWords("A", "Simple", "Heading"),
			Inlines: ast.CreateInlineListNodeFromWords("A", "Simple", "Heading"),
		},
		&ast.ParaNode{
			Inlines: ast.CreateInlineSliceFromWords("This", "is", "the", "introduction."),
			Inlines: ast.CreateInlineListNodeFromWords("This", "is", "the", "introduction."),
		},
		&ast.NestedListNode{
			Kind: ast.NestedListUnordered,
			Items: []ast.ItemSlice{
				[]ast.ItemNode{
					&ast.ParaNode{
						Inlines: ast.CreateInlineSliceFromWords("Item", "1"),
						Inlines: ast.CreateInlineListNodeFromWords("Item", "1"),
					},
				},
				[]ast.ItemNode{
					&ast.ParaNode{
						Inlines: ast.CreateInlineSliceFromWords("Item", "2"),
						Inlines: ast.CreateInlineListNodeFromWords("Item", "2"),
					},
				},
			},
		},
		&ast.ParaNode{
			Inlines: ast.CreateInlineSliceFromWords("This", "is", "some", "intermediate", "text."),
			Inlines: ast.CreateInlineListNodeFromWords("This", "is", "some", "intermediate", "text."),
		},
		ast.CreateParaNode(
			&ast.FormatNode{
				Kind: ast.FormatEmph,
				Attrs: &ast.Attributes{
				Attrs: zjson.Attributes(map[string]string{
					"":      "class",
					"color": "green",
				}),
				Inlines: ast.CreateInlineSliceFromWords("This", "is", "some", "emphasized", "text."),
					Attrs: map[string]string{
						"":      "class",
						"color": "green",
					},
				},
				Inlines: ast.CreateInlineListNodeFromWords("This", "is", "some", "emphasized", "text."),
			},
			&ast.SpaceNode{Lexeme: " "},
			&ast.LinkNode{
				Ref: &ast.Reference{
				Ref:     &ast.Reference{Value: "http://zettelstore.de"},
				Inlines: ast.CreateInlineSliceFromWords("URL", "text."),
					Value: "http://zettelstore.de",
				},
				Inlines: ast.CreateInlineListNodeFromWords("URL", "text."),
				OnlyRef: false,
			},
		),
	}
	)
	v := benchVisitor{}
	b.ResetTimer()
	for n := 0; n < b.N; n++ {
		ast.Walk(&v, &root)
		ast.Walk(&v, root)
	}
}

type benchVisitor struct{}

func (bv *benchVisitor) Visit(ast.Node) ast.Visitor { return bv }

Changes to box/box.go.

1
2

3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
1

2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18


19
20
21
22
23
24
25

-
+

-
+














-
-







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package box provides a generic interface to zettel boxes.
package box

import (
	"context"
	"errors"
	"fmt"
	"io"
	"net/url"
	"strconv"
	"time"

	"zettelstore.de/c/api"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
283
284
285
286
287
288
289



290
291
292
293
































-
-
-




-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
// ErrNotFound is returned if a zettel was not found in the box.
var ErrNotFound = errors.New("zettel not found")

// ErrConflict is returned if a box operation detected a conflict..
// One example: if calculating a new zettel identifier takes too long.
var ErrConflict = errors.New("conflict")

// ErrCapacity is returned if a box has reached its capacity.
var ErrCapacity = errors.New("capacity exceeded")

// ErrInvalidID is returned if the zettel id is not appropriate for the box operation.
type ErrInvalidID struct{ Zid id.Zid }

func (err *ErrInvalidID) Error() string { return "invalid Zettel id: " + err.Zid.String() }

// GetQueryBool is a helper function to extract bool values from a box URI.
func GetQueryBool(u *url.URL, key string) bool {
	_, ok := u.Query()[key]
	return ok
}

// GetQueryInt is a helper function to extract int values of a specified range from a box URI.
func GetQueryInt(u *url.URL, key string, min, def, max int) int {
	sVal := u.Query().Get(key)
	if sVal == "" {
		return def
	}
	iVal, err := strconv.Atoi(sVal)
	if err != nil {
		return def
	}
	if iVal < min {
		return min
	}
	if iVal > max {
		return max
	}
	return iVal
}

Changes to box/compbox/compbox.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package compbox provides zettel that have computed content.
177
178
179
180
181
182
183

184
185
186
187
188
189
190
191
192
193
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194







+










func (cb *compBox) ReadStats(st *box.ManagedBoxStats) {
	st.ReadOnly = true
	st.Zettel = len(myZettel)
	cb.log.Trace().Int("zettel", int64(st.Zettel)).Msg("ReadStats")
}

func updateMeta(m *meta.Meta) {
	m.Set(api.KeyNoIndex, api.ValueTrue)
	if _, ok := m.Get(api.KeySyntax); !ok {
		m.Set(api.KeySyntax, api.ValueSyntaxZmk)
	}
	m.Set(api.KeyRole, api.ValueRoleConfiguration)
	m.Set(api.KeyLang, api.ValueLangEN)
	m.Set(api.KeyReadOnly, api.ValueTrue)
	if _, ok := m.Get(api.KeyVisibility); !ok {
		m.Set(api.KeyVisibility, api.ValueVisibilityExpert)
	}
}

Changes to box/constbox/base.css.

25
26
27
28
29
30
31
32
33






34
35
36
37
38
39
40
25
26
27
28
29
30
31


32
33
34
35
36
37
38
39
40
41
42
43
44







-
-
+
+
+
+
+
+







    float:left;
    display: block;
    text-align: center;
    padding:.41rem .5rem;
    text-decoration: none;
    color:black;
  }
  nav.zs-menu > a:hover, .zs-dropdown:hover button { background-color: hsl(210, 28%, 80%) }
  nav.zs-menu form { float: right }
  nav.zs-menu > a:hover, .zs-dropdown:hover button {
    background-color: hsl(210, 28%, 80%);
  }
  nav.zs-menu form {
    float: right;
  }
  nav.zs-menu form input[type=text] {
    padding: .12rem;
    border: none;
    margin-top: .25rem;
    margin-right: .5rem;
  }
  .zs-dropdown {
63
64
65
66
67
68
69
70
71
72
73












74
75
76
77
78
79
80
81
82
83
84

85
86
87
88
89
90
91




















92
93
94
95
96
97
98
99
100






101
102
103
104
105
106
107
108
109
110
111
112



113
114
115
116
117
118
119
120
121
122
123
124
125
126









127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147








148
149


150
151
152
153
154

155
156
157
158
159
160
161
67
68
69
70
71
72
73




74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97







98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124


125
126
127
128
129
130
131
132
133
134
135
136
137
138
139



140
141
142
143
144
145
146
147
148
149
150
151
152
153



154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180



181
182
183
184
185
186
187
188


189
190
191
192
193


194
195
196
197
198
199
200
201







-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+











+
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







-
-
+
+
+
+
+
+









-
-
-
+
+
+











-
-
-
+
+
+
+
+
+
+
+
+


















-
-
-
+
+
+
+
+
+
+
+
-
-
+
+



-
-
+







    float: none;
    color: black;
    padding:.41rem .5rem;
    text-decoration: none;
    display: block;
    text-align: left;
  }
  .zs-dropdown-content > a:hover { background-color: hsl(210, 28%, 75%) }
  .zs-dropdown:hover > .zs-dropdown-content { display: block }
  main { padding: 0 1rem }
  article > * + * { margin-top: .5rem }
  .zs-dropdown-content > a:hover {
    background-color: hsl(210, 28%, 75%);
  }
  .zs-dropdown:hover > .zs-dropdown-content {
    display: block;
  }
  main {
    padding: 0 1rem;
  }
  article > * + * {
    margin-top: .5rem;
  }
  article header {
    padding: 0;
    margin: 0;
  }
  h1,h2,h3,h4,h5,h6 { font-family:sans-serif; font-weight:normal }
  h1 { font-size:1.5rem;  margin:.65rem 0 }
  h2 { font-size:1.25rem; margin:.70rem 0 }
  h3 { font-size:1.15rem; margin:.75rem 0 }
  h4 { font-size:1.05rem; margin:.8rem 0; font-weight: bold }
  h5 { font-size:1.05rem; margin:.8rem 0 }
  h6 { font-size:1.05rem; margin:.8rem 0; font-weight: lighter }
  p {
  p { margin: .5rem 0 0 0 }
  ol,ul { padding-left: 1.1rem }
  li,figure,figcaption,dl { margin: 0 }
  dt { margin: .5rem 0 0 0 }
  dt+dd { margin-top: 0 }
  dd { margin: .5rem 0 0 2rem }
  dd > p:first-child { margin: 0 0 0 0 }
    margin: .5rem 0 0 0;
  }
  ol,ul {
    padding-left: 1.1rem;
  }
  li,figure,figcaption,dl {
    margin: 0;
  }
  dt {
    margin: .5rem 0 0 0;
  }
  dt+dd {
    margin-top: 0;
  }
  dd {
    margin: .5rem 0 0 2rem;
  }
  dd > p:first-child {
    margin: 0 0 0 0;
  }
  blockquote {
    border-left: 0.5rem solid lightgray;
    padding-left: 1rem;
    margin-left: 1rem;
    margin-right: 2rem;
    font-style: italic;
  }
  blockquote p { margin-bottom: .5rem }
  blockquote cite { font-style: normal }
  blockquote p {
    margin-bottom: .5rem;
  }
  blockquote cite {
    font-style: normal;
  }
  table {
    border-collapse: collapse;
    border-spacing: 0;
    max-width: 100%;
  }
  th,td {
    text-align: left;
    padding: .25rem .5rem;
  }
  td { border-bottom: 1px solid hsl(0, 0%, 85%) }
  thead th { border-bottom: 2px solid hsl(0, 0%, 70%) }
  tfoot th { border-top: 2px solid hsl(0, 0%, 70%) }
  td { border-bottom: 1px solid hsl(0, 0%, 85%); }
  thead th { border-bottom: 2px solid hsl(0, 0%, 70%); }
  tfoot th { border-top: 2px solid hsl(0, 0%, 70%); }
  main form {
    padding: 0 .5em;
    margin: .5em 0 0 0;
  }
  main form:after {
    content: ".";
    display: block;
    height: 0;
    clear: both;
    visibility: hidden;
  }
  main form div { margin: .5em 0 0 0 }
  input { font-family: monospace }
  input[type="submit"],button,select { font: inherit }
  main form div {
    margin: .5em 0 0 0
  }
  input {
    font-family: monospace;
  }
  input[type="submit"],button,select {
    font: inherit;
  }
  label { font-family: sans-serif; font-size:.9rem }
  label::after { content:":" }
  textarea {
    font-family: monospace;
    resize: vertical;
    width: 100%;
  }
  .zs-input {
    padding: .5em;
    display:block;
    border:none;
    border-bottom:1px solid #ccc;
    width:100%;
  }
  .zs-button {
    float:right;
    margin: .5em 0 .5em 1em;
  }
  a:not([class]) { text-decoration-skip-ink: auto }
  a.broken { text-decoration: line-through }
  img { max-width: 100% }
  a:not([class]) {
    text-decoration-skip-ink: auto;
  }
  .zs-broken {
    text-decoration: line-through;
  }
  img {
    max-width: 100%;
  img.right { float: right }
  ol.endnotes {
  }
  .zs-endnotes {
    padding-top: .5rem;
    border-top: 1px solid;
  }
  kbd { font-family:monospace }
  code,pre {
  code,pre,kbd {
    font-family: monospace;
    font-size: 85%;
  }
  code {
    padding: .1rem .2rem;
    background: #f0f0f0;
    border: 1px solid #ccc;
177
178
179
180
181
182
183
184



185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207




208
209
210
211
212
213
214







215
216
217
218
219
220
221
222









223
224
225
226
227
228
229
230
231
232
233



234
235
236
237
238
239
240
241
217
218
219
220
221
222
223

224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246



247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269



270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288

289
290
291
292
293
294
295
296
297
298
299







-
+
+
+




















-
-
-
+
+
+
+







+
+
+
+
+
+
+





-
-
-
+
+
+
+
+
+
+
+
+










-
+
+
+








  }
  div.zs-indication {
    padding: .5rem .7rem;
    max-width: 100%;
    border-radius: .5rem;
    border: 1px solid black;
  }
  div.zs-indication p:first-child { margin-top: 0 }
  div.zs-indication p:first-child {
    margin-top: 0;
  }
  span.zs-indication {
    border: 1px solid black;
    border-radius: .25rem;
    padding: .1rem .2rem;
    font-size: 95%;
  }
  .zs-example { border-style: dotted !important }
  .zs-info {
    background-color: lightblue;
    padding: .5rem 1rem;
  }
  .zs-warning {
    background-color: lightyellow;
    padding: .5rem 1rem;
  }
  .zs-error {
    background-color: lightpink;
    border-style: none !important;
    font-weight: bold;
  }
  td.left,th.left { text-align:left }
  td.center,th.center { text-align:center }
  td.right,th.right { text-align:right }
  .zs-ta-left { text-align:left }
  .zs-ta-center { text-align:center }
  .zs-ta-right { text-align:right }
  .zs-monospace { font-family:monospace }
  .zs-font-size-0 { font-size:75% }
  .zs-font-size-1 { font-size:83% }
  .zs-font-size-2 { font-size:100% }
  .zs-font-size-3 { font-size:117% }
  .zs-font-size-4 { font-size:150% }
  .zs-font-size-5 { font-size:200% }
  .zs-deprecated { border-style: dashed; padding: .2rem }
  kbd {
    background: hsl(210, 5%, 100%);
    border: 1px solid hsl(210, 5%, 70%);
    border-radius: .25rem;
    padding: .1rem .2rem;
    font-size: 75%;
  }
  .zs-meta {
    font-size:.75rem;
    color:#444;
    margin-bottom:1rem;
  }
  .zs-meta a { color:#444 }
  h1+.zs-meta { margin-top:-1rem }
  nav > details { margin-top:1rem }
  .zs-meta a {
    color:#444;
  }
  h1+.zs-meta {
    margin-top:-1rem;
  }
  nav > details {
    margin-top:1rem;
  }
  details > summary {
    width: 100%;
    background-color: #eee;
    font-family:sans-serif;
  }
  details > ul {
    margin-top:0;
    padding-left:2rem;
    background-color: #eee;
  }
  footer { padding: 0 1rem }
  footer {
    padding: 0 1rem;
  }
  @media (prefers-reduced-motion: reduce) {
    * {
      animation-duration: 0.01ms !important;
      animation-iteration-count: 1 !important;
      transition-duration: 0.01ms !important;
      scroll-behavior: auto !important;
    }
  }

Changes to box/constbox/constbox.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package constbox puts zettel inside the executable.
146
147
148
149
150
151
152

153
154
155
156
157
158
159
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160







+








var constZettelMap = map[id.Zid]constZettel{
	id.ConfigurationZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Runtime Configuration",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     api.ValueSyntaxNone,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityOwner,
		},
		domain.NewContent(nil)},
	id.MustParse(api.ZidLicense): {
		constHeader{
			api.KeyTitle:      "Zettelstore License",
			api.KeyRole:       api.ValueRoleConfiguration,
184
185
186
187
188
189
190

191
192
193
194
195
196
197
198

199
200
201
202
203
204
205
206

207
208
209
210
211
212
213
214

215
216
217
218
219
220
221
222

223
224
225
226
227
228
229
230

231
232
233
234
235
236
237
238

239
240
241
242
243
244
245
246

247
248
249
250
251
252
253
254

255
256
257
258
259
260
261
262

263
264
265
266
267
268
269
270

271
272
273
274
275
276
277
278

279
280
281
282
283
284
285
286

287
288
289
290
291
292
293
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307







+








+








+








+








+








+








+








+








+








+








+








+








+







		},
		domain.NewContent(contentDependencies)},
	id.BaseTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Base HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentBaseMustache)},
	id.LoginTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Login Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentLoginMustache)},
	id.ZettelTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentZettelMustache)},
	id.InfoTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Info HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentInfoMustache)},
	id.ContextTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Context HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentContextMustache)},
	id.FormTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentFormMustache)},
	id.RenameTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Rename Form HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentRenameMustache)},
	id.DeleteTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Delete HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentDeleteMustache)},
	id.ListTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore List Zettel HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentListZettelMustache)},
	id.RolesTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore List Roles HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentListRolesMustache)},
	id.TagsTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore List Tags HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentListTagsMustache)},
	id.ErrorTemplateZid: {
		constHeader{
			api.KeyTitle:      "Zettelstore Error HTML Template",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     syntaxTemplate,
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityExpert,
		},
		domain.NewContent(contentErrorMustache)},
	id.MustParse(api.ZidBaseCSS): {
		constHeader{
			api.KeyTitle:      "Zettelstore Base CSS",
			api.KeyRole:       api.ValueRoleConfiguration,
			api.KeySyntax:     "css",
			api.KeyNoIndex:    api.ValueTrue,
			api.KeyVisibility: api.ValueVisibilityPublic,
		},
		domain.NewContent(contentBaseCSS)},
	id.MustParse(api.ZidUserCSS): {
		constHeader{
			api.KeyTitle:      "Zettelstore User CSS",
			api.KeyRole:       api.ValueRoleConfiguration,

Changes to box/dirbox/dirbox.go.

1
2
3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
1
2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27



-
+















+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package dirbox provides a directory-based zettel box.
package dirbox

import (
	"context"
	"errors"
	"net/url"
	"os"
	"path/filepath"
	"strconv"
	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/box/notify"
	"zettelstore.de/z/domain"
41
42
43
44
45
46
47
48

49
50
51
52

53
54
55
56
57
58
59
42
43
44
45
46
47
48

49
50
51
52

53
54
55
56
57
58
59
60







-
+



-
+







		if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {
			return nil, err
		}
		dp := dirBox{
			log:        log,
			number:     cdata.Number,
			location:   u.String(),
			readonly:   box.GetQueryBool(u, "readonly"),
			readonly:   getQueryBool(u, "readonly"),
			cdata:      *cdata,
			dir:        path,
			notifySpec: getDirSrvInfo(log, u.Query().Get("type")),
			fSrvs:      makePrime(uint32(box.GetQueryInt(u, "worker", 1, 7, 1499))),
			fSrvs:      makePrime(uint32(getQueryInt(u, "worker", 1, 7, 1499))),
		}
		return &dp, nil
	})
}

func makePrime(n uint32) uint32 {
	for !isPrime(n) {
106
107
108
109
110
111
112























113
114
115
116
117
118
119
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+








func getDirPath(u *url.URL) string {
	if u.Opaque != "" {
		return filepath.Clean(u.Opaque)
	}
	return filepath.Clean(u.Path)
}

func getQueryBool(u *url.URL, key string) bool {
	_, ok := u.Query()[key]
	return ok
}

func getQueryInt(u *url.URL, key string, min, def, max int) int {
	sVal := u.Query().Get(key)
	if sVal == "" {
		return def
	}
	iVal, err := strconv.Atoi(sVal)
	if err != nil {
		return def
	}
	if iVal < min {
		return min
	}
	if iVal > max {
		return max
	}
	return iVal
}

// dirBox uses a directory to store zettel as files.
type dirBox struct {
	log        *logger.Logger
	number     int
	location   string
	readonly   bool

Changes to box/manager/collect.go.

30
31
32
33
34
35
36
37

38
39
40
41


42
43
44
45
46
47
48
30
31
32
33
34
35
36

37
38
39


40
41
42
43
44
45
46
47
48







-
+


-
-
+
+







	data.refs = id.NewSet()
	data.words = store.NewWordSet()
	data.urls = store.NewWordSet()
	data.itags = store.NewWordSet()
}

func collectZettelIndexData(zn *ast.ZettelNode, data *collectData) {
	ast.Walk(data, &zn.Ast)
	ast.Walk(data, zn.Ast)
}

func collectInlineIndexData(is *ast.InlineSlice, data *collectData) {
	ast.Walk(data, is)
func collectInlineIndexData(iln *ast.InlineListNode, data *collectData) {
	ast.Walk(data, iln)
}

func (data *collectData) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.VerbatimNode:
		data.addText(string(n.Content))
	case *ast.TranscludeNode:

Changes to box/manager/indexer.go.

1
2
3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
1
2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26



-
+














+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package manager

import (
	"context"
	"fmt"
	"net/url"
	"time"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager/store"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
162
163
164
165
166
167
168








169
170
171
172
173
174
175
176
177
178
179
180
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180


181
182
183
184
185
186
187







+
+
+
+
+
+
+
+



-
-







		}
		return false
	}
	return true
}

func (mgr *Manager) idxUpdateZettel(ctx context.Context, zettel domain.Zettel) {
	m := zettel.Meta
	if m.GetBool(api.KeyNoIndex) {
		// Zettel maybe in index
		toCheck := mgr.idxStore.DeleteZettel(ctx, m.Zid)
		mgr.idxCheckZettel(toCheck)
		return
	}

	var cData collectData
	cData.initialize()
	collectZettelIndexData(parser.ParseZettel(zettel, "", mgr.rtConfig), &cData)

	m := zettel.Meta
	zi := store.NewZettelIndex(m.Zid)
	mgr.idxCollectFromMeta(ctx, m, zi, &cData)
	mgr.idxProcessData(ctx, zi, &cData)
	toCheck := mgr.idxStore.UpdateReferences(ctx, zi)
	mgr.idxCheckZettel(toCheck)
}

188
189
190
191
192
193
194
195
196

197
198
199
200
201
202
203
195
196
197
198
199
200
201


202
203
204
205
206
207
208
209







-
-
+







		case meta.TypeID:
			mgr.idxUpdateValue(ctx, descr.Inverse, pair.Value, zi)
		case meta.TypeIDSet:
			for _, val := range meta.ListFromValue(pair.Value) {
				mgr.idxUpdateValue(ctx, descr.Inverse, val, zi)
			}
		case meta.TypeZettelmarkup:
			is := parser.ParseMetadata(pair.Value)
			collectInlineIndexData(&is, cData)
			collectInlineIndexData(parser.ParseMetadata(pair.Value), cData)
		case meta.TypeURL:
			if _, err := url.Parse(pair.Value); err == nil {
				cData.urls.Add(pair.Value)
			}
		default:
			for _, word := range strfun.NormalizeWords(pair.Value) {
				cData.words.Add(word)

Changes to box/membox/membox.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package membox stores zettel volatile in main memory.
29
30
31
32
33
34
35
36
37


38
39
40
41
42
43
44
45
46
47



48
49
50
51
52


53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
29
30
31
32
33
34
35


36
37


38
39
40
41
42



43
44
45





46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

63

64
65
66
67
68
69
70
71
72

73




74
75
76





77
78
79
80
81
82
83
84
85
86
87
88

89
90
91
92
93
94
95







-
-
+
+
-
-





-
-
-
+
+
+
-
-
-
-
-
+
+















-

-









-
+
-
-
-
-



-
-
-
-
-












-







func init() {
	manager.Register(
		"mem",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return &memBox{
				log: kernel.Main.GetLogger(kernel.BoxService).Clone().
					Str("box", "mem").Int("boxnum", int64(cdata.Number)).Child(),
				u:         u,
				cdata:     *cdata,
				u:     u,
				cdata: *cdata,
				maxZettel: box.GetQueryInt(u, "max-zettel", 0, 127, 65535),
				maxBytes:  box.GetQueryInt(u, "max-bytes", 0, 65535, (1024*1024*1024)-1),
			}, nil
		})
}

type memBox struct {
	log       *logger.Logger
	u         *url.URL
	cdata     manager.ConnectData
	log    *logger.Logger
	u      *url.URL
	cdata  manager.ConnectData
	maxZettel int
	maxBytes  int
	mx        sync.RWMutex // Protects the following fields
	zettel    map[id.Zid]domain.Zettel
	curBytes  int
	zettel map[id.Zid]domain.Zettel
	mx     sync.RWMutex
}

func (mb *memBox) notifyChanged(reason box.UpdateReason, zid id.Zid) {
	if chci := mb.cdata.Notify; chci != nil {
		chci <- box.UpdateInfo{Reason: reason, Zid: zid}
	}
}

func (mb *memBox) Location() string {
	return mb.u.String()
}

func (mb *memBox) Start(context.Context) error {
	mb.mx.Lock()
	mb.zettel = make(map[id.Zid]domain.Zettel)
	mb.curBytes = 0
	mb.mx.Unlock()
	mb.log.Trace().Int("max-zettel", int64(mb.maxZettel)).Int("max-bytes", int64(mb.maxBytes)).Msg("Start Box")
	return nil
}

func (mb *memBox) Stop(context.Context) {
	mb.mx.Lock()
	mb.zettel = nil
	mb.mx.Unlock()
}

func (mb *memBox) CanCreateZettel(context.Context) bool {
func (*memBox) CanCreateZettel(context.Context) bool { return true }
	mb.mx.RLock()
	defer mb.mx.RUnlock()
	return len(mb.zettel) < mb.maxZettel
}

func (mb *memBox) CreateZettel(_ context.Context, zettel domain.Zettel) (id.Zid, error) {
	mb.mx.Lock()
	newBytes := mb.curBytes + zettel.Length()
	if mb.maxZettel < len(mb.zettel) || mb.maxBytes < newBytes {
		mb.mx.Unlock()
		return id.Invalid, box.ErrCapacity
	}
	zid, err := box.GetNewZid(func(zid id.Zid) (bool, error) {
		_, ok := mb.zettel[zid]
		return !ok, nil
	})
	if err != nil {
		mb.mx.Unlock()
		return id.Invalid, err
	}
	meta := zettel.Meta.Clone()
	meta.Zid = zid
	zettel.Meta = meta
	mb.zettel[zid] = zettel
	mb.curBytes = newBytes
	mb.mx.Unlock()
	mb.notifyChanged(box.OnUpdate, zid)
	mb.log.Trace().Zid(zid).Msg("CreateZettel")
	return zid, nil
}

func (mb *memBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168

169
170
171
172

173
174
175
176

177
178
179



180
181
182
183
184
185
186
187
188
189
190
191
192
193


194
195
196

197
198
199
200
201
202
203
137
138
139
140
141
142
143








144




145



146
147



148
149
150
151













152
153

154

155
156
157
158
159
160
161
162







-
-
-
-
-
-
-
-
+
-
-
-
-
+
-
-
-

+
-
-
-
+
+
+

-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
-

-
+







			mb.cdata.Enricher.Enrich(ctx, m, mb.cdata.Number)
			handle(m)
		}
	}
	return nil
}

func (mb *memBox) CanUpdateZettel(_ context.Context, zettel domain.Zettel) bool {
	mb.mx.RLock()
	defer mb.mx.RUnlock()
	zid := zettel.Meta.Zid
	if !zid.IsValid() {
		return false
	}

func (*memBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return true }
	newBytes := mb.curBytes + zettel.Length()
	if prevZettel, found := mb.zettel[zid]; found {
		newBytes -= prevZettel.Length()
	}

	return newBytes < mb.maxBytes
}

func (mb *memBox) UpdateZettel(_ context.Context, zettel domain.Zettel) error {
	mb.mx.Lock()
	m := zettel.Meta.Clone()
	if !m.Zid.IsValid() {
		return &box.ErrInvalidID{Zid: m.Zid}
	meta := zettel.Meta.Clone()
	if !meta.Zid.IsValid() {
		return &box.ErrInvalidID{Zid: meta.Zid}
	}

	mb.mx.Lock()
	newBytes := mb.curBytes + zettel.Length()
	if prevZettel, found := mb.zettel[m.Zid]; found {
		newBytes -= prevZettel.Length()
	}
	if mb.maxBytes < newBytes {
		mb.mx.Unlock()
		return box.ErrCapacity
	}

	zettel.Meta = m
	mb.zettel[m.Zid] = zettel
	zettel.Meta = meta
	mb.zettel[meta.Zid] = zettel
	mb.curBytes = newBytes
	mb.mx.Unlock()
	mb.notifyChanged(box.OnUpdate, m.Zid)
	mb.notifyChanged(box.OnUpdate, meta.Zid)
	mb.log.Trace().Msg("UpdateZettel")
	return nil
}

func (*memBox) AllowRenameZettel(context.Context, id.Zid) bool { return true }

func (mb *memBox) RenameZettel(_ context.Context, curZid, newZid id.Zid) error {
231
232
233
234
235
236
237
238

239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
190
191
192
193
194
195
196

197

198
199
200
201

202
203
204
205
206
207
208
209
210
211
212
213
214







-
+
-




-













	_, ok := mb.zettel[zid]
	mb.mx.RUnlock()
	return ok
}

func (mb *memBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	mb.mx.Lock()
	oldZettel, found := mb.zettel[zid]
	if _, ok := mb.zettel[zid]; !ok {
	if !found {
		mb.mx.Unlock()
		return box.ErrNotFound
	}
	delete(mb.zettel, zid)
	mb.curBytes -= oldZettel.Length()
	mb.mx.Unlock()
	mb.notifyChanged(box.OnDelete, zid)
	mb.log.Trace().Msg("DeleteZettel")
	return nil
}

func (mb *memBox) ReadStats(st *box.ManagedBoxStats) {
	st.ReadOnly = false
	mb.mx.RLock()
	st.Zettel = len(mb.zettel)
	mb.mx.RUnlock()
	mb.log.Trace().Int("zettel", int64(st.Zettel)).Msg("ReadStats")
}

Changes to cmd/cmd_file.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package cmd

Changes to cmd/cmd_run.go.

71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88


89
90
91
92
93
94






95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112

113
114
115
116
117
118
119
120

121
122
123
124
125
126
127
128
129
71
72
73
74
75
76
77

78
79
80
81
82
83
84
85


86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128

129
130
131
132
133
134
135







-








-
-
+
+






+
+
+
+
+
+


















+








+

-







	ucListTags := usecase.NewListTags(protectedBoxManager)
	ucZettelContext := usecase.NewZettelContext(protectedBoxManager, rtConfig)
	ucDelete := usecase.NewDeleteZettel(ucLog, protectedBoxManager)
	ucUpdate := usecase.NewUpdateZettel(ucLog, protectedBoxManager)
	ucRename := usecase.NewRenameZettel(ucLog, protectedBoxManager)
	ucUnlinkedRefs := usecase.NewUnlinkedReferences(protectedBoxManager, rtConfig)
	ucRefresh := usecase.NewRefresh(ucLog, protectedBoxManager)
	ucVersion := usecase.NewVersion(kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string))

	webSrv.Handle("/", wui.MakeGetRootHandler(protectedBoxManager))

	// Web user interface
	if !authManager.IsReadonly() {
		webSrv.AddZettelRoute('b', server.MethodGet, wui.MakeGetRenameZettelHandler(
			ucGetMeta, &ucEvaluate))
		webSrv.AddZettelRoute('b', server.MethodPost, wui.MakePostRenameZettelHandler(&ucRename))
		webSrv.AddZettelRoute('c', server.MethodGet, wui.MakeGetCreateZettelHandler(
			ucGetZettel, &ucCreateZettel))
		webSrv.AddZettelRoute('c', server.MethodGet, wui.MakeGetCopyZettelHandler(
			ucGetZettel, usecase.NewCopyZettel()))
		webSrv.AddZettelRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('d', server.MethodGet, wui.MakeGetDeleteZettelHandler(
			ucGetMeta, ucGetAllMeta, &ucEvaluate))
		webSrv.AddZettelRoute('d', server.MethodPost, wui.MakePostDeleteZettelHandler(&ucDelete))
		webSrv.AddZettelRoute('e', server.MethodGet, wui.MakeEditGetZettelHandler(ucGetZettel))
		webSrv.AddZettelRoute('e', server.MethodPost, wui.MakeEditSetZettelHandler(&ucUpdate))
		webSrv.AddZettelRoute('f', server.MethodGet, wui.MakeGetFolgeZettelHandler(
			ucGetZettel, usecase.NewFolgeZettel(rtConfig)))
		webSrv.AddZettelRoute('f', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('g', server.MethodGet, wui.MakeGetNewZettelHandler(
			ucGetZettel, usecase.NewNewZettel()))
		webSrv.AddZettelRoute('g', server.MethodPost, wui.MakePostCreateZettelHandler(&ucCreateZettel))
	}
	webSrv.AddListRoute('g', server.MethodGet, wui.MakeGetGoActionHandler(&ucRefresh))
	webSrv.AddListRoute('h', server.MethodGet, wui.MakeListHTMLMetaHandler(
		ucListMeta, ucListRoles, ucListTags, &ucEvaluate))
	webSrv.AddZettelRoute('h', server.MethodGet, wui.MakeGetHTMLZettelHandler(
		&ucEvaluate, ucGetMeta))
	webSrv.AddListRoute('i', server.MethodGet, wui.MakeGetLoginOutHandler())
	webSrv.AddListRoute('i', server.MethodPost, wui.MakePostLoginHandler(&ucAuthenticate))
	webSrv.AddZettelRoute('i', server.MethodGet, wui.MakeGetInfoHandler(
		ucParseZettel, &ucEvaluate, ucGetMeta, ucGetAllMeta, ucUnlinkedRefs))
	webSrv.AddZettelRoute('k', server.MethodGet, wui.MakeZettelContextHandler(
		ucZettelContext, &ucEvaluate))

	// API
	webSrv.AddListRoute('a', server.MethodPost, a.MakePostLoginHandler(&ucAuthenticate))
	webSrv.AddListRoute('a', server.MethodPut, a.MakeRenewAuthHandler())
	webSrv.AddListRoute('j', server.MethodGet, a.MakeListMetaHandler(ucListMeta))
	webSrv.AddZettelRoute('j', server.MethodGet, a.MakeGetZettelHandler(ucGetZettel))
	webSrv.AddZettelRoute('l', server.MethodGet, a.MakeGetLinksHandler(ucEvaluate))
	webSrv.AddZettelRoute('m', server.MethodGet, a.MakeGetMetaHandler(ucGetMeta))
	webSrv.AddZettelRoute('o', server.MethodGet, a.MakeGetOrderHandler(
		usecase.NewZettelOrder(protectedBoxManager, ucEvaluate)))
	webSrv.AddZettelRoute('p', server.MethodGet, a.MakeGetParsedZettelHandler(ucParseZettel))
	webSrv.AddListRoute('r', server.MethodGet, a.MakeListRoleHandler(ucListRoles))
	webSrv.AddListRoute('t', server.MethodGet, a.MakeListTagsHandler(ucListTags))
	webSrv.AddZettelRoute('u', server.MethodGet, a.MakeListUnlinkedMetaHandler(
		ucGetMeta, ucUnlinkedRefs, &ucEvaluate))
	webSrv.AddListRoute('v', server.MethodPost, a.MakePostEncodeInlinesHandler(ucEvaluate))
	webSrv.AddZettelRoute('v', server.MethodGet, a.MakeGetEvalZettelHandler(ucEvaluate))
	webSrv.AddListRoute('x', server.MethodGet, a.MakeGetDataHandler(ucVersion))
	webSrv.AddListRoute('x', server.MethodPost, a.MakePostCommandHandler(&ucIsAuth, &ucRefresh))
	webSrv.AddZettelRoute('x', server.MethodGet, a.MakeZettelContextHandler(ucZettelContext))
	webSrv.AddListRoute('z', server.MethodGet, a.MakeListPlainHandler(ucListMeta))
	webSrv.AddZettelRoute('z', server.MethodGet, a.MakeGetPlainZettelHandler(ucGetZettel))
	if !authManager.IsReadonly() {
		webSrv.AddListRoute('j', server.MethodPost, a.MakePostCreateZettelHandler(&ucCreateZettel))
		webSrv.AddZettelRoute('j', server.MethodPut, a.MakeUpdateZettelHandler(&ucUpdate))

Changes to cmd/register.go.

14
15
16
17
18
19
20

21
22
23
24
25
26
27
28
29
30
31
32
33
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32
33







+



-









// Mention all needed encoders, parsers and stores to have them registered.
import (
	_ "zettelstore.de/z/box/compbox"       // Allow to use computed box.
	_ "zettelstore.de/z/box/constbox"      // Allow to use global internal box.
	_ "zettelstore.de/z/box/dirbox"        // Allow to use directory box.
	_ "zettelstore.de/z/box/filebox"       // Allow to use file box.
	_ "zettelstore.de/z/box/membox"        // Allow to use in-memory box.
	_ "zettelstore.de/z/encoder/djsonenc"  // Allow to use DJSON encoder.
	_ "zettelstore.de/z/encoder/htmlenc"   // Allow to use HTML encoder.
	_ "zettelstore.de/z/encoder/nativeenc" // Allow to use native encoder.
	_ "zettelstore.de/z/encoder/textenc"   // Allow to use text encoder.
	_ "zettelstore.de/z/encoder/zjsonenc"  // Allow to use ZJSON encoder.
	_ "zettelstore.de/z/encoder/zmkenc"    // Allow to use zmk encoder.
	_ "zettelstore.de/z/kernel/impl"       // Allow kernel implementation to create itself
	_ "zettelstore.de/z/parser/blob"       // Allow to use BLOB parser.
	_ "zettelstore.de/z/parser/draw"       // Allow to use draw parser.
	_ "zettelstore.de/z/parser/markdown"   // Allow to use markdown parser.
	_ "zettelstore.de/z/parser/none"       // Allow to use none parser.
	_ "zettelstore.de/z/parser/plain"      // Allow to use plain parser.
	_ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser.
)

Changes to collect/collect.go.

19
20
21
22
23
24
25

26


27
28
29
30
31
32
33
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35







+
-
+
+







	Embeds []*ast.Reference // list of all embedded material
	Cites  []*ast.CiteNode  // list of all referenced citations
}

// References returns all references mentioned in the given zettel. This also
// includes references to images.
func References(zn *ast.ZettelNode) (s Summary) {
	if zn.Ast != nil {
	ast.Walk(&s, &zn.Ast)
		ast.Walk(&s, zn.Ast)
	}
	return s
}

// Visit all node to collect data for the summary.
func (s *Summary) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.TranscludeNode:

Changes to collect/collect_test.go.

32
33
34
35
36
37
38
39

40
41
42
43
44
45

46
47
48
49
50
51
52
53
54

55


56
57
58
59
60
61
32
33
34
35
36
37
38

39
40
41
42
43
44

45
46
47
48
49
50
51
52
53
54
55

56
57
58
59
60
61
62
63







-
+





-
+









+
-
+
+






	summary := collect.References(zn)
	if summary.Links != nil || summary.Embeds != nil {
		t.Error("No links/images expected, but got:", summary.Links, "and", summary.Embeds)
	}

	intNode := &ast.LinkNode{Ref: parseRef("01234567890123")}
	para := ast.CreateParaNode(intNode, &ast.LinkNode{Ref: parseRef("https://zettelstore.de/z")})
	zn.Ast = ast.BlockSlice{para}
	zn.Ast = ast.CreateBlockListNode(para)
	summary = collect.References(zn)
	if summary.Links == nil || summary.Embeds != nil {
		t.Error("Links expected, and no images, but got:", summary.Links, "and", summary.Embeds)
	}

	para.Inlines = append(para.Inlines, intNode)
	para.Inlines.Append(intNode)
	summary = collect.References(zn)
	if cnt := len(summary.Links); cnt != 3 {
		t.Error("Link count does not work. Expected: 3, got", summary.Links)
	}
}

func TestEmbed(t *testing.T) {
	t.Parallel()
	zn := &ast.ZettelNode{
		Ast: ast.CreateBlockListNode(
		Ast: ast.BlockSlice{ast.CreateParaNode(&ast.EmbedRefNode{Ref: parseRef("12345678901234")})},
			ast.CreateParaNode(&ast.EmbedRefNode{Ref: parseRef("12345678901234")}),
		),
	}
	summary := collect.References(zn)
	if summary.Embeds == nil {
		t.Error("Only image expected, but got: ", summary.Embeds)
	}
}

Changes to collect/order.go.

11
12
13
14
15
16
17



18

19
20
21
22
23
24
25
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28







+
+
+
-
+







// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"

// Order of internal reference within the given zettel.
func Order(zn *ast.ZettelNode) (result []*ast.Reference) {
	if zn.Ast == nil {
		return nil
	}
	for _, bn := range zn.Ast {
	for _, bn := range zn.Ast.List {
		ln, ok := bn.(*ast.NestedListNode)
		if !ok {
			continue
		}
		switch ln.Kind {
		case ast.NestedListOrdered, ast.NestedListUnordered:
			for _, is := range ln.Items {
39
40
41
42
43
44
45
46
47





48
49
50
51
52
53
54
42
43
44
45
46
47
48


49
50
51
52
53
54
55
56
57
58
59
60







-
-
+
+
+
+
+







				return ref
			}
		}
	}
	return nil
}

func firstInlineZettelReference(is ast.InlineSlice) (result *ast.Reference) {
	for _, inl := range is {
func firstInlineZettelReference(iln *ast.InlineListNode) (result *ast.Reference) {
	if iln == nil {
		return nil
	}
	for _, inl := range iln.List {
		switch in := inl.(type) {
		case *ast.LinkNode:
			if ref := in.Ref; ref.IsZettel() {
				return ref
			}
			result = firstInlineZettelReference(in.Inlines)
		case *ast.EmbedRefNode:

Changes to docs/development/20210916194900.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
1
2
3
4

5
6
7
8


9
10
11
12
13
14
15




-
+



-
-







id: 20210916194900
title: Checklist for Release
role: zettel
syntax: zmk
modified: 20220309105459
modified: 20211214181017

# Sync with the official repository
#* ``fossil sync -u``
# Make sure that all dependencies are up-to-date.
#* ``cat go.mod``
# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# All internal tests must succeed:
#* ``go run tools/build.go relcheck`` (alternatively: ``make relcheck``).
# The API tests must succeed on every development platform:
#* ``go run tools/build.go testapi`` (alternatively: ``make api``).
# Run [[linkchecker|https://linkchecker.github.io/linkchecker/]] with the manual:

Changes to docs/manual/00000000000100.zettel.

1
2
3
4
5
6
7
8
9
10

11
12
13
1
2
3
4
5
6
7
8
9

10
11
12
13









-
+



id: 00000000000100
title: Zettelstore Runtime Configuration
role: configuration
syntax: none
default-copyright: (c) 2020-2022 by Detlef Stern <ds@zettelstore.de>
default-license: EUPL-1.2-or-later
default-visibility: public
footer-html: <hr><p><a href="/home/doc/trunk/www/impri.wiki">Imprint / Privacy</a></p>
home-zettel: 00001000000000
modified: 20220215171041
no-index: true
site-name: Zettelstore Manual
visibility: owner

Changes to docs/manual/00001003305000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18

19
20
21

22
23
24
25
26
27
28
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17

18
19
20

21
22
23
24
25
26
27
28





-
+











-
+


-
+







id: 00001003305000
title: Enable Zettelstore to start automatically on Windows
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
modified: 20220218125541
modified: 20211125201602

Windows is a complicated beast. There are several ways to automatically start Zettelstore.

=== Startup folder

One way is to use the [[autostart folder|https://support.microsoft.com/en-us/windows/add-an-app-to-run-automatically-at-startup-in-windows-10-150da165-dcd9-7230-517b-cf3c295d89dd]].
Open the folder where you have placed in the Explorer.
Create a shortcut file for the Zettelstore executable.
There are some ways to do this:
* Execute a right-click on the executable, and choose the menu entry ""Create shortcut"",
* Execute a right-click on the executable, and then click Send To > Desktop (Create shortcut).
* Drag the executable to your Desktop with pressing the ''Alt''-Key.
* Drag the executable to your Desktop with pressing the ++Alt++-Key.

If you have created the shortcut file, you must move it into the Startup folder.
Press the Windows logo key and the key ''R'', type ''shell:startup''.
Press the Windows logo key and the key ++R++, type ''shell:startup''.
Select the OK button.
This will open the Startup folder.
Move the shortcut file into this folder.

The next time you log into your computer, Zettelstore will be started automatically.
However, it remains visible, at least in the task bar.

38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
38
39
40
41
42
43
44

45
46
47
48
49
50
51
52







-
+







All you have to do is to open your web browser, enter the appropriate URL, and there you go.

On the negative side, you will not be notified when you enter the wrong data in the Task scheduler and Zettelstore fails to start.
This can be mitigated by first using the command line prompt to start Zettelstore with the appropriate options.
Once everything works, you can register Zettelstore to be automatically started by the task scheduler.
There you should make sure that you have followed the first steps as described on the [[parent page|00001003300000]].

To sart the Task scheduler management console, press the Windows logo key and the key ''R'', type ''taskschd.msc''.
To sart the Task scheduler management console, press the Windows logo key and the key ++R++, type ''taskschd.msc''.
Select the OK button.

{{00001003305102}}

This will start the ""Task Scheduler"".

Now, create a new task with ""Create Task ...""

Changes to docs/manual/00001003315000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
1
2
3
4
5

6
7
8
9
10

11
12
13
14
15
16
17





























-
+




-







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001003315000
title: Enable Zettelstore to start automatically on Linux
role: manual
tags: #installation #manual #zettelstore
syntax: zmk
modified: 20220307104944
modified: 20220131162209

Since there is no such thing as the one Linux, there are too many different ways to automatically start Zettelstore.

* One way is to interpret your Linux desktop system as a server and use the [[recipe to install Zettelstore on a server|00001003600000]].
** See below for a lighter alternative.
* If you are using the [[Gnome Desktop|https://www.gnome.org/]], you could use the tool [[Tweak|https://wiki.gnome.org/action/show/Apps/Tweaks]] (formerly known as ""GNOME Tweak Tool"" or just ""Tweak Tool"").
  It allows to specify application that should run on startup / login.
* [[KDE|https://kde.org/]] provides a system setting to [[autostart|https://docs.kde.org/stable5/en/plasma-workspace/kcontrol/autostart/]] applications.
* [[Xfce|https://xfce.org/]] allows to specify [[autostart applications|https://docs.xfce.org/xfce/xfce4-session/preferences#application_autostart]].
* [[LXDE|https://www.lxde.org/]] uses [[LXSession Edit|https://wiki.lxde.org/en/LXSession_Edit]] to allow users to specify autostart applications.

If you use a different desktop environment, it often helps to to provide its name and the string ""autostart"" to google for it with the search engine of your choice.

Yet another way is to make use of the middleware that is provided.
Many Linux distributions make use of [[systemd|https://systemd.io/]], which allows to start processes on behalf of an user.
On the command line, adapt the following script to your own needs and execute it:
```
# mkdir -p "$HOME/.config/systemd/user"
# cd "$HOME/.config/systemd/user"
# cat <<__EOF__ > zettelstore.service
[Unit]
Description=Zettelstore
After=network.target home.mount

[Service]
ExecStart=/usr/local/bin/zettelstore run -d zettel

[Install]
WantedBy=default.target
__EOF__
# systemctl --user daemon-reload
# systemctl --user enable zettelstore.service
# systemctl --user start zettelstore.service
# systemctl --user status zettelstore.service
```
The last command should output some lines to indicate success.

Changes to docs/manual/00001004010000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19

20
21

22
23
24

25
26
27


28
29
30
31
32

33
34
35


36
37
38
39
40
41
42


43
44
45
46
47
48



49
50
51
52


53
54

55
56
57


58
59
60
61

62
63
64


65
66
67
68
69


70
71
72
73
74

75
76
77
78
79



80
81
82
83


84
85
86
87
88

89
90

91
92
93

94
95

96
97

98
99
100
101


102
103

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18

19
20

21
22
23

24
25


26
27
28
29
30
31

32
33


34
35
36
37
38
39
40


41
42
43
44
45



46
47
48
49
50


51
52
53

54
55


56
57
58
59
60

61
62


63
64
65
66
67


68
69
70
71
72
73

74
75
76



77
78
79
80



81
82
83
84
85
86

87
88
89
90
91


92
93

94


95
96
97


98
99


100





-
+












-
+

-
+


-
+

-
-
+
+




-
+

-
-
+
+





-
-
+
+



-
-
-
+
+
+


-
-
+
+

-
+

-
-
+
+



-
+

-
-
+
+



-
-
+
+




-
+


-
-
-
+
+
+

-
-
-
+
+




-
+


+

-
-
+

-
+
-
-
+


-
-
+
+
-
-
+
id: 00001004010000
title: Zettelstore startup configuration
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20220304115353
modified: 20211212143318

The configuration file, as specified by the ''-c CONFIGFILE'' [[command line option|00001004051000]], allows you to specify some startup options.
These options cannot be stored in a [[configuration zettel|00001004020000]] because either they are needed before Zettelstore can start or because of security reasons.
For example, Zettelstore need to know in advance, on which network address is must listen or where zettel are stored.
An attacker that is able to change the owner can do anything.
Therefore only the owner of the computer on which Zettelstore runs can change this information.

The file for startup configuration must be created via a text editor in advance.

The syntax of the configuration file is the same as for any zettel metadata.
The following keys are supported:

; [!admin-port|''admin-port'']
; [!admin-port]''admin-port''
: Specifies the TCP port through which you can reach the [[administrator console|00001004100000]].
  A value of ""0"" (the default) disables the administrator console.
  A value of ''0'' (the default) disables the administrator console.
  The administrator console will only be enabled if Zettelstore is started with the [[''run'' sub-command|00001004051000]].

  On most operating systems, the value must be greater than ""1024"" unless you start Zettelstore with the full privileges of a system administrator (which is not recommended).
  On most operating systems, the value must be greater than ''1024'' unless you start Zettelstore with the full privileges of a system administrator (which is not recommended).

  Default: ""0""
; [!box-uri-x|''box-uri-X''], where __X__ is a number greater or equal to one
  Default: ''0''
; [!box-uri-x]''box-uri-__X__'', where __X__ is a number greater or equal to one
: Specifies a [[box|00001004011200]] where zettel are stored.
  During startup __X__ is counted up, starting with one, until no key is found.
  This allows to configure more than one box.

  If no ''box-uri-1'' key is given, the overall effect will be the same as if only ''box-uri-1'' was specified with the value ""dir://.zettel"".
  If no ''box-uri-1'' key is given, the overall effect will be the same as if only ''box-uri-1'' was specified with the value ''dir://.zettel''.
  In this case, even a key ''box-uri-2'' will be ignored.
; [!debug-mode|''debug-mode'']
: Allows to debug the Zettelstore software (mostly used by the developers) if set to [[true|00001006030500]]
; [!debug-mode]''debug-mode''
: Allows to debug the Zettelstore software (mostly used by the developers).
  Disables any timeout values of the internal web server and does not send some security-related data.
  Sets [[''log-level''|#log-level]] to ""debug"".

  Do not enable it for a production server.

  Default: ""false""
; [!default-dir-box-type|''default-dir-box-type'']
  Default: ''false''
; [!default-dir-box-type]''default-dir-box-type''
: Specifies the default value for the (sub-) type of [[directory boxes|00001004011400#type]].
  Zettel are typically stored in such boxes.

  Default: ""notify""
; [!insecure-cookie|''insecure-cookie'']
: Must be set to [[true|00001006030500]], if authentication is enabled and Zettelstore is not accessible not via HTTPS (but via HTTP).
  Default: ''notify''
; [!insecure-cookie]''insecure-cookie''
: Must be set to ''true'', if authentication is enabled and Zettelstore is not accessible not via HTTPS (but via HTTP).
  Otherwise web browser are free to ignore the authentication cookie.

  Default: ""false""
; [!listen-addr|''listen-addr'']
  Default: ''false''
; [!listen-addr]''listen-addr''
: Configures the network address, where the Zettelstore service is listening for requests.
  Syntax is: ''[NETWORKIP]:PORT'', where ''NETWORKIP'' is the IP-address of the networking interface (or something like ""0.0.0.0"" if you want to listen on all network interfaces, and ''PORT'' is the TCP port.
  Syntax is: ''[NETWORKIP]:PORT'', where ''NETWORKIP'' is the IP-address of the networking interface (or something like ''0.0.0.0'' if you want to listen on all network interfaces, and ''PORT'' is the TCP port.

  Default value: ""127.0.0.1:23123""
; [!log-level|''log-level'']
  Default value: ''"127.0.0.1:23123"''
; [!log-level]''log-level''
: Specify the global [[logging level|00001004059700]] for the whole application, overwriting the level ""debug"" set by configuration [[''debug-mode''|#debug-mode]].
  Can be changed at runtime, even for specific internal services, with the ''log-level'' command of the [[administrator console|00001004101000#log-level]].
  
  Default: ""info"".
  Default: ''info''.

  When you are familiar to operate the Zettelstore, you might set the level to ""warn"" or ""error"" to receive less noisy messages from the Zettelstore.
; [!owner|''owner'']
  When you are familiar to operate the Zettelstore, you might set the level to ''warn'' or ''error'' to receive less noisy messages from the Zettelstore.
; [!owner]''owner''
: [[Identifier|00001006050000]] of a zettel that contains data about the owner of the Zettelstore.
  The owner has full authorization for the Zettelstore.
  Only if owner is set to some value, user [[authentication|00001010000000]] is enabled.
; [!persistent-cookie|''persistent-cookie'']
: A [[boolean value|00001006030500]] to make the access cookie persistent.
; [!persistent-cookie]''persistent-cookie''
: A boolean value to make the access cookie persistent.
  This is helpful if you access the Zettelstore via a mobile device.
  On these devices, the operating system is free to stop the web browser and to remove temporary cookies.
  Therefore, an authenticated user will be logged off.

  If ""true"", a persistent cookie is used.
  If ''true'', a persistent cookie is used.
  Its lifetime exceeds the lifetime of the authentication token (see option ''token-lifetime-html'') by 30 seconds.

  Default: ""false""
; [!read-only-mode|''read-only-mode'']
: Puts the Zettelstore service into a read-only mode, if set to a [[true value|00001006030500]].
  Default: ''false''
; [!read-only-mode]''read-only-mode''
: Puts the Zettelstore service into a read-only mode.
  No changes are possible.

  Default: ""false"".
; [!token-lifetime-api|''token-lifetime-api''], [!token-lifetime-html|''token-lifetime-html'']
  Default: false.
; [!token-lifetime-api]''token-lifetime-api'', [!token-lifetime-html]''token-lifetime-html''
: Define lifetime of access tokens in minutes.
  Values are only valid if authentication is enabled, i.e. key ''owner'' is set.

  ''token-lifetime-api'' is for accessing Zettelstore via its [[API|00001012000000]].
  Default: ""10"".
  Default: 10.

  ''token-lifetime-html'' specifies the lifetime for the HTML views.
  Default: 60.
  It is automatically extended, when a new HTML view is rendered.
  Default: ""60"".
; [!url-prefix|''url-prefix'']
; [!url-prefix]''url-prefix''
: Add the given string as a prefix to the local part of a Zettelstore local URL/URI when rendering zettel representations.
  Must begin and end with a slash character (""''/''"", U+002F).
  Must begin and end with a slash character (""''/''"", ''U+002F'').

  Default: ""/"".
  Default: ''"/"''.

  This allows to use a forwarding proxy [[server|00001010090100]] in front of the Zettelstore.
; [!verbose-mode|''verbose-mode'']
: Be more verbose when logging data, if set to a [[true value|00001006030500]].
; [!verbose-mode]''verbose-mode''
: Be more verbose when logging data.

  Default: ""false""
  Default: false

Changes to docs/manual/00001004011200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

23
24
25

26
27
28
29
30

31
32
33
34
35

36
37
38
39
40
41
42
43
44
45
46
47
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

22
23
24

25
26
27
28
29

30
31
32
33
34

35
36
37

38
39
40
41
42
43
44
45
46





-
+















-
+


-
+




-
+




-
+


-









id: 00001004011200
title: Zettelstore boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20220307121547
modified: 20211103163225

A Zettelstore must store its zettel somehow and somewhere.
In most cases you want to store your zettel as files in a directory.
Under certain circumstances you may want to store your zettel elsewhere.

An example are the [[predefined zettel|00001005090000]] that come with a Zettelstore.
They are stored within the software itself.
In another situation you may want to store your zettel volatile, e.g. if you want to provide a sandbox for experimenting.

To cope with these (and more) situations, you configure Zettelstore to use one or more __boxes__.
This is done via the ''box-uri-X'' keys of the [[startup configuration|00001004010000#box-uri-X]] (X is a number).
Boxes are specified using special [[URIs|https://en.wikipedia.org/wiki/Uniform_Resource_Identifier]], somehow similar to web addresses.

The following box URIs are supported:

; [!dir|''dir://DIR'']
; [!dir]''dir:\//DIR''
: Specifies a directory where zettel files are stored.
  ''DIR'' is the file path.
  Although it is possible to use relative file paths, such as ''./zettel'' (&rarr; URI is ''dir://.zettel''), it is preferable to use absolute file paths, e.g. ''/home/user/zettel''.
  Although it is possible to use relative file paths, such as ''./zettel'' (&rarr; URI is ''dir:\//.zettel''), it is preferable to use absolute file paths, e.g. ''/home/user/zettel''.

  The directory must exist before starting the Zettelstore[^There is one exception: when Zettelstore is [[started without any parameter|00001004050000]], e.g. via double-clicking its icon, an directory called ''./zettel'' will be created.].

  It is possible to [[configure|00001004011400]] a directory box.
; [!file|''file:FILE.zip'' or ''file:///path/to/file.zip'']
; [!file]''file:FILE.zip'' oder ''file:/\//path/to/file.zip''
: Specifies a ZIP file which contains files that store zettel.
  You can create such a ZIP file, if you zip a directory full of zettel files.

  This box is always read-only.
; [!mem|''mem:'']
; [!mem]''mem:''
: Stores all its zettel in volatile memory.
  If you stop the Zettelstore, all changes are lost.
  To limit usage of volatile memory, you should [[configure|00001004011600]] this type of box, although the default values might be valid for your use case.

All boxes that you configure via the ''box-uri-X'' keys form a chain of boxes.
If a zettel should be retrieved, a search starts in the box specified with the ''box-uri-2'' key, then ''box-uri-3'' and so on.
If a zettel is created or changed, it is always stored in the box specified with the ''box-uri-1'' key.
This allows to overwrite zettel from other boxes, e.g. the predefined zettel.

If you use the ''mem:'' box, where zettel are stored in volatile memory, it makes only sense if you configure it as ''box-uri-1''.
Such a box will be empty when Zettelstore starts and only the first box will receive updates.
You must make sure that your computer has enough RAM to store all zettel.

Changes to docs/manual/00001004011400.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
1
2
3
4
5

6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
21





-
+







-
+







id: 00001004011400
title: Configure file directory boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20220307121244
modified: 20211216152540

Under certain circumstances, it is preferable to further configure a file directory box.
This is done by appending query parameters after the base box URI ''dir:\//DIR''.

The following parameters are supported:

|= Parameter:|Description|Default value:|
|type|(Sub-) Type of the directory service|(value of ""[[default-dir-box-type|00001004010000#default-dir-box-type]]"")
|type|(Sub-) Type of the directory service|(value of ''[[default-dir-box-type|00001004010000#default-dir-box-type]]'')
|worker|Number of worker that can access the directory in parallel|7
|readonly|Allow only operations that do not create or change zettel|n/a

=== Type
On some operating systems, Zettelstore tries to detect changes to zettel files outside of Zettelstore's control[^This includes Linux, Windows, and macOS.].
On other operating systems, this may be not possible, due to technical limitations.
Automatic detection of external changes is also not possible, if zettel files are put on an external service, such as a file server accessed via SMD/CIFS or NFS.

Deleted docs/manual/00001004011600.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

























-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001004011600
title: Configure memory boxes
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20220307122554

Under most circumstances, it is preferable to further configure a memory box.
This is done by appending query parameters after the base box URI ''mem:''.

The following parameters are supported:

|= Parameter:|Description|Default value:|Maximum value:
|max-bytes|Maximum number of bytes the box will store|65535|1073741824 (1 GiB)
|max-zettel|Maximum number of zettel|127|65535

The default values are somehow arbitrarily, but applicable for many use cases.

While the number of zettel should be easily calculable by an user, the number of bytes might be a little more difficult.

Metadata consumes 6 bytes for the zettel identifier and for each metadata value one byte for the separator, plus the length of key and data.
Then size of the content is its size in bytes.
For text content, its the number of bytes for its UTF-8 encoding.

If one of the limits are exceeded, Zettelstore will give an error indication, based on the HTTP status code 507.

Changes to docs/manual/00001004020000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15
16
17

18
19
20
21
22
23
24

25
26
27
28

29
30
31

32
33
34

35
36
37
38
39

40
41
42
43


44
45
46

47
48
49
50
51

52
53
54

55
56
57

58
59
60
61

62
63
64
65
66


67
68
69
70

71
72
73
74
75
76
77
78
1
2
3
4
5

6
7
8
9
10
11

12
13
14
15
16

17
18
19
20
21
22
23

24
25
26
27

28
29
30

31
32
33

34
35
36
37
38

39
40
41


42
43
44
45

46
47
48
49
50

51
52
53

54
55
56

57
58
59
60

61
62
63
64


65
66
67
68
69

70
71
72
73
74
75
76
77
78





-
+





-
+




-
+






-
+



-
+


-
+


-
+




-
+


-
-
+
+


-
+




-
+


-
+


-
+



-
+



-
-
+
+



-
+








id: 00001004020000
title: Configure the running Zettelstore
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20220304114412
modified: 20220111103757

You can configure a running Zettelstore by modifying the special zettel with the ID [[00000000000100]].
This zettel is called __configuration zettel__.
The following metadata keys change the appearance / behavior of Zettelstore:

; [!default-copyright|''default-copyright'']
; [!default-copyright]''default-copyright''
: Copyright value to be used when rendering content.
  Can be overwritten in a zettel with [[meta key|00001006020000]] ''copyright''.

  Default: (the empty string).
; [!default-lang|''default-lang'']
; [!default-lang]''default-lang''
: Default language to be used when displaying content.
  Can be overwritten in a zettel with [[meta key|00001006020000]] ''lang''. Default: ""en"".

  This value is also used to specify the language for all non-zettel content, e.g. lists or search results.

  Use values according to the language definition of [[RFC-5646|https://tools.ietf.org/html/rfc5646]].
; [!default-license|''default-license'']
; [!default-license]''default-license''
: License value to be used when rendering content.
  Can be overwritten in a zettel with [[meta key|00001006020000]] ''license''.
  Default: (the empty string).
; [!default-role|''default-role'']
; [!default-role]''default-role''
: Role to be used, if a zettel specifies no ''role'' [[meta key|00001006020000]].
  Default: ""zettel"".
; [!default-syntax|''default-syntax'']
; [!default-syntax]''default-syntax''
: Syntax to be used, if a zettel specifies no ''syntax'' [[meta key|00001006020000]].
  Default: ""zmk"" (""[[Zettelmarkup|00001007000000]]"").
; [!default-title|''default-title'']
; [!default-title]''default-title''
: Title to be used, if a zettel specifies no ''title'' [[meta key|00001006020000]].
  Default: ""Untitled"".

  You can use all [[inline-structured elements|00001007040000]] of Zettelmarkup.
; [!default-visibility|''default-visibility'']
; [!default-visibility]''default-visibility''
: Visibility to be used, if zettel does not specify a value for the [[''visibility''|00001006020000#visibility]] metadata key.
  Default: ""login"".
; [!expert-mode|''expert-mode'']
: If set to a [[boolean true value|00001006030500]], all zettel with [[visibility ""expert""|00001010070200]] will be shown (to the owner, if [[authentication is enabled|00001010040100]]; to all, otherwise).
; [!expert-mode]''expert-mode''
: If set to a boolean true value, all zettel with [[visibility ""expert""|00001010070200]] will be shown (to the owner, if [[authentication is enabled|00001010040100]]; to all, otherwise).
  This affects most computed zettel.
  Default: ""False"".
; [!footer-html|''footer-html'']
; [!footer-html]''footer-html''
: Contains some HTML code that will be included into the footer of each Zettelstore web page.
  It only affects the [[web user interface|00001014000000]].
  Zettel content, delivered via the [[API|00001012000000]] as JSON, etc. is not affected.
  Default: (the empty string).
; [!home-zettel|''home-zettel'']
; [!home-zettel]''home-zettel''
: Specifies the identifier of the zettel, that should be presented for the default view / home view.
  If not given or if the identifier does not identify a zettel, the zettel with the identifier ''00010000000000'' is shown.
; [!marker-external|''marker-external'']
; [!marker-external]''marker-external''
: Some HTML code that is displayed after a [[reference to external material|00001007040310]].
  Default: ""&\#10138;"", to display a ""&#10138;"" sign.
; [!max-transclusions|''max-transclusions'']
; [!max-transclusions]''max-transclusions''
: Maximum number of indirect transclusion.
  This is used to avoid an exploding ""transclusion bomb"", a form of a [[billion laughs attack|https://en.wikipedia.org/wiki/Billion_laughs_attack]].
  Default: ""1024"".
; [!site-name|''site-name'']
; [!site-name]''site-name''
: Name of the Zettelstore instance.
  Will be used when displaying some lists.
  Default: ""Zettelstore"".
; [!yaml-header|''yaml-header'']
: If [[true|00001006030500]], metadata and content will be separated by ''---\\n'' instead of an empty line (''\\n\\n'').
; [!yaml-header]''yaml-header''
: If true, metadata and content will be separated by ''-\--\\n'' instead of an empty line (''\\n\\n'').
  Default: ""False"".

  You will probably use this key, if you are working with another software processing [[Markdown|https://daringfireball.net/projects/markdown/]] that uses a subset of [[YAML|https://yaml.org/]] to specify metadata.
; [!zettel-file-syntax|''zettel-file-syntax'']
; [!zettel-file-syntax]''zettel-file-syntax''
: If you create a new zettel with a syntax different to ""zmk"", Zettelstore will store the zettel as two files:
  one for the metadata (file without a filename extension) and another for the content (file extension based on the syntax value).
  If you want to specify alternative syntax values, for which you want new zettel to be stored in one file (file extension ''.zettel''), you can use this key.
  All values are case-insensitive, duplicate values are removed.

  For example, you could use this key if you're working with Markdown syntax and you want to store metadata and content in one ''.zettel'' file.

  If ''yaml-header'' evaluates to true, a zettel is always stored in one ''.zettel'' file.

Changes to docs/manual/00001004051000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15

16
17
18

19
20
21
22
23

24
25
26
27

28
29
30
31
32
33

34
35
36
37
38
39
40

41
42
43
44
45

46
47
48
1
2
3
4
5

6
7
8
9
10
11
12
13
14

15
16
17

18
19
20
21
22

23
24
25
26

27
28
29
30
31
32

33
34
35
36
37
38
39

40
41
42
43
44

45
46
47
48





-
+








-
+


-
+




-
+



-
+





-
+






-
+




-
+



id: 00001004051000
title: The ''run'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20220214175947
modified: 20211124140711

=== ``zettelstore run``
This starts the web service.

```
zettelstore run [-a PORT] [-c CONFIGFILE] [-d DIR] [-debug] [-p PORT] [-r] [-v]
```

; [!a|''-a PORT'']
; [!a]''-a PORT''
: Specifies the TCP port through which you can reach the [[administrator console|00001004100000]].
  See the explanation of [[''admin-port''|00001004010000#admin-port]] for more details.
; [!c|''-c CONFIGFILE'']
; [!c]''-c CONFIGFILE''
: Specifies ''CONFIGFILE'' as a file, where [[startup configuration data|00001004010000]] is read.
  It is ignored, when the given file is not available, nor readable.

  Default: ''./.zscfg''. (''.\\.zscfg'' on Windows)), where ''.'' denotes the ""current directory"".
; [!d|''-d DIR'']
; [!d]''-d DIR''
: Specifies ''DIR'' as the directory that contains all zettel.

  Default is ''./zettel'' (''.\\zettel'' on Windows), where ''.'' denotes the ""current directory"".
; [!debug|''-debug'']
; [!debug]''-debug''
: Allows better debugging of the internal web server by disabling any timeout values.
  You should specify this only as a developer.
  Especially do not enable it for a production server.

  [[https://blog.cloudflare.com/exposing-go-on-the-internet/#timeouts]] contains a good explanation for the usefulness of sensitive timeout values.
; [!p|''-p PORT'']
; [!p]''-p PORT''
: Specifies the integer value ''PORT'' as the TCP port, where the Zettelstore web server listens for requests.

  Default: 23123.

  Zettelstore listens only on ''127.0.0.1'', e.g. only requests from the current computer will be processed.
  If you want to listen on network card to process requests from other computer, please use [[''listen-addr''|00001004010000#listen-addr]] of the configuration file as described below.
; [!r|''-r'']
; [!r]''-r''
: Puts the Zettelstore in read-only mode.
  No changes are possible via the [[web user interface|00001014000000]] / via the [[API|00001012000000]].

  This allows to publish your content without any risks of unauthorized changes.
; [!v|''-v'']
; [!v]''-v''
: Be more verbose when writing logs.

Command line options take precedence over [[configuration file|00001004010000]] options.

Changes to docs/manual/00001004051100.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

22
23
24
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24





-
+














-
+



id: 00001004051100
title: The ''run-simple'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20220214180253
modified: 20210712234203

=== ``zettelstore run-simple``
This sub-command is implicitly called, when an user starts Zettelstore by double-clicking on its GUI icon.
It is s simplified variant of the [[''run'' sub-command|00001004051000]].

It allows only to specify a zettel directory.
The directory will be created automatically, if it does not exist.
This is a difference to the ''run'' sub-command, where the directory must exists.
In contrast to the ''run'' sub-command, other command line parameter are not allowed.

```
zettelstore run-simple [-d DIR]
```

; [!d|''-d DIR'']
; [!d]''-d DIR''
: Specifies ''DIR'' as the directory that contains all zettel.

  Default is ''./zettel'' (''.\\zettel'' on Windows), where ''.'' denotes the ""current directory"".

Changes to docs/manual/00001004051200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28





-
+











+


-








id: 00001004051200
title: The ''file'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20220209114650
modified: 20210727120507

Reads zettel data from a file (or from standard input / stdin) and renders it to standard output / stdout.
This allows Zettelstore to render files manually.
```
zettelstore file [-t FORMAT] [file-1 [file-2]]
```

; ''-t FORMAT''
: Specifies the output format.
  Supported values are:
  [[''html''|00001012920510]] (default),
  [[''djson''|00001012920503]],
  [[''native''|00001012920513]],
  [[''text''|00001012920519]],
  [[''zjson''|00001012920503]],
  and [[''zmk''|00001012920522]].
; ''file-1''
: Specifies the file name, where at least metadata is read.
  If ''file-2'' is not given, the zettel content is also read from here.
; ''file-2''
: File name where the zettel content is stored.

If neither ''file-1'' nor ''file-2'' are given, metadata and zettel content are read from standard input / stdin.

Changes to docs/manual/00001004101000.zettel.

1
2
3
4
5
6

7
8

9
10

11
12
13

14
15

16
17
18
19

20
21

22
23
24
25

26
27

28
29

30
31

32
33

34
35
36
37
38
39
40
41

42
43

44
45
46
47
48
49
50
51
52

53
54
55

56
57
58
59
60
61
62
63
64

65
66
67
68
69
70
71
72
73
74
75
76

77
78

79
80

81
82
83
84
85
86
87

88
89
90
91



92
93

94
1
2
3
4
5

6
7

8
9

10
11
12

13
14

15
16
17
18

19
20

21
22
23
24

25
26

27
28

29
30

31
32

33
34
35
36
37
38
39
40

41
42

43
44
45
46
47
48
49
50
51

52
53
54

55
56
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71
72
73
74
75

76
77

78
79

80
81
82
83
84
85
86

87
88



89
90
91
92

93
94





-
+

-
+

-
+


-
+

-
+



-
+

-
+



-
+

-
+

-
+

-
+

-
+







-
+

-
+








-
+


-
+








-
+











-
+

-
+

-
+






-
+

-
-
-
+
+
+

-
+

id: 00001004101000
title: List of supported commands of the administrator console
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20220218133526
modified: 20211210184654

; [!bye|''bye'']
; ''bye''
: Closes the connection to the administrator console.
; [!config|''config SERVICE'']
; ''config SERVICE''
: Displays all valid configuration keys for the given service.

  If a key ends with the hyphen-minus character (""''-''"", U+002D), the key denotes a list value.
  If a key ends with the hyphen-minus character (""''-''"", ''U+002D''), the key denotes a list value.
  Keys of list elements are specified by appending a number greater than zero to the key.
; [!crlf|''crlf'']
; ''crlf''
: Toggles CRLF mode for console output.
  Changes end of line sequences between Windows mode (==\\r\\n==) and non-Windows mode (==\\n==, initial value).
  Often used on Windows telnet clients that otherwise scramble the output of commands.
; [!dump-index|''dump-index'']
; ''dump-index''
: Displays the content of the internal search index.
; [!dump-recover|''dump-recover RECOVER'']
; ''dump-recover RECOVER''
: Displays data about the last given recovered internal activity.

  The value for ''RECOVER'' can be obtained via the command ``stat core``, which lists all overview data about all recoveries.
; [!echo|''echo'']
; ''echo''
: Toggles the echo mode, where each command is printed before execution.
; [!end-profile|''end-profile'']
; ''end-profile''
: Stops profiling the application.
; [!env|''env'']
; ''env''
: Display environment values.
; [!help|''help'']
; ''help''
: Displays a list of all available commands.
; [!get-config|''get-config'']
; ''get-config''
: Displays current configuration data.

  ``get-config`` shows all current configuration data.

  ``get-config SERVICE`` shows only the current configuration data of the given service.

  ``get-config SERVICE KEY`` shows the current configuration data for the given service and key.
; [!header|''header'']
; ''header''
: Toggles the header mode, where each table is show with a header nor not.
; [!log-level|''log-level'']
; [!log-level]''log-level''
: Displays or sets the [[logging level|00001004059700]] for the kernel or a service.

  ``log-level`` shows all known log level.

  ``log-level NAME`` shows log level for the given service or for the kernel.

  ``log-level NAME VALUE`` sets the log level for the given service or for the kernel.
  ''VALUE'' is either the name of the log level or its numerical value.
; [!metrics|''metrics'']
; ''metrics''
: Displays some values that reflect the inner workings of Zettelstore.
  See [[here|https://golang.org/pkg/runtime/metrics/]] for a technical description of these values.
; [!next-config|''next-config'']
; ''next-config''
: Displays next configuration data.
  It will be the current configuration, if the corresponding services is restarted.

  ``next-config`` shows all next configuration data.

  ``next-config SERVICE`` shows only the next configuration data of the given service.

  ``next-config SERVICE KEY`` shows the next configuration data for the given service and key.
; [!profile|''profile [PROFILE] [FILE]'']
; ''profile [PROFILE] [FILE]''
: Starts to profile the software with the profile PROFILE and writes profiling data to file FILE.
  If PROFILE is not given, a value ''CPU'' is assumed, which specifies to profile CPU usage.
  If FILE is not given, a value ''PROFILE.prof'' will be used.

  Other values for ''PROFILE'' are: ''goroutine'', ''heap'', ''allocs'', ''threadcreate'', ''block'', and ''mutex''.
  In the future, more values may be appropriate.
  See the [[Go documentation|https://pkg.go.dev/runtime/pprof#Profile]] for details.

  This feature is dependent on the internal implementation language of Zettelstore, Go.
  It may be removed without any further notice at any time.
  In most cases, it is a tool for software developers to optimize Zettelstore's internal workings.
; [!restart|''restart SERVICE'']
; ''restart SERVICE''
: Restart the given service and all other that depend on this.
; [!services|''services'']
; ''services''
: Displays s list of all available services and their current status.
; [!set-config|''set-config SERVICE KEY VALUE'']
; ''set-config SERVICE KEY VALUE''
: Sets a single configuration value for the next configuration of a given service.
  It will become effective if the service is restarted.

  If the key specifies a list value, all other list values with a number greater than the given key are deleted.
  You can use the special number ""0"" to delete all values.
  E.g. ``set-config box box-uri-0 any_text`` will remove all values of the list __box-uri-__.
; [!shutdown|''shutdown'']
; ''shutdown''
: Terminate the Zettelstore itself (and closes the connection to the administrator console).
; [!start|''start SERVICE'']
: Start the given service and all dependent services.
; [!stat|''stat SERVICE'']
; ''start SERVICE''
: Start the given bservice and all dependent services.
; ''stat SERVICE''
: Display some statistical values for the given service.
; [!stop|''stop SERVICE'']
; ''stop SERVICE''
: Stop the given service and all other that depend on this.

Changes to docs/manual/00001006010000.zettel.

1
2
3
4
5
6

7
8
9
10
11

12
13
14
15
16
17
18
1
2
3
4
5

6
7
8
9
10

11
12
13
14
15
16
17
18





-
+




-
+







id: 00001006010000
title: Syntax of Metadata
role: manual
tags: #manual #syntax #zettelstore
syntax: zmk
modified: 20220218131923
modified: 20211103164152

The metadata of a zettel is a collection of key-value pairs.
The syntax roughly resembles the internal header of an email ([[RFC5322|https://tools.ietf.org/html/rfc5322]]).

The key is a sequence of alphanumeric characters, a hyphen-minus character (""''-''"", U+002D) is also allowed.
The key is a sequence of alphanumeric characters, a hyphen-minus character (""''-''"") is also allowed.
It begins at the first position of a new line.

A key is separated from its value either by
* a colon character (""'':''""),
* a non-empty sequence of space characters,
* a sequence of space characters, followed by a colon, followed by a sequence of space characters.

Changes to docs/manual/00001006020000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13

14
15

16
17
18

19
20
21

22
23
24

25
26
27

28
29
30
31
32

33
34

35
36

37
38

39
40
41

42
43
44
45
46

47
48
49

50
51
52
53
54


55

56
57
58

59
60
61
62
63
64
65
66
67
68

69
70
71

72
73
74
75
76

77
78
79
80

81
82
83


84
85
86
87
88

89
90
91
92

93
94
95
96

97
98
99
100
101

102
103
104
105
106

107
108
109
110
1
2
3
4
5

6
7
8
9
10
11
12

13
14

15
16
17

18
19
20

21
22
23

24
25
26

27
28
29
30
31

32
33

34
35

36
37

38
39
40

41
42
43
44
45

46
47
48

49
50
51
52
53
54
55
56

57
58
59

60
61
62
63
64
65
66
67
68
69

70
71
72

73
74
75
76
77

78
79
80
81

82
83


84
85
86
87
88
89

90
91
92
93

94
95
96
97

98
99
100
101
102

103
104
105
106
107

108
109
110
111
112





-
+






-
+

-
+


-
+


-
+


-
+


-
+




-
+

-
+

-
+

-
+


-
+




-
+


-
+





+
+
-
+


-
+









-
+


-
+




-
+



-
+

-
-
+
+




-
+



-
+



-
+




-
+




-
+




id: 00001006020000
title: Supported Metadata Keys
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20220218130146
modified: 20220111103609

Although you are free to define your own metadata, by using any key (according to the [[syntax|00001006010000]]), some keys have a special meaning that is enforced by Zettelstore.
See the [[computed list of supported metadata keys|00000000000090]] for details.

Most keys conform to a [[type|00001006030000]].

; [!all-tags|''all-tags'']
; [!all-tags]''all-tags''
: A property (a computed values that is not stored) that contains both the value of [[''tags''|#tags]] together with all [[tags|00001007040000#tag]] that are specified within the content.
; [!back|''back'']
; [!back]''back''
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata, that are not referenced by this zettel.
  Basically, it is the value of [[''backward''|#bachward]], but without any zettel identifier that is contained in [[''forward''|#forward]].
; [!backward|''backward'']
; [!backward]''backward''
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata.
  References within invertible values are not included here, e.g. [[''precursor''|#precursor]].
; [!box-number|''box-number'']
; [!box-number]''box-number''
: Is a computed value and contains the number of the box where the zettel was found.
  For all but the [[predefined zettel|00001005090000]], this number is equal to the number __X__ specified in startup configuration key [[''box-uri-__X__''|00001004010000#box-uri-x]].
; [!copyright|''copyright'']
; [!copyright]''copyright''
: Defines a copyright string that will be encoded.
  If not given, the value ''default-copyright'' from the  [[configuration zettel|00001004020000#default-copyright]] will be used.
; [!credential|''credential'']
; [!credential]''credential''
: Contains the hashed password, as it was emitted by [[``zettelstore password``|00001004051400]].
  It is internally created by hashing the password, the [[zettel identifier|00001006050000]], and the value of the ''ident'' key.

  It is only used for zettel with a ''role'' value of ""user"".
; [!dead|''dead'']
; [!dead]''dead''
: Property that contains all references that does __not__ identify a zettel.
; [!folge|''folge'']
; [!folge]''folge''
: Is a property that contains identifier of all zettel that reference this zettel through the [[''precursor''|#precursor]] value.
; [!forward|''forward'']
; [!forward]''forward''
: Property that contains all references that identify another zettel within the content of the zettel.
; [!id|''id'']
; [!id]''id''
: Contains the [[zettel identifier|00001006050000]], as given by the Zettelstore.
  It cannot be set manually, because it is a computed value.
; [!lang|''lang'']
; [!lang]''lang''
: Language for the zettel.
  Mostly used for HTML rendering of the zettel.
  If not given, the value ''default-lang'' from the  [[configuration zettel|00001004020000#default-lang]] will be used.
  Use values according to the language definition of [[RFC-5646|https://tools.ietf.org/html/rfc5646]].
; [!license|''license'']
; [!license]''license''
: Defines a license string that will be rendered.
  If not given, the value ''default-license'' from the [[configuration zettel|00001004020000#default-license]] will be used.
; [!modified|''modified'']
; [!modified]''modified''
: Date and time when a zettel was modified through Zettelstore.
  If you edit a zettel with an editor software outside Zettelstore, you should set it manually to an appropriate value.

  This is a computed value.
  There is no need to set it via Zettelstore.
; [!no-index]''no-index''
: If set to a ""true"" value, the zettel will not be indexed and therefore not be found in full-text searches.
; [!precursor|''precursor'']
; [!precursor]''precursor''
: References zettel for which this zettel is a ""Folgezettel"" / follow-up zettel.
  Basically the inverse of key [[''folge''|#folge]].
; [!published|''published'']
; [!published]''published''
: This property contains the timestamp of the mast modification / creation of the zettel.
  If [[''modified''|#modified]] is set, it contains the same value.
  Otherwise, if the zettel identifier contains a valid timestamp, the identifier is used.
  In all other cases, this property is not set.

  It can be used for [[sorting|00001012052000]] zettel based on their publication date.

  It is a computed value.
  There is no need to set it via Zettelstore.
; [!read-only|''read-only'']
; [!read-only]''read-only''
: Marks a zettel as read-only.
  The interpretation of [[supported values|00001006020400]] for this key depends, whether authentication is [[enabled|00001010040100]] or not.
; [!role|''role'']
; [!role]''role''
: Defines the role of the zettel.
  Can be used for selecting zettel.
  See [[supported zettel roles|00001006020100]].
  If not given, the value ''default-role'' from the [[configuration zettel|00001004020000#default-role]] will be used.
; [!syntax|''syntax'']
; [!syntax]''syntax''
: Specifies the syntax that should be used for interpreting the zettel.
  The zettel about [[other markup languages|00001008000000]] defines supported values.
  If not given, the value ''default-syntax'' from the [[configuration zettel|00001004020000#default-syntax]] will be used.
; [!tags|''tags'']
; [!tags]''tags''
: Contains a space separated list of tags to describe the zettel further.
  Each Tag must begin with the number sign character (""''#''"", U+0023).
; [!title|''title'']
  Each Tag must begin with the number sign character (""''#''"", ''U+0023'').
; [!title]''title''
: Specifies the title of the zettel.
  If not given, the value ''default-title'' from the [[configuration zettel|00001004020000#default-title]] will be used.

  You can use all [[inline-structured elements|00001007040000]] of Zettelmarkup.
; [!url|''url'']
; [!url]''url''
: Defines an URL / URI for this zettel that possibly references external material.
  One use case is to specify the document that the current zettel comments on.
  The URL will be rendered special in the [[web user interface|00001014000000]] if you use the default template.
; [!useless-files|''useless-files'']
; [!useless-files]''useless-files''
: Contains the file names that are rejected to serve the content of a zettel.
  Is used for [[directory boxes|00001004011400]] and [[file boxes|00001004011200#file]].
  If a zettel is renamed or deleted, these files will be deleted.
; [!user-id|''user-id'']
; [!user-id]''user-id''
: Provides some unique user identification for an [[user zettel|00001010040200]].
  It is used as a user name for authentication.

  It is only used for zettel with a ''role'' value of ""user"".
; [!user-role|''user-role'']
; [!user-role]''user-role''
: Defines the basic privileges of an authenticated user, e.g. reading / changing zettel.
  Is only valid in a user zettel.

  See [[User roles|00001010070300]] for more details.
; [!visibility|''visibility'']
; [!visibility]''visibility''
: When you work with authentication, you can give every zettel a value to decide, who can see the zettel.
  Its default value can be set with [[''default-visibility''|00001004020000#default-visibility]] of the configuration zettel.

  See [[visibility rules for zettel|00001010070200]] for more details.

Changes to docs/manual/00001006020100.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15

16
17
18

19
20
21
22
23
24

25
26
27
28

29
30
31

32
33
34
1
2
3
4
5

6
7
8
9
10
11
12
13
14

15
16
17

18
19
20
21
22
23

24
25
26
27

28
29
30

31
32
33
34





-
+








-
+


-
+





-
+



-
+


-
+



id: 00001006020100
title: Supported Zettel Roles
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20220214174553
modified: 20211127174441

The [[''role'' key|00001006020000#role]] defines what kind of zettel you are writing.
You are free to define your own roles.

The role ''zettel'' is predefined as the default role, but you can [[change this|00001004020000#default-role]].

Some roles are defined for technical reasons:

; [!configuration|''configuration'']
; [!configuration]''configuration''
: A zettel that contains some configuration data for the Zettelstore.
  Most prominent is [[00000000000100]], as described in [[00001004020000]].
; [!manual|''manual'']
; [!manual]''manual''
: All zettel that document the inner workings of the Zettelstore software.
  This role is only used in this specific Zettelstore.

If you adhere to the process outlined by Niklas Luhmann, a zettel could have one of the following three roles:

; [!note|''note'']
; [!note]''note''
: A small note, to remember something.
  Notes are not real zettel, they just help to create a real zettel.
  Think of them as Post-it notes.
; [!literature|''literature'']
; [!literature]''literature''
: Contains some remarks about a book, a paper, a web page, etc.
  You should add a citation key for citing it.
; [!zettel|''zettel'']
; [!zettel]''zettel''
: A real zettel that contains your own thoughts.

However, you are free to define additional roles, e.g. ''material'' for literature that is web-based only, ''slide'' for presentation slides, ''paper'' for the text of a scientific paper, ''project'' to define a project, ...

Changes to docs/manual/00001006030000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36
37
38
39
40
1
2
3
4
5

6
7
8
9
10
11
12
13
14


15
16
17

18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38





-
+








-
-



-








+












id: 00001006030000
title: Supported Key Types
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20220304114106
modified: 20220110161544

All [[supported metadata keys|00001006020000]] conform to a type.

User-defined metadata keys conform also to a type, based on the suffix of the key.

|=Suffix|Type
| ''-number'' | [[Number|00001006033000]]
| ''-role'' | [[Word|00001006035500]]
| ''-set'' | [[WordSet|00001006036000]]
| ''-title'' | [[Zettelmarkup|00001006036500]]
| ''-url'' | [[URL|00001006035000]]
| ''-zettel''  | [[Identifier|00001006032000]]
| ''-zid''  | [[Identifier|00001006032000]]
| ''-zids''  | [[IdentifierSet|00001006032500]]
| any other suffix | [[EString|00001006031500]]

The name of the metadata key is bound to the key type

Every key type has an associated validation rule to check values of the given type.
There is also a rule how values are matched, e.g. against a search term when selecting some zettel.
And there is a rule how values compare for sorting.

* [[Boolean|00001006030500]]
* [[Credential|00001006031000]]
* [[EString|00001006031500]]
* [[Identifier|00001006032000]]
* [[IdentifierSet|00001006032500]]
* [[Number|00001006033000]]
* [[String|00001006033500]]
* [[TagSet|00001006034000]]
* [[Timestamp|00001006034500]]
* [[URL|00001006035000]]
* [[Word|00001006035500]]
* [[WordSet|00001006036000]]
* [[Zettelmarkup|00001006036500]]

Changes to docs/manual/00001006030500.zettel.

1
2

3
4

5
6
7
8

9

10
11


12






13
14



1

2
3

4
5

6

7
8
9


10
11
12
13
14
15
16
17
18


19
20
21

-
+

-
+

-

-
+

+
-
-
+
+

+
+
+
+
+
+
-
-
+
+
+
id: 00001006030500
title: Boolean Value
title: Boolean Key Type
role: manual
tags: #manual #reference #zettel #zettelstore
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20220304114040

On some places, metadata values are interpreted as a truth value.
Values of this type denote a truth value.

=== Allowed values
Every character sequence that begins with a ""0"", ""F"", ""N"", ""f"", or a ""n"" is interpreted as the boolean ""false"" value.
All values are interpreted as the boolean ""true"" value.
Every character sequence that begins with a ""''0''"", ""''F''"", ""''N''"", ""''f''"", or a ""''n''"" is interpreted as the ""false"" boolean value.
All other metadata value is interpreted as the ""true"" boolean value.

=== Match operator
The match operator is the equals operator, i.e.
* ``(true == true) == true``
* ``(false == false) == true``
* ``(true == false) == false``
* ``(false == true) == false``
However, there is no full support for a boolean type.
Such values are interpreted as a [[string value|00001006033500]], e.g. with respect to comparing and sorting.

=== Sorting
The ""false"" value is less than the ""true"" value: ``false < true``

Changes to docs/manual/00001006034000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13

14
15
16
17
18
19
20
1
2
3
4
5

6
7
8
9
10
11
12

13
14
15
16
17
18
19
20





-
+






-
+







id: 00001006034000
title: TagSet Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20220218130413
modified: 20220111103723

Values of this type denote a (sorted) set of tags.

A set is different to a list, as no duplicate values are allowed.

=== Allowed values
Every tag must must begin with the number sign character (""''#''"", U+0023), followed by at least one printable character.
Every tag must must begin with the number sign character (""''#''"", ''U+0023''), followed by at least one printable character.
Tags are separated by space characters.

All characters are mapped to their lower case values.

=== Match operator
It depends of the first character of a search string how it is matched against a tag set value:

Changes to docs/manual/00001007020000.zettel.

1
2
3
4
5
6

7
8
9


10
11
12
13
14

15
16
17
18
19
20
21

1
2
3
4
5

6
7


8
9
10
11
12
13

14
15
16
17
18
19
20

21





-
+

-
-
+
+




-
+






-
+
id: 00001007020000
title: Zettelmarkup: Basic Definitions
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218130713
modified: 20211124175237

Every Zettelmarkup content consists of a sequence of Unicode code-points.
Unicode code-points are called in the following as **character**s.
Every zettelmark content consists of a sequence of Unicode codepoints.
Unicode codepoints are called in the following as **character**s.

Characters are encoded with UTF-8.

; Line
: A __line__ is a sequence of characters, except newline (U+000A) and carriage return (U+000D), followed by a line ending sequence or the end of content.
: A __line__ is a sequence of characters, except newline (''U+000A'') and carraige return (''U+000D''), followed by a line ending sequence or the end of content.
; Line ending
: A __line ending__ is either a newline not followed by a carriage return, a newline followed by a carriage return, or a carriage return.
  Different line can be finalized by different line endings.
; Empty line
: An __empty line__ is an empty sequence of characters, followed by a line ending or the end of content.
; Space character
: The __space character__ is the Unicode code-point U+0020.
: The __space character__ is the Unicode codepoint ''U+0020''.

Changes to docs/manual/00001007030100.zettel.

1
2
3
4
5
6

7
8
9
10
11

12
13
14

15
16
17
18
19
20
21
1
2
3
4
5

6
7
8
9
10

11
12
13

14
15
16
17
18
19
20
21





-
+




-
+


-
+







id: 00001007030100
title: Zettelmarkup: Description Lists
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131155
modified: 20211124172247

A description list is a sequence of terms to be described together with the descriptions of each term.
Every term can described in multiple ways.

A description term (short: __term__) is specified with one semicolon (""'';''"", U+003B) at the first position, followed by a space character and the described term, specified as a sequence of line elements.
A description term (short: __term__) is specified with one semicolon (""'';''"", ''U+003B'') at the first position, followed by a space character and the described term, specified as a sequence of line elements.
If the following lines should also be part of the term, exactly two spaces must be given at the beginning of each following line.

The description of a term is given with one colon (""'':''"", U+003A) at the first position, followed by a space character and the description itself, specified as a sequence of [[inline elements|00001007040000]].
The description of a term is given with one colon (""'':''"", ''U+003A'') at the first position, followed by a space character and the description itself, specified as a sequence of [[inline elements|00001007040000]].
Similar to terms, following lines can also be part of the actual description, if they begin at each line with exactly two space characters.

In contrast to terms, the actual descriptions are merged into a paragraph.
This is because, an actual description can contain more than one paragraph.
As usual, paragraphs are separated by an empty line.
Every following paragraph of an actual description must be indented by two space characters.

Changes to docs/manual/00001007030200.zettel.

1
2
3
4
5
6

7
8
9
10

11
12
13
14
15
16
17
1
2
3
4
5

6
7
8
9

10
11
12
13
14
15
16
17





-
+



-
+







id: 00001007030200
title: Zettelmarkup: Nested Lists
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133902
modified: 20211124172337

There are thee kinds of lists that can be nested: ordered lists, unordered lists, and quotation lists.

Ordered lists are specified with the number sign (""''#''"", U+0023), unordered lists use the asterisk (""''*''"", U+002A), and quotation lists are specified with the greater-than sing (""''>''"", U+003E).
Ordered lists are specified with the number sign (""''#''"", ''U+0023''), unordered lists use the asterisk (""''*''"", ''U+002A''), and quotation lists are specified with the greater-than sing (""''>''"", ''U+003E'').
Let's call these three characters __list characters__.

Any nested list item is specified by a non-empty sequence of list characters, followed by a space character and a sequence of [[inline elements|00001007040000]].
In case of a quotation list as the last list character, the space character followed by a sequence of inline elements is optional.
The number / count of list characters gives the nesting of the lists.
If the following lines should also be part of the list item, exactly the same number of spaces must be given at the beginning of each of the following lines as it is the lists are nested, plus one additional space character.
In other words: the inline elements must begin at the same column as it was on the previous line.

Changes to docs/manual/00001007030300.zettel.

1
2
3
4
5
6

7
8
9

10
11
12
13
14
15
16
1
2
3
4
5

6
7
8

9
10
11
12
13
14
15
16





-
+


-
+







id: 00001007030300
title: Zettelmarkup: Headings
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133755
modified: 20211124172401

To specify a (sub-) section of a zettel, you should use the headings syntax: at
the beginning of a new line type at least three equal signs (""''=''"", U+003D), plus at least one
the beginning of a new line type at least three equal signs (""''=''"", ''U+003D''), plus at least one
space and enter the text of the heading as [[inline elements|00001007040000]].

```zmk
=== Level 1 Heading
==== Level 2 Heading
===== Level 3 Heading
====== Level 4 Heading

Changes to docs/manual/00001007030400.zettel.

1
2
3
4
5
6

7
8
9


10
11
12
13

14
15
16
17
18
19

20
21
22
23
24
25
26
27

28
29
30
31
1
2
3
4
5

6
7


8
9
10
11
12

13
14
15
16
17
18

19
20
21
22
23
24
25
26

27
28
29
30
31





-
+

-
-
+
+



-
+





-
+







-
+




id: 00001007030400
title: Zettelmarkup: Horizontal Rules / Thematic Break
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133651
modified: 20211124174251

To signal a thematic break, you can specify a horizontal rule.
This is done by entering at least three hyphen-minus characters (""''-''"", U+002D) at the first position of a line.
To signal a thematic break, you can specify a horizonal rule.
This is done by entering at least three hyphen-minus characters (""''-''"", ''U+002D'') at the first position of a line.
You can add some [[attributes|00001007050000]], although the horizontal rule does not support the default attribute.
Any other character in this line will be ignored

If you do not enter the three hyphen-minus character at the very first position of a line, the are interpreted as [[inline elements|00001007040000]], typically as an ""en-dash" followed by a hyphen-minus.
If you do not enter the three hyphen-minus charachter at the very first position of a line, the are interpreted as [[inline elements|00001007040000]], typically as an ""en-dash" followed by a hyphen-minus.

Example:

```zmk
---
----{.zs-deprecated}
----{color=green}
-----
 --- inline
--- ignored
```
is rendered in HTML as
:::example
---
----{.zs-deprecated}
----{color=green}
-----
 --- inline
--- ignored
:::

Changes to docs/manual/00001007030500.zettel.

1
2
3
4
5
6

7
8
9
10


11
12
13

14
15
16
17
18
19
20
1
2
3
4
5

6
7
8


9
10
11
12

13
14
15
16
17
18
19
20





-
+


-
-
+
+


-
+







id: 00001007030500
title: Zettelmarkup: Verbatim Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131500
modified: 20211124170510

Verbatim blocks are used to enter text that should not be interpreted.
They begin with at least three grave accent characters (""''`''"", U+0060) at the first position of a line.
Alternatively, a modifier letter grave accent (""''ˋ''"", U+02CB) is also allowed[^On some devices, such as an iPhone / iPad, a grave accent character is harder to enter and is often confused with a modifier letter grave accent.].
They begin with at least three grave accent characters (""''`''"", ''U+0060'') at the first position of a line.
Alternatively, a modifier letter grave accent (""''ˋ''"", ''U+02CB'') is also allowed[^On some devices, such as an iPhone / iPad, a grave accent character is harder to enter and is often confused with a modifier letter grave accent.].

You can add some [[attributes|00001007050000]] on the beginning line of a verbatim block, following the initiating characters.
The verbatim block supports the default attribute: when given, all spaces in the text are rendered in HTML as open box characters (U+2423).
The verbatim block supports the default attribute: when given, all spaces in the text are rendered in HTML as open box characters (""''&#x2423;''"", ''U+2423'').
If you want to give only one attribute and this attribute is the generic attribute, you can omit the most of the attribute syntax and just specify the value.
It will be interpreted as a ([[programming|00001007050200]]) language to support colorizing the text when rendered in HTML.

Any other character in this line will be ignored

Text following the beginning line will not be interpreted, until a line begins with at least the same number of the same characters given at the beginning line.
This allows to enter some grave accent characters in the text that should not be interpreted.

Changes to docs/manual/00001007030600.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15
16
17
18
19
1
2
3
4
5

6
7
8
9
10
11

12
13
14
15
16
17
18
19





-
+





-
+







id: 00001007030600
title: Zettelmarkup: Quotation Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131806
modified: 20211124174017

A simple way to enter a quotation is to use the [[quotation list|00001007030200]].
A quotation list loosely follows the convention of quoting text within emails.
However, if you want to attribute the quotation to seomeone, a quotation block is more appropriately.

This kind of line-range block begins with at least three less-than characters (""''<''"", U+003C) at the first position of a line.
This kind of line-range block begins with at least three less-than characters (""''<''"", ''U+003C'') at the first position of a line.
You can add some [[attributes|00001007050000]] on the beginning line of a quotation block, following the initiating characters.
The quotation does not support the default attribute, nor the generic attribute.
Attributes are interpreted on HTML rendering.
Any other character in this line will be ignored

Text following the beginning line will be interpreted, until a line begins with at least the same number of the same characters given at the beginning line.
This allows to enter a quotation block within a quotation block.

Changes to docs/manual/00001007030700.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

33
34
35
36
37
38
39
40
41
42
43
44
45
46

47
48
49
50
51
52
1
2

3
4

5
6
7
8
9
10
11

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43
44

45
46
47
48
49
50
51


-


-
+






-
+


















-
+













-
+






id: 00001007030700
title: Zettelmarkup: Verse Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218132432
role: manual

Sometimes, you want to enter text with significant space characters at the beginning of each line and with significant line endings.
Poetry is one typical example.
Of course, you could help yourself with hard space characters and hard line breaks, by entering a backslash character before a space character and at the end of each line.
Using a verse block might be easier.

This kind of line-range block begins with at least three quotation mark characters (""''"''"", U+0022) at the first position of a line.
This kind of line-range block begins with at least three quotation mark characters (""''"''"", ''U+0022'') at the first position of a line.
You can add some [[attributes|00001007050000]] on the beginning line of a verse block, following the initiating characters.
The verse block does not support the default attribute, nor the generic attribute.
Attributes are interpreted on HTML rendering.
Any other character in this line will be ignored.

Text following the beginning line will be interpreted, until a line begins with at least the same number of the same characters given at the beginning line.
This allows to enter a verse block within a verse block.
At the ending line, you can enter some [[inline elements|00001007040000]] after the quotation mark characters.
These will interpreted as some attribution text.

For example:

```zmk
""""
A verse block with
 an
  embedded
   verse block
"""{.zs-deprecated}
"""{style=color:green}
Embedded
  verse
block
""" Embedded Author
"""" Verse Author
```
will be rendered as:
:::example
""""
A verse block with
 an
  embedded
   verse block
"""{.zs-deprecated}
"""{style=color:green}
Embedded
  verse
block
""" Embedded Author
"""" Verse Author
:::

Changes to docs/manual/00001007030800.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13

14
15
16
17
18
19
20
1
2
3
4
5

6
7
8
9
10
11
12

13
14
15
16
17
18
19
20





-
+






-
+







id: 00001007030800
title: Zettelmarkup: Region Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131131
modified: 20211124173940

Region blocks does not directly have a visual representation.
They just group a range of lines.
You can use region blocks to enter [[attributes|00001007050000]] that apply only to this range of lines.
One example is to enter a multi-line warning that should be visible.

This kind of line-range block begins with at least three colon characters (""'':''"", U+003A) at the first position of a line[^Since a [[description text|00001007030100]] only use exactly one colon character at the first position of a line, there is no possible ambiguity between these elements.].
This kind of line-range block begins with at least three colon characters (""'':''"", ''U+003A'') at the first position of a line[^Since a [[description text|00001007030100]] only use exactly one colon character at the first position of a line, there is no possible ambiguity between these elements.].
You can add some [[attributes|00001007050000]] on the beginning line of a region block, following the initiating characters.
The region block does not support the default attribute, but it supports the generic attribute.
Some generic attributes, like ``=note``, ``=warning`` will be rendered special.
Attributes are interpreted on HTML rendering.
Any other character in this line will be ignored.

Text following the beginning line will be interpreted, until a line begins with at least the same number of the same characters given at the beginning line.

Changes to docs/manual/00001007030900.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15
16
17
18
19
1
2

3
4

5
6
7
8
9
10

11
12
13
14
15
16
17
18


-


-
+





-
+







id: 00001007030900
title: Zettelmarkup: Comment Blocks
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218130330
role: manual

Comment blocks are quite similar to [[verbatim blocks|00001007030500]]: both are used to enter text that should not be interpreted.
While the text entered inside a verbatim block will be processed somehow, text inside a comment block will be ignored[^Well, not completely ignored: text is read, but it will typically not rendered visible.].
Comment blocks are typically used to give some internal comments, e.g. the license of a text or some internal remarks.

Comment blocks begin with at least three percent sign characters (""''%''"", U+0025) at the first position of a line.
Comment blocks begin with at least three percent sign characters (""''%''"", ''U+0025'') at the first position of a line.
You can add some [[attributes|00001007050000]] on the beginning line of a comment block, following the initiating characters.
The comment block supports the default attribute: when given, the text will be rendered, e.g. as an HTML comment.
When rendered to JSON, the comment block will not be ignored but it will output some JSON text.
Same for other renderers.

Any other character in this line will be ignored

Changes to docs/manual/00001007031000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57



58
59
60
61
62
63
64
1
2
3
4
5

6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54



55
56
57
58
59
60
61
62
63
64





-
+







-
+




















-
+



















-
-
-
+
+
+







id: 00001007031000
title: Zettelmarkup: Tables
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131107
modified: 20211124174218

Tables are used to show some data in a two-dimenensional fashion.
In zettelmarkup, table are not specified explicitly, but by entering __table rows__.
Therefore, a table can be seen as a sequence of table rows.
A table row is nothing as a sequence of __table cells__.
The length of a table is the number of table rows, the width of a table is the maximum length of its rows.

The first cell of a row must begin with the vertical bar character (""''|''"", U+007C) at the first position of a line.
The first cell of a row must begin with the vertical bar character (""''|''"", ''U+007C'') at the first position of a line.
The other cells of a row begin with the same vertical bar character at later positions in that line.
A cell is delimited by the vertical bar character of the next cell or by the end of the current line.
A vertical bar character as the last character of a line will not result in a table cell.
It will be ignored.
Inside a cell, you can specify any [[inline elements|00001007040000]].

For example:
```zmk
| a1 | a2 | a3|
| b1 | b2 | b3
| c1 | c2
```
will be rendered in HTML as:
:::example
| a1 | a2 | a3|
| b1 | b2 | b3
| c1 | c2
:::

=== Header row
If any cell in the first row of a table contains an equal sing character (""''=''"", U+003D) as the very first character, then this first row will be interpreted as a __table header__ row.
If any cell in the first row of a table contains an equal sing character (""''=''"", ''U+003D'') as the very first character, then this first row will be interpreted as a __table header__ row.

For example:
```zmk
| a1 | a2 |= a3|
| b1 | b2 | b3
| c1 | c2
```
will be rendered in HTML as:
:::example
| a1 | a2 |= a3|
| b1 | b2 | b3
| c1 | c2
:::

=== Column alignment
Inside a header row, you can specify the alignment of each header cell by a given character as the last character of a cell.
The alignment of a header cell determines the alignment of every cell in the same column.
The following characters specify the alignment:

* the colon character (""'':''"", U+003A) forces a centered alignment,
* the less-than sign character (""''<''"", U+0060) specifies an alignment to the left,
* the greater-than sign character (""''>''"", U+0062) will produce right aligned cells.
* the colon character (""'':''"", ''U+003A'') forces a centered aligment,
* the less-than sign character (""''<''"", ''U+0060'') specifies an alignment to the left,
* the greater-than sign character (""''>''"", ''U+0062'') will produce right aligned cells.

If no alignment character is given, a default alignment is used.

For example:
```zmk
|=Left<|Right>|Center:|Default
|123456|123456|123456|123456|
86
87
88
89
90
91
92
93

94
95
96
97
98
99
100
86
87
88
89
90
91
92

93
94
95
96
97
98
99
100







-
+







|=Left<|Right>|Center:|Default
|>R|:C|<L
|123456|123456|123456|123456|
|123|123|123|123
:::

=== Rows to be ignored
A line that begins with the sequence ''|%'' (vertical bar character (""''|''"", U+007C), followed by a percent sign character (“%”, U+0025)) will be ignored.
A line that begins with the sequence ''|%'' (vertical bar character (""''|''"", ''U+007C''), followed by a percent sign character (“%”, U+0025)) will be ignored.
For example, this allows to specify a horizontal rule that is not rendered.
Such tables are emitted by some commands of the [[administrator console|00001004100000]].
For example, the command ``get-config box`` will emit:
```
|=Key        | Value  | Description
|%-----------+--------+---------------------------
| defdirtype | notify | Default directory box type

Changes to docs/manual/00001007031100.zettel.

1
2
3
4
5
6

7
8
9
10

11
12
13
14
15
16
17
1
2
3
4
5

6
7
8
9

10
11
12
13
14
15
16
17





-
+



-
+







id: 00001007031100
title: Zettelmarkup: Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133058
modified: 20220201133837

A transclusion allows to include the content of another zettel into the current zettel just by referencing the other zettel.

The transclusion specification begins with three consecutive left curly bracket characters (""''{''"", U+007B) at the first position of a line and ends with three consecutive right curly bracket characters (""''}''"", U+007D).
The transclusion specification begins with three consecutive left curly bracket characters (""''{''"", ''U+007B'') at the first position of a line and ends with three consecutive right curly bracket characters (""''}''"", ''U+007D'').
The curly brackets delimit the [[zettel identifier|00001006050000]] to be included.

First, the referenced zettel is read.
If it contains some transclusions itself, these will be expanded, recursively.
When a recursion is detected, expansion does not take place.
Instead an error message replaces the transclude specification.

Changes to docs/manual/00001007031200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
1
2
3
4
5

6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
21





-
+







-
+







id: 00001007031200
title: Zettelmarkup: Inline-Zettel Block
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218172121
modified: 20220201151458

An inline-zettel block allows to specify some content with another syntax without creating a new zettel.
This is useful, for example, if you want to specify a [[simple drawing|00001008050000]] within your zettel and you are sure that you do not need the drawing in another context.
Another example is to embed some [[Markdown|00001008010500]] content, because you are too lazy to translate Markdown into Zettelmarkup.[^However, translating into Zettelmarkup is quite easy with the [[zmk encoder|00001012920522]].]
A last example is to specify HTML code to use it for some kind of web frontend framework.

As all other [[line-range blocks|00001007030000#line-range-blocks]], an inline-zettel block begins with at least three identical characters, starting at the first position of a line.
For inline-zettel blocks, the at-sign character (""''@''"", U+0040) is used.
For inline-zettel blocks, the at-sign character (""''@''"", ''U+0040'') is used.

You can add some [[attributes|00001007050000]] on the beginning line of a verbatim block, following the initiating characters.
The inline-zettel block uses the attribute key ""syntax"" to specify the [[syntax|00001008000000]] of the inline-zettel.
Alternatively, you can use the generic attribute to specify the syntax value.
If no value is provided, ""draw"" is assumed.

Any other character in this first line will be ignored.
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
53
54
55
56
57
58
59

60
61
62
63
64
65

66
67
68
69
70
71
72







-






-







@@@
:::

Using HTML:
```zmk
@@@html
<h1>H1 Heading</h1>
Alea iacta est
@@@
```
will a section heading of level 1, which is not allowed within Zettelmarkup:
:::example
@@@html
<h1>H1 Heading</h1>
Alea iacta est
@@@
:::
:::note
Please note: some HTML code will not be fully rendered because of possible security implications.
This include HTML lines that contain a ''<script>'' tag or an ''<iframe>'' tag.
:::
Of course, you do not need to switch the syntax and you are allowed to nest inline-zettel blocks:

Changes to docs/manual/00001007040000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
1
2
3
4
5

6
7
8
9
10
11
12
13





-
+







id: 00001007040000
title: Zettelmarkup: Inline-Structured Elements
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131736
modified: 20220201150815

Most characters you type is concerned with inline-structured elements.
The content of a zettel contains is many cases just ordinary text, lightly formatted.
Inline-structured elements allow to format your text and add some helpful links or images.
Sometimes, you want to enter characters that have no representation on your keyboard.

; Text formatting
24
25
26
27
28
29
30
31
32


33
34
35
36
37


38
39
40
41
42
43

44
45
46
47
48
49
50

51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

67
68
24
25
26
27
28
29
30


31
32
33
34
35


36
37
38
39
40
41
42

43
44
45
46
47
48
49

50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

66
67
68







-
-
+
+



-
-
+
+





-
+






-
+















-
+


  This kind of reference may be a link, or an images that is display inline when the zettel is rendered.
  Footnotes sometimes factor out some useful text that hinders the flow of reading text.
  Internal marks allow to reference something within a zettel.
  An important aspect of all knowledge work is to reference others work, e.g. with citation keys.
  All these elements can be subsumed under [[reference-like text|00001007040300]].

=== Other inline elements
==== Comment
A comment begins with two consecutive percent sign characters (""''%''"", U+0025).
==== Comments
A comment begins with two consecutive percent sign characters (""''%''"", ''U+0025'').
It ends at the end of the line where it begins.

==== Backslash
The backslash character (""''\\''"", U+005C) gives the next character another meaning.
* If a space character follows, it is converted in a non-breaking space (U+00A0).
The backslash character (""''\\''"", ''U+005C'') gives the next character another meaning.
* If a space character follows, it is converted in a non-breaking space (''U+00A0'').
* If a line ending follows the backslash character, the line break is converted from a __soft break__ into a __hard break__.
* Every other character is taken as itself, but without the interpretation of a Zettelmarkup element.
  For example, if you want to enter a ""'']''"" into a [[footnote text|00001007040330]], you should escape it with a backslash.

==== Tag
Any text that begins with a number sign character (""''#''"", U+0023), followed by a non-empty sequence of Unicode letters, Unicode digits, the hyphen-minus character (""''-''"", U+002D), or the low line character (""''_''"", U+005F) is interpreted as an __inline tag__.
Any text that begins with a number sign character (""''#''"", ''U+0023''), followed by a non-empty sequence of Unicode letters, Unicode digits, the hyphen-minus character (""''-''"", ''U+002D''), or the low line character (""''_''"", ''U+005F'') is interpreted as an __inline tag__.
They are be considered equivalent to tags in metadata.

==== Entities & more
Sometimes it is not easy to enter special characters.
If you know the Unicode code point of that character, or its name according to the [[HTML standard|https://html.spec.whatwg.org/multipage/named-characters.html]], you can enter it by number or by name.

Regardless which method you use, an entity always begins with an ampersand character (""''&''"", U+0026) and ends with a semicolon character (""'';''"", U+003B).
Regardless which method you use, an entity always begins with an ampersand character (""''&''"", ''U+0026'') and ends with a semicolon character (""'';''"", ''U+003B'').
If you know the HTML name of the character you want to enter, put it between these two character.
Example: ``&amp;`` is rendered as ::&amp;::{=example}.

If you want to enter its numeric code point, a number sign character must follow the ampersand character, followed by digits to base 10.
Example: ``&#38;`` is rendered in HTML as ::&#38;::{=example}.

You also can enter its numeric code point as a hex number, if you put the letter ""x"" after the numeric sign character.
Example: ``&#x26;`` is rendered in HTML as ::&#x26;::{=example}.

Since some Unicode character are used quite often, a special notation is introduced for them:

* Two consecutive hyphen-minus characters result in an __en-dash__ character.
  It is typically used in numeric ranges.
  ``pages 4--7`` will be rendered in HTML as: ::pages 4--7::{=example}.
  Alternative specifications are: ``&ndash;``, ``&x8211``, and ``&#x2013``.
* Three consecutive full stop characters (""''.''"", U+002E) after a space result in an horizontal ellipsis character.
* Three consecutive full stop characters (""''.''"", ''U+002E'') after a space result in an horizontal ellipsis character.
  ``to be continued ... later`` will be rendered in HTML as: ::to be continued, ... later::{=example}.
  Alternative specifications are: ``&hellip;``, ``&x8230``, and ``&#x2026``.

Changes to docs/manual/00001007040100.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17

18
19

20
21

22
23

24


25

26
27

28


29

30
31
32

33
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16

17
18

19
20

21
22

23
24
25
26

27
28

29
30
31
32

33
34
35

36
37





-
+










-
+

-
+

-
+

-
+

+
+
-
+

-
+

+
+
-
+


-
+

id: 00001007040100
title: Zettelmarkup: Text Formatting
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131003
modified: 20211113193440

Text formatting is the way to make your text visually different.
Every text formatting element begins with two same characters.
It ends when these two same characters occur the second time.
It is possible that some [[attributes|00001007050000]] follow immediately, without any separating character.

Text formatting can be nested, up to a reasonable limit.

The following characters begin a text formatting:

* The low line character (""''_''"", U+005F) emphasizes its text.
* The low line character (""''_''"", ''U+005F'') emphasizes its text.
** Example: ``abc __def__ ghi`` is rendered in HTML as: ::abc __def__ ghi::{=example}.
* The asterisk character (""''*''"", U+002A) strongly emphasized its enclosed text.
* The asterisk character (""''*''"", ''U+002A'') strongly emphasized its enclosed text.
** Example: ``abc **def** ghi`` is rendered in HTML as: ::abc **def** ghi::{=example}.
* The greater-than sign character (""''>''"", U+003E) marks text as inserted.
* The greater-than sign character (""''>''"", ''U+003E'') marks text as inserted.
** Example: ``abc >>def>> ghi`` is rendered in HTML as: ::abc >>def>> ghi::{=example}.
* Similar, the tilde character (""''~''"", U+007E) marks deleted text.
* Similar, the tilde character (""''~''"", ''U+007E'') marks deleted text.
** Example: ``abc ~~def~~ ghi`` is rendered in HTML as: ::abc ~~def~~ ghi::{=example}.
* The apostrophe character (""''\'''"", ''U+0027'') renders text in mono-space / fixed font width.
** Example: ``abc ''def'' ghi`` is rendered in HTML as: ::abc ''def'' ghi::{=example}.
* The circumflex accent character (""''^''"", U+005E) allows to enter superscripted text.
* The circumflex accent character (""''^''"", ''U+005E'') allows to enter superscripted text.
** Example: ``e=mc^^2^^`` is rendered in HTML as: ::e=mc^^2^^::{=example}.
* The comma character (""'',''"", U+002C) produces subscripted text.
* The comma character (""'',''"", ''U+002C'') produces subscripted text.
** Example: ``H,,2,,O`` is rendered in HTML as: ::H,,2,,O::{=example}.
* The less-than sign character (""''<''"", ''U+003C'') marks an inline quotation.
** Example: ``<<To be or not<<`` is rendered in HTML as: ::<<To be or not<<::{=example}.
* The quotation mark character (""''"''"", U+0022) marks an inline quotation, according to the [[specified language|00001007050100]].
* The quotation mark character (""''"''"", ''U+0022'') produces the right typographic quotation marks according to the [[specified language|00001007050100]].
** Example: ``""To be or not""`` is rendered in HTML as: ::""To be or not""::{=example}.
** Example: ``""Sein oder nicht""{lang=de}`` is rendered in HTML as: ::""Sein oder nicht""{lang=de}::{=example}.
* The colon character (""'':''"", U+003A) mark some text that should belong together. It fills a similar role as [[region blocks|00001007030800]], but just for inline elements.
* The colon character (""'':''"", ''U+003A'') mark some text that should belong together. It fills a similar role as [[region blocks|00001007030800]], but just for inline elements.
** Example: ``abc ::def::{=example} ghi`` is rendered in HTML as: abc ::def::{=example} ghi.

Changes to docs/manual/00001007040200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16


17
18
19

20
21
22
23
24
25
26
27
28
29
30
31
32


33
34
35
36


37
38
39
40
41

42
43
44
45
46
47
48
49
50

51
52
53
54
55
56
57
58
59
60
61
1
2
3
4
5

6
7
8
9
10
11
12
13
14


15
16
17
18

19
20
21
22
23
24
25
26
27
28
29
30


31
32
33
34


35
36
37
38
39
40

41
42
43
44
45
46
47
48
49

50
51
52
53
54
55
56
57
58
59
60
61





-
+








-
-
+
+


-
+











-
-
+
+


-
-
+
+




-
+








-
+











id: 00001007040200
title: Zettelmarkup: Literal-like formatting
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131420
modified: 20220201152059

There are some reasons to mark text that should be rendered as uninterpreted:
* Mark text as literal, sometimes as part of a program.
* Mark text as input you give into a computer via a keyboard.
* Mark text as output from some computer, e.g. shown at the command line.

=== Literal text
Literal text somehow relates to [[verbatim blocks|00001007030500]]: their content should not be interpreted further, but may be rendered special.
It is specified by two grave accent characters (""''`''"", U+0060), followed by the text, followed by again two grave accent characters, optionally followed by an [[attribute|00001007050000]] specification.
Similar to the verbatim block, the literal element allows also a modifier letter grave accent (""''ˋ''"", U+02CB) as an alternative to the grave accent character[^On some devices, such as an iPhone / iPad, a grave accent character is harder to enter and is often confused with a modifier letter grave accent.].
It is specified by two grave accent characters (""''`''"", ''U+0060''), followed by the text, followed by again two grave accent characters, optionally followed by an [[attribute|00001007050000]] specification.
Similar to the verbatim block, the literal element allows also a modifier letter grave accent (""''ˋ''"", ''U+02CB'') as an alternative to the grave accent character[^On some devices, such as an iPhone / iPad, a grave accent character is harder to enter and is often confused with a modifier letter grave accent.].
However, all four characters must be the same.

The literal element supports the default attribute: when given, all spaces in the text are rendered in HTML as open box characters (U+2423).
The literal element supports the default attribute: when given, all spaces in the text are rendered in HTML as open box characters (""''&#x2423;''"", ''U+2423'').
The use of a generic attribute allows to specify a ([[programming|00001007050200]]) language that controls syntax coloring when rendered in HTML.

If you want to specify a grave accent character in the text, either use modifier grave accent characters as delimiters for the element, or put a backslash character before the grave accent character you want to use inside the element.
If you want to enter a backslash character, you need to enter two of these.

Examples:
* ``\`\`abc def\`\``` is rendered in HTML as ::``abc def``::{=example}.
* ``\`\`abc def\`\`{-}`` is rendered in HTML as ::``abc def``{-}::{=example}.
* ``\`\`abc\\\`def\`\``` is rendered in HTML as ::``abc\`def``::{=example}.
* ``\`\`abc\\\\def\`\``` is rendered in HTML as ::``abc\\def``::{=example}.

=== Computer input
To mark text as input into a computer program, delimit your text with two apostrophe characters (""''\'''"", U+0027) on each side.
=== Keyboard input
To mark text as input into a computer program, delimit your text with two plus sign characters (""''+''"", ''U+002B'') on each side.

Example:
* ``''STRG-C''`` renders in HTML as ::''STRG-C''::{=example}.
* ``''STRG C''{-}`` renders in HTML as ::''STRG C''{-}::{=example}.
* ``++STRG-C++`` renders in HTML as ::++STRG-C++::{=example}.
* ``++STRG C++{-}`` renders in HTML as ::++STRG C++{-}::{=example}.

Attributes can be specified, the default attribute has the same semantic as for literal text.

=== Computer output
To mark text as output from a computer program, delimit your text with two equal sign characters (""''=''"", U+003D) on each side.
To mark text as output from a computer program, delimit your text with two equal sign characters (""''=''"", ''U+003D'') on each side.

Examples:
* ``==The result is: 42==`` renders in HTML as ::==The result is: 42==::{=example}.
* ``==The result is: 42=={-}`` renders in HTML as ::==The result is: 42=={-}::{=example}.

Attributes can be specified, the default attribute has the same semantic as for literal text.

=== Inline-zettel snippet
To specify an inline snippet in a different [[syntax|00001008000000]], delimit your text with two at-sign characters (""''@''"", U+0040) on each side.
To specify an inline snippet in a different [[syntax|00001008000000]], delimit your text with two at-sign characters (""''@''"", ''U+0040'') on each side.

You can add some [[attributes|00001007050000]] immediate after the two closing at-sign characters to specify the syntax to use.
Either use the attribute key ""syntax"" or use the generic attribute to specify the syntax value.
If no value is provided, ""draw"" is assumed.

Examples:
* ``A @@-->@@ B`` renders in HTML as ::A @@-->@@ B::{=example}.
* ``@@<small>@@{=html}Small@@</small>@@{=html}`` renders in HTML as ::@@<small>@@{=html}Small@@</small>@@{=html}::{=example}.

To some degree, an inline-zettel snippet is the @@<small>@@{=html}smaller@@</small>@@{=html} sibling of the [[inline-zettel block|00001007031200]].
For HTML syntax, the same rules apply.

Changes to docs/manual/00001007040310.zettel.

1
2
3
4
5
6

7
8
9

10
11

12
13
14
15
16
17

18
19
20
21

22
23
24
1
2
3
4
5

6
7
8

9
10

11
12
13
14
15
16

17
18
19
20

21
22
23
24





-
+


-
+

-
+





-
+



-
+



id: 00001007040310
title: Zettelmarkup: Links
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131639
modified: 20211124172143

There are two kinds of links, regardless of links to (internal) other zettel or to (external) material.
Both kinds begin with two consecutive left square bracket characters (""''[''"", U+005B) and ends with two consecutive right square bracket characters (""'']''"", U+005D).
Both kinds begin with two consecutive left square bracket characters (""''[''"", ''U+005B'') and ends with two consecutive right square bracket characters (""'']''"", ''U+005D'').

The first form provides some text plus the link specification, delimited by a vertical bar character (""''|''"", U+007C): ``[[text|linkspecification]]``.
The first form provides some text plus the link specification, delimited by a vertical bar character (""''|''"", ''U+007C''): ``[[text|linkspecification]]``.

The second form just provides a link specification between the square brackets.
Its text is derived from the link specification, e.g. by interpreting the link specification as text: ``[[linkspecification]]``.

The link specification for another zettel within the same Zettelstore is just the [[zettel identifier|00001006050000]].
To reference some content within a zettel, you can append a number sign character (""''#''"", U+0023) and the name of the mark to the zettel identifier.
To reference some content within a zettel, you can append a number sign character (""''#''"", ''U+0023'') and the name of the mark to the zettel identifier.
The resulting reference is called ""zettel reference"".

To specify some material outside the Zettelstore, just use an normal Uniform Resource Identifier (URI) as defined by [[RFC\ 3986|https://tools.ietf.org/html/rfc3986]].
If the URL begins with the slash character (""/"", U+002F), or if it begins with ""./"" or with ""../"", i.e. without scheme, user info, and host name, the reference will be treated as a ""local reference"", otherwise as an ""external reference"".
If the URL begins with the slash character (""/"", ''U+002F''), or if it begins with ""./"" or with ""../"", i.e. without scheme, user info, and host name, the reference will be treated as a ""local reference"", otherwise as an ""external reference"".
If the URL begins with two slash characters, it will be interpreted relative to the value of [[''url-prefix''|00001004010000#url-prefix]].

The text in the second form is just a sequence of [[inline elements|00001007040000]].

Changes to docs/manual/00001007040320.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23
24
25
1
2
3
4
5

6
7
8
9
10
11
12

13
14
15
16
17
18
19
20
21
22
23
24
25





-
+






-
+












id: 00001007040320
title: Zettelmarkup: Inline Embedding / Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133039
modified: 20220131153953

To some degree, an specification for embedded material is conceptually not too far away from a specification for [[linked material|00001007040310]].
Both contain a reference specification and optionally some text.
In contrast to a link, the specification of embedded material must currently resolve to some kind of real content.
This content replaces the embed specification.

An embed specification begins with two consecutive left curly bracket characters (""''{''"", U+007B) and ends with two consecutive right curly bracket characters (""''}''"", U+007D).
An embed specification begins with two consecutive left curly bracket characters (""''{''"", ''U+007B'') and ends with two consecutive right curly bracket characters (""''}''"", ''U+007D'').
The curly brackets delimits either a reference specification or some text, a vertical bar character and the link specification, similar to a link.

One difference to a link: if the text was not given, an empty string is assumed.

The reference must point to some content, either zettel content or URL-referenced content.
If the referenced zettel does not exist, or is not readable, a [[spinning emoji|00000000040001]] is presented as a visual hint:
 
Example: ``{{00000000000000}}`` will be rendered as ::{{00000000000000}}::{=example}.

There are two kind of content:
# [[image content|00001007040322]],
# [[textual content|00001007040324]].

Changes to docs/manual/00001007040322.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26





-
+

















-
+


id: 00001007040322
title: Zettelmarkup: Image Embedding
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220214180955
modified: 20211103163657

Image content is assumed, if an URL is used or if the referenced zettel contains an image.

Supported formats are:

* Portable Network Graphics (""PNG""), as defined by [[RFC\ 2083|https://tools.ietf.org/html/rfc2083]].
* Graphics Interchange Format (""GIF"), as defined by [[https://www.w3.org/Graphics/GIF/spec-gif89a.txt]].
* JPEG / JPG, defined by the __Joint Photographic Experts Group__.
* Scalable Vector Graphics (SVG), defined by [[https://www.w3.org/Graphics/SVG/]]

If the text is given, it will be interpreted as an alternative textual representation, to help persons with some visual disabilities.

[[Attributes|00001007050000]] are supported.
They must follow the last right curly bracket character immediately.
One prominent example is to specify an explicit title attribute that is shown on certain web browsers when the zettel is rendered in HTML:

Examples:
* [!spin|``{{Spinning Emoji|00000000040001}}{title=Emoji width=30}``] is rendered as ::{{Spinning Emoji|00000000040001}}{title=Emoji width=30}::{=example}.
* [!spin] ``{{Spinning Emoji|00000000040001}}{title=Emoji width=30}`` is rendered as ::{{Spinning Emoji|00000000040001}}{title=Emoji width=30}::{=example}.
* The above image is also the placeholder for a non-existent zettel:
** ``{{00000000009999}}`` will be rendered as ::{{00000000009999}}::{=example}.

Changes to docs/manual/00001007040330.zettel.

1
2
3
4
5
6
7
8

9
10
11
12
1
2
3
4
5

6

7
8
9
10
11





-

-
+




id: 00001007040330
title: Zettelmarkup: Footnotes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218130100

A footnote begins with a left square bracket, followed by a circumflex accent (""''^''"", U+005E), followed by some text, and ends with a right square bracket.
A footnote begins with a left square bracket, followed by a circumflex accent (""''^''"", ''U+005E''), followed by some text, and ends with a right square bracket.

Example:

``Main text[^Footnote text.].`` is rendered in HTML as: ::Main text[^Footnote text.].::{=example}.

Changes to docs/manual/00001007040340.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14

15
1
2
3
4
5

6
7
8
9
10
11

12
13

14
15





-
+





-
+

-
+

id: 00001007040340
title: Zettelmarkup: Citation Key
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133447
modified: 20211124172111

A citation key references some external material that is part of a bibliografical collection.

Currently, Zettelstore implements this only partially, it is ""work in progress"".

However, the syntax is: beginning with a left square bracket and followed by an at sign character (""''@''"", U+0040), a the citation key is given.
However, the syntax is: beginning with a left square bracket and followed by an at sign character (""''@''"", ''U+0040''), a the citation key is given.
The key is typically a sequence of letters and digits.
If a comma character (""'',''"", U+002C) or a vertical bar character is given, the following is interpreted as [[inline elements|00001007040000]].
If a comma character (""'',''"", ''U+002C'') or a vertical bar character is given, the following is interpreted as [[inline elements|00001007040000]].
A right square bracket ends the text and the citation key element.

Changes to docs/manual/00001007040350.zettel.

1
2
3
4
5
6

7
8
9

10
11

12
13

14
15
16
17
18
19
20
21
1
2
3
4
5

6
7
8

9
10

11
12

13

14
15
16
17
18







-
+


-
+

-
+

-
+
-





-
-
id: 00001007040350
title: Zettelmarkup: Mark
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133206
modified: 20211103163338

A mark allows to name a point within a zettel.
This is useful if you want to reference some content in a zettel, either with a [[link|00001007040310]] or with an [[inline-mode transclusion|00001007040324]].
This is useful if you want to reference some content in a bigger-sized zettel, currently with a [[link|00001007040310]] only[^Other uses of marks will be given, if Zettelmarkup is extended by a concept called __transclusion__.].

A mark begins with a left square bracket, followed by an exclamation mark character (""''!''"", U+0021).
A mark begins with a left square bracket, followed by an exclamation mark character (""''!''"", ''U+0021'').
Now the optional mark name follows.
It is a (possibly empty) sequence of Unicode letters, Unicode digits, the hyphen-minus character (""''-''"", U+002D), or the low-line character (""''_''"", U+005F).
It is a (possibly empty) sequence of Unicode letters, Unicode digits, the hyphen-minus character (""''-''"", ''U+002D''), or the low-line character (""''_''"", ''U+005F'').
An optional text to be explicitly marked is introduced with a vertical bar character (""''|''"", U+007C), followed by some [[inline-structured elements|00001007040000]].
The mark element ends with a right square bracket.

Examples:
* ``[!]`` is a mark without a name, the empty mark.
* ``[!mark]`` is a mark with the name ""mark"".
* ``[!|some text]``is the empty mark with ""some text"".
* ``[!mark|some text]``is a mark with the name ""mark"" and with ""some text"" that is marked.

Changes to docs/manual/00001007050000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19

20
21
22


23
24
25
26
27
28
29
30
31
32
33
34

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55


56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

76
77
78
79

80
81
82
83

84
85
86
87
88

89
90
91
92
93
94
95

96
97
98

99
100
101
102
103

104
105
106
107
108
109

110
111
112
113
114
115
116
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18

19
20


21
22
23
24
25
26
27
28
29
30
31
32
33

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53


54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

75
76
77
78

79
80
81
82

83
84
85
86
87

88
89
90
91
92
93
94

95
96
97

98
99
100
101
102

103
104
105
106
107
108

109
110
111
112
113
114
115
116





-
+












-
+

-
-
+
+











-
+



















-
-
+
+



















-
+



-
+



-
+




-
+






-
+


-
+




-
+





-
+







id: 00001007050000
title: Zettelmarkup: Attributes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218132935
modified: 20211124174947

Attributes allows to modify the way how material is presented.
Alternatively, they provide additional information to markup elements.
To some degree, attributes are similar to [[HTML attributes|https://html.spec.whatwg.org/multipage/dom.html#global-attributes]].

Typical use cases for attributes are to specify the (natural) [[language|00001007050100]] for a text region, to specify the [[programming language|00001007050200]] for highlighting program code, or to make white space visible in plain text.

Attributes are specified within curly brackets ``{...}``.
Of course, more than one attribute can be specified.
Attributes are separated by a sequence of space characters or by a comma character.

An attribute normally consists of an optional key and an optional value.
The key is a sequence of letters, digits, a hyphen-minus (""''-''"", U+002D, and a low line / underscore (""''_''"", U+005D).
The key is a sequence of letters, digits, a hyphen-minus (""''-''"", ''U+002D'', and a low line / underscore (""''_''"", ''U+005D'').
It can be empty.
The value is a sequence of any character, except space and the right curly bracket (""''}''"", U+007D).
If the value must contain a space or the right curly bracket, the value can be specified within two quotation marks (""''"''"", U+0022).
The value is a sequence of any character, except space and the right curly bracket (""''}''"", ''U+007D'').
If the value must contain a space or the right curly bracket, the value can be specified within two quotation marks (""''"''"", ''U+0022'').
Within the quotation marks, the backslash character functions as an escape character to specify the quotation mark (and the backslash character too).

Some examples:

* ``{key=value}`` sets the attribute __key__ to value __value__.
* ``{key="value with space"}`` sets the attribute to the given value.
* ``{key="value with quote \\" (and backslash \\\\)"}``
* ``{name}`` sets the attribute __name__.
  It has no corresponding value.
  It is equivalent to ``{name=}``.
* ``{=key}`` sets the __generic attribute__ to the given value.
  It is mostly used for modifying behavior according to a programming language.
  It is mostly used for modifying behaviour according to a programming language.
* ``{.key}`` sets the __class attribute__ to the given value.
  It is equivalent to ``{class=key}``. 

In these examples, ``key`` must conform the the syntax of attribute keys, even if it is used as a value.

If a key is given more than once in an attribute, the values are concatenated (and separated by a space).

* ``{key=value1 key=value2}`` is the same as ``{key"value1 value2"}``.
* ``{key key}`` is the same as ``{key}``.
* ``{.class1 .class2}`` is equivalent to ``{class="class1 class2"}``.

This is not true for the generic attribute.
In ``{=key1 =key2}``, the first key is ignored.
Therefore it is equivalent to ``{=key2}``.

The key ""''-''"" (just hyphen-minus) is special.
It is called __default attribute__ and has a markup specific meaning.
For example, when used for plain text, it replaces the non-visible space with a visible representation:

* ''``Hello, world``{-}'' produces ==Hello, world=={-}.
* ''``Hello, world``'' produces ==Hello, world==.
* ++``Hello, world``{-}++ produces ==Hello, world=={-}.
* ++``Hello, world``++ produces ==Hello, world==.

For some [[block-structured elements|00001007030000]], there is a syntax variant if you only want to specify a generic attribute.
For all line-range blocks you can specify the generic attributes directly in the first line, after the three (or more) block characters.

```
:::attr
...
:::
```
is equivalent to
```
:::{=attr}
...
:::
```.

For other blocks, the closing curly bracket must be on the same line where the block element begins.
However, spaces are allowed between the blocks characters and the attributes.
```
=== Heading {example}
=== Heading {style=color:green}
```
is allowed and equivalent to
```
=== Heading{example}
=== Heading{style=color:green}
```.
But
```
=== Heading {class=example
=== Heading {style=color:green
background=grey}
```
is not allowed. Same for
```
=== Heading {background=color:"
=== Heading {style=color:"
green"}
```.

For [[inline-structued elements|00001007040000]], the attributes must immediately follow the inline markup.
However, the attributes may be continued on the next line when a space or line ending character is possible.

``::GREEN::{example}`` is allowed, but not ``::GREEN:: {example}``.
``::GREEN::{style=color:green}`` is allowed, but not ``::GREEN:: {style=color:green}``.

```
::GREEN::{class=example
::GREEN::{style=color:green
background=grey}
```
is allowed, but not
```
::GREEN::{background=color:
::GREEN::{style=color:
green}
```.

However,
```
::GREEN::{background=color:"
::GREEN::{style=color:"
green"}
```
is allowed, because line endings are allowed within quotes.

=== Reference material
* [[Supported attribute values for natural languages|00001007050100]]
* [[Supported attribute values for programming languages|00001007050200]]

Changes to docs/manual/00001007060000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15
16
17

18
19
20
21

22
23
24
25
26
27
28

29
30
31
32
33
34
35
1
2
3
4
5

6
7
8
9
10
11

12
13
14
15
16

17
18
19
20

21
22
23
24
25
26
27

28
29
30
31
32
33
34
35





-
+





-
+




-
+



-
+






-
+







id: 00001007060000
title: Zettelmarkup: Summary of Formatting Characters
role: manual
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218124943
modified: 20220201151829

The following table gives an overview about the use of all characters that begin a markup element.

|= Character :|= [[Blocks|00001007030000]] <|= [[Inlines|00001007040000]] <
| ''!''  | (free) | (free)
| ''"''  | [[Verse block|00001007030700]] | [[Short inline quote|00001007040100]]
| ''"''  | [[Verse block|00001007030700]] | [[Typographic quotation mark|00001007040100]]
| ''#''  | [[Ordered list|00001007030200]] | [[Tag|00001007040000]]
| ''$''  | (reserved) | (reserved)
| ''%''  | [[Comment block|00001007030900]] | [[Comment|00001007040000]]
| ''&''  | (free) | [[Entity|00001007040000]]
| ''\'''  | (free)  | [[Computer input|00001007040200]]
| ''\'''  | (free)  | [[Monospace text|00001007040100]]
| ''(''  | (free) | (free)
| '')''  | (free) | (free)
| ''*''  | [[Unordered list|00001007030200]] | [[strongly emphasized text|00001007040100]]
| ''+''  | (free) | (free)
| ''+''  | (free) | [[Keyboard input|00001007040200]]
| '',''  | (free) | [[Subscripted text|00001007040100]]
| ''-''  | [[Horizonal rule|00001007030400]] | ""[[en-dash|00001007040000]]""
| ''.''  | (free) | [[Horizontal ellipsis|00001007040000]]
| ''/''  | (free) | (free)
| '':''  | [[Region block|00001007030800]] / [[description text|00001007030100]] | [[Inline region|00001007040100]]
| '';''  | [[Description term|00001007030100]] | [[Small text|00001007040100]]
| ''<''  | [[Quotation block|00001007030600]] | (free)
| ''<''  | [[Quotation block|00001007030600]] | [[Short inline quote|00001007040100]]
| ''=''  | [[Headings|00001007030300]] | [[Computer output|00001007040200]]
| ''>''  | [[Quotation lists|00001007030200]] | [[Inserted text|00001007040100]]
| ''?''  | (free) | (free)
| ''@''  | [[Inline-Zettel blocks|00001007031200]] | [[Inline-zettel snippets|00001007040200#inline-zettel-snippet]]
| ''[''  | (reserved)  | [[Linked material|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]
| ''\\'' | (blocked by inline meaning) | [[Escape character|00001007040000]]
| '']''  | (reserved) | End of [[link|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]

Changes to docs/manual/00001008000000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25

26
27

28
29
30

31
32
33
34
35
36

37
38
39
40

41
42

43
44
45
46
47

48
49

50
51

52
53
54
55
56
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

23
24

25
26

27
28
29

30
31
32
33
34
35

36
37
38
39

40
41

42
43
44
45
46

47
48

49
50

51
52
53
54
55
56





-
+
















-
+

-
+

-
+


-
+





-
+



-
+

-
+




-
+

-
+

-
+





id: 00001008000000
title: Other Markup Languages
role: manual
tags: #manual #zettelstore
syntax: zmk
modified: 20220214180202
modified: 20220131141205

[[Zettelmarkup|00001007000000]] is not the only markup language you can use to define your content.
Zettelstore is quite agnostic with respect to markup languages.
Of course, Zettelmarkup plays an important role.
However, with the exception of zettel titles, you can use any (markup) language that is supported:

* CSS
* HTML template data
* Image formats: GIF, PNG, JPEG, SVG
* Markdown
* Plain text, not further interpreted

The [[metadata key|00001006020000#syntax]] ""''syntax''"" specifies which language should be used.
If it is not given, the key ""''default-syntax''"" will be used (specified in the [[configuration zettel|00001004020000#default-syntax]]).
The following syntax values are supported:

; [!css|''css'']
; [!css]''css''
: A [[Cascading Style Sheet|https://www.w3.org/Style/CSS/]], to be used when rendering a zettel as HTML.
; [!draw|''draw'']
; [!draw]''draw''
: A simple [[language|00001008050000]] to ""draw"" a graphic by using some simple Unicode characters.
; [!gif|''gif'']; [!jpeg|''jpeg'']; [!jpg|''jpg'']; [!png|''png'']
; [!gif]''gif''; [!jpeg]''jpeg''; [!jpg]''jpg''; [!png]''png''
: The formats for pixel graphics.
  Typically the data is stored in a separate file and the syntax is given in the metafile, which has the same name as the zettel identifier and has no file extension.[^Before version 0.2.0, the metafile had the file extension ''.meta'']
; [!html|''html'']
; [!html]''html''
: Hypertext Markup Language, will not be parsed further.
  Instead, it is treated as [[text|#text]], but will be encoded differently for [[HTML format|00001012920510]] (same for the [[web user interface|00001014000000]]).

  For security reasons, equivocal elements will not be encoded in the HTML format / web user interface, e.g. the ``<script ...`` tag.
  See [[security aspects of Markdown|00001008010000#security-aspects]] for some details.
; [!markdown|''markdown''], [!md|''md'']
; [!markdown]''markdown'', [!md]''md''
: For those who desperately need [[Markdown|https://daringfireball.net/projects/markdown/]].
  Since the world of Markdown is so diverse, a [[CommonMark|00001008010500]] parser is used.
  See [[Use Markdown within Zettelstore|00001008010000]].
; [!mustache|''mustache'']
; [!mustache]''mustache''
: A [[Mustache template|https://mustache.github.io/]], used when rendering a zettel as HTML for the [[web user interface|00001014000000]].
; [!none|''none'']
; [!none]''none''
: Only the metadata of a zettel is ""parsed"".
  Useful for displaying the full metadata.
  The [[runtime configuration zettel|00000000000100]] uses this syntax.
  The zettel content is ignored.
; [!svg|''svg'']
; [!svg]''svg''
: A [[Scalable Vector Graphics|https://www.w3.org/TR/SVG2/]].
; [!text|''text''], [!plain|''plain''], [!txt|''txt'']
; [!text]''text'', [!plain]''plain'', [!txt]''txt''
: Just plain text that must not be interpreted further.
; [!zmk|''zmk'']
; [!zmk]''zmk''
: [[Zettelmarkup|00001007000000]].

The actual values are also listed in a zettel named [[Zettelstore Supported Parser|00000000000092]].

If you specify something else, your content will be interpreted as plain text.

Changes to docs/manual/00001008050000.zettel.

1
2
3
4
5
6

7
8

9
10
11
12
13
14
15
1
2
3
4
5

6
7

8
9
10
11
12
13
14
15





-
+

-
+







id: 00001008050000
title: The ""draw"" language
role: manual
tags: #graphic #manual #zettelstore
syntax: zmk
modified: 20220217180713
modified: 20220201133222

Sometimes, ""a picture is worth a thousand words"".
Sometimes, <<a picture is worth a thousand words<<.
To create some graphical representations, Zettelstore provides a simple mechanism.
Characters like ""''|''"" or ""''-''"" already provide some visual feedback.
For example, to create a picture containing two boxes that are connected via an arrow, the following representation is possible:
```
+-------+       +-------+
| Box 1 | ----> | Box 2 |
+-------+       +-------+

Changes to docs/manual/00001010070200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14

15
16
17
18

19
20

21
22
23
24
25


26
27
28
29
30
31
32
1
2
3
4
5

6
7
8
9
10
11

12
13

14
15
16
17

18
19

20
21
22
23


24
25
26
27
28
29
30
31
32





-
+





-
+

-
+



-
+

-
+



-
-
+
+







id: 00001010070200
title: Visibility rules for zettel
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
modified: 20220304114501
modified: 20211126185655

For every zettel you can specify under which condition the zettel is visible to others.
This is controlled with the metadata key [[''visibility''|00001006020000#visibility]].
The following values are supported:

; [!public|""public""]
; [!public]""public""
: The zettel is visible to everybody, even if the user is not authenticated.
; [!login|""login""]
; [!login]""login""
: Only an authenticated user can access the zettel.

  This is the default value for [[''default-visibility''|00001004020000#default-visibility]].
; [!creator|""creator""]
; [!creator]""creator""
: Only an authenticated user that is allowed to create new zettel can access the zettel.
; [!owner|""owner""]
; [!owner]""owner""
: Only the owner of the Zettelstore can access the zettel.

  This is for zettel with sensitive content, e.g. the [[configuration zettel|00001004020000]] or the various zettel that contains the templates for rendering zettel in HTML.
; [!expert|""expert""]
: Only the owner of the Zettelstore can access the zettel, if runtime configuration [[''expert-mode''|00001004020000#expert-mode]] is set to a [[boolean true value|00001006030500]].
; [!expert]""expert""
: Only the owner of the Zettelstore can access the zettel, if runtime configuration [[''expert-mode''|00001004020000#expert-mode]] is set to a boolean true value.

  This is for zettel with sensitive content that might irritate the owner.
  Computed zettel with internal runtime information are examples for such a zettel.

When you install a Zettelstore, only [[some zettel|//h?visibility=public]] have visibility ""public"".
One is the zettel that contains [[CSS|00000000020001]] for displaying the [[web user interface|00001014000000]].
This is to ensure that the web interface looks nice even for not authenticated users.

Changes to docs/manual/00001010070300.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15

16
17

18
19
20
21
22
23
24
25
26
27
28
29
1
2
3
4
5

6
7
8
9
10
11

12
13
14

15
16

17
18
19
20
21
22
23
24
25
26
27
28
29





-
+





-
+


-
+

-
+












id: 00001010070300
title: User roles
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
modified: 20220214175212
modified: 20211124141938

Every user is associated with some basic privileges.
These are specified in the [[user zettel|00001010040200]] with the key ''user-role''.
The following values are supported:

; [!reader|""reader""]
; [!reader]""reader""
: The user is allowed to read zettel.
  This is the default value for any user except the owner of the Zettelstore.
; [!writer|""writer""]
; [!writer]""writer""
: The user is allowed to create new zettel and to change existing zettel.
; [!creator|""creator""]
; [!creator]""creator""
: The user is only allowed to create new zettel.
  It is also allowed to change its own user zettel.

There are two other user roles, implicitly defined:

; The anonymous user
: This role is assigned to any user that is not authenticated.
  Can only read zettel with visibility [[public|00001010070200]], but cannot change them.
; The owner
: The user that is configured to be the owner of the Zettelstore.
  Does not need to specify a user role in its user zettel.
  Is not restricted in the use of Zettelstore, except when a zettel is marked as [[read-only|00001006020400]].

Changes to docs/manual/00001010090100.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18


19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16


17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46





-
+










-
-
+
+




















-
+







id: 00001010090100
title: External server to encrypt message transport
role: manual
tags: #configuration #encryption #manual #security #zettelstore
syntax: zmk
modified: 20220217180826
modified: 20211027125733

Since Zettelstore does not encrypt the messages it exchanges with its clients, you may need some additional software to enable encryption.

=== Public-key encryption
To enable encryption, you probably use some kind of encryption keys.
In most cases, you need to deploy a ""public-key encryption"" process, where your side publish a public encryption key that only works with a corresponding private decryption key.
Technically, this is not trivial.
Any client who wants to communicate with your Zettelstore must trust the public encryption key.
Otherwise the client cannot be sure that it is communication with your Zettelstore.
This problem is solved in part with [[Let's Encrypt|https://letsencrypt.org/]],
""a free, automated, and open certificate authority (CA), run for the public’s benefit.
It is a service provided by the [[Internet Security Research Group|https://www.abetterinternet.org/]]"".
<<a free, automated, and open certificate authority (CA), run for the public’s benefit.
It is a service provided by the [[Internet Security Research Group|https://www.abetterinternet.org/]]<<.

Alternatively, you can buy these keys for public-key encryption at ""certificate authorities"" or its dealers.

=== Server software for encryption
The solution of placing a server for encryption in front of an encryption-unaware server is a relatively old one.
There are many different alternatives to choose.

First, there are web servers.
Business-grade web servers must enable encryption.
Most of them allow to forward a request unencrypted to another web server.
Some examples:

* [[Apache Web Server|https://httpd.apache.org/]]: enable [[mod_proxy|http://httpd.apache.org/docs/current/mod/mod_proxy.html]] and configure a reverse proxy.
* [[nginx|https://nginx.org/]]: set-up a reverse proxy with the [[''proxy_pass''|https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass]] directive.
* [[Caddy|https://caddyserver.com/]]: see below for details.

Other software is also possible.
There exists software dedicated for this task of handling the encryption part.
Some examples:

* [[stunnel|https://www.stunnel.org/]] (""a proxy designed to add TLS encryption functionality to existing clients and servers without any changes in the programs' code."")
* [[stunnel|https://www.stunnel.org/]] (<<a proxy designed to add TLS encryption functionality to existing clients and servers without any changes in the programs' code.<<)
* [[Traefik|https://traefik.io/]]: set-up a [[router|https://docs.traefik.io/routing/routers/]].

=== Example configuration for Caddy
For the inexperienced owner of a Zettelstore, [[Caddy|https://caddyserver.com/]] is a good option[^In fact, the [[server-based installation procedure|00001003000000]] of Zettelstore was inspired by Caddy.].
Caddy has the capability to automatically fetch appropriately encryption key from Let's Encrypt, without any further configuration.
The only requirement of doing this is that the server must be publicly accessible.

Changes to docs/manual/00001012000000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
1
2
3
4
5

6
7
8
9
10
11
12
13





-
+







id: 00001012000000
title: API
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220304173249
modified: 20220201165200

The API (short for ""**A**pplication **P**rogramming **I**nterface"") is the primary way to communicate with a running Zettelstore.
Most integration with other systems and services is done through the API.
The [[web user interface|00001014000000]] is just an alternative, secondary way of interacting with a Zettelstore.

=== Background
The API is HTTP-based and uses plain text and JSON as its main encoding format for exchanging messages between a Zettelstore and its client software.
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46
47

48
49
50
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

48
49
50
51







+








-
+




=== Working with zettel
* [[Create a new zettel|00001012053200]]
* [[Retrieve metadata and content of an existing zettel|00001012053300]]
* [[Retrieve metadata of an existing zettel|00001012053400]]
* [[Retrieve evaluated metadata and content of an existing zettel in various encodings|00001012053500]]
* [[Retrieve parsed metadata and content of an existing zettel in various encodings|00001012053600]]
* [[Retrieve references of an existing zettel|00001012053700]]
* [[Retrieve context of an existing zettel|00001012053800]]
* [[Retrieve unlinked references to an existing zettel|00001012053900]]
* [[Retrieve zettel order within an existing zettel|00001012054000]]
* [[Update metadata and content of a zettel|00001012054200]]
* [[Rename a zettel|00001012054400]]
* [[Delete a zettel|00001012054600]]

=== Various helper methods
* [[Retrieve administrative data|00001012070500]]
* [[Encode Zettelmarkup inline material as HTML/Text|00001012070500]]
* [[Execute some commands|00001012080100]]
** [[Check for authentication|00001012080200]]
** [[Refresh internal data|00001012080500]]

Changes to docs/manual/00001012050600.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19

20
21

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18

19
20

21





-
+












-
+

-
+
id: 00001012050600
title: API: Provide an access token
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220218130020
modified: 20210830161320

The [[authentication process|00001012050200]] provides you with an [[access token|00001012921000]].
Most API calls need such an access token, so that they know the identity of the caller.

You send the access token in the ""Authorization"" request header field, as described in [[RFC 6750, section 2.1|https://tools.ietf.org/html/rfc6750#section-2.1]].
You need to use the ""Bearer"" authentication scheme to transmit the access token.

For example (in plain text HTTP):
```
GET /z HTTP/1.0
Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg
```
Note, that there is exactly one space character (""'' ''{-}"", U+0020) between the string ""Bearer"" and the access token: ``Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.ey...``{-}.
Note, that there is exactly one space character (''U+0020'') between the string ""Bearer"" and the access token: ``Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.ey...``{-}.

If you use the [[curl|https://curl.haxx.se/]] tool, you can use the ''-H'' command line parameter to set this header field.
If you use the [[curl|https://curl.haxx.se/]] tool, you can use the ++-H++ command line parameter to set this header field.

Changes to docs/manual/00001012051810.zettel.

1
2
3
4
5
6

7
8

9
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
1
2
3
4
5

6
7

8
9
10
11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26





-
+

-
+










-
+







id: 00001012051810
title: API: Select zettel based on their metadata
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220218133305
modified: 20211121170308

Every query parameter that does __not__ begin with the low line character (""_"", U+005F) is treated as the name of a [[metadata|00001006010000]] key.
Every query parameter that does __not__ begin with the low line character (""_"", ''U+005F'') is treated as the name of a [[metadata|00001006010000]] key.
According to the [[type|00001006030000]] of a metadata key, zettel are possibly selected.
All [[supported|00001006020000]] metadata keys have a well-defined type.
User-defined keys have the type ''e'' (string, possibly empty).

For example, if you want to retrieve all zettel that contain the string ""API"" in its title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/j?title=API'
{"query":"title MATCH API","list":[{"id":"00001012921000","meta":{"title":"API: JSON structure of an access token","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920500","meta":{"title":"Formats available by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920000","meta":{"title":"Endpoints used by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}}, ...
```

However, if you want all zettel that does not match a given value, you must prefix the value with the exclamation mark character (""!"", U+0021).
However, if you want all zettel that does not match a given value, you must prefix the value with the exclamation mark character (""!"", ''U+0021'').
For example, if you want to retrieve all zettel that do not contain the string ""API"" in their title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/j?title=!API'
{"query":"title NOT MATCH API","list":[{"id":"00010000000000","meta":{"back":"00001003000000 00001005090000","backward":"00001003000000 00001005090000","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00000000000001 00000000000003 00000000000096 00000000000100","lang":"en","license":"EUPL-1.2-or-later","role":"zettel","syntax":"zmk","title":"Home"}},{"id":"00001014000000","meta":{"back":"00001000000000 00001004020000 00001012920510","backward":"00001000000000 00001004020000 00001012000000 00001012920510","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00001012000000","lang":"en","license":"EUPL-1.2-or-later","published":"00001014000000","role":"manual","syntax":"zmk","tags":"#manual #webui #zettelstore","title":"Web user interface"}},
...
```

Changes to docs/manual/00001012051890.zettel.

1
2
3
4
5
6

7
8

9
10
11
12
13
14
15
16

17
18
19
20
21
22
23

24
25
26

27
28
29

30
31
32

33
34
35
36
37
38
39
1
2
3
4
5

6
7

8
9
10
11
12
13
14
15

16
17
18
19
20
21
22

23
24
25

26
27
28

29
30
31

32
33
34
35
36
37
38
39





-
+

-
+







-
+






-
+


-
+


-
+


-
+







id: 00001012051890
title: API: Search syntax (simple)
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220218130900
modified: 20211124135846

If the search string starts with the exclamation mark character (""!"", U+0021), it will be removed and the query matches all values that **do not match** the search string.
If the search string starts with the exclamation mark character (""!"", ''U+0021''), it will be removed and the query matches all values that **do not match** the search string.

In the next step, the first character of the search string will be inspected.
If it contains one of the characters ""'':''"", ""''=''"", ""''>''"", ""''<''"", or ""''~''"", this will modify how the search will be performed.
The character will be removed from the start of the search string.

For example, assume the search string is ""def"":

; The colon character (""'':''"", U+003A) (or none of these characters)
; The colon character (""'':''"", ''U+003A'') (or none of these characters)
: This is the __default__ comparison.
  The comparison depends on the type of the underlying values.
  For a content search, it is equal to the tilde character ""''~''"", which returns true if a word within the content just contains the search string.
  For metadata, it depends on the key [[type|00001006030000]].

  It you omit the the comparison character, the default comparison is also used.
; The tilde character (""''~''"", U+007E)
; The tilde character (""''~''"", ''U+007E'')
: The inspected text[^Either all words of the zettel content and/or some metadata values] contains the search string.
  ""def"", ""defghi"", and ""abcdefghi"" are matching the search string.
; The equal sign character (""''=''"", U+003D)
; The equal sign character (""''=''"", ''U+003D'')
: The inspected text must contain a word that is equal to the search string.
  Only the word ""def"" matches the search string.
; The greater-than sign character (""''>''"", U+003E)
; The greater-than sign character (""''>''"", ''U+003E'')
: The inspected text must contain a word with the search string as a prefix.
  A word like ""def"" or ""defghi"" matches the search string.
; The less-than sign character (""''<''"", U+003C)
; The less-than sign character (""''<''"", ''U+003C'')
: The inspected text must contain a word with the search string as a suffix.
  A word like ""def"" or ""abcdef"" matches the search string.

If you want to include an initial ""''!''"" into the search string, you must prefix that with the escape character ""''\\''"".
For example ""\\!abc"" will search for the string ""!abc"".
A similar rule applies to the characters that specify the way how the search will be done.
For example, ""!\\=abc"" will search for content that does not contains the string ""=abc"".

Changes to docs/manual/00001012052000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
1
2
3
4
5

6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23





-
+







-
+









id: 00001012052000
title: API: Sort the list of zettel metadata
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220218131937
modified: 20211124175920

If not specified, the list of zettel is sorted descending by the value of the [[zettel identifier|00001006050000]].
The highest zettel identifier, which is a number, comes first.
You change that with the ""''_sort''"" query parameter.
Alternatively, you can also use the ""''_order''"" query parameter.
It is an alias.

Its value is the name of a metadata key, optionally prefixed with a hyphen-minus character (""''-''"", U+002D).
Its value is the name of a metadata key, optionally prefixed with a hyphen-minus character (""-"", ''U+002D'').
According to the [[type|00001006030000]] of a metadata key, the list of zettel is sorted.
If hyphen-minus is given, the order is descending, else ascending.

If you want a random list of zettel, specify the value ""_random"" in place of the metadata key.
""''_sort=_random''"" (or ""''_order=_random''"") is the query parameter in this case.
If can be combined with ""[[''_limit=1''|00001012051830]]"" to obtain just one random zettel.

Currently, only the first occurrence of ''_sort'' is recognized.
In the future it will be possible to specify a combined sort key.

Changes to docs/manual/00001012053500.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58


59
60
61
62
63
64
65
66
67
68
69
70





-
+











-
+



















-
+




















-
-












id: 00001012053500
title: API: Retrieve evaluated metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220301174012
modified: 20211124180519

The [[endpoint|00001012920000]] to work with evaluated metadata and content of a specific zettel is ''/v/{ID}'', where ''{ID}'' is a placeholder for the [[zettel identifier|00001006050000]].

For example, to retrieve some evaluated data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/v/00001012053500''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
If successful, the output is a JSON object:
```sh
# curl http://127.0.0.1:23123/v/00001012053500
{"meta":{"title":[{"t":"Text","s":"API:"},{"t":"Space"},{"t":"Text","s":"Retrieve"},{"t":"Space"},{"t":"Text","s":"evaluated"},{"t":"Space"},{"t":"Text","s":"metadata"},{"t":"Space"},{"t":"Text","s":"and"},{"t":"Space"},{"t":"Text","s":"content"},{"t":"Space"},{"t":"Text","s":"of"},{"t":"Space"},{"t":"Text","s":"an"},{"t":"Space"},{"t":"Text","s":"existing"},{"t":"Space"},{"t":"Text","s":"zettel"},{"t":"Space"},{"t":"Text","s":"in"},{"t":"Space"}, ...
```

To select another encoding, you can provide a query parameter ''_enc=[[ENCODING|00001012920500]]''.
The default encoding is ""[[zjson|00001012920503]]"".
The default encoding is ""[[djson|00001012920503]]"".
Others are ""[[html|00001012920510]]"", ""[[text|00001012920519]]"", and some more.
```sh
# curl 'http://127.0.0.1:23123/v/00001012053500?_enc=html'
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>API: Retrieve evaluated metadata and content of an existing zettel in various encodings</title>
<meta name="zs-role" content="manual">
<meta name="keywords" content="api, manual, zettelstore">
<meta name="zs-syntax" content="zmk">
<meta name="zs-back" content="00001012000000">
<meta name="zs-backward" content="00001012000000">
<meta name="zs-box-number" content="1">
<meta name="copyright" content="(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>">
<meta name="zs-forward" content="00001010040100 00001012050200 00001012920000 00001012920800">
<meta name="zs-published" content="00001012053500">
</head>
<body>
<p>The <a href="00001012920000">endpoint</a> to work with evaluated metadata and content of a specific zettel is <kbd>/v/{ID}</kbd>, where <kbd>{ID}</kbd> is a placeholder for the <a href="00001006050000">zettel identifier</a>.</p>
<p>The <a href="00001012920000">endpoint</a> to work with evaluated metadata and content of a specific zettel is <span class="zs-monospace">/v/{ID}</span>, where <span class="zs-monospace">{ID}</span> is a placeholder for the <a href="00001006050000">zettel identifier</a>.</p>
...
```

You also can use the query parameter ''_part=[[PART|00001012920800]]'' to specify which parts of a zettel must be encoded.
In this case, its default value is ''content''.
```sh
# curl 'http://127.0.0.1:23123/v/00001012053500?_enc=html&_part=meta'
<meta name="zs-title" content="API: Retrieve evaluated metadata and content of an existing zettel in various encodings">
<meta name="zs-role" content="manual">
<meta name="keywords" content="api, manual, zettelstore">
<meta name="zs-syntax" content="zmk">
<meta name="zs-back" content="00001012000000">
<meta name="zs-backward" content="00001012000000">
<meta name="zs-box-number" content="1">
<meta name="copyright" content="(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>">
<meta name="zs-forward" content="00001010040100 00001012050200 00001012920000 00001012920800">
<meta name="zs-lang" content="en">
<meta name="zs-published" content="00001012053500">
```

The optional query parameter ''_embed'' will embed all images into the returned encoding.

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the zettel identifier did not consist of exactly 14 digits or ''_enc'' / ''_part'' contained illegal values.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Added docs/manual/00001012053700.zettel.


























































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001012053700
title: API: Retrieve references of an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20211124180813

The web of zettel is one important value of a Zettelstore.
Many zettel references other zettel, embedded material, external/local material or, via citations, external literature.
By using the [[endpoint|00001012920000]] ''/l/{ID}'' you are able to retrieve these references.

````
# curl http://127.0.0.1:23123/l/00001012053700
{"id":"00001012053700","linked":{"outgoing":["00001012920000","00001007040300#links","00001007040300#embedded-material","00001007040300#citation-key"]},"embedded":{}}
````
Formatted, this translates into:
````json
{
  "id": "00001012053700",
  "linked": {
    "outgoing": [
      "00001012920000",
      "00001007040300#links",
      "00001007040300#embedded-material",
      "00001007040300#citation-key"
    ]
  },
  "embedded": {}
}
````
=== Kind
The following top-level JSON keys are returned:
; ''id''
: The [[zettel identifier|00001006050000]] for which the references were requested.
; ''linked''
: A JSON object that contains information about incoming and outgoing [[links|00001007040300#links]].
; ''embedded''
: A JSON object that contains information about referenced [[embedded material|00001007040300#embedded-material]].
; ''cite''
: A JSON list of [[citation keys|00001007040300#citation-key]] (as JSON strings).

Incoming and outgoing references are basically zettel.
Therefore the list elements are JSON objects with key ''id'', optionally with an appended fragment.
Local and external references are strings.

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the zettel identifier did not consist of exactly 14 digits.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Changes to docs/manual/00001012070500.zettel.

1
2

3
4
5
6

7
8


9









10




11

12

13
14
15

16
17






18
19


20
21




22
23



24
25
26
27







1

2
3
4
5

6
7

8
9
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25

26



27
28

29
30
31
32
33
34
35

36
37
38

39
40
41
42
43

44
45
46
47
48
49

50
51
52
53
54
55
56

-
+



-
+

-
+
+

+
+
+
+
+
+
+
+
+
-
+
+
+
+

+
-
+
-
-
-
+

-
+
+
+
+
+
+

-
+
+

-
+
+
+
+

-
+
+
+



-
+
+
+
+
+
+
+
id: 00001012070500
title: Retrieve administrative data
title: API: Encode Zettelmarkup inline material as HTML/Text
role: zettel
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220304174027
modified: 20211230231944

The [[endpoint|00001012920000]] ''/x'' allows you to retrieve some (administrative) data.
To encode [[Zettelmarkup inline material|00001007040000]] send a HTTP POST request to the [[endpoint|00001012920000]] ''/v''.
The POST body must contain a JSON encoded list of Zettelmarkup inline material to be encoded:

; ''first-zmk''
: Contains the first [[Zettelmarkup encoded|00001007000000]] inline-structured elements.
  This will be encoded as [[HTML|00001012920510]] and [[Text|00001012920519]].
; ''other-zmk''
: Contain more material.
  The list can be empty.
  These will be encoded in HTML only.
; ''lang''
: Specifies the language for HTML encoding.
Currently, you can only request Zettelstore version data.
  If empty, the default language of the Zettelstore instance will be used.
; ''no-links''
: A boolean value, which specifies whether links should be encoded (``"no-links":false``) or should be not encoded (``"no-links":true``).
  Default: ''false''.

Typically, this call will be used to encode the [[title|00001006020000#title]] of a zettel.
````

# curl 'http://127.0.0.1:23123/x'
{"major":0,"minor":4,"patch":0,"info":"dev","hash":"cb121cc980-dirty"}
````
If successful, the call will return the following JSON document:

Zettelstore conforms somehow to the Standard [[Semantic Versioning|https://semver.org/]].
; ''first-html''
: HTML encoding of ''first-zmk''
; ''first-text''
: Text encoding of ''first-zmk''
; ''other_html''
: HTML encoding of the corresponding value in ''other-zmk''.

The names ""major"", ""minor"", and ""patch"" are described in this standard.
Encoding takes place in the context of all other zettel in the Zettelstore.
For example, [[links|00001007040310]] and images are evaluated according to this context.

The name ""info"" contains sometimes some additional information, e.g. ""dev"" for a development version, or ""preview"" for a preview version.
A simple example:
```sh
# curl -X POST --data '{"first-zmk":"hallo [[00000000000001]]"}' http://127.0.0.1:23123/v
{"first-html":"hallo <a href=\"00000000000001\">00000000000001</a>","first-text":"hallo ","other-html":null}

The name ""hash"" contains some data to identify the version from a developers perspective.
# curl -X POST --data '{"first-zmk":"hallo [[00000000000001]]","no-links":true}' http://127.0.0.1:23123/v
{"first-html":"hallo <span>00000000000001</span>","first-text":"hallo ","other-html":null}
```

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
: Operation was successful, the body contains a JSON object as described above.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Most likely, the JSON was not formed according to above rules.
; ''403''
: You are not allowed to perform this operation.

Changes to docs/manual/00001012920000.zettel.

1
2
3
4
5
6

7
8
9
10

11
12

13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32


33
34
35
36
37
38
39

40
41
42


1
2
3
4
5

6
7
8
9

10
11

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


32
33

34
35
36
37
38

39
40


41
42





-
+



-
+

-
+












+






-
-
+
+
-





-
+

-
-
+
+
id: 00001012920000
title: Endpoints used by the API
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20220304173423
modified: 20211230234616

All API endpoints conform to the pattern ''[PREFIX]LETTER[/ZETTEL-ID]'', where:
; ''PREFIX''
: is the URL prefix (default: ""/""), configured via the ''url-prefix'' [[startup configuration|00001004010000]],
: is the URL prefix (default: ''/''), configured via the ''url-prefix'' [[startup configuration|00001004010000]],
; ''LETTER''
: is a single letter that specifies the resource type,
: is a single letter that specifies the ressource type,
; ''ZETTEL-ID''
: is an optional 14 digits string that uniquely [[identify a zettel|00001006050000]].

The following letters are currently in use:

|= Letter:| Without zettel identifier | With [[zettel identifier|00001006050000]] | Mnemonic
| ''a'' | POST: [[client authentication|00001012050200]] | | **A**uthenticate
|       | PUT: [[renew access token|00001012050400]] |
| ''j'' | GET: [[list zettel AS JSON|00001012051200]] | GET: [[retrieve zettel AS JSON|00001012053300]] | **J**SON
|       | POST: [[create new zettel|00001012053200]] | PUT: [[update a zettel|00001012054200]]
|       |  | DELETE: [[delete the zettel|00001012054600]]
|       |  | MOVE: [[rename the zettel|00001012054400]]
| ''l'' |  | GET: [[list references|00001012053700]] | **L**inks
| ''m'' |  | GET: [[retrieve metadata|00001012053400]] | **M**etadata
| ''o'' |  | GET: [[list zettel order|00001012054000]] | **O**rder
| ''p'' |  | GET: [[retrieve parsed zettel|00001012053600]]| **P**arsed
| ''r'' | GET: [[list roles|00001012052600]] | | **R**oles
| ''t'' | GET: [[list tags|00001012052400]] || **T**ags
| ''u'' |  | GET [[unlinked references|00001012053900]] | **U**nlinked
| ''v'' |  | GET: [[retrieve evaluated zettel|00001012053500]] | E**v**aluated
| ''x'' | GET: [[retrieve administrative data|00001012070500]] | GET: [[list zettel context|00001012053800]] | Conte**x**t
| ''v'' | POST: [[encode inlines|00001012070500]] | GET: [[retrieve evaluated zettel|00001012053500]] | E**v**aluated
| ''x'' | POST: [[execute command|00001012080100]] | GET: [[list zettel context|00001012053800]] | Conte**x**t
|       | POST: [[execute command|00001012080100]]
| ''z'' | GET: [[list zettel|00001012051200#plain]] | GET: [[retrieve zettel|00001012053300#plain]] | **Z**ettel
|       | POST: [[create new zettel|00001012053200#plain]] | PUT: [[update a zettel|00001012054200#plain]]
|       |  | DELETE: [[delete zettel|00001012054600#plain]]
|       |  | MOVE: [[rename zettel|00001012054400#plain]]

The full URL will contain either the ""http"" oder ""https"" scheme, a host name, and an optional port number.
The full URL will contain either the ''http'' oder ''https'' scheme, a host name, and an optional port number.

The API examples will assume the ""http"" schema, the local host ""127.0.0.1"", the default port ""23123"", and the default empty ''PREFIX'' ""/"".
Therefore, all URLs in the API documentation will begin with ""http://127.0.0.1:23123/"".
The API examples will assume the ''http'' schema, the local host ''127.0.0.1'', the default port ''23123'', and the default empty ''PREFIX''.
Therefore, all URLs in the API documentation will begin with ''http://127.0.0.1:23123''.

Changes to docs/manual/00001012920500.zettel.

1
2
3
4
5
6

7
8
9
10

11
12
13
14
1
2
3
4
5

6
7
8
9

10
11
12
13
14





-
+



-
+




id: 00001012920500
title: Encodings available via the [[API|00001012000000]]
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20220209114459
modified: 20211124140517

A zettel representation can be encoded in various formats for further processing.

* [[zjson|00001012920503]] (default)
* [[djson|00001012920503]] (default)
* [[html|00001012920510]]
* [[native|00001012920513]]
* [[text|00001012920519]]
* [[zmk|00001012920522]]

Changes to docs/manual/00001012920503.zettel.

1
2

3
4
5
6

7
8
9
10
11
12
13
14
15



16
17
18
19
20
21

22
23
24
25
26
27
28
29
30
31
32
33
34
35
1

2
3
4
5

6
7
8
9
10
11
12



13
14
15
16
17
18



19















-
+



-
+






-
-
-
+
+
+



-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012920503
title: ZJSON Encoding
title: DJSON Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20220223185826
modified: 20211124134305

A zettel representation that allows to process the syntactic structure of a zettel.
It is a JSON-based encoding format, but different to the structures returned by [[endpoint|00001012920000]] ''/j/{ID}''.

For an example, take a look at the JSON encoding of this page, which is available via the ""Info"" sub-page of this zettel: 

* [[//v/00001012920503?_enc=zjson&_part=zettel]],
* [[//v/00001012920503?_enc=zjson&_part=meta]],
* [[//v/00001012920503?_enc=zjson&_part=content]].
* [[//v/00001012920503?_enc=djson&_part=zettel]],
* [[//v/00001012920503?_enc=djson&_part=meta]],
* [[//v/00001012920503?_enc=djson&_part=content]].

If transferred via HTTP, the content type will be ''application/json''.

A full zettel encoding results in a JSON object with two keys: ''"meta"'' and ''"content"''.
Both values are the same as if you have requested just the appropriate [[part|00001012920800]].

TODO: detailed description.
=== Encoding of metadata
Metadata encoding results in a JSON object, where each metadata key is mapped to the same JSON object name.
The associated value is itself a JSON object with two names.
The first name ``""`` references the [[metadata key type|00001006030000]].
Depending on the key type, the other name denotes the value of the metadata element.
The meaning of these names is [[well defined|00001012920582]], as well as the [[mapping of key types to used object names|00001012920584]].

=== Encoding of zettel content
The content encoding results in a JSON array of objects, where each objects represents a Zettelmarkup element.

Every [!zettelmarkup|Zettelmarkup] element is encoded as a JSON object.
These objects always contain the empty name ''""'' with a string value describing the type of Zettelmarkup element.
Depending on the type, other one letter names denotes the details of the element.
The meaning of these names is [[well defined|00001012920588]].

Deleted docs/manual/00001012920582.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14














-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012920582
title: ZJSON Encoding: List of Valid Metadata Value Objects Names
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20220223184324

Every Metadata value element is mapped to a JSON object with some well defined names / keys.

|=Name | JSON Value | Meaning
| ''"\"'' | string | The type of the Zettelmarkup element.
| ''"i"'' | array  | A sequence of [[inline-structured|00001007040000]] elements.
| ''"s"'' | string | The first / major string value of an element.
| ''"y"'' | array  | A set of string values.

Deleted docs/manual/00001012920584.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28




























-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012920584
title: ZJSON Encoding: Mapping of Metadata Key Types to Object Names
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20220304114135

Every [[Metadata key|00001006030000]] is mapped to an [[object name|00001012920582]] where its value is encoded.

|=Type | JSON Object Name | Remark
| [[Credential|00001006031000]] | ''"s"'' | A string with the decrypted credential.
| [[EString|00001006031500]] | ''"s"'' | A possibly empty string.
| [[Identifier|00001006032000]] | ''"s"'' | A string containing a [[zettel identifier|00001006050000]].
| [[IdentifierSet|00001006032500]] | ''"y"'' | An array of strings containing [[zettel identifier|00001006050000]].
| [[Number|00001006033000]] | ''"s"'' | A string containing a numeric value.
| [[String|00001006033500]] | ''"s"'' | A non-empty string.
| [[TagSet|00001006034000]] | ''"y"'' | An array of string containing zettel tags.
| [[Timestamp|00001006034500]] | ''"s"''  | A string containing a timestamp in the format YYYYMMDDHHmmSS.
| [[URL|00001006035000]] | ''"s"'' | A string containing an URL.
| [[Word|00001006035500]] | ''"s"'' | A string containing a word (no space characters)
| [[WordSet|00001006036000]] | ''"y"'' | An array of strings containing words.
| [[Zettelmarkup|00001006036500]] | ''"i"'' | A sequence of [[inline-structured|00001007040000]] elements.

Please note, that metadata is weakly typed.
Every metadata key expects a certain type.
But the user is free to enter something different.
For example, even if the metadata type is ""number"", its value could still be ""abc"".
However, the mapping itself is always valid.

Deleted docs/manual/00001012920588.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
























-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012920588
title: ZJSON Encoding: List of Valid Zettelmarkup Element Objects Names
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20220301102447

Every [[Zettelmarkup|00001007000000]] element is mapped to a JSON object with some well defined names / keys.

|=Name | JSON Value | Meaning
| ''"\"'' | string | The type of the Zettelmarkup element.
| ''"a"'' | object | Additional attributes of the element.
| ''"b"'' | array  | A sequence of [[block-structured|00001007030000]] elements.
| ''"c"'' | array  | A sequence of a sequence of (sub-) list elements or [[inline-structured|00001007040000]] elements. Used for nested lists.
| ''"d"'' | array  | A sequence of description list elements, where each element is an object of a definition term and a list of descriptions.
| ''"e"'' | array  | A sequence of descriptions: a JSON array of simple description, which is itself a JSON array of block structured elements.
| ''"i"'' | array  | A sequence of [[inline-structured|00001007040000]] elements.
| ''"j"'' | object | An objects describing a BLOB element.
| ''"n"'' | number | A numeric value, e.g. for specifying the [[heading|00001007030300]] level.
| ''"o"'' | string | A base64 encoded binary value. Used in some BLOB elements.
| ''"p"'' | array  | A sequence of two elements: a sequence of [[table|00001007031000]] header value, followed by a sequence of sequence of table row values.
| ''"q"'' | string | A second string value, if ''""s""'' is already used.
| ''"s"'' | string | The first / major string value of an element.
| ''"v"'' | string | A third string value, if ''""q""'' is already used.

Changes to docs/manual/00001012920800.zettel.

1
2
3
4
5
6

7
8
9

10
11
12

13
14
15

16
17
1
2
3
4
5

6
7
8

9
10
11

12
13
14

15
16
17





-
+


-
+


-
+


-
+


id: 00001012920800
title: Values to specify zettel parts
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20220214175335
modified: 20210727125306

When working with [[zettel|00001006000000]], you could work with the whole zettel, with its metadata, or with its content:
; [!zettel|''zettel'']
; [!zettel]''zettel''
: Specifies that you work with a zettel as a whole.
  Contains identifier, metadata, and content of a zettel.
; [!meta|''meta'']
; [!meta]''meta''
: Specifies that you only want to cope with the metadata of a zettel.
  Contains identifier and metadata of a zettel.
; [!content|''content'']
; [!content]''content''
: Specifies that you are only interested in the zettel content.
  Contains identifier and content of a zettel.

Changes to docs/manual/00001018000000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23


24
25
26

1
2
3
4
5

6
7
8
9
10
11
12

13
14
15
16
17
18
19
20
21


22
23
24
25

26





-
+






-
+








-
-
+
+


-
+
id: 00001018000000
title: Troubleshooting
role: zettel
tags: #manual #zettelstore
syntax: zmk
modified: 20220218125940
modified: 20220119140045

This page lists some problems and their solutions that may occur when using your Zettelstore.

=== Installation
* **Problem:** When you double-click on the Zettelstore executable icon, macOS complains that Zettelstore is an application from an unknown developer.
  Therefore, it will not start Zettelstore.
** **Solution:** Press the ''Ctrl'' key while opening the context menu of the Zettelstore executable with a right-click.
** **Solution:** Press the ++Ctrl++ while opening the context menu of the Zettelstore executable with a right-click.
   A dialog is then opened where you can acknowledge that you understand the possible risks when you start Zettelstore.
   This dialog is only resented once for a given Zettelstore executable.
* **Problem:** When you double-click on the Zettelstore executable icon, Windows complains that Zettelstore is an application from an unknown developer.
** **Solution:** Windows displays a dialog where you can acknowledge possible risks and allows to start Zettelstore.

=== Authentication
* **Problem:** [[Authentication is enabled|00001010040100]] for a local running Zettelstore and there is a valid [[user zettel|00001010040200]] for the owner.
  But entering user name and password at the [[web user interface|00001014000000]] seems to be ignored, while entering a wrong password will result in an error message.
** **Explanation:** A local running Zettelstore typically means, that you are accessing the Zettelstore using an URL with schema ''http://'', and not ''https://'', for example ''http://localhost:23123''.
   The difference between these two is the missing encryption of user name / password and for the answer of the Zettelstore if you use the ''http://'' schema.
** **Explanation:** A local running Zettelstore typically means, that you are accessing the Zettelstore using an URL with schema ''http:/\/'', and not ''http**s**:/\/'', for example ''http:/\/localhost:23123''.
   The difference between these two is the missing encryption of user name / password and for the answer of the Zettelstore if you use the ''http:/\/'' schema.
   To be secure by default, the Zettelstore will not work in an insecure environment.
** **Solution 1:** If you are sure that your communication medium is safe, even if you use the ''http:/\/'' schema (for example, you are running the Zettelstore on the same computer you are working on, or if the Zettelstore is running on a computer in your protected local network), then you could add the entry ''insecure-cookie: true'' in you [[startup configuration|00001004010000#insecure-cookie]] file.
** **Solution 2:** If you are not sure about the security of your communication medium (for example, if unknown persons might use your local network), then you should run an [[external server|00001010090100]] in front of your Zettelstore to enable the use of the ''https://'' schema.
** **Solution 2:** If you are not sure about the security of your communication medium (for example, if unknown persons might use your local network), then you should run an [[external server|00001010090100]] in front of your Zettelstore to enable the use of the ''http**s**:/\/'' schema.

Changes to domain/content.go.

29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
29
30
31
32
33
34
35



36
37
38
39
40
41
42







-
-
-







}

// NewContent creates a new content from a string.
func NewContent(data []byte) Content {
	return Content{data: data, isBinary: calcIsBinary(data)}
}

// Length returns the number of bytes stored.
func (zc *Content) Length() int { return len(zc.data) }

// Equal compares two content values.
func (zc *Content) Equal(o *Content) bool {
	if zc == nil {
		return o == nil
	}
	if zc.isBinary != o.isBinary {
		return false

Changes to domain/meta/meta.go.

135
136
137
138
139
140
141

142
143
144
145
146
147
148
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149







+







	registerKey(api.KeyCredential, TypeCredential, usageUser, "")
	registerKey(api.KeyDead, TypeIDSet, usageProperty, "")
	registerKey(api.KeyFolge, TypeIDSet, usageProperty, "")
	registerKey(api.KeyForward, TypeIDSet, usageProperty, "")
	registerKey(api.KeyLang, TypeWord, usageUser, "")
	registerKey(api.KeyLicense, TypeEmpty, usageUser, "")
	registerKey(api.KeyModified, TypeTimestamp, usageComputed, "")
	registerKey(api.KeyNoIndex, TypeBool, usageUser, "")
	registerKey(api.KeyPrecursor, TypeIDSet, usageUser, api.KeyFolge)
	registerKey(api.KeyPublished, TypeTimestamp, usageProperty, "")
	registerKey(api.KeyReadOnly, TypeWord, usageUser, "")
	registerKey(api.KeySummary, TypeZettelmarkup, usageUser, "")
	registerKey(api.KeyURL, TypeURL, usageUser, "")
	registerKey(api.KeyUselessFiles, TypeString, usageProperty, "")
	registerKey(api.KeyUserID, TypeWord, usageUser, "")
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
171
172
173
174
175
176
177












178
179
180
181
182
183
184







-
-
-
-
-
-
-
-
-
-
-
-







	pairs := make(map[string]string, len(data))
	for k, v := range data {
		pairs[k] = v
	}
	return &Meta{Zid: zid, pairs: pairs}
}

// Length returns the number of bytes stored for the metadata.
func (m *Meta) Length() int {
	if m == nil {
		return 0
	}
	result := 6 // storage needed for Zid
	for k, v := range m.pairs {
		result += len(k) + len(v) + 1 // 1 because separator
	}
	return result
}

// Clone returns a new copy of the metadata.
func (m *Meta) Clone() *Meta {
	return &Meta{
		Zid:     m.Zid,
		pairs:   m.Map(),
		YamlSep: m.YamlSep,
	}

Changes to domain/meta/type.go.

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
14
15
16
17
18
19
20

21
22
23
24
25
26
27







-







import (
	"strconv"
	"strings"
	"sync"
	"time"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
)

// DescriptionType is a description of a specific key type.
type DescriptionType struct {
	Name  string
	IsSet bool
}
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53
54
55
56
57












58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
38
39
40
41
42
43
44
45












46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71


72
73
74

75
76
77
78
79
80
81







+
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+














-
-



-







	t := &DescriptionType{name, isSet}
	registeredTypes[name] = t
	return t
}

// Supported key types.
var (
	TypeBool         = registerType("Boolean", false)
	TypeCredential   = registerType(zjson.MetaCredential, false)
	TypeEmpty        = registerType(zjson.MetaEmpty, false)
	TypeID           = registerType(zjson.MetaID, false)
	TypeIDSet        = registerType(zjson.MetaIDSet, true)
	TypeNumber       = registerType(zjson.MetaNumber, false)
	TypeString       = registerType(zjson.MetaString, false)
	TypeTagSet       = registerType(zjson.MetaTagSet, true)
	TypeTimestamp    = registerType(zjson.MetaTimestamp, false)
	TypeURL          = registerType(zjson.MetaURL, false)
	TypeWord         = registerType(zjson.MetaWord, false)
	TypeWordSet      = registerType(zjson.MetaWordSet, true)
	TypeZettelmarkup = registerType(zjson.MetaZettelmarkup, false)
	TypeCredential   = registerType("Credential", false)
	TypeEmpty        = registerType("EString", false)
	TypeID           = registerType("Identifier", false)
	TypeIDSet        = registerType("IdentifierSet", true)
	TypeNumber       = registerType("Number", false)
	TypeString       = registerType("String", false)
	TypeTagSet       = registerType("TagSet", true)
	TypeTimestamp    = registerType("Timestamp", false)
	TypeURL          = registerType("URL", false)
	TypeWord         = registerType("Word", false)
	TypeWordSet      = registerType("WordSet", true)
	TypeZettelmarkup = registerType("Zettelmarkup", false)
)

// Type returns a type hint for the given key. If no type hint is specified,
// TypeUnknown is returned.
func (*Meta) Type(key string) *DescriptionType {
	return Type(key)
}

var (
	cachedTypedKeys = make(map[string]*DescriptionType)
	mxTypedKey      sync.RWMutex
	suffixTypes     = map[string]*DescriptionType{
		"-number": TypeNumber,
		"-role":   TypeWord,
		"-set":    TypeWordSet,
		"-title":  TypeZettelmarkup,
		"-url":    TypeURL,
		"-zettel": TypeID,
		"-zid":    TypeID,
		"-zids":   TypeIDSet,
	}
)

// Type returns a type hint for the given key. If no type hint is specified,
// TypeEmpty is returned.
func Type(key string) *DescriptionType {
	if k, ok := registeredKeys[key]; ok {

Changes to domain/zettel.go.

1
2

3
4

5
6
7
8
9
10
11
12
13

14


15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
1

2
3

4
5
6
7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23




24
25
26
27

-
+

-
+









+
-
+
+







-
-
-
-




//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package domain provides domain specific types, constants, and functions.
package domain

import (
import "zettelstore.de/z/domain/meta"
	"zettelstore.de/z/domain/meta"
)

// Zettel is the main data object of a zettelstore.
type Zettel struct {
	Meta    *meta.Meta // Some additional meta-data.
	Content Content    // The content of the zettel itself.
}

// Length returns the number of bytes to store the zettel (in a domain view,
// not in a technical view).
func (z Zettel) Length() int { return z.Meta.Length() + z.Content.Length() }

// Equal compares two zettel for equality.
func (z Zettel) Equal(o Zettel, allowComputed bool) bool {
	return z.Meta.Equal(o.Meta, allowComputed) && z.Content.Equal(&o.Content)
}

Added encoder/buffer.go.














































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package encoder provides a generic interface to encode the abstract syntax
// tree into some text form.
package encoder

import (
	"encoding/base64"
	"io"
)

// BufWriter is a specialized buffered writer for encoding zettel.
type BufWriter struct {
	w      io.Writer // The io.Writer to write to
	err    error     // Collect error
	length int       // Sum length
	buf    []byte    // Buffer to collect bytes
}

// NewBufWriter creates a new BufWriter
func NewBufWriter(w io.Writer) BufWriter {
	return BufWriter{w: w, buf: make([]byte, 0, 4096)}
}

// Write writes the contents of p into the buffer.
func (w *BufWriter) Write(p []byte) (int, error) {
	if w.err != nil {
		return 0, w.err
	}
	w.buf = append(w.buf, p...)
	if len(w.buf) > 2048 {
		w.flush()
		if w.err != nil {
			return 0, w.err
		}
	}
	return len(p), nil
}

// WriteString writes the contents of s into the buffer.
func (w *BufWriter) WriteString(s string) {
	if w.err != nil {
		return
	}
	w.buf = append(w.buf, s...)
	if len(w.buf) > 2048 {
		w.flush()
	}
}

// WriteStrings writes the contents of sl into the buffer.
func (w *BufWriter) WriteStrings(sl ...string) {
	for _, s := range sl {
		w.WriteString(s)
	}
}

// WriteByte writes the content of b into the buffer.
func (w *BufWriter) WriteByte(b byte) error {
	w.buf = append(w.buf, b)
	return nil
}

// WriteBytes writes the content of bs into the buffer.
func (w *BufWriter) WriteBytes(bs ...byte) {
	w.buf = append(w.buf, bs...)
}

// WriteBase64 writes the content of p into the buffer, encoded with base64.
func (w *BufWriter) WriteBase64(p []byte) {
	if w.err == nil {
		w.flush()
	}
	if w.err == nil {
		encoder := base64.NewEncoder(base64.StdEncoding, w.w)
		length, err := encoder.Write(p)
		w.length += length
		err1 := encoder.Close()
		if err == nil {
			w.err = err1
		} else {
			w.err = err
		}
	}
}

// Flush writes any buffered data to the underlying io.Writer. It returns the
// number of bytes written and an error if something went wrong.
func (w *BufWriter) Flush() (int, error) {
	if w.err == nil {
		w.flush()
	}
	return w.length, w.err
}

func (w *BufWriter) flush() {
	length, err := w.w.Write(w.buf)
	w.buf = w.buf[:0]
	w.length += length
	w.err = err
}

Added encoder/djsonenc/djsonenc.go.




























































































































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package jsonenc encodes the abstract syntax tree into JSON.
package jsonenc

import (
	"fmt"
	"io"
	"sort"
	"strconv"

	"zettelstore.de/c/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

func init() {
	encoder.Register(api.EncoderDJSON, encoder.Info{
		Create: func(env *encoder.Environment) encoder.Encoder { return &jsonDetailEncoder{env: env} },
	})
}

type jsonDetailEncoder struct {
	env *encoder.Environment
}

// WriteZettel writes the encoded zettel to the writer.
func (je *jsonDetailEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newDetailVisitor(w, je)
	v.b.WriteString(`{"meta":{`)
	v.writeMeta(zn.InhMeta, evalMeta)
	v.b.WriteByte('}')
	v.b.WriteString(`,"content":`)
	ast.Walk(v, zn.Ast)
	v.b.WriteByte('}')
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as JSON.
func (je *jsonDetailEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newDetailVisitor(w, je)
	v.b.WriteByte('{')
	v.writeMeta(m, evalMeta)
	v.b.WriteByte('}')
	length, err := v.b.Flush()
	return length, err
}

func (je *jsonDetailEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return je.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes a block slice to the writer
func (je *jsonDetailEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newDetailVisitor(w, je)
	ast.Walk(v, bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (je *jsonDetailEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newDetailVisitor(w, je)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b   encoder.BufWriter
	env *encoder.Environment
}

func newDetailVisitor(w io.Writer, je *jsonDetailEncoder) *visitor {
	return &visitor{b: encoder.NewBufWriter(w), env: je.env}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockListNode:
		v.visitBlockList(n)
		return nil
	case *ast.InlineListNode:
		v.walkInlineList(n)
		return nil
	case *ast.ParaNode:
		v.writeNodeStart("Para")
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.visitHeading(n)
	case *ast.HRuleNode:
		v.writeNodeStart("Hrule")
		v.visitAttributes(n.Attrs)
	case *ast.NestedListNode:
		v.visitNestedList(n)
	case *ast.DescriptionListNode:
		v.visitDescriptionList(n)
	case *ast.TableNode:
		v.visitTable(n)
	case *ast.TranscludeNode:
		v.writeNodeStart("Transclude")
		v.writeContentStart('q')
		writeEscaped(&v.b, mapRefState[n.Ref.State])
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Ref.String())
	case *ast.BLOBNode:
		v.visitBLOB(n)
	case *ast.TextNode:
		v.writeNodeStart("Text")
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Text)
	case *ast.TagNode:
		v.writeNodeStart("Tag")
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Tag)
	case *ast.SpaceNode:
		v.writeNodeStart("Space")
		if l := len(n.Lexeme); l > 1 {
			v.writeContentStart('n')
			v.b.WriteString(strconv.Itoa(l))
		}
	case *ast.BreakNode:
		if n.Hard {
			v.writeNodeStart("Hard")
		} else {
			v.writeNodeStart("Soft")
		}
	case *ast.LinkNode:
		v.writeNodeStart("Link")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('q')
		writeEscaped(&v.b, mapRefState[n.Ref.State])
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Ref.String())
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.EmbedBLOBNode:
		v.visitEmbedBLOB(n)
	case *ast.CiteNode:
		v.writeNodeStart("Cite")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Key)
		if n.Inlines != nil {
			v.writeContentStart('i')
			ast.Walk(v, n.Inlines)
		}
	case *ast.FootnoteNode:
		v.writeNodeStart("Footnote")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.MarkNode:
		v.visitMark(n)
	case *ast.FormatNode:
		v.writeNodeStart(mapFormatKind[n.Kind])
		v.visitAttributes(n.Attrs)
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.LiteralNode:
		kind, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown literal kind %v", n.Kind))
		}
		v.writeNodeStart(kind)
		v.visitAttributes(n.Attrs)
		v.writeContentStart('s')
		writeEscaped(&v.b, string(n.Content))
	default:
		return v
	}
	v.b.WriteByte('}')
	return nil
}

var mapVerbatimKind = map[ast.VerbatimKind]string{
	ast.VerbatimZettel:  "ZettelBlock",
	ast.VerbatimProg:    "CodeBlock",
	ast.VerbatimComment: "CommentBlock",
	ast.VerbatimHTML:    "HTMLBlock",
}

func (v *visitor) visitVerbatim(vn *ast.VerbatimNode) {
	kind, ok := mapVerbatimKind[vn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown verbatim kind %v", vn.Kind))
	}
	v.writeNodeStart(kind)
	v.visitAttributes(vn.Attrs)
	v.writeContentStart('s')
	writeEscaped(&v.b, string(vn.Content))
}

var mapRegionKind = map[ast.RegionKind]string{
	ast.RegionSpan:  "SpanBlock",
	ast.RegionQuote: "QuoteBlock",
	ast.RegionVerse: "VerseBlock",
}

func (v *visitor) visitRegion(rn *ast.RegionNode) {
	kind, ok := mapRegionKind[rn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown region kind %v", rn.Kind))
	}
	v.writeNodeStart(kind)
	v.visitAttributes(rn.Attrs)
	v.writeContentStart('b')
	ast.Walk(v, rn.Blocks)
	if rn.Inlines != nil {
		v.writeContentStart('i')
		ast.Walk(v, rn.Inlines)
	}
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	v.writeNodeStart("Heading")
	v.visitAttributes(hn.Attrs)
	v.writeContentStart('n')
	v.b.WriteString(strconv.Itoa(hn.Level))
	if fragment := hn.Fragment; fragment != "" {
		v.writeContentStart('s')
		v.b.WriteStrings(`"`, fragment, `"`)
	}
	v.writeContentStart('i')
	ast.Walk(v, hn.Inlines)
}

var mapNestedListKind = map[ast.NestedListKind]string{
	ast.NestedListOrdered:   "OrderedList",
	ast.NestedListUnordered: "BulletList",
	ast.NestedListQuote:     "QuoteList",
}

func (v *visitor) visitNestedList(ln *ast.NestedListNode) {
	v.writeNodeStart(mapNestedListKind[ln.Kind])
	v.writeContentStart('c')
	for i, item := range ln.Items {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, in := range item {
			v.writeComma(j)
			ast.Walk(v, in)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.writeNodeStart("DescriptionList")
	v.writeContentStart('g')
	for i, def := range dn.Descriptions {
		v.writeComma(i)
		v.b.WriteByte('[')
		ast.Walk(v, def.Term)

		if len(def.Descriptions) > 0 {
			for _, b := range def.Descriptions {
				v.b.WriteString(",[")
				for j, dn := range b {
					v.writeComma(j)
					ast.Walk(v, dn)
				}
				v.b.WriteByte(']')
			}
		}
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitTable(tn *ast.TableNode) {
	v.writeNodeStart("Table")
	v.writeContentStart('p')

	// Table header
	v.b.WriteByte('[')
	for i, cell := range tn.Header {
		v.writeComma(i)
		v.writeCell(cell)
	}
	v.b.WriteString("],")

	// Table rows
	v.b.WriteByte('[')
	for i, row := range tn.Rows {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, cell := range row {
			v.writeComma(j)
			v.writeCell(cell)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteString("]]")
}

var alignmentCode = map[ast.Alignment]string{
	ast.AlignDefault: `["",`,
	ast.AlignLeft:    `["<",`,
	ast.AlignCenter:  `[":",`,
	ast.AlignRight:   `[">",`,
}

func (v *visitor) writeCell(cell *ast.TableCell) {
	v.b.WriteString(alignmentCode[cell.Align])
	ast.Walk(v, cell.Inlines)
	v.b.WriteByte(']')
}

func (v *visitor) visitBLOB(bn *ast.BLOBNode) {
	v.writeNodeStart("Blob")
	if bn.Title != "" {
		v.writeContentStart('q')
		writeEscaped(&v.b, bn.Title)
	}
	v.writeContentStart('s')
	writeEscaped(&v.b, bn.Syntax)
	if bn.Syntax == api.ValueSyntaxSVG {
		v.writeContentStart('v')
		writeEscaped(&v.b, string(bn.Blob))
	} else {
		v.writeContentStart('o')
		v.b.WriteBase64(bn.Blob)
		v.b.WriteByte('"')
	}
}

var mapRefState = map[ast.RefState]string{
	ast.RefStateInvalid:  "invalid",
	ast.RefStateZettel:   "zettel",
	ast.RefStateSelf:     "self",
	ast.RefStateFound:    "found",
	ast.RefStateBroken:   "broken",
	ast.RefStateHosted:   "local",
	ast.RefStateBased:    "based",
	ast.RefStateExternal: "external",
}

func (v *visitor) visitEmbedRef(en *ast.EmbedRefNode) {
	v.writeNodeStart("Embed")
	v.visitAttributes(en.Attrs)
	v.writeContentStart('s')
	writeEscaped(&v.b, en.Ref.String())

	if en.Inlines != nil {
		v.writeContentStart('i')
		ast.Walk(v, en.Inlines)
	}
}

func (v *visitor) visitEmbedBLOB(en *ast.EmbedBLOBNode) {
	v.writeNodeStart("EmbedBLOB")
	v.visitAttributes(en.Attrs)
	v.writeContentStart('j')
	v.b.WriteString(`"s":`)
	writeEscaped(&v.b, en.Syntax)
	if en.Syntax == api.ValueSyntaxSVG {
		v.writeContentStart('q')
		writeEscaped(&v.b, string(en.Blob))
	} else {
		v.writeContentStart('o')
		v.b.WriteBase64(en.Blob)
		v.b.WriteByte('"')
	}
	v.b.WriteByte('}')

	if en.Inlines != nil {
		v.writeContentStart('i')
		ast.Walk(v, en.Inlines)
	}
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	v.writeNodeStart("Mark")
	if text := mn.Text; text != "" {
		v.writeContentStart('s')
		writeEscaped(&v.b, text)
	}
	if fragment := mn.Fragment; fragment != "" {
		v.writeContentStart('q')
		v.b.WriteByte('"')
		v.b.WriteString(fragment)
		v.b.WriteByte('"')
	}
}

var mapFormatKind = map[ast.FormatKind]string{
	ast.FormatEmph:      "Emph",
	ast.FormatStrong:    "Strong",
	ast.FormatMonospace: "Mono",
	ast.FormatDelete:    "Delete",
	ast.FormatInsert:    "Insert",
	ast.FormatSuper:     "Super",
	ast.FormatSub:       "Sub",
	ast.FormatQuote:     "Quote",
	ast.FormatQuotation: "Quotation",
	ast.FormatSpan:      "Span",
}

var mapLiteralKind = map[ast.LiteralKind]string{
	ast.LiteralZettel:  "Zettel",
	ast.LiteralProg:    "Code",
	ast.LiteralKeyb:    "Input",
	ast.LiteralOutput:  "Output",
	ast.LiteralComment: "Comment",
	ast.LiteralHTML:    "HTML",
}

func (v *visitor) visitBlockList(bln *ast.BlockListNode) {
	v.b.WriteByte('[')
	for i, bn := range bln.List {
		v.writeComma(i)
		ast.Walk(v, bn)
	}
	v.b.WriteByte(']')
}

func (v *visitor) walkInlineList(iln *ast.InlineListNode) {
	v.b.WriteByte('[')
	for i, in := range iln.List {
		v.writeComma(i)
		ast.Walk(v, in)
	}
	v.b.WriteByte(']')
}

// visitAttributes write JSON attributes
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)
	}
	sort.Strings(keys)

	v.b.WriteString(`,"a":{"`)
	for i, k := range keys {
		if i > 0 {
			v.b.WriteString(`","`)
		}
		strfun.JSONEscape(&v.b, k)
		v.b.WriteString(`":"`)
		strfun.JSONEscape(&v.b, a.Attrs[k])
	}
	v.b.WriteString(`"}`)
}

func (v *visitor) writeNodeStart(t string) {
	v.b.WriteStrings(`{"":"`, t, `"`)
}

var contentCode = map[rune][]byte{
	'b': []byte(`,"b":`),  // List of blocks
	'c': []byte(`,"c":[`), // List of list of blocks
	'g': []byte(`,"g":[`), // General list
	'i': []byte(`,"i":`),  // List of inlines
	'j': []byte(`,"j":{`), // Embedded JSON object
	'n': []byte(`,"n":`),  // Number
	'o': []byte(`,"o":"`), // Byte object
	'p': []byte(`,"p":[`), // Generic tuple
	'q': []byte(`,"q":`),  // String, if 's' is also needed
	's': []byte(`,"s":`),  // String
	't': []byte("Content code 't' is not allowed"),
	'v': []byte(`,"v":`),                           // String, if 'q' is also needed
	'y': []byte("Content code 'y' is not allowed"), // field after 'j'
}

func (v *visitor) writeContentStart(code rune) {
	if b, ok := contentCode[code]; ok {
		v.b.Write(b)
		return
	}
	panic("Unknown content code " + strconv.Itoa(int(code)))
}

func (v *visitor) writeMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) {
	for i, p := range m.ComputedPairs() {
		if i > 0 {
			v.b.WriteByte(',')
		}
		v.b.WriteByte('"')
		key := p.Key
		strfun.JSONEscape(&v.b, key)
		v.b.WriteString(`":`)
		t := m.Type(key)
		if t.IsSet {
			v.writeSetValue(p.Value)
			continue
		}
		if t == meta.TypeZettelmarkup {
			ast.Walk(v, evalMeta(p.Value))
			continue
		}
		writeEscaped(&v.b, p.Value)
	}
}

func (v *visitor) writeSetValue(value string) {
	v.b.WriteByte('[')
	for i, val := range meta.ListFromValue(value) {
		v.writeComma(i)
		writeEscaped(&v.b, val)
	}
	v.b.WriteByte(']')
}

func (v *visitor) writeComma(pos int) {
	if pos > 0 {
		v.b.WriteByte(',')
	}
}

func writeEscaped(b *encoder.BufWriter, s string) {
	b.WriteByte('"')
	strfun.JSONEscape(b, s)
	b.WriteByte('"')
}

Changes to encoder/encoder.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package encoder provides a generic interface to encode the abstract syntax
23
24
25
26
27
28
29
30
31


32
33
34
35
36

37
38
39
40
41
42
43
23
24
25
26
27
28
29


30
31
32
33
34
35

36
37
38
39
40
41
42
43







-
-
+
+




-
+







)

// Encoder is an interface that allows to encode different parts of a zettel.
type Encoder interface {
	WriteZettel(io.Writer, *ast.ZettelNode, EvalMetaFunc) (int, error)
	WriteMeta(io.Writer, *meta.Meta, EvalMetaFunc) (int, error)
	WriteContent(io.Writer, *ast.ZettelNode) (int, error)
	WriteBlocks(io.Writer, *ast.BlockSlice) (int, error)
	WriteInlines(io.Writer, *ast.InlineSlice) (int, error)
	WriteBlocks(io.Writer, *ast.BlockListNode) (int, error)
	WriteInlines(io.Writer, *ast.InlineListNode) (int, error)
}

// EvalMetaFunc is a function that takes a string of metadata and returns
// a list of syntax elements.
type EvalMetaFunc func(string) ast.InlineSlice
type EvalMetaFunc func(string) *ast.InlineListNode

// Some errors to signal when encoder methods are not implemented.
var (
	ErrNoWriteZettel  = errors.New("method WriteZettel is not implemented")
	ErrNoWriteMeta    = errors.New("method WriteMeta is not implemented")
	ErrNoWriteContent = errors.New("method WriteContent is not implemented")
	ErrNoWriteBlocks  = errors.New("method WriteBlocks is not implemented")
85
86
87
88
89
90
91
92
93


94
95
96
85
86
87
88
89
90
91


92
93
94
95
96







-
-
+
+



}

// GetDefaultEncoding returns the encoding that should be used as default.
func GetDefaultEncoding() api.EncodingEnum {
	if defEncoding != api.EncoderUnknown {
		return defEncoding
	}
	if _, ok := registry[api.EncoderZJSON]; ok {
		return api.EncoderZJSON
	if _, ok := registry[api.EncoderDJSON]; ok {
		return api.EncoderDJSON
	}
	panic("No default encoding given")
}

Changes to encoder/encoder_blob_test.go.

35
36
37
38
39
40
41
42

43
44
45
46
47
48
49
50
51
52
53
54
55
56

57
58
59
35
36
37
38
39
40
41

42
43
44
45
46
47
48
49
50
51
52
53
54
55

56
57
58
59







-
+













-
+



			0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d, 0x49, 0x48, 0x44, 0x52,
			0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01, 0x08, 0x00, 0x00, 0x00, 0x00, 0x3a, 0x7e, 0x9b,
			0x55, 0x00, 0x00, 0x00, 0x0a, 0x49, 0x44, 0x41, 0x54, 0x78, 0x9c, 0x63, 0x62, 0x00, 0x00, 0x00,
			0x06, 0x00, 0x03, 0x36, 0x37, 0x7c, 0xa8, 0x00, 0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae,
			0x42, 0x60, 0x82,
		},
		expect: expectMap{
			encoderZJSON:  `[{"":"BLOB","q":"PNG","s":"png","o":"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="}]`,
			encoderDJSON:  `[{"":"Blob","q":"PNG","s":"png","o":"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="}]`,
			encoderHTML:   `<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg==" title="PNG">`,
			encoderNative: `[BLOB "PNG" "png" "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAAAAAA6fptVAAAACklEQVR4nGNiAAAABgADNjd8qAAAAABJRU5ErkJggg=="]`,
			encoderText:   "",
			encoderZmk:    `%% Unable to display BLOB with title 'PNG' and syntax 'png'.`,
		},
	},
}

func TestBlob(t *testing.T) {
	m := meta.New(id.Invalid)
	m.Set(api.KeyTitle, "PNG")
	for testNum, tc := range pngTestCases {
		inp := input.NewInput(tc.blob)
		pe := &peBlocks{bs: parser.ParseBlocks(inp, m, "png")}
		pe := &peBlocks{bln: parser.ParseBlocks(inp, m, "png")}
		checkEncodings(t, testNum, pe, tc.descr, tc.expect, "???")
	}
}

Changes to encoder/encoder_block_test.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28
29

30
31
32
33
34
35
36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51

52
53
54
55
56
57
58
59
60
61
62

63
64
65
66
67
68
69
70
71
72
73

74
75
76
77
78
79
80
81
82
83
84
85
86
87

88
89
90
91
92
93
94
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
47
48
49
50

51
52
53
54
55
56
57
58
59
60
61

62
63
64
65
66
67
68
69
70
71
72

73
74
75
76
77
78
79
80
81
82
83
84
85
86

87
88
89
90
91
92
93
94







-
+










-
+










-
+










-
+










-
+










-
+













-
+







package encoder_test

var tcsBlock = []zmkTestCase{
	{
		descr: "Empty Zettelmarkup should produce near nothing",
		zmk:   "",
		expect: expectMap{
			encoderZJSON:  `[]`,
			encoderDJSON:  `[]`,
			encoderHTML:   "",
			encoderNative: ``,
			encoderText:   "",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple text: Hello, world",
		zmk:   "Hello, world",
		expect: expectMap{
			encoderZJSON:  `[{"":"Para","i":[{"":"Text","s":"Hello,"},{"":"Space"},{"":"Text","s":"world"}]}]`,
			encoderDJSON:  `[{"":"Para","i":[{"":"Text","s":"Hello,"},{"":"Space"},{"":"Text","s":"world"}]}]`,
			encoderHTML:   "<p>Hello, world</p>",
			encoderNative: `[Para Text "Hello,",Space,Text "world"]`,
			encoderText:   "Hello, world",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple block comment",
		zmk:   "%%%\nNo\nrender\n%%%",
		expect: expectMap{
			encoderZJSON:  `[{"":"CommentBlock","s":"No\nrender"}]`,
			encoderDJSON:  `[{"":"CommentBlock","s":"No\nrender"}]`,
			encoderHTML:   ``,
			encoderNative: `[CommentBlock "No\nrender"]`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Rendered block comment",
		zmk:   "%%%{-}\nRender\n%%%",
		expect: expectMap{
			encoderZJSON:  `[{"":"CommentBlock","a":{"-":""},"s":"Render"}]`,
			encoderDJSON:  `[{"":"CommentBlock","a":{"-":""},"s":"Render"}]`,
			encoderHTML:   "<!--\nRender\n-->",
			encoderNative: `[CommentBlock ("",[-]) "Render"]`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple Heading",
		zmk:   `=== Top`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Heading","n":1,"s":"top","i":[{"":"Text","s":"Top"}]}]`,
			encoderDJSON:  `[{"":"Heading","n":1,"s":"top","i":[{"":"Text","s":"Top"}]}]`,
			encoderHTML:   "<h2 id=\"top\">Top</h2>",
			encoderNative: `[Heading 1 #top Text "Top"]`,
			encoderText:   `Top`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Einfache Liste",
		zmk:   "* A\n* B\n* C",
		expect: expectMap{
			encoderZJSON: `[{"":"Bullet","c":[[{"":"Para","i":[{"":"Text","s":"A"}]}],[{"":"Para","i":[{"":"Text","s":"B"}]}],[{"":"Para","i":[{"":"Text","s":"C"}]}]]}]`,
			encoderDJSON: `[{"":"BulletList","c":[[{"":"Para","i":[{"":"Text","s":"A"}]}],[{"":"Para","i":[{"":"Text","s":"B"}]}],[{"":"Para","i":[{"":"Text","s":"C"}]}]]}]`,
			encoderHTML:  "<ul>\n<li>A</li>\n<li>B</li>\n<li>C</li>\n</ul>",
			encoderNative: `[BulletList
 [[Para Text "A"]],
 [[Para Text "B"]],
 [[Para Text "C"]]]`,
			encoderText: "A\nB\nC",
			encoderZmk:  useZmk,
		},
	},
	{
		descr: "Schachtelliste",
		zmk:   "* T1\n** T2\n* T3\n** T4\n* T5",
		expect: expectMap{
			encoderZJSON: `[{"":"Bullet","c":[[{"":"Para","i":[{"":"Text","s":"T1"}]},{"":"Bullet","c":[[{"":"Para","i":[{"":"Text","s":"T2"}]}]]}],[{"":"Para","i":[{"":"Text","s":"T3"}]},{"":"Bullet","c":[[{"":"Para","i":[{"":"Text","s":"T4"}]}]]}],[{"":"Para","i":[{"":"Text","s":"T5"}]}]]}]`,
			encoderDJSON: `[{"":"BulletList","c":[[{"":"Para","i":[{"":"Text","s":"T1"}]},{"":"BulletList","c":[[{"":"Para","i":[{"":"Text","s":"T2"}]}]]}],[{"":"Para","i":[{"":"Text","s":"T3"}]},{"":"BulletList","c":[[{"":"Para","i":[{"":"Text","s":"T4"}]}]]}],[{"":"Para","i":[{"":"Text","s":"T5"}]}]]}]`,
			encoderHTML: `<ul>
<li>
<p>T1</p>
<ul>
<li>T2</li>
</ul>
</li>
114
115
116
117
118
119
120
121

122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137

138
139
140
141
142
143
144
145
146
147
148

149
150
151
152
153
154
155
156
157

158
159
160


161
162

163
164
165
166
167
168
169
170
171
172

173
174
175
176
177
178
179
180
181
182
183
184
185

186
187
188
189
190
191
192
114
115
116
117
118
119
120

121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136

137
138
139
140
141
142
143
144
145
146
147

148
149
150
151
152
153
154
155
156

157
158


159
160
161

162
163
164
165
166
167
168
169
170
171

172
173
174
175
176
177
178
179
180
181
182
183
184

185
186
187
188
189
190
191
192







-
+















-
+










-
+








-
+

-
-
+
+

-
+









-
+












-
+







			encoderZmk:  useZmk,
		},
	},
	{
		descr: "Zwei Listen hintereinander",
		zmk:   "* Item1.1\n* Item1.2\n* Item1.3\n\n* Item2.1\n* Item2.2",
		expect: expectMap{
			encoderZJSON: `[{"":"Bullet","c":[[{"":"Para","i":[{"":"Text","s":"Item1.1"}]}],[{"":"Para","i":[{"":"Text","s":"Item1.2"}]}],[{"":"Para","i":[{"":"Text","s":"Item1.3"}]}],[{"":"Para","i":[{"":"Text","s":"Item2.1"}]}],[{"":"Para","i":[{"":"Text","s":"Item2.2"}]}]]}]`,
			encoderDJSON: `[{"":"BulletList","c":[[{"":"Para","i":[{"":"Text","s":"Item1.1"}]}],[{"":"Para","i":[{"":"Text","s":"Item1.2"}]}],[{"":"Para","i":[{"":"Text","s":"Item1.3"}]}],[{"":"Para","i":[{"":"Text","s":"Item2.1"}]}],[{"":"Para","i":[{"":"Text","s":"Item2.2"}]}]]}]`,
			encoderHTML:  "<ul>\n<li>Item1.1</li>\n<li>Item1.2</li>\n<li>Item1.3</li>\n<li>Item2.1</li>\n<li>Item2.2</li>\n</ul>",
			encoderNative: `[BulletList
 [[Para Text "Item1.1"]],
 [[Para Text "Item1.2"]],
 [[Para Text "Item1.3"]],
 [[Para Text "Item2.1"]],
 [[Para Text "Item2.2"]]]`,
			encoderText: "Item1.1\nItem1.2\nItem1.3\nItem2.1\nItem2.2",
			encoderZmk:  "* Item1.1\n* Item1.2\n* Item1.3\n* Item2.1\n* Item2.2",
		},
	},
	{
		descr: "Simple horizontal rule",
		zmk:   `---`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Thematic"}]`,
			encoderDJSON:  `[{"":"Hrule"}]`,
			encoderHTML:   "<hr>",
			encoderNative: `[Hrule]`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "No list after paragraph",
		zmk:   "Text\n*abc",
		expect: expectMap{
			encoderZJSON:  `[{"":"Para","i":[{"":"Text","s":"Text"},{"":"Soft"},{"":"Text","s":"*abc"}]}]`,
			encoderDJSON:  `[{"":"Para","i":[{"":"Text","s":"Text"},{"":"Soft"},{"":"Text","s":"*abc"}]}]`,
			encoderHTML:   "<p>Text\n*abc</p>",
			encoderNative: `[Para Text "Text",Space,Text "*abc"]`,
			encoderText:   `Text *abc`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "A list after paragraph",
		zmk:   "Text\n# abc",
		zmk:   "Text\n* abc",
		expect: expectMap{
			encoderZJSON: `[{"":"Para","i":[{"":"Text","s":"Text"}]},{"":"Ordered","c":[[{"":"Para","i":[{"":"Text","s":"abc"}]}]]}]`,
			encoderHTML:  "<p>Text</p>\n<ol>\n<li>abc</li>\n</ol>",
			encoderDJSON: `[{"":"Para","i":[{"":"Text","s":"Text"}]},{"":"BulletList","c":[[{"":"Para","i":[{"":"Text","s":"abc"}]}]]}]`,
			encoderHTML:  "<p>Text</p>\n<ul>\n<li>abc</li>\n</ul>",
			encoderNative: `[Para Text "Text"],
[OrderedList
[BulletList
 [[Para Text "abc"]]]`,
			encoderText: "Text\nabc",
			encoderZmk:  useZmk,
		},
	},
	{
		descr: "Simple Quote Block",
		zmk:   "<<<\nToBeOrNotToBe\n<<< Romeo",
		expect: expectMap{
			encoderZJSON: `[{"":"Excerpt","b":[{"":"Para","i":[{"":"Text","s":"ToBeOrNotToBe"}]}],"i":[{"":"Text","s":"Romeo"}]}]`,
			encoderDJSON: `[{"":"QuoteBlock","b":[{"":"Para","i":[{"":"Text","s":"ToBeOrNotToBe"}]}],"i":[{"":"Text","s":"Romeo"}]}]`,
			encoderHTML:  "<blockquote>\n<p>ToBeOrNotToBe</p>\n<cite>Romeo</cite>\n</blockquote>",
			encoderNative: `[QuoteBlock
 [[Para Text "ToBeOrNotToBe"]],
 [Cite Text "Romeo"]]`,
			encoderText: "ToBeOrNotToBe\nRomeo",
			encoderZmk:  useZmk,
		},
	},
	{
		descr: "Quote Block with multiple paragraphs",
		zmk:   "<<<\nToBeOr\n\nNotToBe\n<<< Romeo",
		expect: expectMap{
			encoderZJSON: `[{"":"Excerpt","b":[{"":"Para","i":[{"":"Text","s":"ToBeOr"}]},{"":"Para","i":[{"":"Text","s":"NotToBe"}]}],"i":[{"":"Text","s":"Romeo"}]}]`,
			encoderDJSON: `[{"":"QuoteBlock","b":[{"":"Para","i":[{"":"Text","s":"ToBeOr"}]},{"":"Para","i":[{"":"Text","s":"NotToBe"}]}],"i":[{"":"Text","s":"Romeo"}]}]`,
			encoderHTML:  "<blockquote>\n<p>ToBeOr</p>\n<p>NotToBe</p>\n<cite>Romeo</cite>\n</blockquote>",
			encoderNative: `[QuoteBlock
 [[Para Text "ToBeOr"],
  [Para Text "NotToBe"]],
 [Cite Text "Romeo"]]`,
			encoderText: "ToBeOr\nNotToBe\nRomeo",
			encoderZmk:  useZmk,
200
201
202
203
204
205
206
207

208
209
210


211
212
213
214
215
216
217
218
219
220
221
222

223
224
225
226
227
228
229
230
231
232
233
234

235
236
237
238
239
240
241
242
243
244
245

246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264

265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285

286
287
288

289
290
291
292


293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309

310
311
312
313
314
315
316
317
318
319
320
200
201
202
203
204
205
206

207
208


209
210
211
212
213
214
215
216
217
218
219
220
221

222
223
224
225
226
227
228
229
230
231
232
233

234
235
236
237
238
239
240
241
242
243
244

245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263

264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284

285
286
287

288
289
290


291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308

309
310
311
312
313
314
315
316
317
318
319
320







-
+

-
-
+
+











-
+











-
+










-
+


















-
+




















-
+


-
+


-
-
+
+
















-
+











Back

Paragraph

    Spacy  Para
""" Author`,
		expect: expectMap{
			encoderZJSON:  "[{\"\":\"Poem\",\"b\":[{\"\":\"Para\",\"i\":[{\"\":\"Text\",\"s\":\"A\"},{\"\":\"Space\",\"s\":\"\u00a0\"},{\"\":\"Text\",\"s\":\"line\"},{\"\":\"Hard\"},{\"\":\"Space\",\"s\":\"\u00a0\u00a0\"},{\"\":\"Text\",\"s\":\"another\"},{\"\":\"Space\",\"s\":\"\u00a0\"},{\"\":\"Text\",\"s\":\"line\"},{\"\":\"Hard\"},{\"\":\"Text\",\"s\":\"Back\"}]},{\"\":\"Para\",\"i\":[{\"\":\"Text\",\"s\":\"Paragraph\"}]},{\"\":\"Para\",\"i\":[{\"\":\"Space\",\"s\":\"\u00a0\u00a0\u00a0\u00a0\"},{\"\":\"Text\",\"s\":\"Spacy\"},{\"\":\"Space\",\"s\":\"\u00a0\u00a0\"},{\"\":\"Text\",\"s\":\"Para\"}]}],\"i\":[{\"\":\"Text\",\"s\":\"Author\"}]}]",
			encoderDJSON:  "[{\"\":\"VerseBlock\",\"b\":[{\"\":\"Para\",\"i\":[{\"\":\"Text\",\"s\":\"A\u00a0line\"},{\"\":\"Hard\"},{\"\":\"Text\",\"s\":\"\u00a0\u00a0another\u00a0line\"},{\"\":\"Hard\"},{\"\":\"Text\",\"s\":\"Back\"}]},{\"\":\"Para\",\"i\":[{\"\":\"Text\",\"s\":\"Paragraph\"}]},{\"\":\"Para\",\"i\":[{\"\":\"Text\",\"s\":\"\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\"}]}],\"i\":[{\"\":\"Text\",\"s\":\"Author\"}]}]",
			encoderHTML:   "<div>\n<p>A\u00a0line<br>\n\u00a0\u00a0another\u00a0line<br>\nBack</p>\n<p>Paragraph</p>\n<p>\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para</p>\n<cite>Author</cite>\n</div>",
			encoderNative: "[VerseBlock\n [[Para Text \"A\",Space,Text \"line\",Break,Space 2,Text \"another\",Space,Text \"line\",Break,Text \"Back\"],\n  [Para Text \"Paragraph\"],\n  [Para Space 4,Text \"Spacy\",Space 2,Text \"Para\"]],\n [Cite Text \"Author\"]]",
			encoderText:   "A line\n another line\nBack\nParagraph\n Spacy Para\nAuthor",
			encoderNative: "[VerseBlock\n [[Para Text \"A\u00a0line\",Break,Text \"\u00a0\u00a0another\u00a0line\",Break,Text \"Back\"],\n  [Para Text \"Paragraph\"],\n  [Para Text \"\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\"]],\n [Cite Text \"Author\"]]",
			encoderText:   "A\u00a0line\n\u00a0\u00a0another\u00a0line\nBack\nParagraph\n\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\nAuthor",
			encoderZmk:    "\"\"\"\nA\u00a0line\\\n\u00a0\u00a0another\u00a0line\\\nBack\nParagraph\n\u00a0\u00a0\u00a0\u00a0Spacy\u00a0\u00a0Para\n\"\"\" Author",
		},
	},
	{
		descr: "Span Block",
		zmk: `:::
A simple
   span
and much more
:::`,
		expect: expectMap{
			encoderZJSON: `[{"":"Block","b":[{"":"Para","i":[{"":"Text","s":"A"},{"":"Space"},{"":"Text","s":"simple"},{"":"Soft"},{"":"Space"},{"":"Text","s":"span"},{"":"Soft"},{"":"Text","s":"and"},{"":"Space"},{"":"Text","s":"much"},{"":"Space"},{"":"Text","s":"more"}]}]}]`,
			encoderDJSON: `[{"":"SpanBlock","b":[{"":"Para","i":[{"":"Text","s":"A"},{"":"Space"},{"":"Text","s":"simple"},{"":"Soft"},{"":"Space","n":3},{"":"Text","s":"span"},{"":"Soft"},{"":"Text","s":"and"},{"":"Space"},{"":"Text","s":"much"},{"":"Space"},{"":"Text","s":"more"}]}]}]`,
			encoderHTML:  "<div>\n<p>A simple\n span\nand much more</p>\n</div>",
			encoderNative: `[SpanBlock
 [[Para Text "A",Space,Text "simple",Space,Space 3,Text "span",Space,Text "and",Space,Text "much",Space,Text "more"]]]`,
			encoderText: `A simple  span and much more`,
			encoderZmk:  useZmk,
		},
	},
	{
		descr: "Simple Verbatim",
		zmk:   "```\nHello\nWorld\n```",
		expect: expectMap{
			encoderZJSON:  `[{"":"CodeBlock","s":"Hello\nWorld"}]`,
			encoderDJSON:  `[{"":"CodeBlock","s":"Hello\nWorld"}]`,
			encoderHTML:   "<pre><code>Hello\nWorld</code></pre>",
			encoderNative: `[CodeBlock "Hello\nWorld"]`,
			encoderText:   "Hello\nWorld",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple Description List",
		zmk:   "; Zettel\n: Paper\n: Note\n; Zettelkasten\n: Slip box",
		expect: expectMap{
			encoderZJSON: `[{"":"Description","d":[{"i":[{"":"Text","s":"Zettel"}],"e":[[{"":"Para","i":[{"":"Text","s":"Paper"}]}],[{"":"Para","i":[{"":"Text","s":"Note"}]}]]},{"i":[{"":"Text","s":"Zettelkasten"}],"e":[[{"":"Para","i":[{"":"Text","s":"Slip"},{"":"Space"},{"":"Text","s":"box"}]}]]}]}]`,
			encoderDJSON: `[{"":"DescriptionList","g":[[[{"":"Text","s":"Zettel"}],[{"":"Para","i":[{"":"Text","s":"Paper"}]}],[{"":"Para","i":[{"":"Text","s":"Note"}]}]],[[{"":"Text","s":"Zettelkasten"}],[{"":"Para","i":[{"":"Text","s":"Slip"},{"":"Space"},{"":"Text","s":"box"}]}]]]}]`,
			encoderHTML:  "<dl>\n<dt>Zettel</dt>\n<dd>Paper</dd>\n<dd>Note</dd>\n<dt>Zettelkasten</dt>\n<dd>Slip box</dd>\n</dl>",
			encoderNative: `[DescriptionList
 [Term [Text "Zettel"],
  [Description
   [Para Text "Paper"]],
  [Description
   [Para Text "Note"]]],
 [Term [Text "Zettelkasten"],
  [Description
   [Para Text "Slip",Space,Text "box"]]]]`,
			encoderText: "Zettel\nPaper\nNote\nZettelkasten\nSlip box",
			encoderZmk:  useZmk,
		},
	},
	{
		descr: "Simple Table",
		zmk:   "|c1|c2|c3\n|d1||d3",
		expect: expectMap{
			encoderZJSON: `[{"":"Table","p":[[],[[{"i":[{"":"Text","s":"c1"}]},{"i":[{"":"Text","s":"c2"}]},{"i":[{"":"Text","s":"c3"}]}],[{"i":[{"":"Text","s":"d1"}]},{"i":[]},{"i":[{"":"Text","s":"d3"}]}]]]}]`,
			encoderDJSON: `[{"":"Table","p":[[],[[["",[{"":"Text","s":"c1"}]],["",[{"":"Text","s":"c2"}]],["",[{"":"Text","s":"c3"}]]],[["",[{"":"Text","s":"d1"}]],["",[]],["",[{"":"Text","s":"d3"}]]]]]}]`,
			encoderHTML: `<table>
<tbody>
<tr><td>c1</td><td>c2</td><td>c3</td></tr>
<tr><td>d1</td><td></td><td>d3</td></tr>
</tbody>
</table>`,
			encoderNative: `[Table
 [Row [Cell Default Text "c1"],[Cell Default Text "c2"],[Cell Default Text "c3"]],
 [Row [Cell Default Text "d1"],[Cell Default],[Cell Default Text "d3"]]]`,
			encoderText: "c1 c2 c3\nd1  d3",
			encoderZmk:  useZmk,
		},
	},
	{
		descr: "Table with alignment and comment",
		zmk: `|h1>|=h2|h3:|
|%--+---+---+
|<c1|c2|:c3|
|f1|f2|=f3`,
		expect: expectMap{
			encoderZJSON: `[{"":"Table","p":[[{"s":">","i":[{"":"Text","s":"h1"}]},{"i":[{"":"Text","s":"h2"}]},{"s":":","i":[{"":"Text","s":"h3"}]}],[[{"s":"<","i":[{"":"Text","s":"c1"}]},{"i":[{"":"Text","s":"c2"}]},{"s":":","i":[{"":"Text","s":"c3"}]}],[{"s":">","i":[{"":"Text","s":"f1"}]},{"i":[{"":"Text","s":"f2"}]},{"s":":","i":[{"":"Text","s":"=f3"}]}]]]}]`,
			encoderDJSON: `[{"":"Table","p":[[[">",[{"":"Text","s":"h1"}]],["",[{"":"Text","s":"h2"}]],[":",[{"":"Text","s":"h3"}]]],[[["<",[{"":"Text","s":"c1"}]],["",[{"":"Text","s":"c2"}]],[":",[{"":"Text","s":"c3"}]]],[[">",[{"":"Text","s":"f1"}]],["",[{"":"Text","s":"f2"}]],[":",[{"":"Text","s":"=f3"}]]]]]}]`,
			encoderHTML: `<table>
<thead>
<tr><th class="right">h1</th><th>h2</th><th class="center">h3</th></tr>
<tr><th class="zs-ta-right">h1</th><th>h2</th><th class="zs-ta-center">h3</th></tr>
</thead>
<tbody>
<tr><td class="left">c1</td><td>c2</td><td class="center">c3</td></tr>
<tr><td class="right">f1</td><td>f2</td><td class="center">=f3</td></tr>
<tr><td class="zs-ta-left">c1</td><td>c2</td><td class="zs-ta-center">c3</td></tr>
<tr><td class="zs-ta-right">f1</td><td>f2</td><td class="zs-ta-center">=f3</td></tr>
</tbody>
</table>`,
			encoderNative: `[Table
 [Header [Cell Right Text "h1"],[Cell Default Text "h2"],[Cell Center Text "h3"]],
 [Row [Cell Left Text "c1"],[Cell Default Text "c2"],[Cell Center Text "c3"]],
 [Row [Cell Right Text "f1"],[Cell Default Text "f2"],[Cell Center Text "=f3"]]]`,
			encoderText: "h1 h2 h3\nc1 c2 c3\nf1 f2 =f3",
			encoderZmk: `|=h1>|=h2|=h3:
|<c1|c2|c3
|f1|f2|=f3`,
		},
	},
	{
		descr: "",
		zmk:   ``,
		expect: expectMap{
			encoderZJSON:  `[]`,
			encoderDJSON:  `[]`,
			encoderHTML:   ``,
			encoderNative: ``,
			encoderText:   "",
			encoderZmk:    useZmk,
		},
	},
}

// func TestEncoderBlock(t *testing.T) {
// 	executeTestCases(t, tcsBlock)
// }

Changes to encoder/encoder_inline_test.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28
29

30
31
32
33
34
35
36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51

52
53
54
55
56
57
58
59
60
61
62

63
64
65
66
67
68
69
70
71
72
73

74
75
76
77
78
79
80
81
82
83
84

85
86
87
88
89
90











91
92
93
94
95

96
97
98
99
100
101
102
103
104
105
106

107
108
109
110
111
112
113
114
115
116
117
118


119
120
121
122
123
124
125
126
127
128
129


130
131
132
133
134











135
136
137
138
139

140
141
142
143
144
145
146
147
148
149
150

151
152
153
154
155
156
157
158
159

160
161

162
163
164
165
166
167
168
169
170
171
172

173
174
175
176
177
178
179
180
181
182
183
184


185
186
187
188
189
190
191
192
193
194

195
196
197
198
199
200
201
202
203
204
205

206
207
208
209
210
211
212
213
214
215
216

217
218
219
220
221
222
223
224
225
226
227

228
229
230
231
232
233
234
235
236
237
238
239


240
241
242
243
244
245
246
247
248
249
250
251



252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273



274
275
276
277
278
279
280
281
282
283
284



285
286
287
288
289
290
291
292
293
294


295
296
297
298
299
300
301
302
303
304

305
306

307
308
309
310
311
312
313
314
315

316
317
318
319
320
321
322
323
324
325
326

327
328

329
330
331
332
333
334
335
336
337

338
339
340
341
342
343
344
345
346
347
348

349
350

351
352
353
354
355
356
357
358
359

360
361
362
363
364
365
366
367
368
369
370

371
372
373
374
375
376
377
378
379
380
381

382
383
384
385
386
387
388
389
390
391
392

393
394
395
396
397
398
399
400
401
402
403

404
405
406
407
408
409
410
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
47
48
49
50

51
52
53
54
55
56
57
58
59
60
61

62
63
64
65
66
67
68
69
70
71
72

73
74
75
76
77
78
79
80
81
82
83

84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105

106
107
108
109
110
111
112
113
114
115
116

117
118
119
120
121
122
123
124
125
126
127


128
129
130
131
132
133
134
135
136
137
138


139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160

161
162
163
164
165
166
167
168
169
170
171

172
173
174
175
176
177
178
179
180

181
182

183
184
185
186
187
188
189
190
191
192
193

194
195
196
197
198
199
200
201
202
203
204


205
206
207
208
209
210
211
212
213
214
215

216
217
218
219
220
221
222
223
224
225
226

227
228
229
230
231
232
233
234
235
236
237

238
239
240
241
242
243
244
245
246
247
248

249
250
251
252
253
254
255
256
257
258
259


260
261
262
263
264
265
266
267
268
269
270



271
272
273
274
275
276
277
278











279
280
281



282
283
284
285
286
287
288
289
290
291
292



293
294
295
296
297
298
299
300
301
302
303


304
305
306
307
308
309
310
311
312
313
314

315
316

317
318
319
320
321
322
323
324
325

326
327
328
329
330
331
332
333
334
335
336

337
338

339
340
341
342
343
344
345
346
347

348
349
350
351
352
353
354
355
356
357
358

359
360

361
362
363
364
365
366
367
368
369

370
371
372
373
374
375
376
377
378
379
380

381
382
383
384
385
386
387
388
389
390
391

392
393
394
395
396
397
398
399
400
401
402

403
404
405
406
407
408
409
410
411
412
413

414
415
416
417
418
419
420
421







-
+










-
+










-
+










-
+










-
+










-
+










-
+






+
+
+
+
+
+
+
+
+
+
+




-
+










-
+










-
-
+
+









-
-
+
+





+
+
+
+
+
+
+
+
+
+
+




-
+










-
+








-
+

-
+










-
+










-
-
+
+









-
+










-
+










-
+










-
+










-
-
+
+









-
-
-
+
+
+





-
-
-
-
-
-
-
-
-
-
-



-
-
-
+
+
+








-
-
-
+
+
+








-
-
+
+









-
+

-
+








-
+










-
+

-
+








-
+










-
+

-
+








-
+










-
+










-
+










-
+










-
+







package encoder_test

var tcsInline = []zmkTestCase{
	{
		descr: "Empty Zettelmarkup should produce near nothing (inline)",
		zmk:   "",
		expect: expectMap{
			encoderZJSON:  `[]`,
			encoderDJSON:  `[]`,
			encoderHTML:   "",
			encoderNative: ``,
			encoderText:   "",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple text: Hello, world (inline)",
		zmk:   `Hello, world`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Text","s":"Hello,"},{"":"Space"},{"":"Text","s":"world"}]`,
			encoderDJSON:  `[{"":"Text","s":"Hello,"},{"":"Space"},{"":"Text","s":"world"}]`,
			encoderHTML:   "Hello, world",
			encoderNative: `Text "Hello,",Space,Text "world"`,
			encoderText:   "Hello, world",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Emphasized formatting",
		zmk:   "__emph__",
		expect: expectMap{
			encoderZJSON:  `[{"":"Emph","i":[{"":"Text","s":"emph"}]}]`,
			encoderDJSON:  `[{"":"Emph","i":[{"":"Text","s":"emph"}]}]`,
			encoderHTML:   "<em>emph</em>",
			encoderNative: `Emph [Text "emph"]`,
			encoderText:   "emph",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Strong formatting",
		zmk:   "**strong**",
		expect: expectMap{
			encoderZJSON:  `[{"":"Strong","i":[{"":"Text","s":"strong"}]}]`,
			encoderDJSON:  `[{"":"Strong","i":[{"":"Text","s":"strong"}]}]`,
			encoderHTML:   "<strong>strong</strong>",
			encoderNative: `Strong [Text "strong"]`,
			encoderText:   "strong",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Insert formatting",
		zmk:   ">>insert>>",
		expect: expectMap{
			encoderZJSON:  `[{"":"Insert","i":[{"":"Text","s":"insert"}]}]`,
			encoderDJSON:  `[{"":"Insert","i":[{"":"Text","s":"insert"}]}]`,
			encoderHTML:   "<ins>insert</ins>",
			encoderNative: `Insert [Text "insert"]`,
			encoderText:   "insert",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Delete formatting",
		zmk:   "~~delete~~",
		expect: expectMap{
			encoderZJSON:  `[{"":"Delete","i":[{"":"Text","s":"delete"}]}]`,
			encoderDJSON:  `[{"":"Delete","i":[{"":"Text","s":"delete"}]}]`,
			encoderHTML:   "<del>delete</del>",
			encoderNative: `Delete [Text "delete"]`,
			encoderText:   "delete",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Update formatting",
		zmk:   "~~old~~>>new>>",
		expect: expectMap{
			encoderZJSON:  `[{"":"Delete","i":[{"":"Text","s":"old"}]},{"":"Insert","i":[{"":"Text","s":"new"}]}]`,
			encoderDJSON:  `[{"":"Delete","i":[{"":"Text","s":"old"}]},{"":"Insert","i":[{"":"Text","s":"new"}]}]`,
			encoderHTML:   "<del>old</del><ins>new</ins>",
			encoderNative: `Delete [Text "old"],Insert [Text "new"]`,
			encoderText:   "oldnew",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Monospace formatting",
		zmk:   "''monospace''",
		expect: expectMap{
			encoderDJSON:  `[{"":"Mono","i":[{"":"Text","s":"monospace"}]}]`,
			encoderHTML:   `<span class="zs-monospace">monospace</span>`,
			encoderNative: `Mono [Text "monospace"]`,
			encoderText:   "monospace",
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Superscript formatting",
		zmk:   "^^superscript^^",
		expect: expectMap{
			encoderZJSON:  `[{"":"Super","i":[{"":"Text","s":"superscript"}]}]`,
			encoderDJSON:  `[{"":"Super","i":[{"":"Text","s":"superscript"}]}]`,
			encoderHTML:   `<sup>superscript</sup>`,
			encoderNative: `Super [Text "superscript"]`,
			encoderText:   `superscript`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Subscript formatting",
		zmk:   ",,subscript,,",
		expect: expectMap{
			encoderZJSON:  `[{"":"Sub","i":[{"":"Text","s":"subscript"}]}]`,
			encoderDJSON:  `[{"":"Sub","i":[{"":"Text","s":"subscript"}]}]`,
			encoderHTML:   `<sub>subscript</sub>`,
			encoderNative: `Sub [Text "subscript"]`,
			encoderText:   `subscript`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Quotes formatting",
		zmk:   `""quotes""`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Quote","i":[{"":"Text","s":"quotes"}]}]`,
			encoderHTML:   "<q>quotes</q>",
			encoderDJSON:  `[{"":"Quote","i":[{"":"Text","s":"quotes"}]}]`,
			encoderHTML:   `"quotes"`,
			encoderNative: `Quote [Text "quotes"]`,
			encoderText:   `quotes`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Quotes formatting (german)",
		zmk:   `""quotes""{lang=de}`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Quote","a":{"lang":"de"},"i":[{"":"Text","s":"quotes"}]}]`,
			encoderHTML:   `<q lang="de">quotes</q>`,
			encoderDJSON:  `[{"":"Quote","a":{"lang":"de"},"i":[{"":"Text","s":"quotes"}]}]`,
			encoderHTML:   `<span lang="de">&bdquo;quotes&ldquo;</span>`,
			encoderNative: `Quote ("",[lang="de"]) [Text "quotes"]`,
			encoderText:   `quotes`,
			encoderZmk:    `""quotes""{lang="de"}`,
		},
	},
	{
		descr: "Quotation formatting",
		zmk:   `<<quotation<<`,
		expect: expectMap{
			encoderDJSON:  `[{"":"Quotation","i":[{"":"Text","s":"quotation"}]}]`,
			encoderHTML:   `<q>quotation</q>`,
			encoderNative: `Quotation [Text "quotation"]`,
			encoderText:   `quotation`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Span formatting",
		zmk:   `::span::`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Span","i":[{"":"Text","s":"span"}]}]`,
			encoderDJSON:  `[{"":"Span","i":[{"":"Text","s":"span"}]}]`,
			encoderHTML:   `<span>span</span>`,
			encoderNative: `Span [Text "span"]`,
			encoderText:   `span`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Code formatting",
		zmk:   "``code``",
		expect: expectMap{
			encoderZJSON:  `[{"":"Code","s":"code"}]`,
			encoderDJSON:  `[{"":"Code","s":"code"}]`,
			encoderHTML:   `<code>code</code>`,
			encoderNative: `Code "code"`,
			encoderText:   `code`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Input formatting",
		zmk:   `''input''`,
		zmk:   `++input++`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Input","s":"input"}]`,
			encoderDJSON:  `[{"":"Input","s":"input"}]`,
			encoderHTML:   `<kbd>input</kbd>`,
			encoderNative: `Input "input"`,
			encoderText:   `input`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Output formatting",
		zmk:   `==output==`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Output","s":"output"}]`,
			encoderDJSON:  `[{"":"Output","s":"output"}]`,
			encoderHTML:   `<samp>output</samp>`,
			encoderNative: `Output "output"`,
			encoderText:   `output`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Nested Span Quote formatting",
		zmk:   `::""abc""::{lang=fr}`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Span","a":{"lang":"fr"},"i":[{"":"Quote","i":[{"":"Text","s":"abc"}]}]}]`,
			encoderHTML:   `<span lang="fr"><q>abc</q></span>`,
			encoderDJSON:  `[{"":"Span","a":{"lang":"fr"},"i":[{"":"Quote","i":[{"":"Text","s":"abc"}]}]}]`,
			encoderHTML:   `<span lang="fr">&laquo;&nbsp;abc&nbsp;&raquo;</span>`,
			encoderNative: `Span ("",[lang="fr"]) [Quote [Text "abc"]]`,
			encoderText:   `abc`,
			encoderZmk:    `::""abc""::{lang="fr"}`,
		},
	},
	{
		descr: "Simple Citation",
		zmk:   `[@Stern18]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Cite","s":"Stern18"}]`,
			encoderDJSON:  `[{"":"Cite","s":"Stern18"}]`,
			encoderHTML:   `Stern18`, // TODO
			encoderNative: `Cite "Stern18"`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "No comment",
		zmk:   `% comment`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Text","s":"%"},{"":"Space"},{"":"Text","s":"comment"}]`,
			encoderDJSON:  `[{"":"Text","s":"%"},{"":"Space"},{"":"Text","s":"comment"}]`,
			encoderHTML:   `% comment`,
			encoderNative: `Text "%",Space,Text "comment"`,
			encoderText:   `% comment`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Line comment",
		zmk:   `%% line comment`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Comment","s":"line comment"}]`,
			encoderDJSON:  `[{"":"Comment","s":"line comment"}]`,
			encoderHTML:   `<!-- line comment -->`,
			encoderNative: `Comment "line comment"`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Comment after text",
		zmk:   `Text %% comment`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Text","s":"Text"},{"":"Comment","s":"comment"}]`,
			encoderDJSON:  `[{"":"Text","s":"Text"},{"":"Comment","s":"comment"}]`,
			encoderHTML:   `Text <!-- comment -->`,
			encoderNative: `Text "Text",Comment "comment"`,
			encoderText:   `Text`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple footnote",
		zmk:   `[^footnote]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Footnote","i":[{"":"Text","s":"footnote"}]}]`,
			encoderHTML:   `<sup id="fnref:0"><a href="#fn:0" class="footnote-ref" role="doc-noteref">0</a></sup>`,
			encoderDJSON:  `[{"":"Footnote","i":[{"":"Text","s":"footnote"}]}]`,
			encoderHTML:   `<sup id="fnref:0"><a href="#fn:0" class="zs-footnote-ref" role="doc-noteref">0</a></sup>`,
			encoderNative: `Footnote [Text "footnote"]`,
			encoderText:   `footnote`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple mark",
		zmk:   `[!mark]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Mark","s":"mark","q":"mark"}]`,
			encoderHTML:   `<a id="mark"></a>`,
			encoderNative: `Mark "mark" #mark`,
			encoderDJSON:  `[{"":"Mark","s":"mark"}]`,
			encoderHTML:   ``,
			encoderNative: `Mark "mark"`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Mark with text",
		zmk:   `[!mark|with text]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Mark","s":"mark","q":"mark","i":[{"":"Text","s":"with"},{"":"Space"},{"":"Text","s":"text"}]}]`,
			encoderHTML:   `<a id="mark">with text</a>`,
			encoderNative: `Mark "mark" #mark [Text "with",Space,Text "text"]`,
			encoderText:   `with text`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Dummy Link",
		zmk:   `[[abc]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"external","s":"abc"}]`,
			encoderHTML:   `<a href="abc" class="external">abc</a>`,
			encoderNative: `Link EXTERNAL "abc"`,
			encoderDJSON:  `[{"":"Link","q":"external","s":"abc","i":[{"":"Text","s":"abc"}]}]`,
			encoderHTML:   `<a href="abc" class="zs-external">abc</a>`,
			encoderNative: `Link EXTERNAL "abc" []`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple URL",
		zmk:   `[[https://zettelstore.de]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"external","s":"https://zettelstore.de"}]`,
			encoderHTML:   `<a href="https://zettelstore.de" class="external">https://zettelstore.de</a>`,
			encoderNative: `Link EXTERNAL "https://zettelstore.de"`,
			encoderDJSON:  `[{"":"Link","q":"external","s":"https://zettelstore.de","i":[{"":"Text","s":"https://zettelstore.de"}]}]`,
			encoderHTML:   `<a href="https://zettelstore.de" class="zs-external">https://zettelstore.de</a>`,
			encoderNative: `Link EXTERNAL "https://zettelstore.de" []`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "URL with Text",
		zmk:   `[[Home|https://zettelstore.de]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"external","s":"https://zettelstore.de","i":[{"":"Text","s":"Home"}]}]`,
			encoderHTML:   `<a href="https://zettelstore.de" class="external">Home</a>`,
			encoderDJSON:  `[{"":"Link","q":"external","s":"https://zettelstore.de","i":[{"":"Text","s":"Home"}]}]`,
			encoderHTML:   `<a href="https://zettelstore.de" class="zs-external">Home</a>`,
			encoderNative: `Link EXTERNAL "https://zettelstore.de" [Text "Home"]`,
			encoderText:   `Home`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple Zettel ID",
		zmk:   `[[00000000000100]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"zettel","s":"00000000000100"}]`,
			encoderDJSON:  `[{"":"Link","q":"zettel","s":"00000000000100","i":[{"":"Text","s":"00000000000100"}]}]`,
			encoderHTML:   `<a href="00000000000100">00000000000100</a>`,
			encoderNative: `Link ZETTEL "00000000000100"`,
			encoderNative: `Link ZETTEL "00000000000100" []`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Zettel ID with Text",
		zmk:   `[[Config|00000000000100]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"zettel","s":"00000000000100","i":[{"":"Text","s":"Config"}]}]`,
			encoderDJSON:  `[{"":"Link","q":"zettel","s":"00000000000100","i":[{"":"Text","s":"Config"}]}]`,
			encoderHTML:   `<a href="00000000000100">Config</a>`,
			encoderNative: `Link ZETTEL "00000000000100" [Text "Config"]`,
			encoderText:   `Config`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Simple Zettel ID with fragment",
		zmk:   `[[00000000000100#frag]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"zettel","s":"00000000000100#frag"}]`,
			encoderDJSON:  `[{"":"Link","q":"zettel","s":"00000000000100#frag","i":[{"":"Text","s":"00000000000100#frag"}]}]`,
			encoderHTML:   `<a href="00000000000100#frag">00000000000100#frag</a>`,
			encoderNative: `Link ZETTEL "00000000000100#frag"`,
			encoderNative: `Link ZETTEL "00000000000100#frag" []`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Zettel ID with Text and fragment",
		zmk:   `[[Config|00000000000100#frag]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"zettel","s":"00000000000100#frag","i":[{"":"Text","s":"Config"}]}]`,
			encoderDJSON:  `[{"":"Link","q":"zettel","s":"00000000000100#frag","i":[{"":"Text","s":"Config"}]}]`,
			encoderHTML:   `<a href="00000000000100#frag">Config</a>`,
			encoderNative: `Link ZETTEL "00000000000100#frag" [Text "Config"]`,
			encoderText:   `Config`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Fragment link to self",
		zmk:   `[[#frag]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"self","s":"#frag"}]`,
			encoderDJSON:  `[{"":"Link","q":"self","s":"#frag","i":[{"":"Text","s":"#frag"}]}]`,
			encoderHTML:   `<a href="#frag">#frag</a>`,
			encoderNative: `Link SELF "#frag"`,
			encoderNative: `Link SELF "#frag" []`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Hosted link",
		zmk:   `[[H|/hosted]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"local","s":"/hosted","i":[{"":"Text","s":"H"}]}]`,
			encoderDJSON:  `[{"":"Link","q":"local","s":"/hosted","i":[{"":"Text","s":"H"}]}]`,
			encoderHTML:   `<a href="/hosted">H</a>`,
			encoderNative: `Link LOCAL "/hosted" [Text "H"]`,
			encoderText:   `H`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Based link",
		zmk:   `[[B|/based]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"local","s":"/based","i":[{"":"Text","s":"B"}]}]`,
			encoderDJSON:  `[{"":"Link","q":"local","s":"/based","i":[{"":"Text","s":"B"}]}]`,
			encoderHTML:   `<a href="/based">B</a>`,
			encoderNative: `Link LOCAL "/based" [Text "B"]`,
			encoderText:   `B`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Relative link",
		zmk:   `[[R|../relative]]`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Link","q":"local","s":"../relative","i":[{"":"Text","s":"R"}]}]`,
			encoderDJSON:  `[{"":"Link","q":"local","s":"../relative","i":[{"":"Text","s":"R"}]}]`,
			encoderHTML:   `<a href="../relative">R</a>`,
			encoderNative: `Link LOCAL "../relative" [Text "R"]`,
			encoderText:   `R`,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "Dummy Embed",
		zmk:   `{{abc}}`,
		expect: expectMap{
			encoderZJSON:  `[{"":"Embed","s":"abc"}]`,
			encoderDJSON:  `[{"":"Embed","s":"abc"}]`,
			encoderHTML:   `<img src="abc" alt="">`,
			encoderNative: `Embed EXTERNAL "abc"`,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
	{
		descr: "",
		zmk:   ``,
		expect: expectMap{
			encoderZJSON:  `[]`,
			encoderDJSON:  `[]`,
			encoderHTML:   ``,
			encoderNative: ``,
			encoderText:   ``,
			encoderZmk:    useZmk,
		},
	},
}

Changes to encoder/encoder_test.go.

1
2

3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

66
67
68
69

70
71
72
73
74
75
76
1

2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

28

29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

64


65

66
67
68
69
70
71
72
73

-
+

-
+



















+



-

-















-
+



















-
+
-
-

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
// Copyright (c) 2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package encoder_test

import (
	"bytes"
	"fmt"
	"testing"

	"zettelstore.de/c/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"

	_ "zettelstore.de/z/encoder/djsonenc"  // Allow to use DJSON encoder.
	_ "zettelstore.de/z/encoder/htmlenc"   // Allow to use HTML encoder.
	_ "zettelstore.de/z/encoder/nativeenc" // Allow to use native encoder.
	_ "zettelstore.de/z/encoder/textenc"   // Allow to use text encoder.
	_ "zettelstore.de/z/encoder/zjsonenc"  // Allow to use ZJSON encoder.
	_ "zettelstore.de/z/encoder/zmkenc"    // Allow to use zmk encoder.
	"zettelstore.de/z/parser/cleaner"
	_ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser.
)

type zmkTestCase struct {
	descr  string
	zmk    string
	inline bool
	expect expectMap
}

type expectMap map[api.EncodingEnum]string

const useZmk = "\000"

const (
	encoderZJSON  = api.EncoderZJSON
	encoderDJSON  = api.EncoderDJSON
	encoderHTML   = api.EncoderHTML
	encoderNative = api.EncoderNative
	encoderText   = api.EncoderText
	encoderZmk    = api.EncoderZmk
)

func TestEncoder(t *testing.T) {
	for i := range tcsInline {
		tcsInline[i].inline = true
	}
	executeTestCases(t, append(tcsBlock, tcsInline...))
}

func executeTestCases(t *testing.T, testCases []zmkTestCase) {
	t.Helper()
	for testNum, tc := range testCases {
		inp := input.NewInput([]byte(tc.zmk))
		var pe parserEncoder
		if tc.inline {
			is := parser.ParseInlines(inp, api.ValueSyntaxZmk)
			pe = &peInlines{iln: parser.ParseInlines(inp, api.ValueSyntaxZmk)}
			cleaner.CleanInlineSlice(&is)
			pe = &peInlines{is: is}
		} else {
			pe = &peBlocks{bs: parser.ParseBlocks(inp, nil, api.ValueSyntaxZmk)}
			pe = &peBlocks{bln: parser.ParseBlocks(inp, nil, api.ValueSyntaxZmk)}
		}
		checkEncodings(t, testNum, pe, tc.descr, tc.expect, tc.zmk)
	}
}

func checkEncodings(t *testing.T, testNum int, pe parserEncoder, descr string, expected expectMap, zmkDefault string) {
	t.Helper()
97
98
99
100
101
102
103
104

105
106
107
108
109

110
111
112
113
114
115
116
117
118

119
120
121
122
123

124
125
126
127
128
129
94
95
96
97
98
99
100

101
102
103
104
105

106
107
108
109
110
111
112
113
114

115
116
117
118
119

120
121
122
123
124
125
126







-
+




-
+








-
+




-
+







type parserEncoder interface {
	encode(encoder.Encoder) (string, error)
	mode() string
}

type peInlines struct {
	is ast.InlineSlice
	iln *ast.InlineListNode
}

func (in peInlines) encode(encdr encoder.Encoder) (string, error) {
	var buf bytes.Buffer
	if _, err := encdr.WriteInlines(&buf, &in.is); err != nil {
	if _, err := encdr.WriteInlines(&buf, in.iln); err != nil {
		return "", err
	}
	return buf.String(), nil
}

func (peInlines) mode() string { return "inline" }

type peBlocks struct {
	bs ast.BlockSlice
	bln *ast.BlockListNode
}

func (bl peBlocks) encode(encdr encoder.Encoder) (string, error) {
	var buf bytes.Buffer
	if _, err := encdr.WriteBlocks(&buf, &bl.bs); err != nil {
	if _, err := encdr.WriteBlocks(&buf, bl.bln); err != nil {
		return "", err
	}
	return buf.String(), nil

}
func (peBlocks) mode() string { return "block" }

Changes to encoder/htmlenc/block.go.

8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
8
9
10
11
12
13
14
15
16
17
18
19
20


21
22
23
24
25
26
27







+





-
-







// under this license.
//-----------------------------------------------------------------------------

// Package htmlenc encodes the abstract syntax tree into HTML5.
package htmlenc

import (
	"bytes"
	"fmt"
	"strconv"
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/c/html"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/strfun"
)

func (v *visitor) visitVerbatim(vn *ast.VerbatimNode) {
	switch vn.Kind {
	case ast.VerbatimZettel:
46
47
48
49
50
51
52
53
54






55
56
57
58
59

















60
61
62
63

64
65
66


67

68
69
70
71
72
73
74
45
46
47
48
49
50
51


52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

83
84
85
86
87
88

89
90
91
92
93
94
95
96







-
-
+
+
+
+
+
+





+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+



-
+



+
+
-
+







		if vn.Attrs.HasDefault() {
			v.b.WriteString("<!--\n")
			v.writeHTMLEscaped(string(vn.Content))
			v.b.WriteString("\n-->")
		}

	case ast.VerbatimHTML:
		if html.IsSave(string(vn.Content)) {
			v.b.Write(vn.Content)
		lines := bytes.Split(vn.Content, []byte{'\n'})
		for _, line := range lines {
			sLine := string(line)
			if !ignoreHTMLText(sLine) {
				v.b.WriteStrings(sLine, "\n")
			}
		}
	default:
		panic(fmt.Sprintf("Unknown verbatim kind %v", vn.Kind))
	}
}

var htmlSnippetsIgnore = []string{
	"<script",
	"</script",
	"<iframe",
	"</iframe",
}

func ignoreHTMLText(s string) bool {
	lower := strings.ToLower(s)
	for _, snippet := range htmlSnippetsIgnore {
		if strings.Contains(lower, snippet) {
			return true
		}
	}
	return false
}

var specialSpanAttr = strfun.NewSet("example", "note", "tip", "important", "caution", "warning")

func processSpanAttributes(attrs zjson.Attributes) zjson.Attributes {
func processSpanAttributes(attrs *ast.Attributes) *ast.Attributes {
	if attrVal, ok := attrs.Get(""); ok {
		attrVal = strings.ToLower(attrVal)
		if specialSpanAttr.Has(attrVal) {
			attrs = attrs.Clone()
			attrs.Remove("")
			attrs = attrs.Clone().Remove("").AddClass("zs-indication").AddClass("zs-" + attrVal)
			attrs = attrs.AddClass("zs-indication").AddClass("zs-" + attrVal)
		}
	}
	return attrs
}

func (v *visitor) visitRegion(rn *ast.RegionNode) {
	var code string
82
83
84
85
86
87
88



89
90
91
92
93
94


95
96

97
98
99
100
101
102
103



104
105
106
107
108
109
110
111
112
113
114
115
116
117

118
119
120
121
122
123
124
125
126



127
128
129
130
131
132
133
104
105
106
107
108
109
110
111
112
113
114
115
116
117


118
119
120

121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144

145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164







+
+
+




-
-
+
+

-
+







+
+
+













-
+









+
+
+







		v.inVerse = true
		code = "div"
	case ast.RegionQuote:
		code = "blockquote"
	default:
		panic(fmt.Sprintf("Unknown region kind %v", rn.Kind))
	}

	v.lang.push(attrs)
	defer v.lang.pop()

	v.b.WriteStrings("<", code)
	v.visitAttributes(attrs)
	v.b.WriteString(">\n")
	ast.Walk(v, &rn.Blocks)
	if len(rn.Inlines) > 0 {
	ast.Walk(v, rn.Blocks)
	if rn.Inlines != nil {
		v.b.WriteString("\n<cite>")
		ast.Walk(v, &rn.Inlines)
		ast.Walk(v, rn.Inlines)
		v.b.WriteString("</cite>")
	}
	v.b.WriteStrings("\n</", code, ">")
	v.inVerse = oldVerse
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	v.lang.push(hn.Attrs)
	defer v.lang.pop()

	lvl := hn.Level + 1
	if lvl > 6 {
		lvl = 6 // HTML has H1..H6
	}
	strLvl := strconv.Itoa(lvl)
	v.b.WriteStrings("<h", strLvl)
	v.visitAttributes(hn.Attrs)
	if _, ok := hn.Attrs.Get("id"); !ok {
		if fragment := hn.Fragment; fragment != "" {
			v.b.WriteStrings(" id=\"", fragment, "\"")
		}
	}
	v.b.WriteByte('>')
	ast.Walk(v, &hn.Inlines)
	ast.Walk(v, hn.Inlines)
	v.b.WriteStrings("</h", strLvl, ">")
}

var mapNestedListKind = map[ast.NestedListKind]string{
	ast.NestedListOrdered:   "ol",
	ast.NestedListUnordered: "ul",
}

func (v *visitor) visitNestedList(ln *ast.NestedListNode) {
	v.lang.push(ln.Attrs)
	defer v.lang.pop()

	if ln.Kind == ast.NestedListQuote {
		// NestedListQuote -> HTML <blockquote> doesn't use <li>...</li>
		v.writeQuotationList(ln)
		return
	}

	code, ok := mapNestedListKind[ln.Kind]
154
155
156
157
158
159
160
161

162
163
164
165
166
167
168
185
186
187
188
189
190
191

192
193
194
195
196
197
198
199







-
+







		if pn := getParaItem(item); pn != nil {
			if inPara {
				v.b.WriteByte('\n')
			} else {
				v.b.WriteString("<p>")
				inPara = true
			}
			ast.Walk(v, &pn.Inlines)
			ast.Walk(v, pn.Inlines)
		} else {
			if inPara {
				v.writeEndPara()
				inPara = false
			}
			ast.WalkItemSlice(v, item)
		}
209
210
211
212
213
214
215
216

217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232

233
234
235
236
237
238
239
240
241
242
243

244
245
246
247
248
249
250
240
241
242
243
244
245
246

247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262

263
264
265
266
267
268
269
270
271
272
273

274
275
276
277
278
279
280
281







-
+















-
+










-
+








// writeItemSliceOrPara emits the content of a paragraph if the paragraph is
// the only element of the block slice and if compact mode is true. Otherwise,
// the item slice is emitted normally.
func (v *visitor) writeItemSliceOrPara(ins ast.ItemSlice, compact bool) {
	if compact && len(ins) == 1 {
		if para, ok := ins[0].(*ast.ParaNode); ok {
			ast.Walk(v, &para.Inlines)
			ast.Walk(v, para.Inlines)
			return
		}
	}
	for i, in := range ins {
		if i >= 0 {
			v.b.WriteByte('\n')
		}
		ast.Walk(v, in)
	}
	v.b.WriteByte('\n')
}

func (v *visitor) writeDescriptionsSlice(ds ast.DescriptionSlice) {
	if len(ds) == 1 {
		if para, ok := ds[0].(*ast.ParaNode); ok {
			ast.Walk(v, &para.Inlines)
			ast.Walk(v, para.Inlines)
			return
		}
	}
	ast.WalkDescriptionSlice(v, ds)
}

func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.b.WriteString("<dl>\n")
	for _, descr := range dn.Descriptions {
		v.b.WriteString("<dt>")
		ast.Walk(v, &descr.Term)
		ast.Walk(v, descr.Term)
		v.b.WriteString("</dt>\n")

		for _, b := range descr.Descriptions {
			v.b.WriteString("<dd>")
			v.writeDescriptionsSlice(b)
			v.b.WriteString("</dd>\n")
		}
267
268
269
270
271
272
273
274
275
276



277
278
279
280
281
282
283

284
285
286
287

288
289
290
291
292
293
294
298
299
300
301
302
303
304



305
306
307
308
309
310
311
312
313

314
315
316
317

318
319
320
321
322
323
324
325







-
-
-
+
+
+






-
+



-
+







		v.b.WriteString("</tbody>\n")
	}
	v.b.WriteString("</table>")
}

var alignStyle = map[ast.Alignment]string{
	ast.AlignDefault: ">",
	ast.AlignLeft:    " class=\"left\">",
	ast.AlignCenter:  " class=\"center\">",
	ast.AlignRight:   " class=\"right\">",
	ast.AlignLeft:    " class=\"zs-ta-left\">",
	ast.AlignCenter:  " class=\"zs-ta-center\">",
	ast.AlignRight:   " class=\"zs-ta-right\">",
}

func (v *visitor) writeRow(row ast.TableRow, cellStart, cellEnd string) {
	v.b.WriteString("<tr>")
	for _, cell := range row {
		v.b.WriteString(cellStart)
		if len(cell.Inlines) == 0 {
		if cell.Inlines.IsEmpty() {
			v.b.WriteByte('>')
		} else {
			v.b.WriteString(alignStyle[cell.Align])
			ast.Walk(v, &cell.Inlines)
			ast.Walk(v, cell.Inlines)
		}
		v.b.WriteString(cellEnd)
	}
	v.b.WriteString("</tr>\n")
}

func (v *visitor) visitBLOB(bn *ast.BLOBNode) {

Changes to encoder/htmlenc/htmlenc.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package htmlenc encodes the abstract syntax tree into HTML5.
45
46
47
48
49
50
51
52

53
54

55
56
57
58

59
60
61
62
63
64
65
45
46
47
48
49
50
51

52
53

54
55
56
57

58
59
60
61
62
63
64
65







-
+

-
+



-
+







	plainTitle, hasTitle := zn.InhMeta.Get(api.KeyTitle)
	if hasTitle {
		v.b.WriteStrings("<title>", v.evalValue(plainTitle, evalMeta), "</title>")
	}
	v.acceptMeta(zn.InhMeta, evalMeta)
	v.b.WriteString("\n</head>\n<body>\n")
	if hasTitle {
		if isTitle := evalMeta(plainTitle); len(isTitle) > 0 {
		if ilnTitle := evalMeta(plainTitle); ilnTitle != nil {
			v.b.WriteString("<h1>")
			ast.Walk(v, &isTitle)
			ast.Walk(v, ilnTitle)
			v.b.WriteString("</h1>\n")
		}
	}
	ast.Walk(v, &zn.Ast)
	ast.Walk(v, zn.Ast)
	v.writeEndnotes()
	v.b.WriteString("</body>\n</html>")
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as HTML5.
76
77
78
79
80
81
82
83

84
85
86
87

88
89

90
91
92
93
94
95
96

97
98
99
100
101

102
103
104
76
77
78
79
80
81
82

83
84
85
86

87
88

89
90
91
92
93
94
95

96
97
98
99
100

101
102
103
104







-
+



-
+

-
+






-
+




-
+



	// Write other metadata
	v.acceptMeta(m, evalMeta)
	length, err := v.b.Flush()
	return length, err
}

func (he *htmlEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return he.WriteBlocks(w, &zn.Ast)
	return he.WriteBlocks(w, zn.Ast)
}

// WriteBlocks encodes a block slice.
func (he *htmlEncoder) WriteBlocks(w io.Writer, bs *ast.BlockSlice) (int, error) {
func (he *htmlEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(he, w)
	ast.Walk(v, bs)
	ast.Walk(v, bln)
	v.writeEndnotes()
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (he *htmlEncoder) WriteInlines(w io.Writer, is *ast.InlineSlice) (int, error) {
func (he *htmlEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(he, w)
	if env := he.env; env != nil {
		v.inInteractive = env.Interactive
	}
	ast.Walk(v, is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

Changes to encoder/htmlenc/inline.go.

10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36



37
38
39

40
41
42

43
44

45
46
47

48
49
50
51

52
53
54
55
56
57

58
59
60
61
62
63
64

65

66
67
68
69
70
71

72
73

74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

91
92


93
94
95



96
97
98

99


100
101
102
103
104
105
106
107
108
109



110
111
112
113
114
115
116
117
118

119


120
121
122
123
124
125
126
127
128
129


130
131

132
133

134
135
136
137


138
139
140
141
142
143

144
145
146
147
148
149
150
151
152
153
154
155
156

157
158
159
160



161

162
163
164
165
166
167
168
169
170
171
172
173
174
175

176
177
178






179
180
181
182
183
184

185
186

187
188
189
190

191
192
193

































194
195
196





197
198
199
200
201
202
203

204
205
206
207
208
209
210
211
212
213
214
215

216
217
218
219
220
221
222
223

224
225
226
227
228
229
230
231
232
233
234
10
11
12
13
14
15
16
17
18
19


20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

41
42
43

44
45

46
47
48

49
50
51
52

53
54
55
56
57
58

59
60
61
62
63
64
65
66
67

68
69
70
71
72
73

74
75

76
77
78
79
80
81
82
83





84




85


86
87
88
89
90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122

123
124
125
126
127
128
129
130
131
132
133
134
135
136
137

138
139

140
141
142
143
144
145
146
147
148
149
150
151

152
153
154
155
156
157
158
159
160





161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183

184
185
186

187
188
189
190
191
192
193
194
195
196
197

198
199

200
201
202
203

204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240



241
242
243
244
245
246
247
248
249
250
251

252
253
254
255
256
257
258
259
260
261
262
263

264
265
266
267
268
269
270
271

272
273
274
275
276
277
278
279
280
281
282
283







+


-
-
















+
+
+


-
+


-
+

-
+


-
+



-
+





-
+







+
-
+





-
+

-
+







-
-
-
-
-

-
-
-
-
+
-
-
+
+



+
+
+



+
-
+
+










+
+
+









+
-
+
+










+
+

-
+

-
+




+
+





-
+








-
-
-
-
-
+




+
+
+

+













-
+


-
+
+
+
+
+
+





-
+

-
+



-
+



+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
+
+
+
+
+






-
+











-
+







-
+












// Package htmlenc encodes the abstract syntax tree into HTML5.
package htmlenc

import (
	"fmt"
	"strconv"
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/c/html"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
)

func (v *visitor) visitBreak(bn *ast.BreakNode) {
	if bn.Hard {
		if v.env.IsXHTML() {
			v.b.WriteString("<br />\n")
		} else {
			v.b.WriteString("<br>\n")
		}
	} else {
		v.b.WriteByte('\n')
	}
}

func (v *visitor) visitLink(ln *ast.LinkNode) {
	v.lang.push(ln.Attrs)
	defer v.lang.pop()

	switch ln.Ref.State {
	case ast.RefStateSelf, ast.RefStateFound, ast.RefStateHosted, ast.RefStateBased:
		v.writeAHref(ln.Ref, ln.Attrs, &ln.Inlines)
		v.writeAHref(ln.Ref, ln.Attrs, ln.Inlines)
	case ast.RefStateBroken:
		attrs := ln.Attrs.Clone()
		attrs = attrs.Set("class", "broken")
		attrs = attrs.Set("class", "zs-broken")
		attrs = attrs.Set("title", "Zettel not found") // l10n
		v.writeAHref(ln.Ref, attrs, &ln.Inlines)
		v.writeAHref(ln.Ref, attrs, ln.Inlines)
	case ast.RefStateExternal:
		attrs := ln.Attrs.Clone()
		attrs = attrs.Set("class", "external")
		attrs = attrs.Set("class", "zs-external")
		if v.env.HasNewWindow() {
			attrs = attrs.Set("target", "_blank").Set("rel", "noopener noreferrer")
		}
		v.writeAHref(ln.Ref, attrs, &ln.Inlines)
		v.writeAHref(ln.Ref, attrs, ln.Inlines)
		if v.env != nil {
			v.b.WriteString(v.env.MarkerExternal)
		}
	default:
		if v.env.IsInteractive(v.inInteractive) {
			v.writeSpan(&ln.Inlines, ln.Attrs)
			v.writeSpan(ln.Inlines, ln.Attrs)
			return
		}
		v.b.WriteString("<a href=\"")
		v.writeQuotedEscaped(ln.Ref.Value)
		v.b.WriteByte('"')
		v.visitAttributes(ln.Attrs)
		v.b.WriteByte('>')
		v.inInteractive = true
		v.writeLinkInlines(&ln.Inlines, ln.Ref)
		ast.Walk(v, ln.Inlines)
		v.inInteractive = false
		v.b.WriteString("</a>")
	}
}

func (v *visitor) writeAHref(ref *ast.Reference, attrs zjson.Attributes, is *ast.InlineSlice) {
func (v *visitor) writeAHref(ref *ast.Reference, attrs *ast.Attributes, iln *ast.InlineListNode) {
	if v.env.IsInteractive(v.inInteractive) {
		v.writeSpan(is, attrs)
		v.writeSpan(iln, attrs)
		return
	}
	v.b.WriteString("<a href=\"")
	v.writeReference(ref)
	v.b.WriteByte('"')
	v.visitAttributes(attrs)
	v.b.WriteByte('>')
	v.writeLinkInlines(is, ref)
	v.b.WriteString("</a>")
}
func (v *visitor) writeLinkInlines(is *ast.InlineSlice, ref *ast.Reference) {
	saveInteractive := v.inInteractive
	v.inInteractive = true
	if len(*is) == 0 {
		v.writeHTMLEscaped(ref.Value)
	} else {
		ast.Walk(v, is)
	ast.Walk(v, iln)
	}
	v.inInteractive = saveInteractive
	v.inInteractive = false
	v.b.WriteString("</a>")
}

func (v *visitor) visitEmbedRef(en *ast.EmbedRefNode) {
	v.lang.push(en.Attrs)
	defer v.lang.pop()

	v.b.WriteString("<img src=\"")
	v.writeReference(en.Ref)
	v.b.WriteString("\" alt=\"")
	if en.Inlines != nil {
	ast.Walk(v, &en.Inlines) // TODO: wrong, must use textenc
		ast.Walk(v, en.Inlines)
	}
	v.b.WriteByte('"')
	v.visitAttributes(en.Attrs)
	if v.env.IsXHTML() {
		v.b.WriteString(" />")
	} else {
		v.b.WriteByte('>')
	}
}

func (v *visitor) visitEmbedBLOB(en *ast.EmbedBLOBNode) {
	v.lang.push(en.Attrs)
	defer v.lang.pop()

	if en.Syntax == api.ValueSyntaxSVG {
		v.b.Write(en.Blob)
		return
	}

	v.b.WriteString("<img src=\"data:image/")
	v.b.WriteStrings(en.Syntax, ";base64,")
	v.b.WriteBase64(en.Blob)
	v.b.WriteString("\" alt=\"")
	if en.Inlines != nil {
	ast.Walk(v, &en.Inlines)
		ast.Walk(v, en.Inlines)
	}
	v.b.WriteByte('"')
	v.visitAttributes(en.Attrs)
	if v.env.IsXHTML() {
		v.b.WriteString(" />")
	} else {
		v.b.WriteByte('>')
	}
}

func (v *visitor) visitCite(cn *ast.CiteNode) {
	v.lang.push(cn.Attrs)
	defer v.lang.pop()
	v.b.WriteString(cn.Key)
	if len(cn.Inlines) > 0 {
	if cn.Inlines != nil {
		v.b.WriteString(", ")
		ast.Walk(v, &cn.Inlines)
		ast.Walk(v, cn.Inlines)
	}
}

func (v *visitor) visitFootnote(fn *ast.FootnoteNode) {
	v.lang.push(fn.Attrs)
	defer v.lang.pop()
	if v.env.IsInteractive(v.inInteractive) {
		return
	}

	n := strconv.Itoa(v.env.AddFootnote(fn))
	v.b.WriteStrings("<sup id=\"fnref:", n, "\"><a href=\"#fn:", n, "\" class=\"footnote-ref\" role=\"doc-noteref\">", n, "</a></sup>")
	v.b.WriteStrings("<sup id=\"fnref:", n, "\"><a href=\"#fn:", n, "\" class=\"zs-footnote-ref\" role=\"doc-noteref\">", n, "</a></sup>")
	// TODO: what to do with Attrs?
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	if v.env.IsInteractive(v.inInteractive) {
		return
	}
	if fragment := mn.Fragment; fragment != "" {
		v.b.WriteStrings(`<a id="`, fragment, `">`)
		if len(mn.Inlines) > 0 {
			ast.Walk(v, &mn.Inlines)
		}
		v.b.WriteString("</a>")
		v.b.WriteStrings("<a id=\"", fragment, "\"></a>")
	}
}

func (v *visitor) visitFormat(fn *ast.FormatNode) {
	v.lang.push(fn.Attrs)
	defer v.lang.pop()

	var code string
	attrs := fn.Attrs.Clone()
	switch fn.Kind {
	case ast.FormatEmph:
		code = "em"
	case ast.FormatStrong:
		code = "strong"
	case ast.FormatInsert:
		code = "ins"
	case ast.FormatDelete:
		code = "del"
	case ast.FormatSuper:
		code = "sup"
	case ast.FormatSub:
		code = "sub"
	case ast.FormatQuote:
	case ast.FormatQuotation:
		code = "q"
	case ast.FormatSpan:
		v.writeSpan(&fn.Inlines, processSpanAttributes(fn.Attrs))
		v.writeSpan(fn.Inlines, processSpanAttributes(attrs))
		return
	case ast.FormatMonospace:
		code, attrs = "span", attrs.AddClass("zs-monospace")
	case ast.FormatQuote:
		v.visitQuotes(fn)
		return
	default:
		panic(fmt.Sprintf("Unknown format kind %v", fn.Kind))
	}
	v.b.WriteStrings("<", code)
	v.visitAttributes(fn.Attrs)
	v.visitAttributes(attrs)
	v.b.WriteByte('>')
	ast.Walk(v, &fn.Inlines)
	ast.Walk(v, fn.Inlines)
	v.b.WriteStrings("</", code, ">")
}

func (v *visitor) writeSpan(is *ast.InlineSlice, attrs zjson.Attributes) {
func (v *visitor) writeSpan(iln *ast.InlineListNode, attrs *ast.Attributes) {
	v.b.WriteString("<span")
	v.visitAttributes(attrs)
	v.b.WriteByte('>')
	ast.Walk(v, iln)
	v.b.WriteString("</span>")

}

var langQuotes = map[string][2]string{
	api.ValueLangEN: {"&ldquo;", "&rdquo;"},
	"de":            {"&bdquo;", "&ldquo;"},
	"fr":            {"&laquo;&nbsp;", "&nbsp;&raquo;"},
}

func getQuotes(lang string) (string, string) {
	langFields := strings.FieldsFunc(lang, func(r rune) bool { return r == '-' || r == '_' })
	for len(langFields) > 0 {
		langSup := strings.Join(langFields, "-")
		quotes, ok := langQuotes[langSup]
		if ok {
			return quotes[0], quotes[1]
		}
		langFields = langFields[0 : len(langFields)-1]
	}
	return "\"", "\""
}

func (v *visitor) visitQuotes(fn *ast.FormatNode) {
	_, withSpan := fn.Attrs.Get("lang")
	if withSpan {
		v.b.WriteString("<span")
		v.visitAttributes(fn.Attrs)
		v.b.WriteByte('>')
	}
	openingQ, closingQ := getQuotes(v.lang.top())
	v.b.WriteString(openingQ)
	ast.Walk(v, is)
	v.b.WriteString("</span>")

	ast.Walk(v, fn.Inlines)
	v.b.WriteString(closingQ)
	if withSpan {
		v.b.WriteString("</span>")
	}
}

func (v *visitor) visitLiteral(ln *ast.LiteralNode) {
	switch ln.Kind {
	case ast.LiteralProg:
		v.writeLiteral("<code", "</code>", ln.Attrs, ln.Content)
	case ast.LiteralInput:
	case ast.LiteralKeyb:
		v.writeLiteral("<kbd", "</kbd>", ln.Attrs, ln.Content)
	case ast.LiteralOutput:
		v.writeLiteral("<samp", "</samp>", ln.Attrs, ln.Content)
	case ast.LiteralZettel, ast.LiteralComment:
		if v.inlinePos > 0 {
			v.b.WriteByte(' ')
		}
		v.b.WriteString("<!-- ")
		v.writeHTMLEscaped(string(ln.Content)) // writeCommentEscaped
		v.b.WriteString(" -->")
	case ast.LiteralHTML:
		if html.IsSave(string(ln.Content)) {
		if !ignoreHTMLText(string(ln.Content)) {
			v.b.Write(ln.Content)
		}
	default:
		panic(fmt.Sprintf("Unknown literal kind %v", ln.Kind))
	}
}

func (v *visitor) writeLiteral(codeS, codeE string, attrs zjson.Attributes, content []byte) {
func (v *visitor) writeLiteral(codeS, codeE string, attrs *ast.Attributes, content []byte) {
	oldVisible := v.visibleSpace
	if attrs != nil {
		v.visibleSpace = attrs.HasDefault()
	}
	v.b.WriteString(codeS)
	v.visitAttributes(attrs)
	v.b.WriteByte('>')
	v.writeHTMLEscaped(string(content))
	v.b.WriteString(codeE)
	v.visibleSpace = oldVisible
}

Added encoder/htmlenc/langstack.go.





































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package htmlenc encodes the abstract syntax tree into HTML5.
package htmlenc

import "zettelstore.de/z/ast"

type langStack struct {
	items []string
}

func newLangStack(lang string) langStack {
	items := make([]string, 1, 16)
	items[0] = lang
	return langStack{items}
}

func (s langStack) top() string { return s.items[len(s.items)-1] }

func (s *langStack) pop() { s.items = s.items[0 : len(s.items)-1] }

func (s *langStack) push(attrs *ast.Attributes) {
	if value, ok := attrs.Get("lang"); ok {
		s.items = append(s.items, value)
	} else {
		s.items = append(s.items, s.top())
	}
}

Added encoder/htmlenc/langstack_test.go.















































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package htmlenc encodes the abstract syntax tree into HTML5.
package htmlenc

import (
	"testing"

	"zettelstore.de/z/ast"
)

func TestStackSimple(t *testing.T) {
	t.Parallel()
	exp := "de"
	s := newLangStack(exp)
	if got := s.top(); got != exp {
		t.Errorf("Init: expected %q, but got %q", exp, got)
		return
	}

	a := &ast.Attributes{}
	s.push(a)
	if got := s.top(); exp != got {
		t.Errorf("Empty push: expected %q, but got %q", exp, got)
	}

	exp2 := "en"
	a = a.Set("lang", exp2)
	s.push(a)
	if got := s.top(); exp2 != got {
		t.Errorf("Full push: expected %q, but got %q", exp2, got)
	}

	s.pop()
	if got := s.top(); exp != got {
		t.Errorf("pop: expected %q, but got %q", exp, got)
	}
}

Changes to encoder/htmlenc/visitor.go.

9
10
11
12
13
14
15

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

32
33
34

35
36
37
38
39




40
41
42


43
44
45
46
47
48
49
50


51
52
53
54
55
56
57


58
59
60
61
62
63
64

65
66
67
68
69
70
71
9
10
11
12
13
14
15
16
17
18
19
20


21
22
23
24
25
26
27
28
29

30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53


54
55
56
57
58
59
60


61
62
63
64
65
66
67
68

69
70
71
72
73
74
75
76







+




-
-









-
+



+





+
+
+
+


-
+
+






-
-
+
+





-
-
+
+






-
+







//-----------------------------------------------------------------------------

package htmlenc

import (
	"bytes"
	"io"
	"sort"
	"strconv"
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/c/html"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	env           *encoder.Environment
	b             encoder.EncWriter
	b             encoder.BufWriter
	visibleSpace  bool // Show space character in plain text
	inVerse       bool // In verse block
	inInteractive bool // Rendered interactive HTML code
	lang          langStack
	textEnc       encoder.Encoder
	inlinePos     int // Element position in inline list node
}

func newVisitor(he *htmlEncoder, w io.Writer) *visitor {
	var lang string
	if he.env != nil {
		lang = he.env.Lang
	}
	return &visitor{
		env:     he.env,
		b:       encoder.NewEncWriter(w),
		b:       encoder.NewBufWriter(w),
		lang:    newLangStack(lang),
		textEnc: encoder.Create(api.EncoderText, nil),
	}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockSlice:
		for i, bn := range *n {
	case *ast.BlockListNode:
		for i, bn := range n.List {
			if i > 0 {
				v.b.WriteByte('\n')
			}
			ast.Walk(v, bn)
		}
	case *ast.InlineSlice:
		for i, in := range *n {
	case *ast.InlineListNode:
		for i, in := range n.List {
			v.inlinePos = i
			ast.Walk(v, in)
		}
		v.inlinePos = 0
	case *ast.ParaNode:
		v.b.WriteString("<p>")
		ast.Walk(v, &n.Inlines)
		ast.Walk(v, n.Inlines)
		v.writeEndPara()
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.visitHeading(n)
157
158
159
160
161
162
163
164
165

166
167
168
169
170
171
172
162
163
164
165
166
167
168


169
170
171
172
173
174
175
176







-
-
+







			v.writeMeta("zs-", key, value)
		}
	}
}

func (v *visitor) evalValue(value string, evalMeta encoder.EvalMetaFunc) string {
	var buf bytes.Buffer
	is := evalMeta(value)
	_, err := v.textEnc.WriteInlines(&buf, &is)
	_, err := v.textEnc.WriteInlines(&buf, evalMeta(value))
	if err == nil {
		return buf.String()
	}
	return ""
}

func (v *visitor) setupIgnoreSet() strfun.Set {
198
199
200
201
202
203
204
205

206
207
208
209

210
211
212
213

214
215
216
217
218
219
220

221
222
223



224
225
226






227
228
229
230
231

232
233
234
235
236
237
238
239
240
241
242

243
244

245
246
247
248
249

250
251
252
253
254
255
256
257
258
202
203
204
205
206
207
208

209
210
211
212

213
214
215
216

217
218
219
220
221
222
223

224
225
226
227
228
229
230



231
232
233
234
235
236
237
238
239
240

241
242
243
244
245
246
247
248
249
250
251

252
253

254
255
256
257
258

259
260
261
262
263
264
265
266
267
268







-
+



-
+



-
+






-
+



+
+
+
-
-
-
+
+
+
+
+
+




-
+










-
+

-
+




-
+









}

func (v *visitor) writeEndnotes() {
	fn, fnNum := v.env.PopFootnote()
	if fn == nil {
		return
	}
	v.b.WriteString("\n<ol class=\"endnotes\">\n")
	v.b.WriteString("\n<ol class=\"zs-endnotes\">\n")
	for fn != nil {
		n := strconv.Itoa(fnNum)
		v.b.WriteStrings("<li value=\"", n, "\" id=\"fn:", n, "\" role=\"doc-endnote\">")
		ast.Walk(v, &fn.Inlines)
		ast.Walk(v, fn.Inlines)
		v.b.WriteStrings(
			" <a href=\"#fnref:",
			n,
			"\" class=\"footnote-backref\" role=\"doc-backlink\">&#x21a9;&#xfe0e;</a></li>\n")
			"\" class=\"zs-footnote-backref\" role=\"doc-backlink\">&#x21a9;&#xfe0e;</a></li>\n")
		fn, fnNum = v.env.PopFootnote()
	}
	v.b.WriteString("</ol>\n")
}

// visitAttributes write HTML attributes
func (v *visitor) visitAttributes(a zjson.Attributes) {
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		if k != "-" {
	a = a.Clone().RemoveDefault()

	for _, k := range a.Keys() {
			keys = append(keys, k)
		}
	}
	sort.Strings(keys)

	for _, k := range keys {
		if k == "" || k == "-" {
			continue
		}
		v.b.WriteStrings(" ", k)
		vl := a[k]
		vl := a.Attrs[k]
		if len(vl) > 0 {
			v.b.WriteString("=\"")
			v.writeQuotedEscaped(vl)
			v.b.WriteByte('"')
		}
	}
}

func (v *visitor) writeHTMLEscaped(s string) {
	if v.visibleSpace {
		html.EscapeVisible(&v.b, s)
		strfun.HTMLEscapeVisible(&v.b, s)
	} else {
		html.Escape(&v.b, s)
		strfun.HTMLEscape(&v.b, s)
	}
}

func (v *visitor) writeQuotedEscaped(s string) {
	html.AttributeEscape(&v.b, s)
	strfun.HTMLAttrEscape(&v.b, s)
}

func (v *visitor) writeReference(ref *ast.Reference) {
	if ref.URL == nil {
		v.writeHTMLEscaped(ref.Value)
		return
	}
	v.b.WriteString(ref.URL.String())
}

Changes to encoder/nativeenc/nativeenc.go.

10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

42
43
44
45
46
47
48
49
50
51
52
53
54
55

56
57
58
59

60
61

62
63
64
65
66
67

68
69

70
71
72
73
74
75
76

77
78
79
80
81
82

83
84
85
86
87
88
89
90




91
92
93

94
95
96
97
98
99
100
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51
52
53
54

55
56
57
58

59
60

61
62
63
64
65
66

67
68

69
70
71
72
73
74
75

76
77
78
79
80
81

82
83
84
85
86




87
88
89
90
91
92

93
94
95
96
97
98
99
100







+



-




















-
+













-
+



-
+

-
+





-
+

-
+






-
+





-
+




-
-
-
-
+
+
+
+


-
+








// Package nativeenc encodes the abstract syntax tree into native format.
package nativeenc

import (
	"fmt"
	"io"
	"sort"
	"strconv"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
)

func init() {
	encoder.Register(api.EncoderNative, encoder.Info{
		Create: func(env *encoder.Environment) encoder.Encoder { return &nativeEncoder{env: env} },
	})
}

type nativeEncoder struct {
	env *encoder.Environment
}

// WriteZettel encodes the zettel to the writer.
func (ne *nativeEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(w, ne)
	v.acceptMeta(zn.InhMeta, evalMeta)
	v.b.WriteByte('\n')
	ast.Walk(v, &zn.Ast)
	ast.Walk(v, zn.Ast)
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data in native format.
func (ne *nativeEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(w, ne)
	v.acceptMeta(m, evalMeta)
	length, err := v.b.Flush()
	return length, err
}

func (ne *nativeEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return ne.WriteBlocks(w, &zn.Ast)
	return ne.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes a block slice to the writer
func (ne *nativeEncoder) WriteBlocks(w io.Writer, bs *ast.BlockSlice) (int, error) {
func (ne *nativeEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(w, ne)
	ast.Walk(v, bs)
	ast.Walk(v, bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (ne *nativeEncoder) WriteInlines(w io.Writer, is *ast.InlineSlice) (int, error) {
func (ne *nativeEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(w, ne)
	ast.Walk(v, is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b     encoder.EncWriter
	b     encoder.BufWriter
	level int
	env   *encoder.Environment
}

func newVisitor(w io.Writer, enc *nativeEncoder) *visitor {
	return &visitor{b: encoder.NewEncWriter(w), env: enc.env}
	return &visitor{b: encoder.NewBufWriter(w), env: enc.env}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockSlice:
		v.visitBlockSlice(n)
	case *ast.InlineSlice:
		v.walkInlineSlice(n)
	case *ast.BlockListNode:
		v.visitBlockList(n)
	case *ast.InlineListNode:
		v.walkInlineList(n)
	case *ast.ParaNode:
		v.b.WriteString("[Para ")
		ast.Walk(v, &n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.visitHeading(n)
122
123
124
125
126
127
128
129

130
131
132
133
134
135
136
122
123
124
125
126
127
128

129
130
131
132
133
134
135
136







-
+







		v.b.WriteByte('"')
	case *ast.TagNode:
		v.b.WriteString("Tag \"")
		v.writeEscaped(n.Tag)
		v.b.WriteByte('"')
	case *ast.SpaceNode:
		v.b.WriteString("Space")
		if l := n.Count(); l > 1 {
		if l := len(n.Lexeme); l > 1 {
			v.b.WriteByte(' ')
			v.b.WriteString(strconv.Itoa(l))
		}
	case *ast.BreakNode:
		if n.Hard {
			v.b.WriteString("Break")
		} else {
144
145
146
147
148
149
150
151

152
153

154
155
156
157
158
159
160

161
162
163
164
165
166
167
168

169
170
171
172
173
174
175
144
145
146
147
148
149
150

151
152

153
154
155
156
157
158
159

160
161
162
163
164
165
166
167

168
169
170
171
172
173
174
175







-
+

-
+






-
+







-
+







		v.visitEmbedBLOB(n)
	case *ast.CiteNode:
		v.b.WriteString("Cite")
		v.visitAttributes(n.Attrs)
		v.b.WriteString(" \"")
		v.writeEscaped(n.Key)
		v.b.WriteByte('"')
		if len(n.Inlines) > 0 {
		if n.Inlines != nil {
			v.b.WriteString(" [")
			ast.Walk(v, &n.Inlines)
			ast.Walk(v, n.Inlines)
			v.b.WriteByte(']')
		}
	case *ast.FootnoteNode:
		v.b.WriteString("Footnote")
		v.visitAttributes(n.Attrs)
		v.b.WriteString(" [")
		ast.Walk(v, &n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
	case *ast.MarkNode:
		v.visitMark(n)
	case *ast.FormatNode:
		v.b.Write(mapFormatKind[n.Kind])
		v.visitAttributes(n.Attrs)
		v.b.WriteString(" [")
		ast.Walk(v, &n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
	case *ast.LiteralNode:
		kind, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown literal kind %v", n.Kind))
		}
		v.b.Write(kind)
217
218
219
220
221
222
223
224
225

226
227
228
229
230
231
232
217
218
219
220
221
222
223


224
225
226
227
228
229
230
231







-
-
+







	v.b.WriteByte(']')
}

func (v *visitor) writeZettelmarkup(key, value string, evalMeta encoder.EvalMetaFunc) {
	v.b.WriteByte('[')
	v.b.WriteString(key)
	v.b.WriteByte(' ')
	is := evalMeta(value)
	ast.Walk(v, &is)
	ast.Walk(v, evalMeta(value))
	v.b.WriteByte(']')
}

func (v *visitor) writeMetaString(m *meta.Meta, key, native string) {
	if val, ok := m.Get(key); ok && len(val) > 0 {
		v.b.WriteStrings("\n[", native, " \"", val, "\"]")
	}
275
276
277
278
279
280
281
282

283
284
285

286
287
288
289

290
291
292
293
294
295
296
297
298
299
300
301
302
303

304
305
306
307
308
309
310
274
275
276
277
278
279
280

281
282
283

284
285
286
287

288
289
290
291
292
293
294
295
296
297
298
299
300
301

302
303
304
305
306
307
308
309







-
+


-
+



-
+













-
+







	}
	v.b.Write(kind)
	v.visitAttributes(rn.Attrs)
	v.level++
	v.writeNewLine()
	v.b.WriteByte('[')
	v.level++
	ast.Walk(v, &rn.Blocks)
	ast.Walk(v, rn.Blocks)
	v.level--
	v.b.WriteByte(']')
	if len(rn.Inlines) > 0 {
	if rn.Inlines != nil {
		v.b.WriteByte(',')
		v.writeNewLine()
		v.b.WriteString("[Cite ")
		ast.Walk(v, &rn.Inlines)
		ast.Walk(v, rn.Inlines)
		v.b.WriteByte(']')
	}
	v.level--
	v.b.WriteByte(']')
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	v.b.WriteStrings("[Heading ", strconv.Itoa(hn.Level))
	if fragment := hn.Fragment; fragment != "" {
		v.b.WriteStrings(" #", fragment)
	}
	v.visitAttributes(hn.Attrs)
	v.b.WriteByte(' ')
	ast.Walk(v, &hn.Inlines)
	ast.Walk(v, hn.Inlines)
	v.b.WriteByte(']')
}

var mapNestedListKind = map[ast.NestedListKind][]byte{
	ast.NestedListOrdered:   []byte("[OrderedList"),
	ast.NestedListUnordered: []byte("[BulletList"),
	ast.NestedListQuote:     []byte("[QuoteList"),
335
336
337
338
339
340
341
342

343
344
345
346
347
348
349
334
335
336
337
338
339
340

341
342
343
344
345
346
347
348







-
+







func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.b.WriteString("[DescriptionList")
	v.level++
	for i, descr := range dn.Descriptions {
		v.writeComma(i)
		v.writeNewLine()
		v.b.WriteString("[Term [")
		ast.Walk(v, &descr.Term)
		ast.Walk(v, descr.Term)
		v.b.WriteByte(']')

		if len(descr.Descriptions) > 0 {
			v.level++
			for _, b := range descr.Descriptions {
				v.b.WriteByte(',')
				v.writeNewLine()
399
400
401
402
403
404
405
406

407
408

409
410
411
412
413
414
415
398
399
400
401
402
403
404

405
406

407
408
409
410
411
412
413
414







-
+

-
+







	ast.AlignLeft:    " Left",
	ast.AlignCenter:  " Center",
	ast.AlignRight:   " Right",
}

func (v *visitor) writeCell(cell *ast.TableCell) {
	v.b.WriteStrings("[Cell", alignString[cell.Align])
	if len(cell.Inlines) > 0 {
	if !cell.Inlines.IsEmpty() {
		v.b.WriteByte(' ')
		ast.Walk(v, &cell.Inlines)
		ast.Walk(v, cell.Inlines)
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitBLOB(bn *ast.BLOBNode) {
	v.b.WriteString("[BLOB \"")
	v.writeEscaped(bn.Title)
438
439
440
441
442
443
444
445
446
447
448



449
450

451
452
453
454
455
456
457
458
459
460
461
462

463
464

465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481

482
483

484
485
486
487
488
489
490

491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514










515
516
517
518
519
520

521
522
523
524
525
526
527


528
529
530
531
532
533
534
535
536


537
538
539
540
541
542
543

544
545
546



547



548
549

550
551
552
553

554
555
556
557
558
559

560
561
562
563
564
565
566
437
438
439
440
441
442
443




444
445
446

447
448
449
450
451
452
453
454
455
456
457
458
459

460
461

462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478

479
480

481
482
483
484
485
486
487

488
489
490
491
492
493
494
495
496





497
498
499








500
501
502
503
504
505
506
507
508
509
510
511
512
513
514

515
516
517
518
519
520


521
522
523
524
525
526
527
528
529


530
531
532
533
534
535
536
537

538
539
540
541
542
543
544

545
546
547
548

549
550
551
552

553
554
555
556
557
558

559
560
561
562
563
564
565
566







-
-
-
-
+
+
+
-

+











-
+

-
+
















-
+

-
+






-
+








-
-
-
-
-



-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+





-
+





-
-
+
+







-
-
+
+






-
+



+
+
+
-
+
+
+

-
+



-
+





-
+







func (v *visitor) visitLink(ln *ast.LinkNode) {
	v.b.WriteString("Link")
	v.visitAttributes(ln.Attrs)
	v.b.WriteByte(' ')
	v.b.WriteString(mapRefState[ln.Ref.State])
	v.b.WriteString(" \"")
	v.writeEscaped(ln.Ref.String())
	v.b.WriteByte('"')
	if len(ln.Inlines) > 0 {
		v.b.WriteString(" [")
		ast.Walk(v, &ln.Inlines)
	v.b.WriteString("\" [")
	if !ln.OnlyRef {
		ast.Walk(v, ln.Inlines)
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitEmbedRef(en *ast.EmbedRefNode) {
	v.b.WriteString("Embed")
	v.visitAttributes(en.Attrs)
	v.b.WriteByte(' ')
	v.b.WriteString(mapRefState[en.Ref.State])
	v.b.WriteString(" \"")
	v.writeEscaped(en.Ref.String())
	v.b.WriteByte('"')

	if len(en.Inlines) > 0 {
	if en.Inlines != nil {
		v.b.WriteString(" [")
		ast.Walk(v, &en.Inlines)
		ast.Walk(v, en.Inlines)
		v.b.WriteByte(']')
	}
}

func (v *visitor) visitEmbedBLOB(en *ast.EmbedBLOBNode) {
	v.b.WriteString("EmbedBLOB")
	v.visitAttributes(en.Attrs)
	v.b.WriteStrings(" {\"", en.Syntax, "\" \"")
	if en.Syntax == api.ValueSyntaxSVG {
		v.writeEscaped(string(en.Blob))
	} else {
		v.b.WriteString("\" \"")
		v.b.WriteBase64(en.Blob)
	}
	v.b.WriteString("\"}")

	if len(en.Inlines) > 0 {
	if en.Inlines != nil {
		v.b.WriteString(" [")
		ast.Walk(v, &en.Inlines)
		ast.Walk(v, en.Inlines)
		v.b.WriteByte(']')
	}
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	v.b.WriteString("Mark")
	if text := mn.Mark; text != "" {
	if text := mn.Text; text != "" {
		v.b.WriteString(" \"")
		v.writeEscaped(text)
		v.b.WriteByte('"')
	}
	if fragment := mn.Fragment; fragment != "" {
		v.b.WriteString(" #")
		v.writeEscaped(fragment)
	}
	if len(mn.Inlines) > 0 {
		v.b.WriteString(" [")
		ast.Walk(v, &mn.Inlines)
		v.b.WriteByte(']')
	}
}

var mapFormatKind = map[ast.FormatKind][]byte{
	ast.FormatEmph:   []byte("Emph"),
	ast.FormatStrong: []byte("Strong"),
	ast.FormatInsert: []byte("Insert"),
	ast.FormatDelete: []byte("Delete"),
	ast.FormatSuper:  []byte("Super"),
	ast.FormatSub:    []byte("Sub"),
	ast.FormatQuote:  []byte("Quote"),
	ast.FormatSpan:   []byte("Span"),
	ast.FormatEmph:      []byte("Emph"),
	ast.FormatStrong:    []byte("Strong"),
	ast.FormatInsert:    []byte("Insert"),
	ast.FormatMonospace: []byte("Mono"),
	ast.FormatDelete:    []byte("Delete"),
	ast.FormatSuper:     []byte("Super"),
	ast.FormatSub:       []byte("Sub"),
	ast.FormatQuote:     []byte("Quote"),
	ast.FormatQuotation: []byte("Quotation"),
	ast.FormatSpan:      []byte("Span"),
}

var mapLiteralKind = map[ast.LiteralKind][]byte{
	ast.LiteralZettel:  []byte("Zettel"),
	ast.LiteralProg:    []byte("Code"),
	ast.LiteralInput:   []byte("Input"),
	ast.LiteralKeyb:    []byte("Input"),
	ast.LiteralOutput:  []byte("Output"),
	ast.LiteralComment: []byte("Comment"),
	ast.LiteralHTML:    []byte("HTML"),
}

func (v *visitor) visitBlockSlice(bs *ast.BlockSlice) {
	for i, bn := range *bs {
func (v *visitor) visitBlockList(bln *ast.BlockListNode) {
	for i, bn := range bln.List {
		if i > 0 {
			v.b.WriteByte(',')
			v.writeNewLine()
		}
		ast.Walk(v, bn)
	}
}
func (v *visitor) walkInlineSlice(is *ast.InlineSlice) {
	for i, in := range *is {
func (v *visitor) walkInlineList(iln *ast.InlineListNode) {
	for i, in := range iln.List {
		v.writeComma(i)
		ast.Walk(v, in)
	}
}

// visitAttributes write native attributes
func (v *visitor) visitAttributes(a zjson.Attributes) {
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)

	}
	sort.Strings(keys)

	v.b.WriteString(" (\"")
	if val, ok := a[""]; ok {
	if val, ok := a.Attrs[""]; ok {
		v.writeEscaped(val)
	}
	v.b.WriteString("\",[")
	for i, k := range a.Keys() {
	for i, k := range keys {
		if k == "" {
			continue
		}
		v.writeComma(i)
		v.b.WriteString(k)
		val := a[k]
		val := a.Attrs[k]
		if len(val) > 0 {
			v.b.WriteString("=\"")
			v.writeEscaped(val)
			v.b.WriteByte('"')
		}
	}
	v.b.WriteString("])")

Changes to encoder/textenc/textenc.go.

28
29
30
31
32
33
34
35

36
37
38
39
40
41
42

43
44


45
46
47
48
49

50
51
52
53
54
55
56
57
58








59

60
61
62
63
64
65
66
67
68
69
70

71
72
73
74

75
76

77
78
79
80
81
82

83
84

85
86
87
88
89
90
91

92
93
94
95
96

97
98
99
100
101
102
103
104




105
106
107
108
109
110
111
112
113
114
115


116
117

118
119
120
121
122
123
124
28
29
30
31
32
33
34

35
36
37
38
39
40
41

42
43
44
45
46
47
48
49


50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

68
69
70
71
72
73
74
75
76
77
78

79
80
81
82

83
84

85
86
87
88
89
90

91
92

93
94
95
96
97
98
99

100
101
102
103
104

105
106
107
108
109




110
111
112
113
114
115
116
117
118
119
120
121
122


123
124
125

126
127
128
129
130
131
132
133







-
+






-
+


+
+



-
-
+









+
+
+
+
+
+
+
+
-
+










-
+



-
+

-
+





-
+

-
+






-
+




-
+




-
-
-
-
+
+
+
+









-
-
+
+

-
+








type textEncoder struct{}

// WriteZettel writes metadata and content.
func (te *textEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(w)
	te.WriteMeta(&v.b, zn.InhMeta, evalMeta)
	v.visitBlockSlice(&zn.Ast)
	v.visitBlockList(zn.Ast)
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes metadata as text.
func (te *textEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	buf := encoder.NewEncWriter(w)
	buf := encoder.NewBufWriter(w)
	for _, pair := range m.ComputedPairs() {
		switch meta.Type(pair.Key) {
		case meta.TypeBool:
			writeBool(&buf, pair.Value)
		case meta.TypeTagSet:
			writeTagSet(&buf, meta.ListFromValue(pair.Value))
		case meta.TypeZettelmarkup:
			is := evalMeta(pair.Value)
			te.WriteInlines(&buf, &is)
			te.WriteInlines(&buf, evalMeta(pair.Value))
		default:
			buf.WriteString(pair.Value)
		}
		buf.WriteByte('\n')
	}
	length, err := buf.Flush()
	return length, err
}

func writeBool(buf *encoder.BufWriter, val string) {
	if meta.BoolValue(val) {
		buf.WriteString("true")
	} else {
		buf.WriteString("false")
	}
}

func writeTagSet(buf *encoder.EncWriter, tags []string) {
func writeTagSet(buf *encoder.BufWriter, tags []string) {
	for i, tag := range tags {
		if i > 0 {
			buf.WriteByte(' ')
		}
		buf.WriteString(meta.CleanTag(tag))
	}

}

func (te *textEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return te.WriteBlocks(w, &zn.Ast)
	return te.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes the content of a block slice to the writer.
func (*textEncoder) WriteBlocks(w io.Writer, bs *ast.BlockSlice) (int, error) {
func (*textEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(w)
	v.visitBlockSlice(bs)
	v.visitBlockList(bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (*textEncoder) WriteInlines(w io.Writer, is *ast.InlineSlice) (int, error) {
func (*textEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(w)
	ast.Walk(v, is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b         encoder.EncWriter
	b         encoder.BufWriter
	inlinePos int
}

func newVisitor(w io.Writer) *visitor {
	return &visitor{b: encoder.NewEncWriter(w)}
	return &visitor{b: encoder.NewBufWriter(w)}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockSlice:
		v.visitBlockSlice(n)
	case *ast.InlineSlice:
		for i, in := range *n {
	case *ast.BlockListNode:
		v.visitBlockList(n)
	case *ast.InlineListNode:
		for i, in := range n.List {
			v.inlinePos = i
			ast.Walk(v, in)
		}
		v.inlinePos = 0
		return nil
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
		return nil
	case *ast.RegionNode:
		v.visitBlockSlice(&n.Blocks)
		if len(n.Inlines) > 0 {
		v.visitBlockList(n.Blocks)
		if n.Inlines != nil {
			v.b.WriteByte('\n')
			ast.Walk(v, &n.Inlines)
			ast.Walk(v, n.Inlines)
		}
		return nil
	case *ast.NestedListNode:
		v.visitNestedList(n)
		return nil
	case *ast.DescriptionListNode:
		v.visitDescriptionList(n)
141
142
143
144
145
146
147
148
149


150
151
152
153
154
155
156
157
158
159
160
161
150
151
152
153
154
155
156


157
158





159
160
161
162
163
164
165







-
-
+
+
-
-
-
-
-







		if n.Hard {
			v.b.WriteByte('\n')
		} else {
			v.b.WriteByte(' ')
		}
		return nil
	case *ast.LinkNode:
		if len(n.Inlines) > 0 {
			ast.Walk(v, &n.Inlines)
		if !n.OnlyRef {
			ast.Walk(v, n.Inlines)
		}
		return nil
	case *ast.MarkNode:
		if len(n.Inlines) > 0 {
			ast.Walk(v, &n.Inlines)
		}
		return nil
	case *ast.FootnoteNode:
		if v.inlinePos > 0 {
			v.b.WriteByte(' ')
		}
		// No 'return nil' to write text
183
184
185
186
187
188
189
190

191
192
193
194
195
196
197
187
188
189
190
191
192
193

194
195
196
197
198
199
200
201







-
+







		}
	}
}

func (v *visitor) visitDescriptionList(dl *ast.DescriptionListNode) {
	for i, descr := range dl.Descriptions {
		v.writePosChar(i, '\n')
		ast.Walk(v, &descr.Term)
		ast.Walk(v, descr.Term)
		for _, b := range descr.Descriptions {
			v.b.WriteByte('\n')
			for k, d := range b {
				v.writePosChar(k, '\n')
				ast.Walk(v, d)
			}
		}
208
209
210
211
212
213
214
215

216
217
218
219
220


221
222
223
224
225
226
227
228
229
230
212
213
214
215
216
217
218

219
220
221
222


223
224
225
226
227
228
229
230
231
232
233
234







-
+



-
-
+
+










		v.writeRow(row)
	}
}

func (v *visitor) writeRow(row ast.TableRow) {
	for i, cell := range row {
		v.writePosChar(i, ' ')
		ast.Walk(v, &cell.Inlines)
		ast.Walk(v, cell.Inlines)
	}
}

func (v *visitor) visitBlockSlice(bs *ast.BlockSlice) {
	for i, bn := range *bs {
func (v *visitor) visitBlockList(bns *ast.BlockListNode) {
	for i, bn := range bns.List {
		v.writePosChar(i, '\n')
		ast.Walk(v, bn)
	}
}

func (v *visitor) writePosChar(pos int, ch byte) {
	if pos > 0 {
		v.b.WriteByte(ch)
	}
}

Deleted encoder/write.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85





















































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package encoder

import (
	"encoding/base64"
	"io"
)

// EncWriter is a specialized writer for encoding zettel.
type EncWriter struct {
	w      io.Writer // The io.Writer to write to
	err    error     // Collect error
	length int       // Collected length
}

// NewEncWriter creates a new EncWriter
func NewEncWriter(w io.Writer) EncWriter {
	return EncWriter{w: w}
}

// Write writes the content of p.
func (w *EncWriter) Write(p []byte) (l int, err error) {
	if w.err != nil {
		return 0, w.err
	}
	l, w.err = w.w.Write(p)
	w.length += l
	return l, w.err
}

// WriteString writes the content of s.
func (w *EncWriter) WriteString(s string) {
	if w.err != nil {
		return
	}
	var l int
	l, w.err = io.WriteString(w.w, s)
	w.length += l
}

// WriteStrings writes the content of sl.
func (w *EncWriter) WriteStrings(sl ...string) {
	for _, s := range sl {
		w.WriteString(s)
	}
}

// WriteByte writes the content of b.
func (w *EncWriter) WriteByte(b byte) error {
	var l int
	l, w.err = w.Write([]byte{b})
	w.length += l
	return w.err
}

// WriteBytes writes the content of bs.
func (w *EncWriter) WriteBytes(bs ...byte) {
	w.Write(bs)
}

// WriteBase64 writes the content of p, encoded with base64.
func (w *EncWriter) WriteBase64(p []byte) {
	if w.err == nil {
		encoder := base64.NewEncoder(base64.StdEncoding, w.w)
		var l int
		l, w.err = encoder.Write(p)
		w.length += l
		err1 := encoder.Close()
		if w.err == nil {
			w.err = err1
		}
	}
}

// Flush returns the collected length and error.
func (w *EncWriter) Flush() (int, error) { return w.length, w.err }

Deleted encoder/zjsonenc/zjsonenc.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554










































































































































































































































































































































































































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package zjsonenc encodes the abstract syntax tree into JSON.
package zjsonenc

import (
	"fmt"
	"io"
	"strconv"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

func init() {
	encoder.Register(api.EncoderZJSON, encoder.Info{
		Create: func(env *encoder.Environment) encoder.Encoder { return &jsonDetailEncoder{env: env} },
	})
}

type jsonDetailEncoder struct {
	env *encoder.Environment
}

// WriteZettel writes the encoded zettel to the writer.
func (je *jsonDetailEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newDetailVisitor(w, je)
	v.b.WriteString(`{"meta":`)
	v.writeMeta(zn.InhMeta, evalMeta)
	v.b.WriteString(`,"content":`)
	ast.Walk(v, &zn.Ast)
	v.b.WriteByte('}')
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as JSON.
func (je *jsonDetailEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newDetailVisitor(w, je)
	v.writeMeta(m, evalMeta)
	length, err := v.b.Flush()
	return length, err
}

func (je *jsonDetailEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return je.WriteBlocks(w, &zn.Ast)
}

// WriteBlocks writes a block slice to the writer
func (je *jsonDetailEncoder) WriteBlocks(w io.Writer, bs *ast.BlockSlice) (int, error) {
	v := newDetailVisitor(w, je)
	ast.Walk(v, bs)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (je *jsonDetailEncoder) WriteInlines(w io.Writer, is *ast.InlineSlice) (int, error) {
	v := newDetailVisitor(w, je)
	ast.Walk(v, is)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b       encoder.EncWriter
	env     *encoder.Environment
	inVerse bool // Visiting a verse block: save spaces in ZJSON object
}

func newDetailVisitor(w io.Writer, je *jsonDetailEncoder) *visitor {
	return &visitor{b: encoder.NewEncWriter(w), env: je.env}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockSlice:
		v.visitBlockSlice(n)
		return nil
	case *ast.InlineSlice:
		v.walkInlineSlice(n)
		return nil
	case *ast.ParaNode:
		v.writeNodeStart(zjson.TypeParagraph)
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &n.Inlines)
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.visitHeading(n)
	case *ast.HRuleNode:
		v.writeNodeStart(zjson.TypeBreakThematic)
		v.visitAttributes(n.Attrs)
	case *ast.NestedListNode:
		v.visitNestedList(n)
	case *ast.DescriptionListNode:
		v.visitDescriptionList(n)
	case *ast.TableNode:
		v.visitTable(n)
	case *ast.TranscludeNode:
		v.writeNodeStart(zjson.TypeTransclude)
		v.writeContentStart(zjson.NameString2)
		writeEscaped(&v.b, mapRefState[n.Ref.State])
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, n.Ref.String())
	case *ast.BLOBNode:
		v.visitBLOB(n)
	case *ast.TextNode:
		v.writeNodeStart(zjson.TypeText)
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, n.Text)
	case *ast.TagNode:
		v.writeNodeStart(zjson.TypeTag)
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, n.Tag)
	case *ast.SpaceNode:
		v.writeNodeStart(zjson.TypeSpace)
		if v.inVerse {
			v.writeContentStart(zjson.NameString)
			writeEscaped(&v.b, n.Lexeme)
		}
	case *ast.BreakNode:
		if n.Hard {
			v.writeNodeStart(zjson.TypeBreakHard)
		} else {
			v.writeNodeStart(zjson.TypeBreakSoft)
		}
	case *ast.LinkNode:
		v.writeNodeStart(zjson.TypeLink)
		v.visitAttributes(n.Attrs)
		v.writeContentStart(zjson.NameString2)
		writeEscaped(&v.b, mapRefState[n.Ref.State])
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, n.Ref.String())
		if len(n.Inlines) > 0 {
			v.writeContentStart(zjson.NameInline)
			ast.Walk(v, &n.Inlines)
		}
	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.EmbedBLOBNode:
		v.visitEmbedBLOB(n)
	case *ast.CiteNode:
		v.writeNodeStart(zjson.TypeCitation)
		v.visitAttributes(n.Attrs)
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, n.Key)
		if len(n.Inlines) > 0 {
			v.writeContentStart(zjson.NameInline)
			ast.Walk(v, &n.Inlines)
		}
	case *ast.FootnoteNode:
		v.writeNodeStart(zjson.TypeFootnote)
		v.visitAttributes(n.Attrs)
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &n.Inlines)
	case *ast.MarkNode:
		v.visitMark(n)
	case *ast.FormatNode:
		v.writeNodeStart(mapFormatKind[n.Kind])
		v.visitAttributes(n.Attrs)
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &n.Inlines)
	case *ast.LiteralNode:
		kind, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown literal kind %v", n.Kind))
		}
		v.writeNodeStart(kind)
		v.visitAttributes(n.Attrs)
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, string(n.Content))
	default:
		return v
	}
	v.b.WriteByte('}')
	return nil
}

var mapVerbatimKind = map[ast.VerbatimKind]string{
	ast.VerbatimZettel:  zjson.TypeVerbatimZettel,
	ast.VerbatimProg:    zjson.TypeVerbatimCode,
	ast.VerbatimComment: zjson.TypeVerbatimComment,
	ast.VerbatimHTML:    zjson.TypeVerbatimHTML,
}

func (v *visitor) visitVerbatim(vn *ast.VerbatimNode) {
	kind, ok := mapVerbatimKind[vn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown verbatim kind %v", vn.Kind))
	}
	v.writeNodeStart(kind)
	v.visitAttributes(vn.Attrs)
	v.writeContentStart(zjson.NameString)
	writeEscaped(&v.b, string(vn.Content))
}

var mapRegionKind = map[ast.RegionKind]string{
	ast.RegionSpan:  zjson.TypeBlock,
	ast.RegionQuote: zjson.TypeExcerpt,
	ast.RegionVerse: zjson.TypePoem,
}

func (v *visitor) visitRegion(rn *ast.RegionNode) {
	kind, ok := mapRegionKind[rn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown region kind %v", rn.Kind))
	}
	saveInVerse := v.inVerse
	if rn.Kind == ast.RegionVerse {
		v.inVerse = true
	}
	v.writeNodeStart(kind)
	v.visitAttributes(rn.Attrs)
	v.writeContentStart(zjson.NameBlock)
	ast.Walk(v, &rn.Blocks)
	if len(rn.Inlines) > 0 {
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &rn.Inlines)
	}
	v.inVerse = saveInVerse
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	v.writeNodeStart(zjson.TypeHeading)
	v.visitAttributes(hn.Attrs)
	v.writeContentStart(zjson.NameNumeric)
	v.b.WriteString(strconv.Itoa(hn.Level))
	if fragment := hn.Fragment; fragment != "" {
		v.writeContentStart(zjson.NameString)
		v.b.WriteStrings(`"`, fragment, `"`)
	}
	v.writeContentStart(zjson.NameInline)
	ast.Walk(v, &hn.Inlines)
}

var mapNestedListKind = map[ast.NestedListKind]string{
	ast.NestedListOrdered:   zjson.TypeListOrdered,
	ast.NestedListUnordered: zjson.TypeListBullet,
	ast.NestedListQuote:     zjson.TypeListQuotation,
}

func (v *visitor) visitNestedList(ln *ast.NestedListNode) {
	v.writeNodeStart(mapNestedListKind[ln.Kind])
	v.writeContentStart(zjson.NameList)
	for i, item := range ln.Items {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, in := range item {
			v.writeComma(j)
			ast.Walk(v, in)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.writeNodeStart(zjson.TypeDescrList)
	v.writeContentStart(zjson.NameDescrList)
	for i, def := range dn.Descriptions {
		v.writeComma(i)
		v.b.WriteStrings(`{"`, zjson.NameInline, `":`)
		ast.Walk(v, &def.Term)

		if len(def.Descriptions) > 0 {
			v.writeContentStart(zjson.NameDescription)
			for j, b := range def.Descriptions {
				v.writeComma(j)
				v.b.WriteByte('[')
				for k, dn := range b {
					v.writeComma(k)
					ast.Walk(v, dn)
				}
				v.b.WriteByte(']')
			}
			v.b.WriteByte(']')
		}
		v.b.WriteByte('}')
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitTable(tn *ast.TableNode) {
	v.writeNodeStart(zjson.TypeTable)
	v.writeContentStart(zjson.NameTable)

	// Table header
	v.b.WriteByte('[')
	for i, cell := range tn.Header {
		v.writeComma(i)
		v.writeCell(cell)
	}
	v.b.WriteString("],")

	// Table rows
	v.b.WriteByte('[')
	for i, row := range tn.Rows {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, cell := range row {
			v.writeComma(j)
			v.writeCell(cell)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteString("]]")
}

var alignmentCode = map[ast.Alignment]string{
	ast.AlignDefault: "",
	ast.AlignLeft:    "<",
	ast.AlignCenter:  ":",
	ast.AlignRight:   ">",
}

func (v *visitor) writeCell(cell *ast.TableCell) {
	if aCode := alignmentCode[cell.Align]; aCode != "" {
		v.b.WriteStrings(`{"`, zjson.NameString, `":"`, aCode, `","`, zjson.NameInline, `":`)
	} else {
		v.b.WriteStrings(`{"`, zjson.NameInline, `":`)
	}
	ast.Walk(v, &cell.Inlines)
	v.b.WriteByte('}')
}

func (v *visitor) visitBLOB(bn *ast.BLOBNode) {
	v.writeNodeStart(zjson.TypeBLOB)
	if bn.Title != "" {
		v.writeContentStart(zjson.NameString2)
		writeEscaped(&v.b, bn.Title)
	}
	v.writeContentStart(zjson.NameString)
	writeEscaped(&v.b, bn.Syntax)
	if bn.Syntax == api.ValueSyntaxSVG {
		v.writeContentStart(zjson.NameString3)
		writeEscaped(&v.b, string(bn.Blob))
	} else {
		v.writeContentStart(zjson.NameBinary)
		v.b.WriteBase64(bn.Blob)
		v.b.WriteByte('"')
	}
}

var mapRefState = map[ast.RefState]string{
	ast.RefStateInvalid:  zjson.RefStateInvalid,
	ast.RefStateZettel:   zjson.RefStateZettel,
	ast.RefStateSelf:     zjson.RefStateSelf,
	ast.RefStateFound:    zjson.RefStateFound,
	ast.RefStateBroken:   zjson.RefStateBroken,
	ast.RefStateHosted:   zjson.RefStateHosted,
	ast.RefStateBased:    zjson.RefStateBased,
	ast.RefStateExternal: zjson.RefStateExternal,
}

func (v *visitor) visitEmbedRef(en *ast.EmbedRefNode) {
	v.writeNodeStart(zjson.TypeEmbed)
	v.visitAttributes(en.Attrs)
	v.writeContentStart(zjson.NameString)
	writeEscaped(&v.b, en.Ref.String())

	if len(en.Inlines) > 0 {
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &en.Inlines)
	}
	if en.Syntax != "" {
		v.writeContentStart(zjson.NameString2)
		writeEscaped(&v.b, en.Syntax)
	}
}

func (v *visitor) visitEmbedBLOB(en *ast.EmbedBLOBNode) {
	v.writeNodeStart(zjson.TypeEmbedBLOB)
	v.visitAttributes(en.Attrs)
	v.writeContentStart(zjson.NameString)
	writeEscaped(&v.b, en.Syntax)
	if en.Syntax == api.ValueSyntaxSVG {
		v.writeContentStart(zjson.NameString3)
		writeEscaped(&v.b, string(en.Blob))
	} else {
		v.writeContentStart(zjson.NameBinary)
		v.b.WriteBase64(en.Blob)
		v.b.WriteByte('"')
	}
	if len(en.Inlines) > 0 {
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &en.Inlines)
	}
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	v.writeNodeStart(zjson.TypeMark)
	if text := mn.Mark; text != "" {
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, text)
	}
	if fragment := mn.Fragment; fragment != "" {
		v.writeContentStart(zjson.NameString2)
		v.b.WriteByte('"')
		v.b.WriteString(fragment)
		v.b.WriteByte('"')
	}
	if len(mn.Inlines) > 0 {
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &mn.Inlines)
	}
}

var mapFormatKind = map[ast.FormatKind]string{
	ast.FormatEmph:   zjson.TypeFormatEmph,
	ast.FormatStrong: zjson.TypeFormatStrong,
	ast.FormatDelete: zjson.TypeFormatDelete,
	ast.FormatInsert: zjson.TypeFormatInsert,
	ast.FormatSuper:  zjson.TypeFormatSuper,
	ast.FormatSub:    zjson.TypeFormatSub,
	ast.FormatQuote:  zjson.TypeFormatQuote,
	ast.FormatSpan:   zjson.TypeFormatSpan,
}

var mapLiteralKind = map[ast.LiteralKind]string{
	ast.LiteralZettel:  zjson.TypeLiteralZettel,
	ast.LiteralProg:    zjson.TypeLiteralCode,
	ast.LiteralInput:   zjson.TypeLiteralInput,
	ast.LiteralOutput:  zjson.TypeLiteralOutput,
	ast.LiteralComment: zjson.TypeLiteralComment,
	ast.LiteralHTML:    zjson.TypeLiteralHTML,
}

func (v *visitor) visitBlockSlice(bs *ast.BlockSlice) {
	v.b.WriteByte('[')
	for i, bn := range *bs {
		v.writeComma(i)
		ast.Walk(v, bn)
	}
	v.b.WriteByte(']')
}

func (v *visitor) walkInlineSlice(is *ast.InlineSlice) {
	v.b.WriteByte('[')
	for i, in := range *is {
		v.writeComma(i)
		ast.Walk(v, in)
	}
	v.b.WriteByte(']')
}

// visitAttributes write JSON attributes
func (v *visitor) visitAttributes(a zjson.Attributes) {
	if a.IsEmpty() {
		return
	}

	v.writeContentStart(zjson.NameAttribute)
	for i, k := range a.Keys() {
		if i > 0 {
			v.b.WriteString(`","`)
		}
		strfun.JSONEscape(&v.b, k)
		v.b.WriteString(`":"`)
		strfun.JSONEscape(&v.b, a[k])
	}
	v.b.WriteString(`"}`)
}

func (v *visitor) writeNodeStart(t string) {
	v.b.WriteStrings(`{"":"`, t, `"`)
}

var valueStart = map[string]string{
	zjson.NameBlock:       "",
	zjson.NameAttribute:   `{"`,
	zjson.NameList:        "[",
	zjson.NameDescrList:   "[",
	zjson.NameDescription: "[",
	zjson.NameInline:      "",
	zjson.NameBLOB:        "{",
	zjson.NameNumeric:     "",
	zjson.NameBinary:      `"`,
	zjson.NameTable:       "[",
	zjson.NameString2:     "",
	zjson.NameString:      "",
	zjson.NameString3:     "",
}

func (v *visitor) writeContentStart(jsonName string) {
	s, ok := valueStart[jsonName]
	if !ok {
		panic("Unknown object name " + jsonName)
	}
	v.b.WriteStrings(`,"`, jsonName, `":`, s)
}

func (v *visitor) writeMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) {
	v.b.WriteByte('{')
	for i, p := range m.ComputedPairs() {
		if i > 0 {
			v.b.WriteByte(',')
		}
		v.b.WriteByte('"')
		key := p.Key
		strfun.JSONEscape(&v.b, key)
		t := m.Type(key)
		v.b.WriteStrings(`":{"`, zjson.NameType, `":"`, t.Name, `","`)
		if t.IsSet {
			v.b.WriteStrings(zjson.NameSet, `":`)
			v.writeSetValue(p.Value)
		} else if t == meta.TypeZettelmarkup {
			v.b.WriteStrings(zjson.NameInline, `":`)
			is := evalMeta(p.Value)
			ast.Walk(v, &is)
		} else {
			v.b.WriteStrings(zjson.NameString, `":`)
			writeEscaped(&v.b, p.Value)
		}
		v.b.WriteByte('}')
	}
	v.b.WriteByte('}')
}

func (v *visitor) writeSetValue(value string) {
	v.b.WriteByte('[')
	for i, val := range meta.ListFromValue(value) {
		v.writeComma(i)
		writeEscaped(&v.b, val)
	}
	v.b.WriteByte(']')
}

func (v *visitor) writeComma(pos int) {
	if pos > 0 {
		v.b.WriteByte(',')
	}
}

func writeEscaped(b *encoder.EncWriter, s string) {
	b.WriteByte('"')
	strfun.JSONEscape(b, s)
	b.WriteByte('"')
}

Changes to encoder/zmkenc/zmkenc.go.

10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25
26







+


-








// Package zmkenc encodes the abstract syntax tree back into Zettelmarkup.
package zmkenc

import (
	"fmt"
	"io"
	"sort"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

func init() {
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

63
64
65
66
67
68
69
70
71

72
73
74
75

76
77

78
79
80
81
82
83

84
85

86
87
88
89
90
91
92

93
94
95
96
97
98
99
100
101

102
103
104
105
106
107
108
109
110
111




112
113
114
115
116
117
118
36
37
38
39
40
41
42

43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60


61
62
63
64
65
66
67
68
69

70
71
72
73

74
75

76
77
78
79
80
81

82
83

84
85
86
87
88
89
90

91
92
93
94
95
96
97
98
99

100
101
102
103
104
105
106




107
108
109
110
111
112
113
114
115
116
117







-
+

















-
-
+








-
+



-
+

-
+





-
+

-
+






-
+








-
+






-
-
-
-
+
+
+
+







	v := newVisitor(w, ze)
	v.acceptMeta(zn.InhMeta, evalMeta)
	if zn.InhMeta.YamlSep {
		v.b.WriteString("---\n")
	} else {
		v.b.WriteByte('\n')
	}
	ast.Walk(v, &zn.Ast)
	ast.Walk(v, zn.Ast)
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as zmk.
func (ze *zmkEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(w, ze)
	v.acceptMeta(m, evalMeta)
	length, err := v.b.Flush()
	return length, err
}

func (v *visitor) acceptMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) {
	for _, p := range m.ComputedPairs() {
		key := p.Key
		v.b.WriteStrings(key, ": ")
		if meta.Type(key) == meta.TypeZettelmarkup {
			is := evalMeta(p.Value)
			ast.Walk(v, &is)
			ast.Walk(v, evalMeta(p.Value))
		} else {
			v.b.WriteString(p.Value)
		}
		v.b.WriteByte('\n')
	}
}

func (ze *zmkEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return ze.WriteBlocks(w, &zn.Ast)
	return ze.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes the content of a block slice to the writer.
func (ze *zmkEncoder) WriteBlocks(w io.Writer, bs *ast.BlockSlice) (int, error) {
func (ze *zmkEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(w, ze)
	ast.Walk(v, bs)
	ast.Walk(v, bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (ze *zmkEncoder) WriteInlines(w io.Writer, is *ast.InlineSlice) (int, error) {
func (ze *zmkEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(w, ze)
	ast.Walk(v, is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b         encoder.EncWriter
	b         encoder.BufWriter
	prefix    []byte
	enc       *zmkEncoder
	inVerse   bool
	inlinePos int
}

func newVisitor(w io.Writer, enc *zmkEncoder) *visitor {
	return &visitor{
		b:   encoder.NewEncWriter(w),
		b:   encoder.NewBufWriter(w),
		enc: enc,
	}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockSlice:
		v.visitBlockSlice(n)
	case *ast.InlineSlice:
		for i, in := range *n {
	case *ast.BlockListNode:
		v.visitBlockList(n)
	case *ast.InlineListNode:
		for i, in := range n.List {
			v.inlinePos = i
			ast.Walk(v, in)
		}
		v.inlinePos = 0
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
146
147
148
149
150
151
152
153

154
155
156
157

158
159
160
161
162
163
164
165
166
167
168

169
170

171
172
173
174
175
176
177
145
146
147
148
149
150
151

152
153
154
155

156
157
158
159
160
161
162
163
164
165
166

167
168

169
170
171
172
173
174
175
176







-
+



-
+










-
+

-
+







		v.visitEmbedRef(n)
	case *ast.EmbedBLOBNode:
		v.visitEmbedBLOB(n)
	case *ast.CiteNode:
		v.visitCite(n)
	case *ast.FootnoteNode:
		v.b.WriteString("[^")
		ast.Walk(v, &n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
		v.visitAttributes(n.Attrs)
	case *ast.MarkNode:
		v.visitMark(n)
		v.b.WriteStrings("[!", n.Text, "]")
	case *ast.FormatNode:
		v.visitFormat(n)
	case *ast.LiteralNode:
		v.visitLiteral(n)
	default:
		return v
	}
	return nil
}

func (v *visitor) visitBlockSlice(bs *ast.BlockSlice) {
func (v *visitor) visitBlockList(bln *ast.BlockListNode) {
	var lastWasParagraph bool
	for i, bn := range *bs {
	for i, bn := range bln.List {
		if i > 0 {
			v.b.WriteByte('\n')
			if lastWasParagraph && !v.inVerse {
				if _, ok := bn.(*ast.ParaNode); ok {
					v.b.WriteByte('\n')
				}
			}
220
221
222
223
224
225
226
227

228
229
230
231

232
233

234
235
236
237
238
239
240

241
242
243
244
245
246
247
219
220
221
222
223
224
225

226
227
228
229

230
231

232
233
234
235
236
237
238

239
240
241
242
243
244
245
246







-
+



-
+

-
+






-
+







		panic(fmt.Sprintf("Unknown region kind %d", rn.Kind))
	}
	v.b.WriteString(kind)
	v.visitAttributes(rn.Attrs)
	v.b.WriteByte('\n')
	saveInVerse := v.inVerse
	v.inVerse = rn.Kind == ast.RegionVerse
	ast.Walk(v, &rn.Blocks)
	ast.Walk(v, rn.Blocks)
	v.inVerse = saveInVerse
	v.b.WriteByte('\n')
	v.b.WriteString(kind)
	if len(rn.Inlines) > 0 {
	if rn.Inlines != nil {
		v.b.WriteByte(' ')
		ast.Walk(v, &rn.Inlines)
		ast.Walk(v, rn.Inlines)
	}
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	const headingSigns = "========= "
	v.b.WriteString(headingSigns[len(headingSigns)-hn.Level-3:])
	ast.Walk(v, &hn.Inlines)
	ast.Walk(v, hn.Inlines)
	v.visitAttributes(hn.Attrs)
}

var mapNestedListKind = map[ast.NestedListKind]byte{
	ast.NestedListOrdered:   '#',
	ast.NestedListUnordered: '*',
	ast.NestedListQuote:     '>',
276
277
278
279
280
281
282
283

284
285
286
287
288
289
290
275
276
277
278
279
280
281

282
283
284
285
286
287
288
289







-
+








func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	for i, descr := range dn.Descriptions {
		if i > 0 {
			v.b.WriteByte('\n')
		}
		v.b.WriteString("; ")
		ast.Walk(v, &descr.Term)
		ast.Walk(v, descr.Term)

		for _, b := range descr.Descriptions {
			v.b.WriteString("\n: ")
			ast.WalkDescriptionSlice(v, b)
		}
	}
}
312
313
314
315
316
317
318
319

320
321
322
323
324
325
326
327
328
329
330
331
332

333
334
335
336
337
338
339
340

341
342
343
344
345
346
347
348

349
350
351
352
353
354
355
311
312
313
314
315
316
317

318
319
320
321
322
323
324
325
326
327
328
329
330

331
332
333
334
335
336
337
338

339
340
341
342
343
344
345
346

347
348
349
350
351
352
353
354







-
+












-
+







-
+







-
+







func (v *visitor) writeTableHeader(header ast.TableRow, align []ast.Alignment) {
	for pos, cell := range header {
		v.b.WriteString("|=")
		colAlign := align[pos]
		if cell.Align != colAlign {
			v.b.WriteString(alignCode[cell.Align])
		}
		ast.Walk(v, &cell.Inlines)
		ast.Walk(v, cell.Inlines)
		if colAlign != ast.AlignDefault {
			v.b.WriteString(alignCode[colAlign])
		}
	}
}

func (v *visitor) writeTableRow(row ast.TableRow, align []ast.Alignment) {
	for pos, cell := range row {
		v.b.WriteByte('|')
		if cell.Align != align[pos] {
			v.b.WriteString(alignCode[cell.Align])
		}
		ast.Walk(v, &cell.Inlines)
		ast.Walk(v, cell.Inlines)
	}
}

func (v *visitor) visitBLOB(bn *ast.BLOBNode) {
	if bn.Syntax == api.ValueSyntaxSVG {
		v.b.WriteStrings("@@@", bn.Syntax, "\n")
		v.b.Write(bn.Blob)
		v.b.WriteString("\n@@@\n")
		v.b.WriteString("@@@\n")
		return
	}
	v.b.WriteStrings(
		"%% Unable to display BLOB with title '", bn.Title, "' and syntax '", bn.Syntax, "'.")
}

var escapeSeqs = strfun.NewSet(
	"\\", "__", "**", "~~", "^^", ",,", ">>", `""`, "::", "''", "``", "++", "==",
	"\\", "__", "**", "~~", "^^", ",,", "<<", ">>", `""`, "::", "''", "``", "++", "==",
)

func (v *visitor) visitText(tn *ast.TextNode) {
	last := 0
	for i := 0; i < len(tn.Text); i++ {
		if b := tn.Text[i]; b == '\\' {
			v.b.WriteString(tn.Text[last:i])
384
385
386
387
388
389
390
391
392


393
394
395
396
397
398
399
400
401


402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419

420
421

422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445










446
447
448
449
450
451
452
453
454

455
456
457
458
459
460
461
462
463
464
465
466


467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482

483
484
485
486
487
488
489
490

491
492
493






494
495

496
497
498
499
500
501
502
503
504

505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524

525
526
383
384
385
386
387
388
389


390
391
392
393
394
395
396
397
398


399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417

418
419

420
421
422
423
424
425










426








427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444

445
446
447
448
449
450
451
452
453
454
455


456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472

473
474
475
476
477
478
479
480

481
482
483
484
485
486
487
488
489
490
491

492
493
494
495
496
497
498
499
500

501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520

521
522
523







-
-
+
+







-
-
+
+

















-
+

-
+





-
-
-
-
-
-
-
-
-
-

-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+








-
+










-
-
+
+















-
+







-
+



+
+
+
+
+
+

-
+








-
+



















-
+


			v.b.WriteByte(' ')
		}
	}
}

func (v *visitor) visitLink(ln *ast.LinkNode) {
	v.b.WriteString("[[")
	if len(ln.Inlines) > 0 {
		ast.Walk(v, &ln.Inlines)
	if !ln.OnlyRef {
		ast.Walk(v, ln.Inlines)
		v.b.WriteByte('|')
	}
	v.b.WriteStrings(ln.Ref.String(), "]]")
}

func (v *visitor) visitEmbedRef(en *ast.EmbedRefNode) {
	v.b.WriteString("{{")
	if len(en.Inlines) > 0 {
		ast.Walk(v, &en.Inlines)
	if en.Inlines != nil {
		ast.Walk(v, en.Inlines)
		v.b.WriteByte('|')
	}
	v.b.WriteStrings(en.Ref.String(), "}}")
}

func (v *visitor) visitEmbedBLOB(en *ast.EmbedBLOBNode) {
	if en.Syntax == api.ValueSyntaxSVG {
		v.b.WriteString("@@")
		v.b.Write(en.Blob)
		v.b.WriteStrings("@@{=", en.Syntax, "}")
		return
	}
	v.b.WriteString("{{TODO: display inline BLOB}}")
}

func (v *visitor) visitCite(cn *ast.CiteNode) {
	v.b.WriteStrings("[@", cn.Key)
	if len(cn.Inlines) > 0 {
	if cn.Inlines != nil {
		v.b.WriteString(", ")
		ast.Walk(v, &cn.Inlines)
		ast.Walk(v, cn.Inlines)
	}
	v.b.WriteByte(']')
	v.visitAttributes(cn.Attrs)
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	v.b.WriteStrings("[!", mn.Mark)
	if len(mn.Inlines) > 0 {
		v.b.WriteByte('|')
		ast.Walk(v, &mn.Inlines)
	}
	v.b.WriteByte(']')

}

var mapFormatKind = map[ast.FormatKind][]byte{
	ast.FormatEmph:   []byte("__"),
	ast.FormatStrong: []byte("**"),
	ast.FormatInsert: []byte(">>"),
	ast.FormatDelete: []byte("~~"),
	ast.FormatSuper:  []byte("^^"),
	ast.FormatSub:    []byte(",,"),
	ast.FormatQuote:  []byte(`""`),
	ast.FormatSpan:   []byte("::"),
	ast.FormatEmph:      []byte("__"),
	ast.FormatStrong:    []byte("**"),
	ast.FormatInsert:    []byte(">>"),
	ast.FormatDelete:    []byte("~~"),
	ast.FormatSuper:     []byte("^^"),
	ast.FormatSub:       []byte(",,"),
	ast.FormatQuotation: []byte("<<"),
	ast.FormatQuote:     []byte("\"\""),
	ast.FormatSpan:      []byte("::"),
	ast.FormatMonospace: []byte("''"),
}

func (v *visitor) visitFormat(fn *ast.FormatNode) {
	kind, ok := mapFormatKind[fn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown format kind %d", fn.Kind))
	}
	v.b.Write(kind)
	ast.Walk(v, &fn.Inlines)
	ast.Walk(v, fn.Inlines)
	v.b.Write(kind)
	v.visitAttributes(fn.Attrs)
}

func (v *visitor) visitLiteral(ln *ast.LiteralNode) {
	switch ln.Kind {
	case ast.LiteralZettel:
		v.writeLiteral('@', ln.Attrs, ln.Content)
	case ast.LiteralProg:
		v.writeLiteral('`', ln.Attrs, ln.Content)
	case ast.LiteralInput:
		v.writeLiteral('\'', ln.Attrs, ln.Content)
	case ast.LiteralKeyb:
		v.writeLiteral('+', ln.Attrs, ln.Content)
	case ast.LiteralOutput:
		v.writeLiteral('=', ln.Attrs, ln.Content)
	case ast.LiteralComment:
		if v.inlinePos > 0 {
			v.b.WriteByte(' ')
		}
		v.b.WriteString("%% ")
		v.b.Write(ln.Content)
	case ast.LiteralHTML:
		v.writeLiteral('x', syntaxToHTML(ln.Attrs), ln.Content)
	default:
		panic(fmt.Sprintf("Unknown literal kind %v", ln.Kind))
	}
}

func (v *visitor) writeLiteral(code byte, attrs zjson.Attributes, content []byte) {
func (v *visitor) writeLiteral(code byte, attrs *ast.Attributes, content []byte) {
	v.b.WriteBytes(code, code)
	v.writeEscaped(string(content), code)
	v.b.WriteBytes(code, code)
	v.visitAttributes(attrs)
}

// visitAttributes write HTML attributes
func (v *visitor) visitAttributes(a zjson.Attributes) {
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)
	}
	sort.Strings(keys)

	v.b.WriteByte('{')
	for i, k := range a.Keys() {
	for i, k := range keys {
		if i > 0 {
			v.b.WriteByte(' ')
		}
		if k == "-" {
			v.b.WriteByte('-')
			continue
		}
		v.b.WriteString(k)
		if vl := a[k]; len(vl) > 0 {
		if vl := a.Attrs[k]; len(vl) > 0 {
			v.b.WriteStrings("=\"", vl)
			v.b.WriteByte('"')
		}
	}
	v.b.WriteByte('}')
}

func (v *visitor) writeEscaped(s string, toEscape byte) {
	last := 0
	for i := 0; i < len(s); i++ {
		if b := s[i]; b == toEscape || b == '\\' {
			v.b.WriteString(s[last:i])
			v.b.WriteBytes('\\', b)
			last = i + 1
		}
	}
	v.b.WriteString(s[last:])
}

func syntaxToHTML(a zjson.Attributes) zjson.Attributes {
func syntaxToHTML(a *ast.Attributes) *ast.Attributes {
	return a.Clone().Set("", api.ValueSyntaxHTML).Remove(api.KeySyntax)
}

Changes to evaluator/evaluator.go.

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
12
13
14
15
16
17
18

19
20

21
22
23
24
25
26
27







-


-







package evaluator

import (
	"context"
	"errors"
	"fmt"
	"strconv"
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
51
52
53
54
55
56
57
58
59


60
61
62
63
64
65
66



67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

97
98
99
100
101
102
103
104
105
106
107
108
109




110
111
112
113
114
115
116
117
118



119
120
121
122

123
124

125
126
127
128
129
130
131
132




133
134

135
136
137
138
139
140
141
49
50
51
52
53
54
55


56
57
58
59
60
61



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78

79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

94
95
96
97
98
99
100
101
102
103




104
105
106
107
108
109
110
111
112
113



114
115
116
117
118
119

120
121

122
123
124
125
126




127
128
129
130
131

132
133
134
135
136
137
138
139







-
-
+
+




-
-
-
+
+
+














-
+














-
+









-
-
-
-
+
+
+
+






-
-
-
+
+
+



-
+

-
+




-
-
-
-
+
+
+
+

-
+







// given ports, and the given environment.
func EvaluateZettel(ctx context.Context, port Port, env *Environment, rtConfig config.Config, zn *ast.ZettelNode) {
	if zn.Syntax == api.ValueSyntaxNone {
		// AST is empty, evaluate to a description list of metadata.
		zn.Ast = evaluateMetadata(zn.Meta)
		return
	}
	evaluateNode(ctx, port, env, rtConfig, &zn.Ast)
	cleaner.CleanBlockSlice(&zn.Ast)
	evaluateNode(ctx, port, env, rtConfig, zn.Ast)
	cleaner.CleanBlockList(zn.Ast)
}

// EvaluateInline evaluates the given inline list in the given context, with
// the given ports, and the given environment.
func EvaluateInline(ctx context.Context, port Port, env *Environment, rtConfig config.Config, is *ast.InlineSlice) {
	evaluateNode(ctx, port, env, rtConfig, is)
	cleaner.CleanInlineSlice(is)
func EvaluateInline(ctx context.Context, port Port, env *Environment, rtConfig config.Config, iln *ast.InlineListNode) {
	evaluateNode(ctx, port, env, rtConfig, iln)
	cleaner.CleanInlineList(iln)
}

func evaluateNode(ctx context.Context, port Port, env *Environment, rtConfig config.Config, n ast.Node) {
	if env == nil {
		env = &emptyEnv
	}
	e := evaluator{
		ctx:             ctx,
		port:            port,
		env:             env,
		rtConfig:        rtConfig,
		transcludeMax:   rtConfig.GetMaxTransclusions(),
		transcludeCount: 0,
		costMap:         map[id.Zid]transcludeCost{},
		embedMap:        map[string]ast.InlineSlice{},
		embedMap:        map[string]*ast.InlineListNode{},
		marker:          &ast.ZettelNode{},
	}
	ast.Walk(&e, n)
}

type evaluator struct {
	ctx             context.Context
	port            Port
	env             *Environment
	rtConfig        config.Config
	transcludeMax   int
	transcludeCount int
	costMap         map[id.Zid]transcludeCost
	marker          *ast.ZettelNode
	embedMap        map[string]ast.InlineSlice
	embedMap        map[string]*ast.InlineListNode
}

type transcludeCost struct {
	zn *ast.ZettelNode
	ec int
}

func (e *evaluator) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockSlice:
		e.visitBlockSlice(n)
	case *ast.InlineSlice:
		e.visitInlineSlice(n)
	case *ast.BlockListNode:
		e.visitBlockList(n)
	case *ast.InlineListNode:
		e.visitInlineList(n)
	default:
		return e
	}
	return nil
}

func (e *evaluator) visitBlockSlice(bs *ast.BlockSlice) {
	for i := 0; i < len(*bs); i++ {
		bn := (*bs)[i]
func (e *evaluator) visitBlockList(bln *ast.BlockListNode) {
	for i := 0; i < len(bln.List); i++ {
		bn := bln.List[i]
		ast.Walk(e, bn)
		switch n := bn.(type) {
		case *ast.VerbatimNode:
			i += transcludeNode(bs, i, e.evalVerbatimNode(n))
			i += transcludeNode(bln, i, e.evalVerbatimNode(n))
		case *ast.TranscludeNode:
			i += transcludeNode(bs, i, e.evalTransclusionNode(n))
			i += transcludeNode(bln, i, e.evalTransclusionNode(n))
		}
	}
}

func transcludeNode(bln *ast.BlockSlice, i int, bn ast.BlockNode) int {
	if ln, ok := bn.(*ast.BlockSlice); ok {
		*bln = replaceWithBlockNodes(*bln, i, *ln)
		return len(*ln) - 1
func transcludeNode(bln *ast.BlockListNode, i int, bn ast.BlockNode) int {
	if ln, ok := bn.(*ast.BlockListNode); ok {
		bln.List = replaceWithBlockNodes(bln.List, i, ln.List)
		return len(ln.List) - 1
	}
	(*bln)[i] = bn
	bln.List[i] = bn
	return 0
}

func replaceWithBlockNodes(bns []ast.BlockNode, i int, replaceBns []ast.BlockNode) []ast.BlockNode {
	if len(replaceBns) == 1 {
		bns[i] = replaceBns[0]
		return bns
161
162
163
164
165
166
167
168

169
170
171

172
173
174
175
176
177
178
159
160
161
162
163
164
165

166
167
168

169
170
171
172
173
174
175
176







-
+


-
+







	m.Set(api.KeySyntax, getSyntax(vn.Attrs, api.ValueSyntaxDraw))
	zettel := domain.Zettel{
		Meta:    m,
		Content: domain.NewContent(vn.Content),
	}
	e.transcludeCount++
	zn := e.evaluateEmbeddedZettel(zettel)
	return &zn.Ast
	return zn.Ast
}

func getSyntax(a zjson.Attributes, defSyntax string) string {
func getSyntax(a *ast.Attributes, defSyntax string) string {
	if a != nil {
		if val, ok := a.Get(api.KeySyntax); ok {
			return val
		}
		if val, ok := a.Get(""); ok {
			return val
		}
225
226
227
228
229
230
231
232

233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249



250
251
252
253

254
255

256
257

258
259

260
261
262
263
264
265
266
267




268
269

270
271
272
273

274
275
276
277
278

279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296

297
298
299
300
301
302
303
223
224
225
226
227
228
229

230
231
232
233
234
235
236
237
238
239
240
241
242
243
244



245
246
247
248
249
250

251
252

253
254

255
256

257
258
259
260
261




262
263
264
265
266

267
268
269
270

271
272
273
274
275

276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293

294
295
296
297
298
299
300
301







-
+














-
-
-
+
+
+



-
+

-
+

-
+

-
+




-
-
-
-
+
+
+
+

-
+



-
+




-
+

















-
+







		e.costMap[zid] = transcludeCost{zn: zn, ec: e.transcludeCount - ec}
		e.transcludeCount = 0 // No stack needed, because embedding is done left-recursive, depth-first.
	}
	e.transcludeCount++
	if ec := cost.ec; ec > 0 {
		e.transcludeCount += cost.ec
	}
	return &zn.Ast
	return zn.Ast
}

func (e *evaluator) checkMaxTransclusions(ref *ast.Reference) ast.InlineNode {
	if maxTrans := e.transcludeMax; e.transcludeCount > maxTrans {
		e.transcludeCount = maxTrans + 1
		return createInlineErrorText(ref,
			"Too", "many", "transclusions", "(must", "be", "at", "most", strconv.Itoa(maxTrans)+",",
			"see", "runtime", "configuration", "key", "max-transclusions)")
	}
	return nil
}

func makeBlockNode(in ast.InlineNode) ast.BlockNode { return ast.CreateParaNode(in) }

func (e *evaluator) visitInlineSlice(is *ast.InlineSlice) {
	for i := 0; i < len(*is); i++ {
		in := (*is)[i]
func (e *evaluator) visitInlineList(iln *ast.InlineListNode) {
	for i := 0; i < len(iln.List); i++ {
		in := iln.List[i]
		ast.Walk(e, in)
		switch n := in.(type) {
		case *ast.TagNode:
			(*is)[i] = e.visitTag(n)
			iln.List[i] = e.visitTag(n)
		case *ast.LinkNode:
			(*is)[i] = e.evalLinkNode(n)
			iln.List[i] = e.evalLinkNode(n)
		case *ast.EmbedRefNode:
			i += embedNode(is, i, e.evalEmbedRefNode(n))
			i += embedNode(iln, i, e.evalEmbedRefNode(n))
		case *ast.LiteralNode:
			i += embedNode(is, i, e.evalLiteralNode(n))
			i += embedNode(iln, i, e.evalLiteralNode(n))
		}
	}
}

func embedNode(is *ast.InlineSlice, i int, in ast.InlineNode) int {
	if ln, ok := in.(*ast.InlineSlice); ok {
		*is = replaceWithInlineNodes(*is, i, *ln)
		return len(*ln) - 1
func embedNode(iln *ast.InlineListNode, i int, in ast.InlineNode) int {
	if ln, ok := in.(*ast.InlineListNode); ok {
		iln.List = replaceWithInlineNodes(iln.List, i, ln.List)
		return len(ln.List) - 1
	}
	(*is)[i] = in
	iln.List[i] = in
	return 0
}

func replaceWithInlineNodes(ins ast.InlineSlice, i int, replaceIns ast.InlineSlice) ast.InlineSlice {
func replaceWithInlineNodes(ins []ast.InlineNode, i int, replaceIns []ast.InlineNode) []ast.InlineNode {
	if len(replaceIns) == 1 {
		ins[i] = replaceIns[0]
		return ins
	}
	newIns := make(ast.InlineSlice, 0, len(ins)+len(replaceIns)-1)
	newIns := make([]ast.InlineNode, 0, len(ins)+len(replaceIns)-1)
	if i > 0 {
		newIns = append(newIns, ins[:i]...)
	}
	if len(replaceIns) > 0 {
		newIns = append(newIns, replaceIns...)
	}
	if i+1 < len(ins) {
		newIns = append(newIns, ins[i+1:]...)
	}
	return newIns
}

func (e *evaluator) visitTag(tn *ast.TagNode) ast.InlineNode {
	if gtr := e.env.GetTagRef; gtr != nil {
		fullTag := "#" + tn.Tag
		return &ast.LinkNode{
			Ref:     e.env.GetTagRef(fullTag),
			Inlines: ast.CreateInlineSliceFromWords(fullTag),
			Inlines: ast.CreateInlineListNodeFromWords(fullTag),
		}
	}
	return tn
}

func (e *evaluator) evalLinkNode(ln *ast.LinkNode) ast.InlineNode {
	ref := ln.Ref
316
317
318
319
320
321
322
323

324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
314
315
316
317
318
319
320

321
322
323
324
325
326
327
328

329
330
331
332
333







334
335
336
337
338
339
340







-
+







-





-
-
-
-
-
-
-








	zid := mustParseZid(ref)
	_, err := e.port.GetMeta(box.NoEnrichContext(e.ctx), zid)
	if errors.Is(err, &box.ErrNotAllowed{}) {
		return &ast.FormatNode{
			Kind:    ast.FormatSpan,
			Attrs:   ln.Attrs,
			Inlines: getLinkInline(ln),
			Inlines: ln.Inlines,
		}
	} else if err != nil {
		ln.Ref.State = ast.RefStateBroken
		return ln
	}

	if gfr := e.env.GetFoundRef; gfr != nil {
		ln.Inlines = getLinkInline(ln)
		ln.Ref = gfr(zid, ref.URL.EscapedFragment())
	}
	return ln
}

func getLinkInline(ln *ast.LinkNode) ast.InlineSlice {
	if ln.Inlines != nil {
		return ln.Inlines
	}
	return ast.InlineSlice{&ast.TextNode{Text: ln.Ref.Value}}
}

func (e *evaluator) evalEmbedRefNode(en *ast.EmbedRefNode) ast.InlineNode {
	ref := en.Ref

	// To prevent e.embedCount from counting
	if errText := e.checkMaxTransclusions(ref); errText != nil {
		return errText
	}
393
394
395
396
397
398
399
400

401
402
403

404
405
406
407
408
409
410
411
412
413

414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430

431
432
433
434
435
436

437
438
439
440
441
442
443
383
384
385
386
387
388
389

390
391
392

393
394
395
396
397
398
399
400
401
402

403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419

420
421
422
423
424
425

426
427
428
429
430
431
432
433







-
+


-
+









-
+
















-
+





-
+







		e.transcludeCount = 0 // No stack needed, because embedding is done left-recursive, depth-first.
	}
	e.transcludeCount++

	result, ok := e.embedMap[ref.Value]
	if !ok {
		// Search for text to be embedded.
		result = findInlineSlice(&zn.Ast, ref.URL.Fragment)
		result = findInlineList(zn.Ast, ref.URL.Fragment)
		e.embedMap[ref.Value] = result
	}
	if len(result) == 0 {
	if result.IsEmpty() {
		return &ast.LiteralNode{
			Kind:    ast.LiteralComment,
			Content: append([]byte("Nothing to transclude: "), ref.String()...),
		}
	}

	if ec := cost.ec; ec > 0 {
		e.transcludeCount += cost.ec
	}
	return &result
	return result
}

func mustParseZid(ref *ast.Reference) id.Zid {
	zid, err := id.Parse(ref.URL.Path)
	if err != nil {
		panic(fmt.Sprintf("%v: %q (state %v) -> %v", err, ref.URL.Path, ref.State, ref))
	}
	return zid
}

func (e *evaluator) evalLiteralNode(ln *ast.LiteralNode) ast.InlineNode {
	if ln.Kind != ast.LiteralZettel {
		return ln
	}
	e.transcludeCount++
	result := e.evaluateEmbeddedInline(ln.Content, getSyntax(ln.Attrs, api.ValueSyntaxDraw))
	if len(result) == 0 {
	if result.IsEmpty() {
		return &ast.LiteralNode{
			Kind:    ast.LiteralComment,
			Content: []byte("Nothing to transclude"),
		}
	}
	return &result
	return result
}

func (e *evaluator) getSyntax(m *meta.Meta) string {
	if cfg := e.rtConfig; cfg != nil {
		return config.GetSyntax(m, cfg)
	}
	return m.GetDefault(api.KeySyntax, "")
454
455
456
457
458
459
460
461

462
463
464
465
466
467










468
469
470

471
472
473
474
475
476
477
478
479
480
481
482
483
484

485
486
487
488
489




490
491
492
493
494
495
496
497









498
499

500
501


502
503
504
505



506
507

508
509

510
511
512
513
514









515
516
517
518




519
520
521
522
523

524
525
526
527

528
529

530
531
532
533
534
535





536
537
538
539


540
541
542

543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559

560
561
562
563

564
565
566
567
568


569
570

571
572
573
574
575
576


577
578
579
580
581
582
583
584

585
586
587
588
589
590
591
592
593
594
595
596

597
598
599
600
601
602
603
604
605
606
444
445
446
447
448
449
450

451
452
453
454
455


456
457
458
459
460
461
462
463
464
465
466
467

468
469
470
471
472
473
474

475






476





477
478
479
480








481
482
483
484
485
486
487
488
489
490

491
492

493
494
495



496
497
498
499

500
501

502
503
504
505
506
507
508
509
510
511
512
513
514
515
516




517
518
519
520
521
522
523
524

525
526
527
528

529
530

531
532





533
534
535
536
537




538
539



540














541
542

543
544
545
546

547
548
549
550


551
552
553

554
555
556
557
558


559
560
561







562

563
564
565
566
567
568
569
570
571
572

573
574
575
576
577
578
579
580
581
582
583







-
+




-
-
+
+
+
+
+
+
+
+
+
+


-
+






-

-
-
-
-
-
-
+
-
-
-
-
-
+
+
+
+
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+

-
+

-
+
+

-
-
-
+
+
+

-
+

-
+





+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
+
+




-
+



-
+

-
+

-
-
-
-
-
+
+
+
+
+
-
-
-
-
+
+
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-


-
+



-
+



-
-
+
+

-
+




-
-
+
+

-
-
-
-
-
-
-
+
-










-
+










	errorZid := id.EmojiZid
	if gim := e.env.GetImageMaterial; gim != nil {
		zettel, err := e.port.GetZettel(box.NoEnrichContext(e.ctx), errorZid)
		if err != nil {
			panic(err)
		}
		inlines := en.Inlines
		if len(inlines) == 0 {
		if inlines == nil {
			if title := e.getTitle(zettel.Meta); title != "" {
				inlines = parser.ParseMetadata(title)
			}
		}
		syntax := e.getSyntax(zettel.Meta)
		return enrichImageNode(gim(zettel, syntax), inlines, en.Attrs, syntax)
		result := gim(zettel, e.getSyntax(zettel.Meta))
		switch er := result.(type) {
		case *ast.EmbedRefNode:
			er.Inlines = inlines
			er.Attrs = en.Attrs
		case *ast.EmbedBLOBNode:
			er.Inlines = inlines
			er.Attrs = en.Attrs
		}
		return result
	}
	en.Ref = ast.ParseReference(errorZid.String())
	if len(en.Inlines) == 0 {
	if en.Inlines == nil {
		en.Inlines = parser.ParseMetadata("Error placeholder")
	}
	return en
}

func (e *evaluator) embedImage(en *ast.EmbedRefNode, zettel domain.Zettel) ast.InlineEmbedNode {
	syntax := e.getSyntax(zettel.Meta)
	if gim := e.env.GetImageMaterial; gim != nil {
		return enrichImageNode(gim(zettel, syntax), en.Inlines, en.Attrs, syntax)
	}
	en.Syntax = syntax
	return en
}

		result := gim(zettel, e.getSyntax(zettel.Meta))
func enrichImageNode(result ast.InlineEmbedNode, in ast.InlineSlice, a zjson.Attributes, syntax string) ast.InlineEmbedNode {
	switch er := result.(type) {
	case *ast.EmbedRefNode:
		er.Inlines = in
		er.Attrs = a
		switch er := result.(type) {
		case *ast.EmbedRefNode:
			er.Inlines = en.Inlines
			er.Attrs = en.Attrs
		er.Syntax = syntax
	case *ast.EmbedBLOBNode:
		er.Inlines = in
		er.Attrs = a
	}
	return result
}

		case *ast.EmbedBLOBNode:
			er.Inlines = en.Inlines
			er.Attrs = en.Attrs
		}
		return result
	}
	return en
}

func createInlineErrorText(ref *ast.Reference, msgWords ...string) ast.InlineNode {
	text := strings.Join(msgWords, " ")
	text := ast.CreateInlineListNodeFromWords(msgWords...)
	if ref != nil {
		text += ": " + ref.String() + "."
		ln := linkNodeToReference(ref)
		text.Append(&ast.TextNode{Text: ":"}, &ast.SpaceNode{Lexeme: " "}, ln, &ast.TextNode{Text: "."}, &ast.SpaceNode{Lexeme: " "})
	}
	ln := &ast.LiteralNode{
		Kind:    ast.LiteralInput,
		Content: []byte(text),
	fn := &ast.FormatNode{
		Kind:    ast.FormatMonospace,
		Inlines: text,
	}
	fn := &ast.FormatNode{
	fn = &ast.FormatNode{
		Kind:    ast.FormatStrong,
		Inlines: ast.InlineSlice{ln},
		Inlines: ast.CreateInlineListNode(fn),
	}
	fn.Attrs = fn.Attrs.AddClass("error")
	return fn
}

func linkNodeToReference(ref *ast.Reference) *ast.LinkNode {
	ln := &ast.LinkNode{
		Ref:     ref,
		Inlines: ast.CreateInlineListNodeFromWords(ref.String()),
		OnlyRef: true,
	}
	return ln
}

func (e *evaluator) evaluateEmbeddedInline(content []byte, syntax string) ast.InlineSlice {
	is := parser.ParseInlines(input.NewInput(content), syntax)
	ast.Walk(e, &is)
	return is
func (e *evaluator) evaluateEmbeddedInline(content []byte, syntax string) *ast.InlineListNode {
	iln := parser.ParseInlines(input.NewInput(content), syntax)
	ast.Walk(e, iln)
	return iln
}

func (e *evaluator) evaluateEmbeddedZettel(zettel domain.Zettel) *ast.ZettelNode {
	zn := parser.ParseZettel(zettel, e.getSyntax(zettel.Meta), e.rtConfig)
	ast.Walk(e, &zn.Ast)
	ast.Walk(e, zn.Ast)
	return zn
}

func findInlineSlice(bs *ast.BlockSlice, fragment string) ast.InlineSlice {
func findInlineList(bnl *ast.BlockListNode, fragment string) *ast.InlineListNode {
	if fragment == "" {
		return firstInlinesToEmbed(*bs)
		return bnl.List.FirstParagraphInlines()
	}
	fs := fragmentSearcher{fragment: fragment}
	ast.Walk(&fs, bs)
	return fs.result
}

	fs := fragmentSearcher{
		fragment: fragment,
		result:   nil,
	}
	ast.Walk(&fs, bnl)
func firstInlinesToEmbed(bs ast.BlockSlice) ast.InlineSlice {
	if ins := bs.FirstParagraphInlines(); ins != nil {
		return ins
	}
	return fs.result
}
	if len(bs) == 0 {
		return nil
	}

	if bn, ok := bs[0].(*ast.BLOBNode); ok {
		var ins ast.InlineSlice
		if bn.Title != "" {
			ins = ast.CreateInlineSliceFromWords(strings.Fields(bn.Title)...)
		}
		return ast.InlineSlice{&ast.EmbedBLOBNode{
			Blob:    bn.Blob,
			Syntax:  bn.Syntax,
			Inlines: ins,
		}}
	}
	return nil
}

type fragmentSearcher struct {
	fragment string
	result   ast.InlineSlice
	result   *ast.InlineListNode
}

func (fs *fragmentSearcher) Visit(node ast.Node) ast.Visitor {
	if len(fs.result) > 0 {
	if fs.result != nil {
		return nil
	}
	switch n := node.(type) {
	case *ast.BlockSlice:
		for i, bn := range *n {
	case *ast.BlockListNode:
		for i, bn := range n.List {
			if hn, ok := bn.(*ast.HeadingNode); ok && hn.Fragment == fs.fragment {
				fs.result = (*n)[i+1:].FirstParagraphInlines()
				fs.result = n.List[i+1:].FirstParagraphInlines()
				return nil
			}
			ast.Walk(fs, bn)
		}
	case *ast.InlineSlice:
		for i, in := range *n {
	case *ast.InlineListNode:
		for i, in := range n.List {
			if mn, ok := in.(*ast.MarkNode); ok && mn.Fragment == fs.fragment {
				ris := skipSpaceNodes((*n)[i+1:])
				if len(mn.Inlines) > 0 {
					fs.result = append(ast.InlineSlice{}, mn.Inlines...)
					fs.result = append(fs.result, &ast.SpaceNode{Lexeme: " "})
					fs.result = append(fs.result, ris...)
				} else {
					fs.result = ris
				fs.result = ast.CreateInlineListNode(skipSpaceNodes(n.List[i+1:])...)
				}
				return nil
			}
			ast.Walk(fs, in)
		}
	default:
		return fs
	}
	return nil
}

func skipSpaceNodes(ins ast.InlineSlice) ast.InlineSlice {
func skipSpaceNodes(ins []ast.InlineNode) []ast.InlineNode {
	for i, in := range ins {
		switch in.(type) {
		case *ast.SpaceNode:
		case *ast.BreakNode:
		default:
			return ins[i:]
		}
	}
	return nil
}

Changes to evaluator/metadata.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24

25
26
27
28
29
30
31




32
33
34
35

36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51

52
53
54
55
56
57
58
59
60

61
62
63
64
65
66

67
11
12
13
14
15
16
17

18
19
20
21
22
23

24
25
26
27

28


29
30
31
32
33
34
35

36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51

52
53
54
55
56
57
58
59
60

61
62
63
64
65
66

67
68







-
+





-
+



-

-
-
+
+
+
+



-
+




-
+










-
+








-
+





-
+

package evaluator

import (
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
)

func evaluateMetadata(m *meta.Meta) ast.BlockSlice {
func evaluateMetadata(m *meta.Meta) *ast.BlockListNode {
	descrlist := &ast.DescriptionListNode{}
	for _, p := range m.Pairs() {
		descrlist.Descriptions = append(
			descrlist.Descriptions, getMetadataDescription(p.Key, p.Value))
	}
	return ast.BlockSlice{descrlist}
	return ast.CreateBlockListNode(descrlist)
}

func getMetadataDescription(key, value string) ast.Description {
	is := convertMetavalueToInlineSlice(value, meta.Type(key))
	return ast.Description{
		Term:         ast.InlineSlice{&ast.TextNode{Text: key}},
		Descriptions: []ast.DescriptionSlice{{&ast.ParaNode{Inlines: is}}},
		Term: ast.CreateInlineListNode(&ast.TextNode{Text: key}),
		Descriptions: []ast.DescriptionSlice{{
			ast.CreateParaNode(convertMetavalueToInlineList(value, meta.Type(key))),
		}},
	}
}

func convertMetavalueToInlineSlice(value string, dt *meta.DescriptionType) ast.InlineSlice {
func convertMetavalueToInlineList(value string, dt *meta.DescriptionType) *ast.InlineListNode {
	var sliceData []string
	if dt.IsSet {
		sliceData = meta.ListFromValue(value)
		if len(sliceData) == 0 {
			return nil
			return &ast.InlineListNode{}
		}
	} else {
		sliceData = []string{value}
	}
	var makeLink bool
	switch dt {
	case meta.TypeID, meta.TypeIDSet:
		makeLink = true
	}

	result := make(ast.InlineSlice, 0, 2*len(sliceData)-1)
	result := make([]ast.InlineNode, 0, 2*len(sliceData)-1)
	for i, val := range sliceData {
		if i > 0 {
			result = append(result, &ast.SpaceNode{Lexeme: " "})
		}
		tn := &ast.TextNode{Text: val}
		if makeLink {
			result = append(result, &ast.LinkNode{
				Ref:     ast.ParseReference(val),
				Inlines: ast.InlineSlice{tn},
				Inlines: ast.CreateInlineListNode(tn),
			})
		} else {
			result = append(result, tn)
		}
	}
	return result
	return ast.CreateInlineListNode(result...)
}

Changes to go.mod.

1
2
3
4
5
6
7
8
9
10



11
12

13
14
15
1
2
3
4
5
6
7



8
9
10
11

12
13
14
15







-
-
-
+
+
+

-
+



module zettelstore.de/z

go 1.17

require (
	github.com/fsnotify/fsnotify v1.5.1
	github.com/pascaldekloe/jwt v1.10.0
	github.com/yuin/goldmark v1.4.8
	golang.org/x/crypto v0.0.0-20220307211146-efcb8507fb70
	golang.org/x/term v0.0.0-20210927222741-03fcf44c2211
	github.com/yuin/goldmark v1.4.5
	golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e
	golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b
	golang.org/x/text v0.3.7
	zettelstore.de/c v0.0.0-20220308145137-122c412c3a99
	zettelstore.de/c v0.0.0-20220201173304-22e447546177
)

require golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c // indirect

Changes to go.sum.

1
2
3
4
5
6
7
8
9





10
11
12
13
14
15
16
17
18



19
20
21
22
23


1
2
3
4





5
6
7
8
9
10

11
12
13
14



15
16
17
18
19
20


21
22




-
-
-
-
-
+
+
+
+
+

-




-
-
-
+
+
+



-
-
+
+
github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI=
github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=
github.com/pascaldekloe/jwt v1.10.0 h1:ktcIUV4TPvh404R5dIBEnPCsSwj0sqi3/0+XafE5gJs=
github.com/pascaldekloe/jwt v1.10.0/go.mod h1:TKhllgThT7TOP5rGr2zMLKEDZRAgJfBbtKyVeRsNB9A=
github.com/yuin/goldmark v1.4.8 h1:zHPiabbIRssZOI0MAzJDHsyvG4MXCGqVaMOwR+HeoQQ=
github.com/yuin/goldmark v1.4.8/go.mod h1:rmuwmfZ0+bvzB24eSC//bk1R1Zp3hM0OXYv/G2LIilg=
golang.org/x/crypto v0.0.0-20220307211146-efcb8507fb70 h1:syTAU9FwmvzEoIYMqcPHOcVm4H3U5u90WsvuYgwpETU=
golang.org/x/crypto v0.0.0-20220307211146-efcb8507fb70/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
github.com/yuin/goldmark v1.4.5 h1:4OEQwtW2uLXjEdgnGM3Vg652Pq37X7NOIRzFWb3BzIc=
github.com/yuin/goldmark v1.4.5/go.mod h1:rmuwmfZ0+bvzB24eSC//bk1R1Zp3hM0OXYv/G2LIilg=
golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e h1:gsTQYXdTw2Gq7RBsWvlQ91b+aEQ6bXFUngBGuR8sPpI=
golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c h1:F1jZWGFhYfh0Ci55sIpILtKKK8p3i2/krTr0H1rg74I=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
zettelstore.de/c v0.0.0-20220308145137-122c412c3a99 h1:0WknFoNBwtwD1pqUq4XPGtvkqyE0nN8tJJAPTCjHt/8=
zettelstore.de/c v0.0.0-20220308145137-122c412c3a99/go.mod h1:Hx/qzHCaQ8zzXEzBglBj/2aGkQpBQG81/4XztCIGJ84=
zettelstore.de/c v0.0.0-20220201173304-22e447546177 h1:sxj86XDtZrY98NoMp4VPA+D8Pet0vm/EJkLCu7bwgK4=
zettelstore.de/c v0.0.0-20220201173304-22e447546177/go.mod h1:Hx/qzHCaQ8zzXEzBglBj/2aGkQpBQG81/4XztCIGJ84=

Changes to kernel/impl/core.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
// Copyright (c) 2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package impl
60
61
62
63
64
65
66
67

68
69
70
71
72
73
74
60
61
62
63
64
65
66

67
68
69
70
71
72
73
74







-
+







		},
		kernel.CoreProgname: {"Program name", nil, false},
		kernel.CoreVerbose:  {"Verbose output", parseBool, true},
		kernel.CoreVersion: {
			"Version",
			cs.noFrozen(func(val string) interface{} {
				if val == "" {
					return kernel.CoreDefaultVersion
					return "unknown"
				}
				return val
			}),
			false,
		},
	}
	cs.next = interfaceMap{

Changes to kernel/kernel.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
// Copyright (c) 2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package kernel provides the main kernel service.
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
131
132
133
134
135
136
137





138
139
140
141
142
143
144







-
-
-
-
-







	CoreHostname  = "hostname"
	CorePort      = "port"
	CoreProgname  = "progname"
	CoreVerbose   = "verbose"
	CoreVersion   = "version"
)

// Defined values for core service.
const (
	CoreDefaultVersion = "unknown"
)

// Constants for config service keys.
const (
	ConfigSimpleMode = "simple-mode"
)

// Constants for authentication service keys.
const (

Changes to parser/blob/blob.go.

42
43
44
45
46
47
48
49

50
51
52
53
54

55
56
57
58

59
60
61



42
43
44
45
46
47
48

49
50
51
52
53

54
55
56
57

58
59
60

61
62
63







-
+




-
+



-
+


-
+
+
+
		IsTextParser:  false,
		IsImageFormat: true,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) *ast.BlockListNode {
	if p := parser.Get(syntax); p != nil {
		syntax = p.Name
	}
	title, _ := m.Get(api.KeyTitle)
	return ast.BlockSlice{&ast.BLOBNode{
	return ast.CreateBlockListNode(&ast.BLOBNode{
		Title:  title,
		Syntax: syntax,
		Blob:   []byte(inp.Src),
	}}
	})
}

func parseInlines(*input.Input, string) ast.InlineSlice { return nil }
func parseInlines(*input.Input, string) *ast.InlineListNode {
	return nil
}

Changes to parser/cleaner/cleaner.go.

1
2

3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


26
27
28


29
30
31
32
33
34
35
1

2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


24
25
26


27
28
29
30
31
32
33
34
35

-
+

-
+



















-
-
+
+

-
-
+
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package cleaner provides funxtions to clean up the parsed AST.
package cleaner

import (
	"bytes"
	"strconv"

	"zettelstore.de/c/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

// CleanBlockSlice cleans the given block list.
func CleanBlockSlice(bs *ast.BlockSlice) { cleanNode(bs) }
// CleanBlockList cleans the given block list.
func CleanBlockList(bln *ast.BlockListNode) { cleanNode(bln) }

// CleanInlineSlice cleans the given inline list.
func CleanInlineSlice(is *ast.InlineSlice) { cleanNode(is) }
// CleanInlineList cleans the given inline list.
func CleanInlineList(iln *ast.InlineListNode) { cleanNode(iln) }

func cleanNode(n ast.Node) {
	cv := cleanVisitor{
		textEnc: encoder.Create(api.EncoderText, nil),
		hasMark: false,
		doMark:  false,
	}
56
57
58
59
60
61
62
63

64
65
66
67
68

69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

92
93
94
95
96
97

98
99
100
101
102
103
104
56
57
58
59
60
61
62

63
64
65
66
67

68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83








84
85
86
87
88
89

90
91
92
93
94
95
96
97







-
+




-
+















-
-
-
-
-
-
-
-
+





-
+







		cv.visitMark(n)
		return nil
	}
	return cv
}

func (cv *cleanVisitor) visitHeading(hn *ast.HeadingNode) {
	if cv.doMark || hn == nil || len(hn.Inlines) == 0 {
	if cv.doMark || hn == nil || hn.Inlines.IsEmpty() {
		return
	}
	if hn.Slug == "" {
		var buf bytes.Buffer
		_, err := cv.textEnc.WriteInlines(&buf, &hn.Inlines)
		_, err := cv.textEnc.WriteInlines(&buf, hn.Inlines)
		if err != nil {
			return
		}
		hn.Slug = strfun.Slugify(buf.String())
	}
	if hn.Slug != "" {
		hn.Fragment = cv.addIdentifier(hn.Slug, hn)
	}
}

func (cv *cleanVisitor) visitMark(mn *ast.MarkNode) {
	if !cv.doMark {
		cv.hasMark = true
		return
	}
	// if mn.Mark == "" && len(mn.Inlines) > 0 {
	// 	var buf bytes.Buffer
	// 	_, err := cv.textEnc.WriteInlines(&buf, &mn.Inlines)
	// 	if err == nil {
	// 		mn.Mark = buf.String()
	// 	}
	// }
	if mn.Mark == "" {
	if mn.Text == "" {
		mn.Slug = ""
		mn.Fragment = cv.addIdentifier("*", mn)
		return
	}
	if mn.Slug == "" {
		mn.Slug = strfun.Slugify(mn.Mark)
		mn.Slug = strfun.Slugify(mn.Text)
	}
	mn.Fragment = cv.addIdentifier(mn.Slug, mn)
}

func (cv *cleanVisitor) addIdentifier(id string, node ast.Node) string {
	if cv.ids == nil {
		cv.ids = map[string]ast.Node{id: node}

Changes to parser/draw/canvas.go.

17
18
19
20
21
22
23

24
25

26

27
28
29
30
31
32
33
34





35
36
37
38
39
40
41
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

37
38
39
40
41
42
43
44
45
46
47
48







+


+

+







-
+
+
+
+
+







// All rights reserved.
//-----------------------------------------------------------------------------

package draw

import (
	"bytes"
	"encoding/json"
	"fmt"
	"image"
	"regexp"
	"sort"
	"strconv"
	"unicode/utf8"
)

// newCanvas returns a new Canvas, initialized from the provided data. If tabWidth is set to a non-negative
// value, that value will be used to convert tabs to spaces within the grid. Creation of the Canvas
// can fail if the diagram contains invalid UTF-8 sequences.
func newCanvas(data []byte, tabWidth int) (*canvas, error) {
	c := &canvas{}
	c := &canvas{
		optMaps: optionMaps{
			"__a2s__closed__options__": {"fill": "#fff"},
		},
	}

	lines := bytes.Split(data, []byte("\n"))
	c.siz.Y = len(lines)

	// Diagrams will often not be padded to a uniform width. To overcome this, we scan over
	// each line and figure out which is the longest. This becomes the width of the canvas.
	for i, line := range lines {
117
118
119
120
121
122
123


124
125
126
127
128
129
130

131
132
133
134
135
136
137
138
139
140
141
142























143
144
145
146
147
148
149
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182







+
+







+












+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







			index++
		}
	}

	return out, nil
}

type optionMaps map[string]map[string]interface{}

// canvas is the parsed source data.
type canvas struct {
	// (0,0) is top left.
	grid           []char
	visited        []bool
	objs           objects
	siz            image.Point
	optMaps        optionMaps
	hasStartMarker bool
	hasEndMarker   bool
}

// String provides a view into the underlying grid.
func (c *canvas) String() string { return fmt.Sprintf("%+v", c.grid) }

// objects returns all the objects found in the underlying grid.
func (c *canvas) objects() objects { return c.objs }

// size returns the visual dimensions of the Canvas.
func (c *canvas) size() image.Point { return c.siz }

// options returns a map of options to apply to Objects based on the object's tag. This
// maps tag name to a map of option names to options.
func (c *canvas) options() optionMaps { return c.optMaps }

// enclosingObjects returns the set of objects that contain this point in order from most
// to least specific.
func (c *canvas) enclosingObjects(p point) (q objects) {
	maxTL := point{x: -1, y: -1}
	for _, o := range c.objs {
		// An object can't really contain another unless it is a polygon.
		if !o.IsClosed() {
			continue
		}

		if o.HasPoint(p) && o.Corners()[0].x > maxTL.x && o.Corners()[0].y > maxTL.y {
			q = append(q, o)
			maxTL.x = o.Corners()[0].x
			maxTL.y = o.Corners()[0].y
		}
	}
	return q
}

// findObjects finds all objects (lines, polygons, and text) within the underlying grid.
func (c *canvas) findObjects() {
	p := point{}

	// Find any new paths by starting with a point that wasn't yet visited, beginning at the top
	// left of the grid.
328
329
330
331
332
333
334



335
336
337
338
339
340




341






342
343
344
345
346
347
348
349
350
351

352
353
354
355
356
357
358
359
















360
361








































362
363
364
365
366
367
368
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396

397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470







+
+
+






+
+
+
+

+
+
+
+
+
+









-
+








+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+


+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







			}
		}
	}

	return out
}

// Used for matching [X, Y]: {...} tag definitions. These definitions target specific objects.
var objTagRE = regexp.MustCompile(`(\d+)\s*,\s*(\d+)$`)

// scanText extracts a line of text.
func (c *canvas) scanText(start point) *object {
	obj := &object{points: []point{start}, isText: true}
	whiteSpaceStreak := 0
	cur := start

	tagged := 0
	tag := []rune{}
	tagDef := []rune{}

	for c.canRight(cur) {
		if cur.x == start.x && c.at(cur).isObjectStartTag() {
			tagged++
		} else if cur.x > start.x && c.at(cur).isObjectEndTag() {
			tagged++
		}

		cur.x++
		if c.isVisited(cur) {
			// If the point is already visited, we hit a polygon or a line.
			break
		}
		ch := c.at(cur)
		if !ch.isTextCont() {
			break
		}
		if ch.isSpace() {
		if tagged == 0 && ch.isSpace() {
			whiteSpaceStreak++
			// Stop when we see 3 consecutive whitespace points.
			if whiteSpaceStreak > 2 {
				break
			}
		} else {
			whiteSpaceStreak = 0
		}

		switch tagged {
		case 1:
			if !c.at(cur).isObjectEndTag() {
				tag = append(tag, rune(ch))
			}
		case 2:
			if c.at(cur).isTagDefinitionSeparator() {
				tagged++
			} else {
				tagged = -1
			}
		case 3:
			tagDef = append(tagDef, rune(ch))
		}

		obj.points = append(obj.points, cur)
	}

	// If we found a start and end tag marker, we either need to assign the tag to the object,
	// or we need to assign the specified options to the global canvas option space.
	if tagged == 2 {
		t := string(tag)
		if container := c.enclosingObjects(start); container != nil {
			container[0].SetTag(t)
		}

		// The tag applies to the text object as well so that properties like
		// a2s:label can be set.
		obj.SetTag(t)
	} else if tagged == 3 {
		t := string(tag)

		// A tag definition targeting an object will not be found within any object; we need
		// to do that calculation here.
		if matches := objTagRE.FindStringSubmatch(t); matches != nil {
			if targetX, err := strconv.ParseInt(matches[1], 10, 0); err == nil {
				if targetY, err1 := strconv.ParseInt(matches[2], 10, 0); err1 == nil {
					for i, o := range c.objs {
						corner := o.Corners()[0]
						if corner.x == int(targetX) && corner.y == int(targetY) {
							c.objs[i].SetTag(t)
							break
						}
					}
				}
			}
		}
		// This is a tag definition. Parse the JSON and assign the options to the canvas.
		var m interface{}
		def := []byte(string(tagDef))
		if err := json.Unmarshal(def, &m); err == nil {
			// The tag applies to the reference object as well, so that properties like
			// a2s:delref can be set.
			obj.SetTag(t)
			c.optMaps[t] = m.(map[string]interface{})
		}
	}

	// Trim the right side of the text object.
	for len(obj.points) != 0 && c.at(obj.points[len(obj.points)-1]).isSpace() {
		obj.points = obj.points[:len(obj.points)-1]
	}

	obj.seal(c)

Changes to parser/draw/char.go.

18
19
20
21
22
23
24












25
26
27
28

29
30
31
32
33
34
35
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
47







+
+
+
+
+
+
+
+
+
+
+
+



-
+







//-----------------------------------------------------------------------------

package draw

import "unicode"

type char rune

func (c char) isObjectStartTag() bool {
	return c == '['
}

func (c char) isObjectEndTag() bool {
	return c == ']'
}

func (c char) isTagDefinitionSeparator() bool {
	return c == ':'
}

func (c char) isTextStart() bool {
	r := rune(c)
	return unicode.IsLetter(r) || unicode.IsNumber(r) || unicode.IsSymbol(r)
	return c.isObjectStartTag() || unicode.IsLetter(r) || unicode.IsNumber(r) || unicode.IsSymbol(r)
}

func (c char) isTextCont() bool {
	return unicode.IsPrint(rune(c))
}

func (c char) isSpace() bool {

Added parser/draw/color.go.








































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// This file was originally created by the ASCIIToSVG contributors under an MIT
// license, but later changed to fulfil the needs of Zettelstore. The following
// statements affects the original code as found on
// https://github.com/asciitosvg/asciitosvg (Commit:
// ca82a5ce41e2190a05e07af6e8b3ea4e3256a283, 2020-11-20):
//
// Copyright 2012 - 2018 The ASCIIToSVG Contributors
// All rights reserved.
//-----------------------------------------------------------------------------

package draw

import (
	"fmt"
	"strconv"
)

func parseHexColor(c string) (r, g, b int, err error) {
	var pr, pg, pb int64

	switch len(c) {
	case 4:
		pr, err = strconv.ParseInt(string(c[1]), 16, 0)
		if err != nil {
			return 0, 0, 0, err
		}

		pg, err = strconv.ParseInt(string(c[2]), 16, 0)
		if err != nil {
			return 0, 0, 0, err
		}

		pb, err = strconv.ParseInt(string(c[3]), 16, 0)
		if err != nil {
			return 0, 0, 0, err
		}

		pr *= 17
		pg *= 17
		pb *= 17
	case 7:
		pr, err = strconv.ParseInt(string(c[1:3]), 16, 0)
		if err != nil {
			return 0, 0, 0, err
		}

		pg, err = strconv.ParseInt(string(c[3:5]), 16, 0)
		if err != nil {
			return 0, 0, 0, err
		}

		pb, err = strconv.ParseInt(string(c[5:7]), 16, 0)
		if err != nil {
			return 0, 0, 0, err
		}

	default:
		return 0, 0, 0, fmt.Errorf("color '%s' not of valid length", c)
	}

	r, g, b = int(pr), int(pg), int(pb)

	return
}

// colorToRGB matches a color string and returns its RGB components.
func colorToRGB(c string) (r, g, b int, err error) {
	if c[0] == '#' {
		return parseHexColor(c)
	}

	return 0, 0, 0, fmt.Errorf("color '%s' can't be parsed", c)
}

// textColor returns an accessible text color to use on top of a supplied background color. The
// formula used for calculating whether the contrast is accessible comes from a W3 working group
// paper on accessibility at http://www.w3.org/TR/AERT. The recommended contrast is a brightness
// difference of at least 125 and a color difference of at least 500. Folks can style their colors
// as they like, but our default text color is black, so the color difference for text is just the
// sum of the components.
func textColor(c string) string {
	r, g, b, err := colorToRGB(c)
	if err != nil {
		return "#000"
	}

	brightness := (r*299 + g*587 + b*114) / 1000
	difference := r + g + b
	if brightness < 125 && difference < 500 {
		return "#fff"
	}

	return "#000"
}

Added parser/draw/color_test.go.
































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//
// This file was originally created by the ASCIIToSVG contributors under an MIT
// license, but later changed to fulfil the needs of Zettelstore. The following
// statements affects the original code as found on
// https://github.com/asciitosvg/asciitosvg (Commit:
// ca82a5ce41e2190a05e07af6e8b3ea4e3256a283, 2020-11-20):
//
// Copyright 2012 - 2018 The ASCIIToSVG Contributors
// All rights reserved.
//-----------------------------------------------------------------------------

package draw

import "testing"

func TestParseHexColor(t *testing.T) {
	t.Parallel()
	data := []struct {
		color   string
		rgb     []int
		isError bool
	}{
		{"#123", []int{17, 34, 51}, false},
		{"#fff", []int{255, 255, 255}, false},
		{"#FFF", []int{255, 255, 255}, false},
		{"#ffffff", []int{255, 255, 255}, false},
		{"#FFFFFF", []int{255, 255, 255}, false},
		{"#fFfFFf", []int{255, 255, 255}, false},
		{"#notacolor", nil, true},
		{"alsonotacolor", nil, true},
		{"#ffg", nil, true},
		{"#FFG", nil, true},
		{"#fffffg", nil, true},
		{"#FFFFFG", nil, true},
	}

	for i, v := range data {
		r, g, b, err := colorToRGB(v.color)
		if v.isError {
			if err == nil {
				t.Errorf("%d: colorToRGB(%q) expected error, but got none", i, v.color)
			}
			continue
		}

		if err != nil {
			t.Errorf("%d: colorToRGB(%q) got error %v", i, v.color, err)
			continue
		}

		if r != v.rgb[0] || g != v.rgb[1] || b != v.rgb[2] {
			t.Errorf("%d: colorToRGB(%q) expected %v, but got [%v,%v,%v]", i, v.color, v.rgb, r, g, b)
		}
	}
}

Changes to parser/draw/draw.go.

33
34
35
36
37
38
39
40

41
42
43
44
45
46

47
48
49
50
51
52
53
54
55
56

57
58

59
60
61
62

63
64
65

66
67
68
69
70
71
72
73
74

75
76
77

78
79
80
81


82
83
84
85


86
33
34
35
36
37
38
39

40
41
42
43
44
45

46
47
48
49
50
51
52
53
54
55

56
57

58
59
60
61

62
63
64

65
66
67
68
69
70
71
72
73

74
75
76

77
78
79


80
81
82
83


84
85
86







-
+





-
+









-
+

-
+



-
+


-
+








-
+


-
+


-
-
+
+


-
-
+
+

const (
	defaultTabSize = 8
	defaultFont    = ""
	defaultScaleX  = 10
	defaultScaleY  = 20
)

func parseBlocks(inp *input.Input, m *meta.Meta, _ string) ast.BlockSlice {
func parseBlocks(inp *input.Input, m *meta.Meta, _ string) *ast.BlockListNode {
	font := m.GetDefault("font", defaultFont)
	scaleX := m.GetNumber("x-scale", defaultScaleX)
	scaleY := m.GetNumber("y-scale", defaultScaleY)
	canvas, err := newCanvas(inp.Src[inp.Pos:], defaultTabSize)
	if err != nil {
		return ast.BlockSlice{ast.CreateParaNode(canvasErrMsg(err)...)}
		return ast.CreateBlockListNode(ast.CreateParaNode(canvasErrMsg(err)))
	}
	if scaleX < 1 || 1000000 < scaleX {
		scaleX = defaultScaleX
	}
	if scaleY < 1 || 1000000 < scaleY {
		scaleY = defaultScaleY
	}
	svg := canvasToSVG(canvas, font, int(scaleX), int(scaleY))
	if len(svg) == 0 {
		return ast.BlockSlice{ast.CreateParaNode(noSVGErrMsg()...)}
		return ast.CreateBlockListNode(ast.CreateParaNode(noSVGErrMsg()))
	}
	return ast.BlockSlice{&ast.BLOBNode{
	return ast.CreateBlockListNode(&ast.BLOBNode{
		Title:  "",
		Syntax: api.ValueSyntaxSVG,
		Blob:   svg,
	}}
	})
}

func parseInlines(inp *input.Input, _ string) ast.InlineSlice {
func parseInlines(inp *input.Input, _ string) *ast.InlineListNode {
	canvas, err := newCanvas(inp.Src[inp.Pos:], defaultTabSize)
	if err != nil {
		return canvasErrMsg(err)
	}
	svg := canvasToSVG(canvas, defaultFont, defaultScaleX, defaultScaleY)
	if len(svg) == 0 {
		return noSVGErrMsg()
	}
	return ast.InlineSlice{&ast.EmbedBLOBNode{
	return ast.CreateInlineListNode(&ast.EmbedBLOBNode{
		Blob:   svg,
		Syntax: api.ValueSyntaxSVG,
	}}
	})
}

func canvasErrMsg(err error) ast.InlineSlice {
	return ast.CreateInlineSliceFromWords("Error:", err.Error())
func canvasErrMsg(err error) *ast.InlineListNode {
	return ast.CreateInlineListNodeFromWords("Error:", err.Error())
}

func noSVGErrMsg() ast.InlineSlice {
	return ast.CreateInlineSliceFromWords("NO", "IMAGE")
func noSVGErrMsg() *ast.InlineListNode {
	return ast.CreateInlineListNodeFromWords("NO", "IMAGE")
}

Changes to parser/draw/object.go.

26
27
28
29
30
31
32

33
34
35
36
37
38




39
40
41




42
43
44




45
46



47
48
49
50
51










52
53
54
55
56
57
58
26
27
28
29
30
31
32
33
34
35
36
37


38
39
40
41
42


43
44
45
46
47


48
49
50
51
52

53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77







+




-
-
+
+
+
+

-
-
+
+
+
+

-
-
+
+
+
+

-
+
+
+





+
+
+
+
+
+
+
+
+
+







	// points always starts with the top most, then left most point, proceeding to the right.
	points   []point
	text     []rune
	corners  []point
	isText   bool
	isClosed bool
	isDashed bool
	tag      string
}

// Points returns all the points occupied by this Object. Every object has at least one point,
// and all points are both in-order and contiguous.
func (o *object) Points() []point { return o.points }

func (o *object) Points() []point {
	return o.points
}

// Corners returns all the corners (change of direction) along the path.
func (o *object) Corners() []point { return o.corners }

func (o *object) Corners() []point {
	return o.corners
}

// IsClosed is true if the object is composed of a closed path.
func (o *object) IsClosed() bool { return o.isClosed }

func (o *object) IsClosed() bool {
	return o.isClosed
}

// IsDashed is true if this object is a path object, and lines should be drawn dashed.
func (o *object) IsDashed() bool { return o.isDashed }
func (o *object) IsDashed() bool {
	return o.isDashed
}

// Text returns the text associated with this object if textual, and nil otherwise.
func (o *object) Text() []rune {
	return o.text
}

// SetTag sets an options tag on this object so the renderer may look up options.
func (o *object) SetTag(s string) {
	o.tag = s
}

// Tag returns the tag of this object, if any.
func (o *object) Tag() string {
	return o.tag
}

func (o *object) isOpenPath() bool   { return !o.isClosed && !o.isText }
func (o *object) isClosedPath() bool { return o.isClosed && !o.isText }
func (o *object) isJustText() bool   { return o.isText }

func (o *object) String() string {
	if o.isJustText() {

Changes to parser/draw/point.go.

37
38
39
40
41
42
43
44



45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62












37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60




61
62
63
64
65
66
67
68
69
70
71
72







-
+
+
+














-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
// of the diagram. The point also provides hints to the renderer as to how it should be interpreted.
type point struct {
	x, y int
	hint renderHint
}

// String implements fmt.Stringer on Point.
func (p point) String() string { return fmt.Sprintf("(%d,%d)", p.x, p.y) }
func (p point) String() string {
	return fmt.Sprintf("(%d,%d)", p.x, p.y)
}

// isHorizontal returns true if p1 and p2 are horizontally aligned.
func isHorizontal(p1, p2 point) bool {
	d := p1.x - p2.x
	return d <= 1 && d >= -1 && p1.y == p2.y
}

// isVertical returns true if p1 and p2 are vertically aligned.
func isVertical(p1, p2 point) bool {
	d := p1.y - p2.y
	return d <= 1 && d >= -1 && p1.x == p2.x
}

// The following functions return true when the diagonals are connected in various compass directions.
func isDiagonalSE(p1, p2 point) bool { return p1.x-p2.x == -1 && p1.y-p2.y == -1 }
func isDiagonalSW(p1, p2 point) bool { return p1.x-p2.x == 1 && p1.y-p2.y == -1 }
func isDiagonalNW(p1, p2 point) bool { return p1.x-p2.x == 1 && p1.y-p2.y == 1 }
func isDiagonalNE(p1, p2 point) bool { return p1.x-p2.x == -1 && p1.y-p2.y == 1 }
func isDiagonalSE(p1, p2 point) bool {
	return p1.x-p2.x == -1 && p1.y-p2.y == -1
}
func isDiagonalSW(p1, p2 point) bool {
	return p1.x-p2.x == 1 && p1.y-p2.y == -1
}
func isDiagonalNW(p1, p2 point) bool {
	return p1.x-p2.x == 1 && p1.y-p2.y == 1
}
func isDiagonalNE(p1, p2 point) bool {
	return p1.x-p2.x == -1 && p1.y-p2.y == 1
}

Changes to parser/draw/svg.go.

65
66
67
68
69
70
71
72

73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88













89

90
91
92
93
94
95
96
65
66
67
68
69
70
71

72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101

102
103
104
105
106
107
108
109







-
+
















+
+
+
+
+
+
+
+
+
+
+
+
+
-
+







		fmt.Fprintf(w, markerTag, nameStartMarker, x, y, "M 10 0 L 10 10 L 0 5 z")
	}
	if c.hasEndMarker {
		fmt.Fprintf(w, markerTag, nameEndMarker, x, y, "M 0 0 L 10 5 L 0 10 z")
	}
}

const pathTag = `<path id="%s%d" %sd="%s" />`
const pathTag = `%s<path id="%s%d" %sd="%s" />%s`

func writeClosedPaths(w io.Writer, c *canvas, scaleX, scaleY int) {
	first := true
	for i, obj := range c.objects() {
		if !obj.isClosedPath() {
			continue
		}
		if first {
			io.WriteString(w, `<g id="closed" stroke="#000" stroke-width="2" fill="none">`)
			first = false
		}
		opts := ""
		if obj.IsDashed() {
			opts = `stroke-dasharray="5 5" `
		}

		tag := obj.Tag()
		if tag == "" {
			tag = "__a2s__closed__options__"
		}
		options := c.options()
		opts += getTagOpts(options, tag)

		startLink, endLink := "", ""
		if link, ok := options[tag]["a2s:link"]; ok {
			startLink = link.(string)
			endLink = "</a>"
		}

		fmt.Fprintf(w, pathTag, "closed", i, opts, flatten(obj.Points(), scaleX, scaleY)+"Z")
		fmt.Fprintf(w, pathTag, startLink, "closed", i, opts, flatten(obj.Points(), scaleX, scaleY)+"Z", endLink)
	}
	if !first {
		io.WriteString(w, "</g>")
	}
}

func writeOpenPaths(w io.Writer, c *canvas, scaleX, scaleY int) {
126
127
128
129
130
131
132










133

134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153




154





















155
156
157
158


159
160
161
162
163
















































164
165
166
167
168
169
170
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155

156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204


205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266







+
+
+
+
+
+
+
+
+
+
-
+




















+
+
+
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+


-
-
+
+





+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







		}
		if points[0].hint == startMarker {
			opts += optStartMarker
		}
		if points[len(points)-1].hint == endMarker {
			opts += optEndMarker
		}

		options := c.options()
		tag := obj.Tag()
		opts += getTagOpts(options, tag)

		startLink, endLink := "", ""
		if link, ok := options[tag]["a2s:link"]; ok {
			startLink = link.(string)
			endLink = "</a>"
		}
		fmt.Fprintf(w, pathTag, "open", i, opts, flatten(points, scaleX, scaleY))
		fmt.Fprintf(w, pathTag, startLink, "open", i, opts, flatten(points, scaleX, scaleY), endLink)
	}
	if !first {
		io.WriteString(w, "</g>")
	}
}

func writeTexts(w io.Writer, c *canvas, font string, scaleX, scaleY int) {
	fontSize := float64(scaleY) * 0.75
	deltaX := float64(scaleX) / 4
	deltaY := float64(scaleY) / 4
	first := true
	for i, obj := range c.objects() {
		if !obj.isJustText() {
			continue
		}
		if first {
			fmt.Fprintf(w, `<g id="text" stroke="none" style="font-family:%s;font-size:%gpx">`, font, fontSize)
			first = false
		}

		// Look up the fill of the containing box to determine what text color to use.
		color := findTextColor(c, obj)

		startLink, endLink := "", ""
		text := string(obj.Text())
		tag := obj.Tag()
		if tag != "" {
			options := c.options()
			if label, ok := options[tag]["a2s:label"]; ok {
				text = label.(string)
			}

			// If we're a reference, the a2s:delref tag informs us to remove our reference.
			// TODO(dhobsd): If text is on column 0 but is not a special reference,
			// we can't really detect that here.
			if obj.Corners()[0].x == 0 {
				if _, ok := options[tag]["a2s:delref"]; ok {
					continue
				}
			}

			if link, ok := options[tag]["a2s:link"]; ok {
				startLink = link.(string)
				endLink = "</a>"
			}
		}
		sp := scale(obj.Points()[0], scaleX, scaleY)
		fmt.Fprintf(w,
			`<text id="obj%d" x="%g" y="%g">%s</text>`,
			i, sp.X-deltaX, sp.Y+deltaY, escape(text))
			`%s<text id="obj%d" x="%g" y="%g" fill="%s">%s</text>%s`,
			startLink, i, sp.X-deltaX, sp.Y+deltaY, color, escape(text), endLink)
	}
	if !first {
		io.WriteString(w, "</g>")
	}
}

func getTagOpts(options optionMaps, tag string) string {
	opts := ""
	if tagOpts, ok := options[tag]; ok {
		for k, v := range tagOpts {
			if strings.HasPrefix(k, "a2s:") {
				continue
			}

			switch v := v.(type) {
			case string:
				opts += fmt.Sprintf(`%s="%s" `, k, v)
			default:
				// TODO(dhobsd): Implement.
				opts += fmt.Sprintf(`%s="UNIMPLEMENTED" `, k)
			}
		}
	}
	return opts
}

func findTextColor(c *canvas, o *object) string {
	// If the tag on the text object is a special reference, that's the color we should use
	// for the text.
	options := c.options()
	if tag := o.Tag(); objTagRE.MatchString(tag) {
		if fill, ok := options[tag]["fill"]; ok {
			return fill.(string)
		}
	}

	// Otherwise, find the most specific fill and calibrate the color based on that.
	if containers := c.enclosingObjects(o.Points()[0]); containers != nil {
		for _, container := range containers {
			if tag := container.Tag(); tag != "" {
				if fill, ok := options[tag]["fill"]; ok {
					if fill == "none" {
						continue
					}
					return textColor(fill.(string))
				}
			}
		}
	}

	// Default to black.
	return "#000"
}

func escape(s string) string {
	b := bytes.Buffer{}
	strfun.XMLEscape(&b, s)
	return b.String()
}

Changes to parser/draw/svg_test.go.

33
34
35
36
37
38
39










40

41
42




































43

44
45
46
47
48
49
50
51
52
53


















54
55
56

57
58

59
60
61
62
63
64
65
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

89
90
91
92
93
94
95
96
97
98

99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121

122
123
124
125
126
127
128
129







+
+
+
+
+
+
+
+
+
+
-
+


+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+









-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+



+

-
+







		// 0 Box with dashed corners and text
		{
			[]string{
				"+--.",
				"|Hi:",
				"+--+",
			},
			510,
		},

		// 1 Box with non-existent ref
		{
			[]string{
				".-----.",
				"|[a]  |",
				"'-----'",
			},
			486,
			596,
		},

		// 2 Box with ref, change background color of container with #RRGGBB
		{
			[]string{
				".-----.",
				"|[a]  |",
				"'-----'",
				"",
				"[a]: {\"fill\":\"#000000\"}",
			},
			691,
		},

		// 3 Box with ref && fill, change label
		{
			[]string{
				".-----.",
				"|[a]  |",
				"'-----'",
				"",
				"[a]: {\"fill\":\"#000000\",\"a2s:label\":\"abcdefg\"}",
			},
			655,
		},

		// 4 Box with ref && fill && label, remove ref
		{
			[]string{
				".-----.",
				"|[a]  |",
				"'-----'",
				"",
				"[a]: {\"fill\":\"#000000\",\"a2s:label\":\"abcd\",\"a2s:delref\":1}",
			},
			597,
		},

		// 2 Ticks and dots in lines.
		// 5 Ticks and dots in lines.
		{
			[]string{
				" ------x----->",
				"",
				" <-----*------",
			},
			1088,
		},

		// 3 Just text
		// 6 Just text
		{
			[]string{
				" foo",
			},
			277,
		},

		// 7 Just text with a deleting reference
		{
			[]string{
				" foo",
				"[1,0]: {\"a2s:delref\":1,\"a2s:label\":\"foo\"}",
			},
			278,
		},

		// 8 Just text with a link
		{
			[]string{
				" foo",
				"[1,0]: {\"a2s:delref\":1, \"a2s:link\":\"https://github.com/asciitosvg/asciitosvg\"}",
			},
			265,
			322,
		},
	}
	for i, line := range data {
		canvas, err := newCanvas([]byte(strings.Join(line.input, "\n")), 9)
		if err != nil {
			t.Fatalf("Error creating canvas: %s", err)
		}

Changes to parser/markdown/markdown.go.

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

43
44
45
46
47
48
49



50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

67
68
69
70

71
72
73
74
75
76

77
78
79
80
81
82
83
16
17
18
19
20
21
22

23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

41
42
43
44
45



46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

65
66
67
68

69
70
71
72
73
74

75
76
77
78
79
80
81
82







-


















-
+




-
-
-
+
+
+
















-
+



-
+





-
+







	"fmt"

	gm "github.com/yuin/goldmark"
	gmAst "github.com/yuin/goldmark/ast"
	gmText "github.com/yuin/goldmark/text"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
	parser.Register(&parser.Info{
		Name:          "markdown",
		AltNames:      []string{"md"},
		IsTextParser:  true,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, _ *meta.Meta, _ string) ast.BlockSlice {
func parseBlocks(inp *input.Input, _ *meta.Meta, _ string) *ast.BlockListNode {
	p := parseMarkdown(inp)
	return p.acceptBlockChildren(p.docNode)
}

func parseInlines(inp *input.Input, syntax string) ast.InlineSlice {
	bs := parseBlocks(inp, nil, syntax)
	return bs.FirstParagraphInlines()
func parseInlines(inp *input.Input, syntax string) *ast.InlineListNode {
	bln := parseBlocks(inp, nil, syntax)
	return bln.List.FirstParagraphInlines()
}

func parseMarkdown(inp *input.Input) *mdP {
	source := []byte(inp.Src[inp.Pos:])
	parser := gm.DefaultParser()
	node := parser.Parse(gmText.NewReader(source))
	textEnc := encoder.Create(api.EncoderText, nil)
	return &mdP{source: source, docNode: node, textEnc: textEnc}
}

type mdP struct {
	source  []byte
	docNode gmAst.Node
	textEnc encoder.Encoder
}

func (p *mdP) acceptBlockChildren(docNode gmAst.Node) ast.BlockSlice {
func (p *mdP) acceptBlockChildren(docNode gmAst.Node) *ast.BlockListNode {
	if docNode.Type() != gmAst.TypeDocument {
		panic(fmt.Sprintf("Expected document, but got node type %v", docNode.Type()))
	}
	result := make(ast.BlockSlice, 0, docNode.ChildCount())
	result := make([]ast.BlockNode, 0, docNode.ChildCount())
	for child := docNode.FirstChild(); child != nil; child = child.NextSibling() {
		if block := p.acceptBlock(child); block != nil {
			result = append(result, block)
		}
	}
	return result
	return ast.CreateBlockListNode(result...)
}

func (p *mdP) acceptBlock(node gmAst.Node) ast.ItemNode {
	if node.Type() != gmAst.TypeBlock {
		panic(fmt.Sprintf("Expected block node, but got node type %v", node.Type()))
	}
	switch n := node.(type) {
100
101
102
103
104
105
106
107
108


109
110
111
112
113
114
115
99
100
101
102
103
104
105


106
107
108
109
110
111
112
113
114







-
-
+
+







	case *gmAst.HTMLBlock:
		return p.acceptHTMLBlock(n)
	}
	panic(fmt.Sprintf("Unhandled block node of kind %v", node.Kind()))
}

func (p *mdP) acceptParagraph(node *gmAst.Paragraph) ast.ItemNode {
	if is := p.acceptInlineChildren(node); len(is) > 0 {
		return &ast.ParaNode{Inlines: is}
	if iln := p.acceptInlineChildren(node); iln != nil && len(iln.List) > 0 {
		return &ast.ParaNode{Inlines: iln}
	}
	return nil
}

func (p *mdP) acceptHeading(node *gmAst.Heading) *ast.HeadingNode {
	return &ast.HeadingNode{
		Level:   node.Level,
129
130
131
132
133
134
135
136

137
138
139
140
141
142
143
128
129
130
131
132
133
134

135
136
137
138
139
140
141
142







-
+







		Kind:    ast.VerbatimProg,
		Attrs:   nil, //TODO
		Content: p.acceptRawText(node),
	}
}

func (p *mdP) acceptFencedCodeBlock(node *gmAst.FencedCodeBlock) *ast.VerbatimNode {
	var attrs zjson.Attributes
	var attrs *ast.Attributes
	if language := node.Language(p.source); len(language) > 0 {
		attrs = attrs.Set("class", "language-"+cleanText(language, true))
	}
	return &ast.VerbatimNode{
		Kind:    ast.VerbatimProg,
		Attrs:   attrs,
		Content: p.acceptRawText(node),
172
173
174
175
176
177
178
179

180
181
182
183
184
185
186
171
172
173
174
175
176
177

178
179
180
181
182
183
184
185







-
+







			p.acceptItemSlice(node),
		},
	}
}

func (p *mdP) acceptList(node *gmAst.List) ast.ItemNode {
	kind := ast.NestedListUnordered
	var attrs zjson.Attributes
	var attrs *ast.Attributes
	if node.IsOrdered() {
		kind = ast.NestedListOrdered
		if node.Start != 1 {
			attrs = attrs.Set("start", fmt.Sprintf("%d", node.Start))
		}
	}
	items := make([]ast.ItemSlice, 0, node.ChildCount())
205
206
207
208
209
210
211
212
213


214
215
216
217
218
219
220
204
205
206
207
208
209
210


211
212
213
214
215
216
217
218
219







-
-
+
+







			result = append(result, item)
		}
	}
	return result
}

func (p *mdP) acceptTextBlock(node *gmAst.TextBlock) ast.ItemNode {
	if is := p.acceptInlineChildren(node); len(is) > 0 {
		return &ast.ParaNode{Inlines: is}
	if iln := p.acceptInlineChildren(node); iln != nil && len(iln.List) > 0 {
		return &ast.ParaNode{Inlines: iln}
	}
	return nil
}

func (p *mdP) acceptHTMLBlock(node *gmAst.HTMLBlock) *ast.VerbatimNode {
	content := p.acceptRawText(node)
	if node.HasClosure() {
229
230
231
232
233
234
235
236
237


238
239
240
241
242
243

244
245
246

247
248
249
250
251
252
253
228
229
230
231
232
233
234


235
236
237
238
239
240
241

242
243
244

245
246
247
248
249
250
251
252







-
-
+
+





-
+


-
+







	}
	return &ast.VerbatimNode{
		Kind:    ast.VerbatimHTML,
		Content: content,
	}
}

func (p *mdP) acceptInlineChildren(node gmAst.Node) ast.InlineSlice {
	result := make(ast.InlineSlice, 0, node.ChildCount())
func (p *mdP) acceptInlineChildren(node gmAst.Node) *ast.InlineListNode {
	result := make([]ast.InlineNode, 0, node.ChildCount())
	for child := node.FirstChild(); child != nil; child = child.NextSibling() {
		if inlines := p.acceptInline(child); inlines != nil {
			result = append(result, inlines...)
		}
	}
	return result
	return ast.CreateInlineListNode(result...)
}

func (p *mdP) acceptInline(node gmAst.Node) ast.InlineSlice {
func (p *mdP) acceptInline(node gmAst.Node) []ast.InlineNode {
	if node.Type() != gmAst.TypeInline {
		panic(fmt.Sprintf("Expected inline node, but got %v", node.Type()))
	}
	switch n := node.(type) {
	case *gmAst.Text:
		return p.acceptText(n)
	case *gmAst.CodeSpan:
262
263
264
265
266
267
268
269

270
271
272
273
274
275

276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291

292
293
294
295

296
297
298
299
300
301
302
261
262
263
264
265
266
267

268
269
270
271
272
273

274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289

290
291
292
293

294
295
296
297
298
299
300
301







-
+





-
+















-
+



-
+







		return p.acceptAutoLink(n)
	case *gmAst.RawHTML:
		return p.acceptRawHTML(n)
	}
	panic(fmt.Sprintf("Unhandled inline node %v", node.Kind()))
}

func (p *mdP) acceptText(node *gmAst.Text) ast.InlineSlice {
func (p *mdP) acceptText(node *gmAst.Text) []ast.InlineNode {
	segment := node.Segment
	if node.IsRaw() {
		return splitText(string(segment.Value(p.source)))
	}
	ins := splitText(string(segment.Value(p.source)))
	result := make(ast.InlineSlice, 0, len(ins)+1)
	result := make([]ast.InlineNode, 0, len(ins)+1)
	for _, in := range ins {
		if tn, ok := in.(*ast.TextNode); ok {
			tn.Text = cleanText([]byte(tn.Text), true)
		}
		result = append(result, in)
	}
	if node.HardLineBreak() {
		result = append(result, &ast.BreakNode{Hard: true})
	} else if node.SoftLineBreak() {
		result = append(result, &ast.BreakNode{Hard: false})
	}
	return result
}

// splitText transform the text into a sequence of TextNode and SpaceNode
func splitText(text string) ast.InlineSlice {
func splitText(text string) []ast.InlineNode {
	if text == "" {
		return nil
	}
	result := make(ast.InlineSlice, 0, 1)
	result := make([]ast.InlineNode, 0, 1)

	state := 0 // 0=unknown,1=non-spaces,2=spaces
	lastPos := 0
	for pos, ch := range text {
		if input.IsSpace(ch) {
			if state == 1 {
				result = append(result, &ast.TextNode{Text: text[lastPos:pos]})
356
357
358
359
360
361
362
363
364


365
366
367
368
369
370
371
355
356
357
358
359
360
361


362
363
364
365
366
367
368
369
370







-
-
+
+







	}
	if lastPos < len(text) {
		buf.Write(text[lastPos:])
	}
	return buf.String()
}

func (p *mdP) acceptCodeSpan(node *gmAst.CodeSpan) ast.InlineSlice {
	return ast.InlineSlice{
func (p *mdP) acceptCodeSpan(node *gmAst.CodeSpan) []ast.InlineNode {
	return []ast.InlineNode{
		&ast.LiteralNode{
			Kind:    ast.LiteralProg,
			Attrs:   nil, //TODO
			Content: cleanCodeSpan(node.Text(p.source)),
		},
	}
}
385
386
387
388
389
390
391
392

393
394
395
396
397

398
399
400
401
402
403
404
405
406

407
408

409
410
411
412

413
414
415

416
417
418
419
420
421

422
423

424
425
426
427

428
429
430

431
432
433
434
435
436
437


438
439

440
441
442
443
444
445
446

447
448
449

450
451
452
453
454





455

456
457
458
459




460
461
462
463
464

465
466
467
468
469
470

471
472
473
474
475
476
477
384
385
386
387
388
389
390

391
392
393
394
395

396
397
398
399
400
401
402
403
404

405
406

407
408
409
410

411
412
413
414
415
416
417
418
419
420

421
422

423
424
425
426

427
428
429

430
431
432
433
434
435


436
437
438

439
440
441
442
443
444
445

446
447
448

449
450
451
452
453
454
455
456
457
458
459

460
461



462
463
464
465
466
467
468
469

470
471
472
473
474
475

476
477
478
479
480
481
482
483







-
+




-
+








-
+

-
+



-
+



+





-
+

-
+



-
+


-
+





-
-
+
+

-
+






-
+


-
+





+
+
+
+
+
-
+

-
-
-
+
+
+
+




-
+





-
+







			lastPos = pos + 1
		}
	}
	buf.Write(text[lastPos:])
	return buf.Bytes()
}

func (p *mdP) acceptEmphasis(node *gmAst.Emphasis) ast.InlineSlice {
func (p *mdP) acceptEmphasis(node *gmAst.Emphasis) []ast.InlineNode {
	kind := ast.FormatEmph
	if node.Level == 2 {
		kind = ast.FormatStrong
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.FormatNode{
			Kind:    kind,
			Attrs:   nil, //TODO
			Inlines: p.acceptInlineChildren(node),
		},
	}
}

func (p *mdP) acceptLink(node *gmAst.Link) ast.InlineSlice {
func (p *mdP) acceptLink(node *gmAst.Link) []ast.InlineNode {
	ref := ast.ParseReference(cleanText(node.Destination, true))
	var attrs zjson.Attributes
	var attrs *ast.Attributes
	if title := node.Title; len(title) > 0 {
		attrs = attrs.Set("title", cleanText(title, true))
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.LinkNode{
			Ref:     ref,
			Inlines: p.acceptInlineChildren(node),
			OnlyRef: false,
			Attrs:   attrs,
		},
	}
}

func (p *mdP) acceptImage(node *gmAst.Image) ast.InlineSlice {
func (p *mdP) acceptImage(node *gmAst.Image) []ast.InlineNode {
	ref := ast.ParseReference(cleanText(node.Destination, true))
	var attrs zjson.Attributes
	var attrs *ast.Attributes
	if title := node.Title; len(title) > 0 {
		attrs = attrs.Set("title", cleanText(title, true))
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.EmbedRefNode{
			Ref:     ref,
			Inlines: p.flattenInlineSlice(node),
			Inlines: p.flattenInlineList(node),
			Attrs:   attrs,
		},
	}
}

func (p *mdP) flattenInlineSlice(node gmAst.Node) ast.InlineSlice {
	is := p.acceptInlineChildren(node)
func (p *mdP) flattenInlineList(node gmAst.Node) *ast.InlineListNode {
	iln := p.acceptInlineChildren(node)
	var buf bytes.Buffer
	_, err := p.textEnc.WriteInlines(&buf, &is)
	_, err := p.textEnc.WriteInlines(&buf, iln)
	if err != nil {
		panic(err)
	}
	if buf.Len() == 0 {
		return nil
	}
	return ast.InlineSlice{&ast.TextNode{Text: buf.String()}}
	return ast.CreateInlineListNode(&ast.TextNode{Text: buf.String()})
}

func (p *mdP) acceptAutoLink(node *gmAst.AutoLink) ast.InlineSlice {
func (p *mdP) acceptAutoLink(node *gmAst.AutoLink) []ast.InlineNode {
	u := node.URL(p.source)
	if node.AutoLinkType == gmAst.AutoLinkEmail &&
		!bytes.HasPrefix(bytes.ToLower(u), []byte("mailto:")) {
		u = append([]byte("mailto:"), u...)
	}
	ref := ast.ParseReference(cleanText(u, false))
	label := node.Label(p.source)
	if len(label) == 0 {
		label = u
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.LinkNode{
			Ref:     ast.ParseReference(cleanText(u, false)),
			Inlines: nil,
			Attrs:   nil, // TODO
			Ref:     ref,
			Inlines: ast.CreateInlineListNode(&ast.TextNode{Text: string(label)}),
			OnlyRef: true,
			Attrs:   nil, //TODO
		},
	}
}

func (p *mdP) acceptRawHTML(node *gmAst.RawHTML) ast.InlineSlice {
func (p *mdP) acceptRawHTML(node *gmAst.RawHTML) []ast.InlineNode {
	segs := make([][]byte, 0, node.Segments.Len())
	for i := 0; i < node.Segments.Len(); i++ {
		segment := node.Segments.At(i)
		segs = append(segs, segment.Value(p.source))
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.LiteralNode{
			Kind:    ast.LiteralHTML,
			Attrs:   nil, // TODO: add HTML as language
			Content: bytes.Join(segs, nil),
		},
	}
}

Changes to parser/none/none.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package none provides a none-parser, e.g. for zettel with just metadata.
25
26
27
28
29
30
31


32

33
34
35

36
37

38
25
26
27
28
29
30
31
32
33

34

35

36
37

38
39







+
+
-
+
-

-
+

-
+

		AltNames:      []string{},
		IsTextParser:  false,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}
func parseBlocks(*input.Input, *meta.Meta, string) *ast.BlockListNode {
	return &ast.BlockListNode{}

}
func parseBlocks(*input.Input, *meta.Meta, string) ast.BlockSlice { return nil }

func parseInlines(inp *input.Input, _ string) ast.InlineSlice {
func parseInlines(inp *input.Input, _ string) *ast.InlineListNode {
	inp.SkipToEOL()
	return nil
	return &ast.InlineListNode{}
}

Changes to parser/parser.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package parser provides a generic interface to a range of different parsers.
29
30
31
32
33
34
35
36
37


38
39
40
41
42
43
44
29
30
31
32
33
34
35


36
37
38
39
40
41
42
43
44







-
-
+
+







// be valid. This can ce achieved on calling inp.Next() after the input stream
// was created.
type Info struct {
	Name          string
	AltNames      []string
	IsTextParser  bool
	IsImageFormat bool
	ParseBlocks   func(*input.Input, *meta.Meta, string) ast.BlockSlice
	ParseInlines  func(*input.Input, string) ast.InlineSlice
	ParseBlocks   func(*input.Input, *meta.Meta, string) *ast.BlockListNode
	ParseInlines  func(*input.Input, string) *ast.InlineListNode
}

var registry = map[string]*Info{}

// Register the parser (info) for later retrieval.
func Register(pi *Info) {
	if _, ok := registry[pi.Name]; ok {
88
89
90
91
92
93
94
95
96
97
98




99
100
101
102

103
104
105
106
107
108
109

110
111
112
113
114
115
116
88
89
90
91
92
93
94




95
96
97
98
99
100
101

102

103
104
105
106
107

108
109
110
111
112
113
114
115







-
-
-
-
+
+
+
+



-
+
-





-
+







	if !ok {
		return false
	}
	return pi.IsImageFormat
}

// ParseBlocks parses some input and returns a slice of block nodes.
func ParseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
	bs := Get(syntax).ParseBlocks(inp, m, syntax)
	cleaner.CleanBlockSlice(&bs)
	return bs
func ParseBlocks(inp *input.Input, m *meta.Meta, syntax string) *ast.BlockListNode {
	bln := Get(syntax).ParseBlocks(inp, m, syntax)
	cleaner.CleanBlockList(bln)
	return bln
}

// ParseInlines parses some input and returns a slice of inline nodes.
func ParseInlines(inp *input.Input, syntax string) ast.InlineSlice {
func ParseInlines(inp *input.Input, syntax string) *ast.InlineListNode {
	// Do not clean, because we don't know the context where this function will be called.
	return Get(syntax).ParseInlines(inp, syntax)
}

// ParseMetadata parses a string as Zettelmarkup, resulting in an inline slice.
// Typically used to parse the title or other metadata of type Zettelmarkup.
func ParseMetadata(value string) ast.InlineSlice {
func ParseMetadata(value string) *ast.InlineListNode {
	return ParseInlines(input.NewInput([]byte(value)), api.ValueSyntaxZmk)
}

// ParseZettel parses the zettel based on the syntax.
func ParseZettel(zettel domain.Zettel, syntax string, rtConfig config.Config) *ast.ZettelNode {
	m := zettel.Meta
	inhMeta := m

Changes to parser/plain/plain.go.

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
11
12
13
14
15
16
17

18
19
20
21
22
23
24







-







// Package plain provides a parser for plain text data.
package plain

import (
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
61
62
63
64
65
66
67
68

69
70
71

72
73
74
75


76
77
78

79
80
81

82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

101
102
103

104
105
106

107
108

109
110

111
112

113
114
115
116
117



118
119
120

121
122
123

124
125
126
127
128

129
130
131

132
133
134
135
136
137
138
139
140
141
142
143
144
60
61
62
63
64
65
66

67
68
69

70
71
72


73
74
75
76

77
78
79

80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98

99
100
101

102
103
104

105
106

107
108

109
110

111
112
113



114
115
116
117
118

119
120
121

122
123
124
125
126

127
128
129

130
131
132
133
134
135
136
137
138
139
140
141
142
143







-
+


-
+


-
-
+
+


-
+


-
+


















-
+


-
+


-
+

-
+

-
+

-
+


-
-
-
+
+
+


-
+


-
+




-
+


-
+













		IsTextParser:  false,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, _ *meta.Meta, syntax string) ast.BlockSlice {
func parseBlocks(inp *input.Input, _ *meta.Meta, syntax string) *ast.BlockListNode {
	return doParseBlocks(inp, syntax, ast.VerbatimProg)
}
func parseBlocksHTML(inp *input.Input, _ *meta.Meta, syntax string) ast.BlockSlice {
func parseBlocksHTML(inp *input.Input, _ *meta.Meta, syntax string) *ast.BlockListNode {
	return doParseBlocks(inp, syntax, ast.VerbatimHTML)
}
func doParseBlocks(inp *input.Input, syntax string, kind ast.VerbatimKind) ast.BlockSlice {
	return ast.BlockSlice{
func doParseBlocks(inp *input.Input, syntax string, kind ast.VerbatimKind) *ast.BlockListNode {
	return ast.CreateBlockListNode(
		&ast.VerbatimNode{
			Kind:    kind,
			Attrs:   zjson.Attributes{"": syntax},
			Attrs:   &ast.Attributes{Attrs: map[string]string{"": syntax}},
			Content: readContent(inp),
		},
	}
	)
}

func readContent(inp *input.Input) []byte {
	result := make([]byte, 0, len(inp.Src)-inp.Pos+1)
	for {
		inp.EatEOL()
		posL := inp.Pos
		if inp.Ch == input.EOS {
			return result
		}
		inp.SkipToEOL()
		if len(result) > 0 {
			result = append(result, '\n')
		}
		result = append(result, inp.Src[posL:inp.Pos]...)
	}
}

func parseInlines(inp *input.Input, syntax string) ast.InlineSlice {
func parseInlines(inp *input.Input, syntax string) *ast.InlineListNode {
	return doParseInlines(inp, syntax, ast.LiteralProg)
}
func parseInlinesHTML(inp *input.Input, syntax string) ast.InlineSlice {
func parseInlinesHTML(inp *input.Input, syntax string) *ast.InlineListNode {
	return doParseInlines(inp, syntax, ast.LiteralHTML)
}
func doParseInlines(inp *input.Input, syntax string, kind ast.LiteralKind) ast.InlineSlice {
func doParseInlines(inp *input.Input, syntax string, kind ast.LiteralKind) *ast.InlineListNode {
	inp.SkipToEOL()
	return ast.InlineSlice{&ast.LiteralNode{
	return ast.CreateInlineListNode(&ast.LiteralNode{
		Kind:    kind,
		Attrs:   zjson.Attributes{"": syntax},
		Attrs:   &ast.Attributes{Attrs: map[string]string{"": syntax}},
		Content: append([]byte(nil), inp.Src[0:inp.Pos]...),
	}}
	})
}

func parseSVGBlocks(inp *input.Input, _ *meta.Meta, syntax string) ast.BlockSlice {
	is := parseSVGInlines(inp, syntax)
	if len(is) == 0 {
func parseSVGBlocks(inp *input.Input, _ *meta.Meta, syntax string) *ast.BlockListNode {
	iln := parseSVGInlines(inp, syntax)
	if iln == nil {
		return nil
	}
	return ast.BlockSlice{ast.CreateParaNode(is...)}
	return ast.CreateBlockListNode(ast.CreateParaNode(iln))
}

func parseSVGInlines(inp *input.Input, syntax string) ast.InlineSlice {
func parseSVGInlines(inp *input.Input, syntax string) *ast.InlineListNode {
	svgSrc := scanSVG(inp)
	if svgSrc == "" {
		return nil
	}
	return ast.InlineSlice{&ast.EmbedBLOBNode{
	return ast.CreateInlineListNode(&ast.EmbedBLOBNode{
		Blob:   []byte(svgSrc),
		Syntax: syntax,
	}}
	})
}

func scanSVG(inp *input.Input) string {
	for input.IsSpace(inp.Ch) {
		inp.Next()
	}
	svgSrc := string(inp.Src[inp.Pos:])
	if !strings.HasPrefix(svgSrc, "<svg ") {
		return ""
	}
	// TODO: check proper end </svg>
	return svgSrc
}

Changes to parser/zettelmark/block.go.

1
2
3
4
5
6
7
8
9
10

11
12
13
14
15
16
17
18
19
20
21


22
23
24

25
26
27
28
29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20


21
22
23
24

25
26
27
28
29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45










+









-
-
+
+


-
+












-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package zettelmark provides a parser for zettelmarkup.
package zettelmark

import (
	"fmt"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/input"
)

// parseBlockSlice parses a sequence of blocks.
func (cp *zmkP) parseBlockSlice() ast.BlockSlice {
// parseBlockList parses a sequence of blocks.
func (cp *zmkP) parseBlockList() *ast.BlockListNode {
	inp := cp.inp
	var lastPara *ast.ParaNode
	bs := make(ast.BlockSlice, 0, 2)
	bs := make([]ast.BlockNode, 0, 2)
	for inp.Ch != input.EOS {
		bn, cont := cp.parseBlock(lastPara)
		if bn != nil {
			bs = append(bs, bn)
		}
		if !cont {
			lastPara, _ = bn.(*ast.ParaNode)
		}
	}
	if cp.nestingLevel != 0 {
		panic("Nesting level was not decremented")
	}
	return bs
	return ast.CreateBlockListNode(bs...)
}

// parseBlock parses one block.
func (cp *zmkP) parseBlock(lastPara *ast.ParaNode) (res ast.BlockNode, cont bool) {
	inp := cp.inp
	pos := inp.Pos
	if cp.nestingLevel <= maxNestingLevel {
93
94
95
96
97
98
99
100

101
102
103
104
105
106
107
94
95
96
97
98
99
100

101
102
103
104
105
106
107
108







-
+







			return bn, false
		}
	}
	inp.SetPos(pos)
	cp.clearStacked()
	pn := cp.parsePara()
	if lastPara != nil {
		lastPara.Inlines = append(lastPara.Inlines, pn.Inlines...)
		lastPara.Inlines.Append(pn.Inlines.List...)
		return nil, true
	}
	return pn, false
}

func (cp *zmkP) cleanupListsAfterEOL() {
	for _, l := range cp.lists {
132
133
134
135
136
137
138
139

140
141
142
143
144
145
146
133
134
135
136
137
138
139

140
141
142
143
144
145
146
147







-
+







func (cp *zmkP) parsePara() *ast.ParaNode {
	pn := ast.NewParaNode()
	for {
		in := cp.parseInline()
		if in == nil {
			return pn
		}
		pn.Inlines = append(pn.Inlines, in)
		pn.Inlines.Append(in)
		if _, ok := in.(*ast.BreakNode); ok {
			ch := cp.inp.Ch
			switch ch {
			// Must contain all cases from above switch in parseBlock.
			case input.EOS, '\n', '\r', '@', '`', runeModGrave, '%', '"', '<', '=', '-', '*', '#', '>', ';', ':', ' ', '|', '{':
				return pn
			}
226
227
228
229
230
231
232
233

234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252

253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273







274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291

292
293
294
295
296
297
298
299
300

301
302
303
304
305
306
307
227
228
229
230
231
232
233

234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252

253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272


273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296

297
298
299
300
301
302
303
304
305

306
307
308
309
310
311
312
313







-
+


















-
+



















-
-
+
+
+
+
+
+
+

















-
+








-
+







	inp.SkipToEOL()
	if inp.Ch == input.EOS {
		return nil, false
	}
	rn = &ast.RegionNode{
		Kind:    kind,
		Attrs:   attrs,
		Blocks:  nil,
		Blocks:  &ast.BlockListNode{},
		Inlines: nil,
	}
	var lastPara *ast.ParaNode
	inp.EatEOL()
	for {
		posL := inp.Pos
		switch inp.Ch {
		case fch:
			if cp.countDelim(fch) >= cnt {
				cp.parseRegionLastLine(rn)
				return rn, true
			}
			inp.SetPos(posL)
		case input.EOS:
			return nil, false
		}
		bn, cont := cp.parseBlock(lastPara)
		if bn != nil {
			rn.Blocks = append(rn.Blocks, bn)
			rn.Blocks.List = append(rn.Blocks.List, bn)
		}
		if !cont {
			lastPara, _ = bn.(*ast.ParaNode)
		}
	}
}

func (cp *zmkP) parseRegionLastLine(rn *ast.RegionNode) {
	cp.clearStacked() // remove any lists defined in the region
	cp.skipSpace()
	for {
		switch cp.inp.Ch {
		case input.EOS, '\n', '\r':
			return
		}
		in := cp.parseInline()
		if in == nil {
			return
		}
		rn.Inlines = append(rn.Inlines, in)
	}
		if rn.Inlines == nil {
			rn.Inlines = ast.CreateInlineListNode(in)
		} else {
			rn.Inlines.Append(in)
		}
	}

}

// parseHeading parses a head line.
func (cp *zmkP) parseHeading() (hn *ast.HeadingNode, success bool) {
	inp := cp.inp
	delims := cp.countDelim(inp.Ch)
	if delims < 3 {
		return nil, false
	}
	if inp.Ch != ' ' {
		return nil, false
	}
	inp.Next()
	cp.skipSpace()
	if delims > 7 {
		delims = 7
	}
	hn = &ast.HeadingNode{Level: delims - 2, Inlines: nil}
	hn = &ast.HeadingNode{Level: delims - 2, Inlines: &ast.InlineListNode{}}
	for {
		if input.IsEOLEOS(inp.Ch) {
			return hn, true
		}
		in := cp.parseInline()
		if in == nil {
			return hn, true
		}
		hn.Inlines = append(hn.Inlines, in)
		hn.Inlines.Append(in)
		if inp.Ch == '{' && inp.Peek() != '{' {
			attrs := cp.parseAttributes(true)
			hn.Attrs = attrs
			inp.SkipToEOL()
			return hn, true
		}
	}
426
427
428
429
430
431
432
433

434
435
436
437



438

439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459

460
461
462
463
464
465
466
432
433
434
435
436
437
438

439
440
441
442
443
444
445
446

447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467

468
469
470
471
472
473
474
475







-
+




+
+
+
-
+




















-
+







	defPos := len(descrl.Descriptions) - 1
	if defPos == 0 {
		res = descrl
	}
	for {
		in := cp.parseInline()
		if in == nil {
			if len(descrl.Descriptions[defPos].Term) == 0 {
			if descrl.Descriptions[defPos].Term == nil {
				return nil, false
			}
			return res, true
		}
		if descrl.Descriptions[defPos].Term == nil {
			descrl.Descriptions[defPos].Term = &ast.InlineListNode{}
		}
		descrl.Descriptions[defPos].Term = append(descrl.Descriptions[defPos].Term, in)
		descrl.Descriptions[defPos].Term.Append(in)
		if _, ok := in.(*ast.BreakNode); ok {
			return res, true
		}
	}
}

// parseDefDescr parses a description of a definition list.
func (cp *zmkP) parseDefDescr() (res ast.BlockNode, success bool) {
	inp := cp.inp
	inp.Next()
	if inp.Ch != ' ' {
		return nil, false
	}
	inp.Next()
	cp.skipSpace()
	descrl := cp.descrl
	if descrl == nil || len(descrl.Descriptions) == 0 {
		return nil, false
	}
	defPos := len(descrl.Descriptions) - 1
	if len(descrl.Descriptions[defPos].Term) == 0 {
	if descrl.Descriptions[defPos].Term == nil {
		return nil, false
	}
	pn := cp.parseLinePara()
	if pn == nil {
		return nil, false
	}
	cp.lists = nil
500
501
502
503
504
505
506
507

508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526

527
528
529
530
531
532
533
534
535
536
537
538
539
540
541

542
543
544
545
546
547
548
549
550
551
552
553
554
555

556
557
558
559
560

561
562
563
564
565
566
567
509
510
511
512
513
514
515

516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534

535
536
537
538
539
540
541
542
543
544
545
546
547
548
549

550
551
552
553
554
555
556
557
558
559
560
561
562
563

564
565
566
567
568

569
570
571
572
573
574
575
576







-
+


















-
+














-
+













-
+




-
+







	ln := cp.lists[cnt-1]
	pn := cp.parseLinePara()
	if pn == nil {
		pn = ast.NewParaNode()
	}
	lbn := ln.Items[len(ln.Items)-1]
	if lpn, ok := lbn[len(lbn)-1].(*ast.ParaNode); ok {
		lpn.Inlines = append(lpn.Inlines, pn.Inlines...)
		lpn.Inlines.Append(pn.Inlines.List...)
	} else {
		ln.Items[len(ln.Items)-1] = append(ln.Items[len(ln.Items)-1], pn)
	}
	return true
}

func (cp *zmkP) parseIndentForDescription(cnt int) bool {
	defPos := len(cp.descrl.Descriptions) - 1
	if cnt < 1 || defPos < 0 {
		return false
	}
	if len(cp.descrl.Descriptions[defPos].Descriptions) == 0 {
		// Continuation of a definition term
		for {
			in := cp.parseInline()
			if in == nil {
				return true
			}
			cp.descrl.Descriptions[defPos].Term = append(cp.descrl.Descriptions[defPos].Term, in)
			cp.descrl.Descriptions[defPos].Term.Append(in)
			if _, ok := in.(*ast.BreakNode); ok {
				return true
			}
		}
	}

	// Continuation of a definition description
	pn := cp.parseLinePara()
	if pn == nil {
		return false
	}
	descrPos := len(cp.descrl.Descriptions[defPos].Descriptions) - 1
	lbn := cp.descrl.Descriptions[defPos].Descriptions[descrPos]
	if lpn, ok := lbn[len(lbn)-1].(*ast.ParaNode); ok {
		lpn.Inlines = append(lpn.Inlines, pn.Inlines...)
		lpn.Inlines.Append(pn.Inlines.List...)
	} else {
		descrPos = len(cp.descrl.Descriptions[defPos].Descriptions) - 1
		cp.descrl.Descriptions[defPos].Descriptions[descrPos] = append(cp.descrl.Descriptions[defPos].Descriptions[descrPos], pn)
	}
	return true
}

// parseLinePara parses one line of inline material.
func (cp *zmkP) parseLinePara() *ast.ParaNode {
	pn := ast.NewParaNode()
	for {
		in := cp.parseInline()
		if in == nil {
			if len(pn.Inlines) == 0 {
			if pn.Inlines == nil {
				return nil
			}
			return pn
		}
		pn.Inlines = append(pn.Inlines, in)
		pn.Inlines.Append(in)
		if _, ok := in.(*ast.BreakNode); ok {
			return pn
		}
	}
}

// parseRow parse one table row.
594
595
596
597
598
599
600
601

602
603
604
605
606
607

608
609
610

611
612
613
614
615
616
617
603
604
605
606
607
608
609

610
611
612
613
614
615

616
617
618

619
620
621
622
623
624
625
626







-
+





-
+


-
+







		// inp.Ch must be '|'
	}
}

// parseCell parses one single cell of a table row.
func (cp *zmkP) parseCell() *ast.TableCell {
	inp := cp.inp
	var l ast.InlineSlice
	var l []ast.InlineNode
	for {
		if input.IsEOLEOS(inp.Ch) {
			if len(l) == 0 {
				return nil
			}
			return &ast.TableCell{Inlines: l}
			return &ast.TableCell{Inlines: ast.CreateInlineListNode(l...)}
		}
		if inp.Ch == '|' {
			return &ast.TableCell{Inlines: l}
			return &ast.TableCell{Inlines: ast.CreateInlineListNode(l...)}
		}
		l = append(l, cp.parseInline())
	}
}

// parseTransclusion parses '{' '{' '{' ZID '}' '}' '}'
func (cp *zmkP) parseTransclusion() (ast.BlockNode, bool) {

Changes to parser/zettelmark/inline.go.

15
16
17
18
19
20
21
22
23


24
25

26
27
28
29
30
31

32
33

34
35
36
37
38
39
40
15
16
17
18
19
20
21


22
23
24

25
26
27
28
29
30

31
32

33
34
35
36
37
38
39
40







-
-
+
+

-
+





-
+

-
+







	"bytes"
	"fmt"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/input"
)

// parseInlineSlice parses a sequence of Inlines until EOS.
func (cp *zmkP) parseInlineSlice() ast.InlineSlice {
// parseInlineList parses a sequence of Inlines until EOS.
func (cp *zmkP) parseInlineList() *ast.InlineListNode {
	inp := cp.inp
	var is ast.InlineSlice
	var ins []ast.InlineNode
	for inp.Ch != input.EOS {
		in := cp.parseInline()
		if in == nil {
			break
		}
		is = append(is, in)
		ins = append(ins, in)
	}
	return is
	return ast.CreateInlineListNode(ins...)
}

func (cp *zmkP) parseInline() ast.InlineNode {
	inp := cp.inp
	pos := inp.Pos
	if cp.nestingLevel <= maxNestingLevel {
		cp.nestingLevel++
66
67
68
69
70
71
72
73

74
75

76
77
78
79
80
81
82
66
67
68
69
70
71
72

73
74

75
76
77
78
79
80
81
82







-
+

-
+







			if inp.Ch == '{' {
				in, success = cp.parseEmbed()
			}
		case '#':
			return cp.parseTag()
		case '%':
			in, success = cp.parseComment()
		case '_', '*', '>', '~', '^', ',', '"', ':':
		case '_', '*', '>', '~', '\'', '^', ',', '<', '"', ':':
			in, success = cp.parseFormat()
		case '@', '\'', '`', '=', runeModGrave:
		case '@', '+', '`', '=', runeModGrave:
			in, success = cp.parseLiteral()
		case '\\':
			return cp.parseBackslash()
		case '-':
			in, success = cp.parseNdash()
		case '&':
			in, success = cp.parseEntity()
96
97
98
99
100
101
102
103

104
105
106
107
108
109
110
96
97
98
99
100
101
102

103
104
105
106
107
108
109
110







-
+







		return cp.parseTextBackslash()
	}
	for {
		inp.Next()
		switch inp.Ch {
		// The following case must contain all runes that occur in parseInline!
		// Plus the closing brackets ] and } and ) and the middle |
		case input.EOS, '\n', '\r', ' ', '\t', '[', ']', '{', '}', '(', ')', '|', '#', '%', '_', '*', '>', '~', '^', ',', '"', ':', '\'', '@', '`', runeModGrave, '=', '\\', '-', '&':
		case input.EOS, '\n', '\r', ' ', '\t', '[', ']', '{', '}', '(', ')', '|', '#', '%', '_', '*', '>', '~', '\'', '^', ',', '<', '"', ':', '+', '@', '`', runeModGrave, '=', '\\', '-', '&':
			return &ast.TextNode{Text: string(inp.Src[pos:inp.Pos])}
		}
	}
}

func (cp *zmkP) parseTextBackslash() *ast.TextNode {
	cp.inp.Next()
152
153
154
155
156
157
158
159

160
161






162
163
164



165
166
167
168
169
170
171
172

173
174
175
176
177
178
179
180

181
182
183
184
185
186
187
188
189
190
191

192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212

213
214
215

216
217
218
219
220
221
222
152
153
154
155
156
157
158

159
160
161
162
163
164
165
166
167
168


169
170
171
172
173
174
175
176
177
178

179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198

199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219

220
221
222

223
224
225
226
227
228
229
230







-
+


+
+
+
+
+
+

-
-
+
+
+







-
+








+










-
+




















-
+


-
+








func (cp *zmkP) parseSoftBreak() *ast.BreakNode {
	cp.inp.EatEOL()
	return &ast.BreakNode{}
}

func (cp *zmkP) parseLink() (*ast.LinkNode, bool) {
	if ref, is, ok := cp.parseReference(']'); ok {
	if ref, iln, ok := cp.parseReference(']'); ok {
		attrs := cp.parseAttributes(false)
		if len(ref) > 0 {
			onlyRef := false
			r := ast.ParseReference(ref)
			if iln == nil {
				iln = ast.CreateInlineListNode(&ast.TextNode{Text: ref})
				onlyRef = true
			}
			return &ast.LinkNode{
				Ref:     ast.ParseReference(ref),
				Inlines: is,
				Ref:     r,
				Inlines: iln,
				OnlyRef: onlyRef,
				Attrs:   attrs,
			}, true
		}
	}
	return nil, false
}

func (cp *zmkP) parseReference(closeCh rune) (ref string, is ast.InlineSlice, ok bool) {
func (cp *zmkP) parseReference(closeCh rune) (ref string, iln *ast.InlineListNode, ok bool) {
	inp := cp.inp
	inp.Next()
	cp.skipSpace()
	pos := inp.Pos
	hasSpace, ok := cp.readReferenceToSep(closeCh)
	if !ok {
		return "", nil, false
	}
	var ins []ast.InlineNode
	if inp.Ch == '|' { // First part must be inline text
		if pos == inp.Pos { // [[| or {{|
			return "", nil, false
		}
		cp.inp = input.NewInput(inp.Src[pos:inp.Pos])
		for {
			in := cp.parseInline()
			if in == nil {
				break
			}
			is = append(is, in)
			ins = append(ins, in)
		}
		cp.inp = inp
		inp.Next()
	} else if hasSpace {
		return "", nil, false
	} else {
		inp.SetPos(pos)
	}

	cp.skipSpace()
	pos = inp.Pos
	if !cp.readReferenceToClose(closeCh) {
		return "", nil, false
	}
	ref = string(inp.Src[pos:inp.Pos])
	inp.Next()
	if inp.Ch != closeCh {
		return "", nil, false
	}
	inp.Next()
	if len(is) == 0 {
	if len(ins) == 0 {
		return ref, nil, true
	}
	return ref, is, true
	return ref, ast.CreateInlineListNode(ins...), true
}

func (cp *zmkP) readReferenceToSep(closeCh rune) (bool, bool) {
	hasSpace := false
	inp := cp.inp
	for {
		switch inp.Ch {
299
300
301
302
303
304
305
306

307
308
309
310



311

312
313
314

315
316

317
318
319
320
321
322
323

324
325
326
327
328
329

330
331
332

333
334
335
336

337
338
339
340
341
342

343
344
345
346
347
348
349
350
351
352
353
354

355
356
357
358
359
360

361
362
363

364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
307
308
309
310
311
312
313

314
315
316
317
318
319
320
321

322
323
324

325
326

327
328
329
330
331
332
333

334
335
336
337
338
339

340
341
342

343
344
345
346

347
348
349
350
351
352

353
354
355
356
357
358
359
360
361
362
363
364

365
366
367
368
369
370

371



372









373
374
375
376
377
378
379







-
+




+
+
+
-
+


-
+

-
+






-
+





-
+


-
+



-
+





-
+











-
+





-
+
-
-
-
+
-
-
-
-
-
-
-
-
-







	}
	attrs := cp.parseAttributes(false)
	return &ast.CiteNode{Key: string(inp.Src[pos:posL]), Inlines: ins, Attrs: attrs}, true
}

func (cp *zmkP) parseFootnote() (*ast.FootnoteNode, bool) {
	cp.inp.Next()
	is, ok := cp.parseLinkLikeRest()
	iln, ok := cp.parseLinkLikeRest()
	if !ok {
		return nil, false
	}
	attrs := cp.parseAttributes(false)
	if iln == nil {
		iln = &ast.InlineListNode{}
	}
	return &ast.FootnoteNode{Inlines: is, Attrs: attrs}, true
	return &ast.FootnoteNode{Inlines: iln, Attrs: attrs}, true
}

func (cp *zmkP) parseLinkLikeRest() (ast.InlineSlice, bool) {
func (cp *zmkP) parseLinkLikeRest() (*ast.InlineListNode, bool) {
	cp.skipSpace()
	var is ast.InlineSlice
	var ins []ast.InlineNode
	inp := cp.inp
	for inp.Ch != ']' {
		in := cp.parseInline()
		if in == nil {
			return nil, false
		}
		is = append(is, in)
		ins = append(ins, in)
		if _, ok := in.(*ast.BreakNode); ok && input.IsEOLEOS(inp.Ch) {
			return nil, false
		}
	}
	inp.Next()
	if len(is) == 0 {
	if len(ins) == 0 {
		return nil, true
	}
	return is, true
	return ast.CreateInlineListNode(ins...), true
}

func (cp *zmkP) parseEmbed() (ast.InlineNode, bool) {
	if ref, is, ok := cp.parseReference('}'); ok {
	if ref, iln, ok := cp.parseReference('}'); ok {
		attrs := cp.parseAttributes(false)
		if len(ref) > 0 {
			r := ast.ParseReference(ref)
			return &ast.EmbedRefNode{
				Ref:     r,
				Inlines: is,
				Inlines: iln,
				Attrs:   attrs,
			}, true
		}
	}
	return nil, false
}

func (cp *zmkP) parseMark() (*ast.MarkNode, bool) {
	inp := cp.inp
	inp.Next()
	pos := inp.Pos
	for inp.Ch != '|' && inp.Ch != ']' {
	for inp.Ch != ']' {
		if !isNameRune(inp.Ch) {
			return nil, false
		}
		inp.Next()
	}
	mark := inp.Src[pos:inp.Pos]
	mn := &ast.MarkNode{Text: string(inp.Src[pos:inp.Pos])}
	var is ast.InlineSlice
	if inp.Ch == '|' {
		inp.Next()
	inp.Next()
		var ok bool
		is, ok = cp.parseLinkLikeRest()
		if !ok {
			return nil, false
		}
	} else {
		inp.Next()
	}
	mn := &ast.MarkNode{Mark: string(mark), Inlines: is}
	return mn, true
}

func (cp *zmkP) parseTag() ast.InlineNode {
	inp := cp.inp
	posH := inp.Pos
	inp.Next()
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420










421
422
423
424
425
426
427
428
429
430
431
432
433
434
435

436
437
438
439
440
441
442
443
444
445
446
447

448
449
450
451
452

453
454
455
456
457
458
459
460
461

462
463
464
465
466
467
468
406
407
408
409
410
411
412








413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436

437
438
439
440
441
442
443
444
445
446
447
448

449
450
451
452
453

454
455
456
457
458
459
460
461
462

463
464
465
466
467
468
469
470







-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+














-
+











-
+




-
+








-
+







			}, true
		}
		inp.Next()
	}
}

var mapRuneFormat = map[rune]ast.FormatKind{
	'_': ast.FormatEmph,
	'*': ast.FormatStrong,
	'>': ast.FormatInsert,
	'~': ast.FormatDelete,
	'^': ast.FormatSuper,
	',': ast.FormatSub,
	'"': ast.FormatQuote,
	':': ast.FormatSpan,
	'_':  ast.FormatEmph,
	'*':  ast.FormatStrong,
	'>':  ast.FormatInsert,
	'~':  ast.FormatDelete,
	'\'': ast.FormatMonospace,
	'^':  ast.FormatSuper,
	',':  ast.FormatSub,
	'<':  ast.FormatQuotation,
	'"':  ast.FormatQuote,
	':':  ast.FormatSpan,
}

func (cp *zmkP) parseFormat() (res ast.InlineNode, success bool) {
	inp := cp.inp
	fch := inp.Ch
	kind, ok := mapRuneFormat[fch]
	if !ok {
		panic(fmt.Sprintf("%q is not a formatting char", fch))
	}
	inp.Next() // read 2nd formatting character
	if inp.Ch != fch {
		return nil, false
	}
	inp.Next()
	fn := &ast.FormatNode{Kind: kind, Inlines: nil}
	fn := &ast.FormatNode{Kind: kind, Inlines: &ast.InlineListNode{}}
	for {
		if inp.Ch == input.EOS {
			return nil, false
		}
		if inp.Ch == fch {
			inp.Next()
			if inp.Ch == fch {
				inp.Next()
				fn.Attrs = cp.parseAttributes(false)
				return fn, true
			}
			fn.Inlines = append(fn.Inlines, &ast.TextNode{Text: string(fch)})
			fn.Inlines.Append(&ast.TextNode{Text: string(fch)})
		} else if in := cp.parseInline(); in != nil {
			if _, ok = in.(*ast.BreakNode); ok && input.IsEOLEOS(inp.Ch) {
				return nil, false
			}
			fn.Inlines = append(fn.Inlines, in)
			fn.Inlines.Append(in)
		}
	}
}

var mapRuneLiteral = map[rune]ast.LiteralKind{
	'@':          ast.LiteralZettel,
	'`':          ast.LiteralProg,
	runeModGrave: ast.LiteralProg,
	'\'':         ast.LiteralInput,
	'+':          ast.LiteralKeyb,
	'=':          ast.LiteralOutput,
}

func (cp *zmkP) parseLiteral() (res ast.InlineNode, success bool) {
	inp := cp.inp
	fch := inp.Ch
	kind, ok := mapRuneLiteral[fch]

Changes to parser/zettelmark/post-processor.go.

1
2
3
4
5
6
7
8
9
10

11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26

27
28

29
30
31
32
33
34
35
36
37
38
39
40
41




42
43
44
45

46
47
48
49
50
51

52
53
54
55
56
57
58
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26

27
28

29
30
31
32
33
34
35
36
37
38




39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61










+









-
+





-
+

-
+









-
-
-
-
+
+
+
+




+






+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package zettelmark provides a parser for zettelmarkup.
package zettelmark

import (
	"strings"

	"zettelstore.de/z/ast"
)

// postProcessBlocks is the entry point for post-processing a list of block nodes.
func postProcessBlocks(bs *ast.BlockSlice) {
func postProcessBlocks(bs *ast.BlockListNode) {
	pp := postProcessor{}
	ast.Walk(&pp, bs)
}

// postProcessInlines is the entry point for post-processing a list of inline nodes.
func postProcessInlines(is *ast.InlineSlice) {
func postProcessInlines(iln *ast.InlineListNode) {
	pp := postProcessor{}
	ast.Walk(&pp, is)
	ast.Walk(&pp, iln)
}

// postProcessor is a visitor that cleans the abstract syntax tree.
type postProcessor struct {
	inVerse bool
}

func (pp *postProcessor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockSlice:
		pp.visitBlockSlice(n)
	case *ast.InlineSlice:
		pp.visitInlineSlice(n)
	case *ast.BlockListNode:
		pp.visitBlockList(n)
	case *ast.InlineListNode:
		pp.visitInlineList(n)
	case *ast.ParaNode:
		return pp
	case *ast.RegionNode:
		pp.visitRegion(n)
		return pp
	case *ast.HeadingNode:
		return pp
	case *ast.NestedListNode:
		pp.visitNestedList(n)
	case *ast.DescriptionListNode:
		pp.visitDescriptionList(n)
		return pp
	case *ast.TableNode:
		pp.visitTable(n)
	case *ast.LinkNode:
		return pp
	case *ast.EmbedRefNode:
		return pp
	case *ast.EmbedBLOBNode:
68
69
70
71
72
73
74
75

76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
71
72
73
74
75
76
77

78



79
80
81
82
83
84
85
86
87
88
89



90
91
92
93
94
95
96







-
+
-
-
-











-
-
-







}

func (pp *postProcessor) visitRegion(rn *ast.RegionNode) {
	oldVerse := pp.inVerse
	if rn.Kind == ast.RegionVerse {
		pp.inVerse = true
	}
	pp.visitBlockSlice(&rn.Blocks)
	pp.visitBlockList(rn.Blocks)
	if len(rn.Inlines) > 0 {
		pp.visitInlineSlice(&rn.Inlines)
	}
	pp.inVerse = oldVerse
}

func (pp *postProcessor) visitNestedList(ln *ast.NestedListNode) {
	for i, item := range ln.Items {
		ln.Items[i] = pp.processItemSlice(item)
	}
}

func (pp *postProcessor) visitDescriptionList(dn *ast.DescriptionListNode) {
	for i, def := range dn.Descriptions {
		if len(def.Term) > 0 {
			ast.Walk(pp, &dn.Descriptions[i].Term)
		}
		for j, b := range def.Descriptions {
			dn.Descriptions[i].Descriptions[j] = pp.processDescriptionSlice(b)
		}
	}
}

func (pp *postProcessor) visitTable(tn *ast.TableNode) {
114
115
116
117
118
119
120
121

122
123
124
125
126
127
128
111
112
113
114
115
116
117

118
119
120
121
122
123
124
125







-
+







		}
	}
	pp.visitTableRows(tn, width)
}

func (*postProcessor) visitTableHeader(tn *ast.TableNode) {
	for pos, cell := range tn.Header {
		ins := cell.Inlines
		ins := cell.Inlines.List
		if len(ins) == 0 {
			continue
		}
		if textNode, ok := ins[0].(*ast.TextNode); ok {
			textNode.Text = strings.TrimPrefix(textNode.Text, "=")
		}
		if textNode, ok := ins[len(ins)-1].(*ast.TextNode); ok {
157
158
159
160
161
162
163
164

165
166
167
168
169
170
171
172
173



174
175
176
177
178
179
180
154
155
156
157
158
159
160

161
162
163
164
165
166
167
168


169
170
171
172
173
174
175
176
177
178







-
+







-
-
+
+
+







	return width
}

func appendCells(row ast.TableRow, width int, colAlign []ast.Alignment) ast.TableRow {
	for len(row) < width {
		row = append(row, &ast.TableCell{
			Align:   colAlign[len(row)],
			Inlines: nil,
			Inlines: &ast.InlineListNode{},
		})
	}
	return row
}

func isHeaderRow(row ast.TableRow) bool {
	for _, cell := range row {
		if is := cell.Inlines; len(is) > 0 {
			if textNode, ok := is[0].(*ast.TextNode); ok {
		iln := cell.Inlines
		if inlines := iln.List; len(inlines) > 0 {
			if textNode, ok := inlines[0].(*ast.TextNode); ok {
				if strings.HasPrefix(textNode.Text, "=") {
					return true
				}
			}
		}
	}
	return false
191
192
193
194
195
196
197


198

199
200
201
202
203
204
205
206
207
208
209

210
211
212

213
214
215
216
217
218
219
220
221
222
223


224
225
226
227


228
229
230

231
232
233
234
235


236
237

238
239

240
241
242
243
244
245
246
247
248
249


250
251

252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270

271
272
273
274
275
276
277
189
190
191
192
193
194
195
196
197

198
199
200
201
202
203
204
205
206
207
208

209
210
211

212
213
214
215
216
217
218
219
220
221


222
223
224
225


226
227
228
229

230
231
232
233


234
235
236

237
238

239
240
241
242
243
244
245
246
247


248
249
250

251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269

270
271
272
273
274
275
276
277







+
+
-
+










-
+


-
+









-
-
+
+


-
-
+
+


-
+



-
-
+
+

-
+

-
+








-
-
+
+

-
+


















-
+







	default:
		return ast.AlignDefault
	}
}

// processCell tries to recognize cell formatting.
func (pp *postProcessor) processCell(cell *ast.TableCell, colAlign ast.Alignment) {
	iln := cell.Inlines
	ins := iln.List
	if tn := initialText(cell.Inlines); tn != nil {
	if tn := initialText(ins); tn != nil {
		align := getAlignment(tn.Text[0])
		if align == ast.AlignDefault {
			cell.Align = colAlign
		} else {
			tn.Text = tn.Text[1:]
			cell.Align = align
		}
	} else {
		cell.Align = colAlign
	}
	ast.Walk(pp, &cell.Inlines)
	ast.Walk(pp, iln)
}

func initialText(ins ast.InlineSlice) *ast.TextNode {
func initialText(ins []ast.InlineNode) *ast.TextNode {
	if len(ins) == 0 {
		return nil
	}
	if tn, ok := ins[0].(*ast.TextNode); ok && len(tn.Text) > 0 {
		return tn
	}
	return nil
}

func (pp *postProcessor) visitBlockSlice(bs *ast.BlockSlice) {
	if bs == nil {
func (pp *postProcessor) visitBlockList(bln *ast.BlockListNode) {
	if bln == nil {
		return
	}
	if len(*bs) == 0 {
		*bs = nil
	if len(bln.List) == 0 {
		bln.List = nil
		return
	}
	for _, bn := range *bs {
	for _, bn := range bln.List {
		ast.Walk(pp, bn)
	}
	fromPos, toPos := 0, 0
	for fromPos < len(*bs) {
		(*bs)[toPos] = (*bs)[fromPos]
	for fromPos < len(bln.List) {
		bln.List[toPos] = bln.List[fromPos]
		fromPos++
		switch bn := (*bs)[toPos].(type) {
		switch bn := bln.List[toPos].(type) {
		case *ast.ParaNode:
			if len(bn.Inlines) > 0 {
			if len(bn.Inlines.List) > 0 {
				toPos++
			}
		case *nullItemNode:
		case *nullDescriptionNode:
		default:
			toPos++
		}
	}
	for pos := toPos; pos < len(*bs); pos++ {
		(*bs)[pos] = nil // Allow excess nodes to be garbage collected.
	for pos := toPos; pos < len(bln.List); pos++ {
		bln.List[pos] = nil // Allow excess nodes to be garbage collected.
	}
	*bs = (*bs)[:toPos:toPos]
	bln.List = bln.List[:toPos:toPos]

}

// processItemSlice post-processes a slice of items.
// It is one of the working horses for post-processing.
func (pp *postProcessor) processItemSlice(ins ast.ItemSlice) ast.ItemSlice {
	if len(ins) == 0 {
		return nil
	}
	for _, in := range ins {
		ast.Walk(pp, in)
	}
	fromPos, toPos := 0, 0
	for fromPos < len(ins) {
		ins[toPos] = ins[fromPos]
		fromPos++
		switch in := ins[toPos].(type) {
		case *ast.ParaNode:
			if in != nil && len(in.Inlines) > 0 {
			if in != nil && len(in.Inlines.List) > 0 {
				toPos++
			}
		case *nullItemNode:
		case *nullDescriptionNode:
		default:
			toPos++
		}
293
294
295
296
297
298
299
300

301
302
303
304
305
306
307
308
309
310
311
312
313
314
315


316
317
318
319


320
321
322

323
324
325

326
327
328
329
330






331
332
333
334
335


336
337
338
339
340
341
342
343
344
345

346
347
348
349

350
351
352
353

354
355
356
357
358
359
360
361
362
363


364

365
366
367
368
369
370
371
372
373














374
375
376
377
378
379
380
381
382
383
384
385

386
387
388
389
390
391
392
393

394
395
396

397
398
399
400
401
402
403
404
405
406
407

408


409
410
411
412

413
414
415






416
417
418
419
420
421
422
423
424

425
426
427
428
429


430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448


449
450
451
452
453
454
455
456
457
293
294
295
296
297
298
299

300
301
302
303
304
305
306
307
308
309
310
311
312
313


314
315
316
317


318
319
320
321

322
323
324
325
326





327
328
329
330
331
332
333
334
335


336
337
338
339
340




341
342

343
344
345
346

347
348
349
350

351
352
353
354
355
356
357
358
359


360
361
362
363









364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385




386
387
388
389
390
391
392
393

394
395
396

397
398
399
400
401
402
403
404
405
406
407
408
409

410
411
412
413
414

415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432

433
434
435
436


437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455


456
457
458
459
460
461
462
463
464
465
466







-
+













-
-
+
+


-
-
+
+


-
+



+
-
-
-
-
-
+
+
+
+
+
+



-
-
+
+



-
-
-
-


-
+



-
+



-
+








-
-
+
+

+
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+








-
-
-
-
+







-
+


-
+











+
-
+
+



-
+



+
+
+
+
+
+








-
+



-
-
+
+

















-
-
+
+









	}
	fromPos, toPos := 0, 0
	for fromPos < len(dns) {
		dns[toPos] = dns[fromPos]
		fromPos++
		switch dn := dns[toPos].(type) {
		case *ast.ParaNode:
			if len(dn.Inlines) > 0 {
			if len(dn.Inlines.List) > 0 {
				toPos++
			}
		case *nullDescriptionNode:
		default:
			toPos++
		}
	}
	for pos := toPos; pos < len(dns); pos++ {
		dns[pos] = nil // Allow excess nodes to be garbage collected.
	}
	return dns[:toPos:toPos]
}

func (pp *postProcessor) visitInlineSlice(is *ast.InlineSlice) {
	if is == nil {
func (pp *postProcessor) visitInlineList(iln *ast.InlineListNode) {
	if iln == nil {
		return
	}
	if len(*is) == 0 {
		*is = nil
	if len(iln.List) == 0 {
		iln.List = nil
		return
	}
	for _, in := range *is {
	for _, in := range iln.List {
		ast.Walk(pp, in)
	}

	if !pp.inVerse {
	pp.processInlineSliceHead(is)
	toPos := pp.processInlineSliceCopy(is)
	toPos = pp.processInlineSliceTail(is, toPos)
	*is = (*is)[:toPos:toPos]
	pp.processInlineSliceInplace(is)
		processInlineSliceHead(iln)
	}
	toPos := pp.processInlineSliceCopy(iln)
	toPos = pp.processInlineSliceTail(iln, toPos)
	iln.List = iln.List[:toPos:toPos]
	pp.processInlineListInplace(iln)
}

// processInlineSliceHead removes leading spaces and empty text.
func (pp *postProcessor) processInlineSliceHead(is *ast.InlineSlice) {
	ins := *is
func processInlineSliceHead(iln *ast.InlineListNode) {
	ins := iln.List
	for i, in := range ins {
		switch in := in.(type) {
		case *ast.SpaceNode:
			if pp.inVerse {
				*is = ins[i:]
				return
			}
		case *ast.TextNode:
			if len(in.Text) > 0 {
				*is = ins[i:]
				iln.List = ins[i:]
				return
			}
		default:
			*is = ins[i:]
			iln.List = ins[i:]
			return
		}
	}
	*is = ins[0:0]
	iln.List = ins[0:0]
}

// processInlineSliceCopy goes forward through the slice and tries to eliminate
// elements that follow the current element.
//
// Two text nodes are merged into one.
//
// Two spaces following a break are merged into a hard break.
func (pp *postProcessor) processInlineSliceCopy(is *ast.InlineSlice) int {
	ins := *is
func (pp *postProcessor) processInlineSliceCopy(iln *ast.InlineListNode) int {
	ins := iln.List
	maxPos := len(ins)
	for {
	toPos := pp.processInlineSliceCopyLoop(is, maxPos)
	for pos := toPos; pos < maxPos; pos++ {
		ins[pos] = nil // Allow excess nodes to be garbage collected.
	}
	return toPos
}

func (pp *postProcessor) processInlineSliceCopyLoop(is *ast.InlineSlice, maxPos int) int {
	ins := *is
		again, toPos := pp.processInlineSliceCopyLoop(iln, maxPos)
		for pos := toPos; pos < maxPos; pos++ {
			ins[pos] = nil // Allow excess nodes to be garbage collected.
		}
		if !again {
			return toPos
		}
		maxPos = toPos
	}
}

func (pp *postProcessor) processInlineSliceCopyLoop(iln *ast.InlineListNode, maxPos int) (bool, int) {
	ins := iln.List
	again := false
	fromPos, toPos := 0, 0
	for fromPos < maxPos {
		ins[toPos] = ins[fromPos]
		fromPos++
		switch in := ins[toPos].(type) {
		case *ast.TextNode:
			fromPos = processTextNode(ins, maxPos, in, fromPos)
		case *ast.SpaceNode:
			if pp.inVerse {
				in.Lexeme = strings.Repeat("\u00a0", in.Count())
			}
			fromPos = processSpaceNode(ins, maxPos, in, toPos, fromPos)
			again, fromPos = pp.processSpaceNode(ins, maxPos, in, toPos, again, fromPos)
		case *ast.BreakNode:
			if pp.inVerse {
				in.Hard = true
			}
		}
		toPos++
	}
	return toPos
	return again, toPos
}

func processTextNode(ins ast.InlineSlice, maxPos int, in *ast.TextNode, fromPos int) int {
func processTextNode(ins []ast.InlineNode, maxPos int, in *ast.TextNode, fromPos int) int {
	for fromPos < maxPos {
		if tn, ok := ins[fromPos].(*ast.TextNode); ok {
			in.Text = in.Text + tn.Text
			fromPos++
		} else {
			break
		}
	}
	return fromPos
}

func (pp *postProcessor) processSpaceNode(
func processSpaceNode(ins ast.InlineSlice, maxPos int, in *ast.SpaceNode, toPos, fromPos int) int {
	ins []ast.InlineNode, maxPos int, in *ast.SpaceNode, toPos int, again bool, fromPos int,
) (bool, int) {
	if fromPos < maxPos {
		switch nn := ins[fromPos].(type) {
		case *ast.BreakNode:
			if in.Count() > 1 {
			if len(in.Lexeme) > 1 {
				nn.Hard = true
				ins[toPos] = nn
				fromPos++
			}
		case *ast.TextNode:
			if pp.inVerse {
				ins[toPos] = &ast.TextNode{Text: strings.Repeat("\u00a0", len(in.Lexeme)) + nn.Text}
				fromPos++
				again = true
			}
		case *ast.LiteralNode:
			if nn.Kind == ast.LiteralComment {
				ins[toPos] = ins[fromPos]
				fromPos++
			}
		}
	}
	return fromPos
	return again, fromPos
}

// processInlineSliceTail removes empty text nodes, breaks and spaces at the end.
func (*postProcessor) processInlineSliceTail(is *ast.InlineSlice, toPos int) int {
	ins := *is
func (*postProcessor) processInlineSliceTail(iln *ast.InlineListNode, toPos int) int {
	ins := iln.List
	for toPos > 0 {
		switch n := ins[toPos-1].(type) {
		case *ast.TextNode:
			if len(n.Text) > 0 {
				return toPos
			}
		case *ast.BreakNode:
		case *ast.SpaceNode:
		default:
			return toPos
		}
		toPos--
		ins[toPos] = nil // Kill node to enable garbage collection
	}
	return toPos
}

func (*postProcessor) processInlineSliceInplace(is *ast.InlineSlice) {
	for _, in := range *is {
func (*postProcessor) processInlineListInplace(iln *ast.InlineListNode) {
	for _, in := range iln.List {
		if n, ok := in.(*ast.TextNode); ok {
			if n.Text == "..." {
				n.Text = "\u2026"
			} else if len(n.Text) == 4 && strings.IndexByte(",;:!?", n.Text[3]) >= 0 && n.Text[:3] == "..." {
				n.Text = "\u2026" + n.Text[3:]
			}
		}
	}
}

Changes to parser/zettelmark/zettelmark.go.

1
2

3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

37
38
39
40



41
42
43

44
45
46
47



48
49
50
51
52
53
54
1

2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

35
36



37
38
39
40
41

42
43



44
45
46
47
48
49
50
51
52
53

-
+

-
+













-

















-
+

-
-
-
+
+
+


-
+

-
-
-
+
+
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package zettelmark provides a parser for zettelmarkup.
package zettelmark

import (
	"unicode"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
	parser.Register(&parser.Info{
		Name:          api.ValueSyntaxZmk,
		AltNames:      nil,
		IsTextParser:  true,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, _ *meta.Meta, _ string) ast.BlockSlice {
func parseBlocks(inp *input.Input, _ *meta.Meta, _ string) *ast.BlockListNode {
	parser := &zmkP{inp: inp}
	bs := parser.parseBlockSlice()
	postProcessBlocks(&bs)
	return bs
	bns := parser.parseBlockList()
	postProcessBlocks(bns)
	return bns
}

func parseInlines(inp *input.Input, _ string) ast.InlineSlice {
func parseInlines(inp *input.Input, _ string) *ast.InlineListNode {
	parser := &zmkP{inp: inp}
	is := parser.parseInlineSlice()
	postProcessInlines(&is)
	return is
	iln := parser.parseInlineList()
	postProcessInlines(iln)
	return iln
}

type zmkP struct {
	inp          *input.Input             // Input stream
	lists        []*ast.NestedListNode    // Stack of lists
	table        *ast.TableNode           // Current table
	descrl       *ast.DescriptionListNode // Current description list
158
159
160
161
162
163
164
165

166
167
168
169
170
171
172
173

174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189

190
191
192
193
194
195

196
197
198
199
200

201
202
203

204
205
206
207
208
209
210
157
158
159
160
161
162
163

164
165
166
167
168
169
170
171

172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187

188
189
190
191
192
193

194
195
196
197
198

199
200
201

202
203
204
205
206
207
208
209







-
+







-
+















-
+





-
+




-
+


-
+








// parseAttributes reads optional attributes.
// If sameLine is True, it is called from block nodes. In this case, a single
// name is allowed. It will parse as {name}. Attributes are not allowed to be
// continued on next line.
// If sameLine is False, it is called from inline nodes. In this case, the next
// rune must be '{'. A continuation on next lines is allowed.
func (cp *zmkP) parseAttributes(sameLine bool) zjson.Attributes {
func (cp *zmkP) parseAttributes(sameLine bool) *ast.Attributes {
	inp := cp.inp
	if sameLine {
		pos := inp.Pos
		for isNameRune(inp.Ch) {
			inp.Next()
		}
		if pos < inp.Pos {
			return zjson.Attributes{"": string(inp.Src[pos:inp.Pos])}
			return &ast.Attributes{Attrs: map[string]string{"": string(inp.Src[pos:inp.Pos])}}
		}

		// No immediate name: skip spaces
		cp.skipSpace()
	}

	pos := inp.Pos
	attrs, success := cp.doParseAttributes(sameLine)
	if sameLine || success {
		return attrs
	}
	inp.SetPos(pos)
	return nil
}

func (cp *zmkP) doParseAttributes(sameLine bool) (res zjson.Attributes, success bool) {
func (cp *zmkP) doParseAttributes(sameLine bool) (res *ast.Attributes, success bool) {
	inp := cp.inp
	if inp.Ch != '{' {
		return nil, false
	}
	inp.Next()
	attrs := zjson.Attributes{}
	attrs := map[string]string{}
	if !cp.parseAttributeValues(sameLine, attrs) {
		return nil, false
	}
	inp.Next()
	return attrs, true
	return &ast.Attributes{Attrs: attrs}, true
}

func (cp *zmkP) parseAttributeValues(sameLine bool, attrs zjson.Attributes) bool {
func (cp *zmkP) parseAttributeValues(sameLine bool, attrs map[string]string) bool {
	inp := cp.inp
	for {
		cp.skipSpaceLine(sameLine)
		switch inp.Ch {
		case input.EOS:
			return false
		case '}':

Changes to parser/zettelmark/zettelmark_test.go.

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
15
16
17
18
19
20
21

22
23
24
25
26
27
28







-







	"bytes"
	"fmt"
	"sort"
	"strings"
	"testing"

	"zettelstore.de/c/api"
	"zettelstore.de/c/zjson"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"

	// Ensure that the text encoder is available.
	// Needed by parser/cleanup.go
	_ "zettelstore.de/z/encoder/textenc"
48
49
50
51
52
53
54
55

56
57
58
59
60
61
62
47
48
49
50
51
52
53

54
55
56
57
58
59
60
61







-
+








	for tcn, tc := range tcs {
		t.Run(fmt.Sprintf("TC=%02d,src=%q", tcn, tc.source), func(st *testing.T) {
			st.Helper()
			inp := input.NewInput([]byte(tc.source))
			bns := parser.ParseBlocks(inp, nil, api.ValueSyntaxZmk)
			var tv TestVisitor
			ast.Walk(&tv, &bns)
			ast.Walk(&tv, bns)
			got := tv.String()
			if tc.want != got {
				st.Errorf("\nwant=%q\n got=%q", tc.want, got)
			}
		})
	}
}
140
141
142
143
144
145
146
147

148
149
150
151


152
153
154
155
156
157
158
159
160
161
162
163
164



165
166
167
168
169

170
171
172
173

174
175
176
177
178
179
180
181
182
183
184
185
186
139
140
141
142
143
144
145

146
147
148


149
150
151
152
153
154
155
156
157
158
159
160



161
162
163
164
165
166
167

168
169
170
171

172
173
174
175

176

177
178
179
180
181
182
183







-
+


-
-
+
+










-
-
-
+
+
+




-
+



-
+



-

-







		{"[[|", "(PARA [[|)"},
		{"[[]", "(PARA [[])"},
		{"[[|]", "(PARA [[|])"},
		{"[[]]", "(PARA [[]])"},
		{"[[|]]", "(PARA [[|]])"},
		{"[[ ]]", "(PARA [[ SP ]])"},
		{"[[\n]]", "(PARA [[ SB ]])"},
		{"[[ a]]", "(PARA (LINK a))"},
		{"[[ a]]", "(PARA (LINK a a))"},
		{"[[a ]]", "(PARA [[a SP ]])"},
		{"[[a\n]]", "(PARA [[a SB ]])"},
		{"[[a]]", "(PARA (LINK a))"},
		{"[[12345678901234]]", "(PARA (LINK 12345678901234))"},
		{"[[a]]", "(PARA (LINK a a))"},
		{"[[12345678901234]]", "(PARA (LINK 12345678901234 12345678901234))"},
		{"[[a]", "(PARA [[a])"},
		{"[[|a]]", "(PARA [[|a]])"},
		{"[[b|]]", "(PARA [[b|]])"},
		{"[[b|a]]", "(PARA (LINK a b))"},
		{"[[b| a]]", "(PARA (LINK a b))"},
		{"[[b%c|a]]", "(PARA (LINK a b%c))"},
		{"[[b%%c|a]]", "(PARA [[b {% c|a]]})"},
		{"[[b|a]", "(PARA [[b|a])"},
		{"[[b\nc|a]]", "(PARA (LINK a b SB c))"},
		{"[[b c|a#n]]", "(PARA (LINK a#n b SP c))"},
		{"[[a]]go", "(PARA (LINK a) go)"},
		{"[[b|a]]{go}", "(PARA (LINK a b)[ATTR go])"},
		{"[[[[a]]|b]]", "(PARA (LINK [[a) |b]])"},
		{"[[a]]go", "(PARA (LINK a a) go)"},
		{"[[a]]{go}", "(PARA (LINK a a)[ATTR go])"},
		{"[[[[a]]|b]]", "(PARA (LINK [[a [[a) |b]])"},
		{"[[a[b]c|d]]", "(PARA (LINK d a[b]c))"},
		{"[[[b]c|d]]", "(PARA (LINK d [b]c))"},
		{"[[a[]c|d]]", "(PARA (LINK d a[]c))"},
		{"[[a[b]|d]]", "(PARA (LINK d a[b]))"},
		{"[[\\|]]", "(PARA (LINK %5C%7C))"},
		{"[[\\|]]", "(PARA (LINK %5C%7C \\|))"},
		{"[[\\||a]]", "(PARA (LINK a |))"},
		{"[[b\\||a]]", "(PARA (LINK a b|))"},
		{"[[b\\|c|a]]", "(PARA (LINK a b|c))"},
		{"[[\\]]]", "(PARA (LINK %5C%5D))"},
		{"[[\\]]]", "(PARA (LINK %5C%5D \\]))"},
		{"[[\\]|a]]", "(PARA (LINK a ]))"},
		{"[[b\\]|a]]", "(PARA (LINK a b]))"},
		{"[[\\]\\||a]]", "(PARA (LINK a ]|))"},
		{"[[http://a]]", "(PARA (LINK http://a))"},
		{"[[http://a|http://a]]", "(PARA (LINK http://a http://a))"},
		{"[[[[a]]]]", "(PARA (LINK [[a) ]])"},
	})
}

func TestCite(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[@", "(PARA [@)"},
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
242
243
244
245
246
247
248

249

250
251
252
253
254
255
256







-

-







		{"{{\\||a}}", "(PARA (EMBED a |))"},
		{"{{b\\||a}}", "(PARA (EMBED a b|))"},
		{"{{b\\|c|a}}", "(PARA (EMBED a b|c))"},
		{"{{\\}}}", "(PARA (EMBED %5C%7D))"},
		{"{{\\}|a}}", "(PARA (EMBED a }))"},
		{"{{b\\}|a}}", "(PARA (EMBED a b}))"},
		{"{{\\}\\||a}}", "(PARA (EMBED a }|))"},
		{"{{http://a}}", "(PARA (EMBED http://a))"},
		{"{{http://a|http://a}}", "(PARA (EMBED http://a http://a))"},
		{"{{{{a}}}}", "(PARA (EMBED %7B%7Ba) }})"},
	})
}

func TestTag(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"#", "(PARA #)"},
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
273
274
275
276
277
278
279




280
281
282
283
284
285
286







-
-
-
-







		{"[! ]", "(PARA [! SP ])"},
		{"[!a]", "(PARA (MARK \"a\" #a))"},
		{"[!a][!a]", "(PARA (MARK \"a\" #a) (MARK \"a\" #a-1))"},
		{"[!a ]", "(PARA [!a SP ])"},
		{"[!a_]", "(PARA (MARK \"a_\" #a))"},
		{"[!a_][!a]", "(PARA (MARK \"a_\" #a) (MARK \"a\" #a-1))"},
		{"[!a-b]", "(PARA (MARK \"a-b\" #a-b))"},
		{"[!a|b]", "(PARA (MARK \"a\" #a b))"},
		{"[!a|]", "(PARA (MARK \"a\" #a))"},
		{"[!|b]", "(PARA (MARK #* b))"},
		{"[!|b c]", "(PARA (MARK #* b SP c))"},
	})
}

func TestComment(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"%", "(PARA %)"},
309
310
311
312
313
314
315
316

317
318
319
320
321
322
323
324

325
326
327
328
329
330
331
300
301
302
303
304
305
306

307
308
309
310
311
312
313
314

315
316
317
318
319
320
321
322







-
+







-
+







		{"100%", "(PARA 100%)"},
	})
}

func TestFormat(t *testing.T) {
	t.Parallel()
	// Not for Insert / '>', because collision with quoted list
	for _, ch := range []string{"_", "*", "~", "^", ",", "\"", ":"} {
	for _, ch := range []string{"_", "*", "~", "'", "^", ",", "<", "\"", ":"} {
		checkTcs(t, replace(ch, TestCases{
			{"$", "(PARA $)"},
			{"$$", "(PARA $$)"},
			{"$$$", "(PARA $$$)"},
			{"$$$$", "(PARA {$})"},
		}))
	}
	for _, ch := range []string{"_", "*", ">", "~", "^", ",", "\"", ":"} {
	for _, ch := range []string{"_", "*", ">", "~", "'", "^", ",", "<", "\"", ":"} {
		checkTcs(t, replace(ch, TestCases{
			{"$$a$$", "(PARA {$ a})"},
			{"$$a$$$", "(PARA {$ a} $)"},
			{"$$$a$$", "(PARA {$ $a})"},
			{"$$$a$$$", "(PARA {$ $a} $)"},
			{"$\\$", "(PARA $$)"},
			{"$\\$$", "(PARA $$$)"},
343
344
345
346
347
348
349
350

351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373




374
375
376
377
378
379
380
381

382
383
384
385
386
387
388
334
335
336
337
338
339
340

341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360




361
362
363
364
365
366
367
368
369
370
371

372
373
374
375
376
377
378
379







-
+



















-
-
-
-
+
+
+
+







-
+







		{"__**a**__", "(PARA {_ {* a}})"},
		{"__**__**", "(PARA __ {* __})"},
	})
}

func TestLiteral(t *testing.T) {
	t.Parallel()
	for _, ch := range []string{"@", "`", "'", "="} {
	for _, ch := range []string{"@", "`", "+", "="} {
		checkTcs(t, replace(ch, TestCases{
			{"$", "(PARA $)"},
			{"$$", "(PARA $$)"},
			{"$$$", "(PARA $$$)"},
			{"$$$$", "(PARA {$})"},
			{"$$a$$", "(PARA {$ a})"},
			{"$$a$$$", "(PARA {$ a} $)"},
			{"$$$a$$", "(PARA {$ $a})"},
			{"$$$a$$$", "(PARA {$ $a} $)"},
			{"$\\$", "(PARA $$)"},
			{"$\\$$", "(PARA $$$)"},
			{"$$\\$", "(PARA $$$)"},
			{"$$a\\$$", "(PARA $$a$$)"},
			{"$$a$\\$", "(PARA $$a$$)"},
			{"$$a\\$$$", "(PARA {$ a$})"},
			{"$$a$${go}", "(PARA {$ a}[ATTR go])"},
		}))
	}
	checkTcs(t, TestCases{
		{"''````''", "(PARA {' ````})"},
		{"''``a``''", "(PARA {' ``a``})"},
		{"''``''``", "(PARA {' ``} ``)"},
		{"''\\'''", "(PARA {' '})"},
		{"++````++", "(PARA {+ ````})"},
		{"++``a``++", "(PARA {+ ``a``})"},
		{"++``++``", "(PARA {+ ``} ``)"},
		{"++\\+++", "(PARA {+ +})"},
	})
}

func TestMixFormatCode(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"__abc__\n**def**", "(PARA {_ abc} SB {* def})"},
		{"''abc''\n==def==", "(PARA {' abc} SB {= def})"},
		{"++abc++\n==def==", "(PARA {+ abc} SB {= def})"},
		{"__abc__\n==def==", "(PARA {_ abc} SB {= def})"},
		{"__abc__\n``def``", "(PARA {_ abc} SB {` def})"},
		{"\"\"ghi\"\"\n::abc::\n``def``\n", "(PARA {\" ghi} SB {: abc} SB {` def})"},
	})
}

func TestNDash(t *testing.T) {
721
722
723
724
725
726
727
728
729


730
731
732

733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752

753
754

755
756

757
758

759
760
761
762
763
764
765

766
767
768
769
770
771
772
712
713
714
715
716
717
718


719
720
721
722

723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742

743
744

745
746

747
748

749
750
751
752
753
754
755

756
757
758
759
760
761
762
763







-
-
+
+


-
+



















-
+

-
+

-
+

-
+






-
+







	buf bytes.Buffer
}

func (tv *TestVisitor) String() string { return tv.buf.String() }

func (tv *TestVisitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.InlineSlice:
		tv.visitInlineSlice(n)
	case *ast.InlineListNode:
		tv.visitInlineList(n)
	case *ast.ParaNode:
		tv.buf.WriteString("(PARA")
		ast.Walk(tv, &n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.buf.WriteByte(')')
	case *ast.VerbatimNode:
		code, ok := mapVerbatimKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown verbatim code %v", n.Kind))
		}
		tv.buf.WriteString(code)
		if len(n.Content) > 0 {
			tv.buf.WriteByte('\n')
			tv.buf.Write(n.Content)
		}
		tv.buf.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.RegionNode:
		code, ok := mapRegionKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown region code %v", n.Kind))
		}
		tv.buf.WriteString(code)
		if len(n.Blocks) > 0 {
		if n.Blocks != nil && len(n.Blocks.List) > 0 {
			tv.buf.WriteByte(' ')
			ast.Walk(tv, &n.Blocks)
			ast.Walk(tv, n.Blocks)
		}
		if len(n.Inlines) > 0 {
		if n.Inlines != nil {
			tv.buf.WriteString(" (LINE")
			ast.Walk(tv, &n.Inlines)
			ast.Walk(tv, n.Inlines)
			tv.buf.WriteByte(')')
		}
		tv.buf.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.HeadingNode:
		fmt.Fprintf(&tv.buf, "(H%d", n.Level)
		ast.Walk(tv, &n.Inlines)
		ast.Walk(tv, n.Inlines)
		if n.Fragment != "" {
			tv.buf.WriteString(" #")
			tv.buf.WriteString(n.Fragment)
		}
		tv.buf.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.HRuleNode:
780
781
782
783
784
785
786
787

788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803

804
805
806
807
808
809
810
811
812
813
814
815
816
817
818

819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838

839
840
841

842
843
844
845
846
847
848
849
850
851

852
853
854
855
856
857


858
859
860
861
862
863
864
865
866


867
868
869
870
871
872

873
874
875
876
877

878
879

880
881
882
883
884
885
886
887
888
889
890
891
892

893
894
895
896
897
898
899
771
772
773
774
775
776
777

778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793

794
795
796
797
798
799
800
801
802
803
804
805
806
807
808

809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828

829
830
831

832
833
834
835
836
837
838
839
840
841

842
843
844
845
846


847
848
849
850
851
852
853
854
855


856
857
858
859
860
861
862

863
864
865
866
867

868
869

870
871
872
873
874
875
876



877
878
879

880
881
882
883
884
885
886
887







-
+















-
+














-
+



















-
+


-
+









-
+




-
-
+
+







-
-
+
+





-
+




-
+

-
+






-
-
-



-
+







			tv.buf.WriteByte('}')
		}
		tv.buf.WriteByte(')')
	case *ast.DescriptionListNode:
		tv.buf.WriteString("(DL")
		for _, def := range n.Descriptions {
			tv.buf.WriteString(" (DT")
			ast.Walk(tv, &def.Term)
			ast.Walk(tv, def.Term)
			tv.buf.WriteByte(')')
			for _, b := range def.Descriptions {
				tv.buf.WriteString(" (DD ")
				ast.WalkDescriptionSlice(tv, b)
				tv.buf.WriteByte(')')
			}
		}
		tv.buf.WriteByte(')')
	case *ast.TableNode:
		tv.buf.WriteString("(TAB")
		if len(n.Header) > 0 {
			tv.buf.WriteString(" (TR")
			for _, cell := range n.Header {
				tv.buf.WriteString(" (TH")
				tv.buf.WriteString(alignString[cell.Align])
				ast.Walk(tv, &cell.Inlines)
				ast.Walk(tv, cell.Inlines)
				tv.buf.WriteString(")")
			}
			tv.buf.WriteString(")")
		}
		if len(n.Rows) > 0 {
			tv.buf.WriteString(" ")
			for _, row := range n.Rows {
				tv.buf.WriteString("(TR")
				for i, cell := range row {
					if i == 0 {
						tv.buf.WriteString(" ")
					}
					tv.buf.WriteString("(TD")
					tv.buf.WriteString(alignString[cell.Align])
					ast.Walk(tv, &cell.Inlines)
					ast.Walk(tv, cell.Inlines)
					tv.buf.WriteString(")")
				}
				tv.buf.WriteString(")")
			}
		}
		tv.buf.WriteString(")")
	case *ast.TranscludeNode:
		fmt.Fprintf(&tv.buf, "(TRANSCLUDE %v)", n.Ref)
	case *ast.BLOBNode:
		tv.buf.WriteString("(BLOB ")
		tv.buf.WriteString(n.Syntax)
		tv.buf.WriteString(")")
	case *ast.TextNode:
		tv.buf.WriteString(n.Text)
	case *ast.TagNode:
		tv.buf.WriteByte('#')
		tv.buf.WriteString(n.Tag)
		tv.buf.WriteByte('#')
	case *ast.SpaceNode:
		if l := n.Count(); l == 1 {
		if len(n.Lexeme) == 1 {
			tv.buf.WriteString("SP")
		} else {
			fmt.Fprintf(&tv.buf, "SP%d", l)
			fmt.Fprintf(&tv.buf, "SP%d", len(n.Lexeme))
		}
	case *ast.BreakNode:
		if n.Hard {
			tv.buf.WriteString("HB")
		} else {
			tv.buf.WriteString("SB")
		}
	case *ast.LinkNode:
		fmt.Fprintf(&tv.buf, "(LINK %v", n.Ref)
		ast.Walk(tv, &n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.buf.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.EmbedRefNode:
		fmt.Fprintf(&tv.buf, "(EMBED %v", n.Ref)
		if len(n.Inlines) > 0 {
			ast.Walk(tv, &n.Inlines)
		if n.Inlines != nil {
			ast.Walk(tv, n.Inlines)
		}
		tv.buf.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.EmbedBLOBNode:
		panic("TODO: zmktest blob")
	case *ast.CiteNode:
		fmt.Fprintf(&tv.buf, "(CITE %s", n.Key)
		if len(n.Inlines) > 0 {
			ast.Walk(tv, &n.Inlines)
		if n.Inlines != nil {
			ast.Walk(tv, n.Inlines)
		}
		tv.buf.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.FootnoteNode:
		tv.buf.WriteString("(FN")
		ast.Walk(tv, &n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.buf.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.MarkNode:
		tv.buf.WriteString("(MARK")
		if n.Mark != "" {
		if n.Text != "" {
			tv.buf.WriteString(" \"")
			tv.buf.WriteString(n.Mark)
			tv.buf.WriteString(n.Text)
			tv.buf.WriteByte('"')
		}
		if n.Fragment != "" {
			tv.buf.WriteString(" #")
			tv.buf.WriteString(n.Fragment)
		}
		if len(n.Inlines) > 0 {
			ast.Walk(tv, &n.Inlines)
		}
		tv.buf.WriteByte(')')
	case *ast.FormatNode:
		fmt.Fprintf(&tv.buf, "{%c", mapFormatKind[n.Kind])
		ast.Walk(tv, &n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.buf.WriteByte('}')
		tv.visitAttributes(n.Attrs)
	case *ast.LiteralNode:
		code, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("No element for code %v", n.Kind))
		}
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947










948
949
950
951
952
953

954
955
956
957
958
959


960
961
962
963
964
965

966
967
968
969
970
971
972


973
974
975
976
977
978
979
980

981
982
983
984
985
986
987
921
922
923
924
925
926
927








928
929
930
931
932
933
934
935
936
937
938
939
940
941
942

943
944
945
946
947


948
949
950
951
952
953
954

955
956
957
958
959
960


961
962
963
964
965
966
967
968
969

970
971
972
973
974
975
976
977







-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+





-
+




-
-
+
+





-
+





-
-
+
+







-
+







	ast.AlignDefault: "",
	ast.AlignLeft:    "l",
	ast.AlignCenter:  "c",
	ast.AlignRight:   "r",
}

var mapFormatKind = map[ast.FormatKind]rune{
	ast.FormatEmph:   '_',
	ast.FormatStrong: '*',
	ast.FormatInsert: '>',
	ast.FormatDelete: '~',
	ast.FormatSuper:  '^',
	ast.FormatSub:    ',',
	ast.FormatQuote:  '"',
	ast.FormatSpan:   ':',
	ast.FormatEmph:      '_',
	ast.FormatStrong:    '*',
	ast.FormatInsert:    '>',
	ast.FormatDelete:    '~',
	ast.FormatMonospace: '\'',
	ast.FormatSuper:     '^',
	ast.FormatSub:       ',',
	ast.FormatQuote:     '"',
	ast.FormatQuotation: '<',
	ast.FormatSpan:      ':',
}

var mapLiteralKind = map[ast.LiteralKind]rune{
	ast.LiteralZettel:  '@',
	ast.LiteralProg:    '`',
	ast.LiteralInput:   '\'',
	ast.LiteralKeyb:    '+',
	ast.LiteralOutput:  '=',
	ast.LiteralComment: '%',
}

func (tv *TestVisitor) visitInlineSlice(is *ast.InlineSlice) {
	for _, in := range *is {
func (tv *TestVisitor) visitInlineList(iln *ast.InlineListNode) {
	for _, in := range iln.List {
		tv.buf.WriteByte(' ')
		ast.Walk(tv, in)
	}
}

func (tv *TestVisitor) visitAttributes(a zjson.Attributes) {
func (tv *TestVisitor) visitAttributes(a *ast.Attributes) {
	if a.IsEmpty() {
		return
	}
	tv.buf.WriteString("[ATTR")

	keys := make([]string, 0, len(a))
	for k := range a {
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)
	}
	sort.Strings(keys)

	for _, k := range keys {
		tv.buf.WriteByte(' ')
		tv.buf.WriteString(k)
		v := a[k]
		v := a.Attrs[k]
		if len(v) > 0 {
			tv.buf.WriteByte('=')
			if strings.ContainsRune(v, ' ') {
				tv.buf.WriteByte('"')
				tv.buf.WriteString(v)
				tv.buf.WriteByte('"')
			} else {

Changes to search/retrieve.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
// Copyright (c) 2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package search
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

73
74

75
76
77
78
79
80
81
82
83
84

85
86

87
88
89
90
91
92
93
23
24
25
26
27
28
29
































30
31
32
33
34
35
36
37
38
39

40
41

42
43
44
45
46
47
48
49
50
51

52
53

54
55
56
57
58
59
60
61







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-










-
+

-
+









-
+

-
+







type searchOp struct {
	s  string
	op compareOp
}
type searchFunc func(string) id.Set
type searchCallMap map[searchOp]searchFunc

var cmpPred = map[compareOp]func(string, string) bool{
	cmpEqual:    func(s, t string) bool { return s == t },
	cmpPrefix:   strings.HasPrefix,
	cmpSuffix:   strings.HasSuffix,
	cmpContains: strings.Contains,
}

func (scm searchCallMap) addSearch(s string, op compareOp, sf searchFunc) {
	pred := cmpPred[op]
	for k := range scm {
		if op == cmpContains {
			if strings.Contains(k.s, s) {
				return
			}
			if strings.Contains(s, k.s) {
				delete(scm, k)
				break
			}
		}
		if k.op != op {
			continue
		}
		if pred(k.s, s) {
			return
		}
		if pred(s, k.s) {
			delete(scm, k)
		}
	}
	scm[searchOp{s: s, op: op}] = sf
}

func alwaysIncluded(id.Zid) bool { return true }
func neverIncluded(id.Zid) bool  { return false }

func prepareRetrieveCalls(searcher Searcher, search []expValue) (normCalls, plainCalls, negCalls searchCallMap) {
	normCalls = make(searchCallMap, len(search))
	negCalls = make(searchCallMap, len(search))
	for _, val := range search {
		for _, word := range strfun.NormalizeWords(val.value) {
			sf := getSearchFunc(searcher, val.op)
			if val.negate {
				negCalls.addSearch(word, val.op, sf)
				negCalls[searchOp{s: word, op: val.op}] = sf
			} else {
				normCalls.addSearch(word, val.op, sf)
				normCalls[searchOp{s: word, op: val.op}] = sf
			}
		}
	}

	plainCalls = make(searchCallMap, len(search))
	for _, val := range search {
		word := strings.ToLower(strings.TrimSpace(val.value))
		sf := getSearchFunc(searcher, val.op)
		if val.negate {
			negCalls.addSearch(word, val.op, sf)
			negCalls[searchOp{s: word, op: val.op}] = sf
		} else {
			plainCalls.addSearch(word, val.op, sf)
			plainCalls[searchOp{s: word, op: val.op}] = sf
		}
	}
	return normCalls, plainCalls, negCalls
}

func hasConflictingCalls(normCalls, plainCalls, negCalls searchCallMap) bool {
	for val := range negCalls {
151
152
153
154
155
156
157


158
159
160
161
162
163
164
165
166
167
168
169
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133


134
135
136
137







+
+






-
-




		negatives = negatives.Add(sf(val.s))
	}
	return negatives
}

func getSearchFunc(searcher Searcher, op compareOp) searchFunc {
	switch op {
	case cmpDefault, cmpContains:
		return searcher.SearchContains
	case cmpEqual:
		return searcher.SearchEqual
	case cmpPrefix:
		return searcher.SearchPrefix
	case cmpSuffix:
		return searcher.SearchSuffix
	case cmpContains:
		return searcher.SearchContains
	default:
		panic(fmt.Sprintf("Unexpected value of comparison operation: %v", op))
	}
}

Changes to search/search.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package search provides a zettel search.
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
65
66
67
68
69
70
71




















72
73
74
75
76
77
78







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







	descending bool   // Sort by order, but descending
	offset     int    // <= 0: no offset
	limit      int    // <= 0: no limit
}

type expTagValues map[string][]expValue

// Clone the search value.
func (s *Search) Clone() *Search {
	if s == nil {
		return nil
	}
	c := new(Search)
	c.preMatch = s.preMatch
	c.tags = make(expTagValues, len(s.tags))
	for k, v := range s.tags {
		c.tags[k] = v
	}
	c.search = append([]expValue{}, s.search...)
	c.negate = s.negate
	c.order = s.order
	c.descending = s.descending
	c.offset = s.offset
	c.limit = s.limit
	return c
}

// RandomOrder is a pseudo metadata key that selects a random order.
const RandomOrder = "_random"

type compareOp uint8

const (
	cmpUnknown compareOp = iota
129
130
131
132
133
134
135
136
137


138
139
140
141
142
143
144
145
146
147
148







149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171

172
173

174
175
176

177
178
179
180
181
182
183
184

185
186
187

188
189
190
191

192
193

194
195

196
197

198
199

200
201

202
203
204
205
206
207
208
109
110
111
112
113
114
115


116
117
118
119
120
121
122
123





124
125
126
127
128
129
130
131
132
133
134



















135
136

137
138
139

140
141

142
143
144
145
146

147
148
149

150
151
152
153

154
155

156
157

158
159

160
161

162
163

164
165
166
167
168
169
170
171







-
-
+
+






-
-
-
-
-
+
+
+
+
+
+
+




-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+

-
+


-
+

-





-
+


-
+



-
+

-
+

-
+

-
+

-
+

-
+







type expValue struct {
	value  string
	op     compareOp
	negate bool
}

// AddExpr adds a match expression to the search.
func (s *Search) AddExpr(key, value string) *Search {
	val := parseOp(strings.TrimSpace(value))
func (s *Search) AddExpr(key, val string) *Search {
	val, negate, op := parseOp(strings.TrimSpace(val))
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	if key == "" {
		s.addSearch(val)
	} else if s.tags == nil {
		s.tags = expTagValues{key: {val}}
	} else {
		s.tags[key] = append(s.tags[key], val)
		s.search = append(s.search, expValue{value: val, op: op, negate: negate})
	} else {
		if s.tags == nil {
			s.tags = expTagValues{key: {{value: val, op: op, negate: negate}}}
		} else {
			s.tags[key] = append(s.tags[key], expValue{value: val, op: op, negate: negate})
		}
	}
	return s
}

func (s *Search) addSearch(val expValue) {
	if val.negate {
		val.op = val.op.negate()
		val.negate = false
	}
	switch val.op {
	case cmpDefault:
		val.op = cmpContains
	case cmpNotDefault:
		val.op = cmpContains
		val.negate = true
	case cmpNotEqual, cmpNoPrefix, cmpNoSuffix, cmpNotContains:
		val.op = val.op.negate()
		val.negate = true
	}
	s.search = append(s.search, val)
}

func parseOp(s string) expValue {
func parseOp(s string) (r string, negate bool, op compareOp) {
	if s == "" {
		return expValue{value: s, op: cmpDefault, negate: false}
		return s, false, cmpDefault
	}
	if s[0] == '\\' {
		return expValue{value: s[1:], op: cmpDefault, negate: false}
		return s[1:], false, cmpDefault
	}
	negate := false
	if s[0] == '!' {
		negate = true
		s = s[1:]
	}
	if s == "" {
		return expValue{value: s, op: cmpDefault, negate: negate}
		return s, negate, cmpDefault
	}
	if s[0] == '\\' {
		return expValue{value: s[1:], op: cmpDefault, negate: negate}
		return s[1:], negate, cmpDefault
	}
	switch s[0] {
	case ':':
		return expValue{value: s[1:], op: cmpDefault, negate: negate}
		return s[1:], negate, cmpDefault
	case '=':
		return expValue{value: s[1:], op: cmpEqual, negate: negate}
		return s[1:], negate, cmpEqual
	case '>':
		return expValue{value: s[1:], op: cmpPrefix, negate: negate}
		return s[1:], negate, cmpPrefix
	case '<':
		return expValue{value: s[1:], op: cmpSuffix, negate: negate}
		return s[1:], negate, cmpSuffix
	case '~':
		return expValue{value: s[1:], op: cmpContains, negate: negate}
		return s[1:], negate, cmpContains
	}
	return expValue{value: s, op: cmpDefault, negate: negate}
	return s, negate, cmpDefault
}

// SetNegate changes the search to reverse its selection.
func (s *Search) SetNegate() *Search {
	if s == nil {
		s = new(Search)
	}
310
311
312
313
314
315
316

317
318


319
320

321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340

341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
273
274
275
276
277
278
279
280


281
282


283


284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310

311
312
313
314
315
316
317







+
-
-
+
+
-
-
+
-
-


















+








-








// RetrieveAndCompileMatch queries the search index and returns a predicate
// for its results and returns a matching predicate.
func (s *Search) RetrieveAndCompileMatch(searcher Searcher) (RetrievePredicate, MetaMatchFunc) {
	if s == nil {
		return alwaysIncluded, matchAlways
	}
	s.mx.Lock()
	s = s.Clone()
	match := s.compileMatch() // Match might add some searches
	pred := s.retrieveIndex(searcher)
	match := s.compileMatch()
	var pred RetrievePredicate
	if searcher != nil {
	s.mx.Unlock()
		pred = s.retrieveIndex(searcher)
	}

	if pred == nil {
		if match == nil {
			if s.negate {
				return neverIncluded, matchNever
			}
			return alwaysIncluded, matchAlways
		}
		return alwaysIncluded, match
	}
	if match == nil {
		return pred, matchAlways
	}
	return pred, match
}

// retrieveIndex and return a predicate to ask for results.
func (s *Search) retrieveIndex(searcher Searcher) RetrievePredicate {
	negate := s.negate
	if len(s.search) == 0 {
		return nil
	}
	normCalls, plainCalls, negCalls := prepareRetrieveCalls(searcher, s.search)
	if hasConflictingCalls(normCalls, plainCalls, negCalls) {
		return s.neverWithNegate()
	}

	negate := s.negate
	positives := retrievePositives(normCalls, plainCalls)
	if positives == nil {
		// No positive search for words, must contain only words for a negative search.
		// Otherwise len(search) == 0 (see above)
		negatives := retrieveNegatives(negCalls)
		return func(zid id.Zid) bool { return negatives.Contains(zid) == negate }
	}
373
374
375
376
377
378
379
380

381
382
383
384
385
386
387
334
335
336
337
338
339
340

341
342
343
344
345
346
347
348







-
+







		return alwaysIncluded
	}
	return neverIncluded
}

// compileMatch returns a function to match metadata based on select specification.
func (s *Search) compileMatch() MetaMatchFunc {
	compMeta := s.compileMeta()
	compMeta := compileMeta(s.tags)
	preMatch := s.preMatch
	if compMeta == nil {
		if preMatch == nil {
			return nil
		}
		return preMatch
	}

Changes to search/select.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package search
24
25
26
27
28
29
30
31
32


33
34
35
36
37
38
39
40
41
42




43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

60
61
62
63
64
65
66
67
68
24
25
26
27
28
29
30


31
32
33
34
35
36
37
38




39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

59


60
61
62
63
64
65
66







-
-
+
+






-
-
-
-
+
+
+
+
















-
+
-
-








type matchSpec struct {
	key   string
	match matchValueFunc
}

// compileMeta calculates a selection func based on the given select criteria.
func (s *Search) compileMeta() MetaMatchFunc {
	posSpecs, negSpecs, nomatch := s.createSelectSpecs()
func compileMeta(tags expTagValues) MetaMatchFunc {
	posSpecs, negSpecs, nomatch := createSelectSpecs(tags)
	if len(posSpecs) > 0 || len(negSpecs) > 0 || len(nomatch) > 0 {
		return makeSearchMetaMatchFunc(posSpecs, negSpecs, nomatch)
	}
	return nil
}

func (s *Search) createSelectSpecs() (posSpecs, negSpecs []matchSpec, nomatch []string) {
	posSpecs = make([]matchSpec, 0, len(s.tags))
	negSpecs = make([]matchSpec, 0, len(s.tags))
	for key, values := range s.tags {
func createSelectSpecs(tags map[string][]expValue) (posSpecs, negSpecs []matchSpec, nomatch []string) {
	posSpecs = make([]matchSpec, 0, len(tags))
	negSpecs = make([]matchSpec, 0, len(tags))
	for key, values := range tags {
		if !meta.KeyIsValid(key) {
			continue
		}
		if always, never := countEmptyValues(values); always+never > 0 {
			if never == 0 {
				posSpecs = append(posSpecs, matchSpec{key, matchValueAlways})
				continue
			}
			if always == 0 {
				negSpecs = append(negSpecs, matchSpec{key, nil})
				continue
			}
			// value must match always AND never, at the same time. This results in a no-match.
			nomatch = append(nomatch, key)
			continue
		}
		posMatch, negMatch := createPosNegMatchFunc(
		posMatch, negMatch := createPosNegMatchFunc(key, values)
			key, values,
			func(val string, op compareOp) { s.addSearch(expValue{value: val, op: op, negate: false}) })
		if posMatch != nil {
			posSpecs = append(posSpecs, matchSpec{key, posMatch})
		}
		if negMatch != nil {
			negSpecs = append(negSpecs, matchSpec{key, negMatch})
		}
	}
78
79
80
81
82
83
84
85
86
87

88
89
90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
105
106

107
108
109
110


111
112
113
114

115
116

117
118

119
120

121
122

123
124

125
126



































127
128


129
130
131
132
133
134
135
136
137
138
139
140


141
142
143
144
145
146
147
148
149
150
151
152
153
154
155


156
157
158
159
160
161
162
76
77
78
79
80
81
82



83
84
85
86
87
88
89
90
91
92

93
94
95
96
97
98
99
100
101

102
103
104
105
106
107
108
109
110
111

112
113

114
115

116
117

118
119

120
121

122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159


160
161
162
163
164
165
166
167
168
169
170
171


172
173
174
175
176
177
178
179
180
181
182
183
184
185
186


187
188
189
190
191
192
193
194
195







-
-
-
+









-
+








-
+




+
+



-
+

-
+

-
+

-
+

-
+

-
+


+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+










-
-
+
+













-
-
+
+







				always++
			}
		}
	}
	return always, never
}

type addSearchFunc func(val string, op compareOp)

func createPosNegMatchFunc(key string, values []expValue, addSearch addSearchFunc) (posMatch, negMatch matchValueFunc) {
func createPosNegMatchFunc(key string, values []expValue) (posMatch, negMatch matchValueFunc) {
	posValues := make([]opValue, 0, len(values))
	negValues := make([]opValue, 0, len(values))
	for _, val := range values {
		if val.negate {
			negValues = append(negValues, opValue{value: val.value, op: val.op.negate()})
		} else {
			posValues = append(posValues, opValue{value: val.value, op: val.op})
		}
	}
	return createMatchFunc(key, posValues, addSearch), createMatchFunc(key, negValues, addSearch)
	return createMatchFunc(key, posValues), createMatchFunc(key, negValues)
}

// opValue is an expValue, but w/o the field "negate"
type opValue struct {
	value string
	op    compareOp
}

func createMatchFunc(key string, values []opValue, addSearch addSearchFunc) matchValueFunc {
func createMatchFunc(key string, values []opValue) matchValueFunc {
	if len(values) == 0 {
		return nil
	}
	switch meta.Type(key) {
	case meta.TypeBool:
		return createMatchBoolFunc(values)
	case meta.TypeCredential:
		return matchValueNever
	case meta.TypeID, meta.TypeTimestamp: // ID and timestamp use the same layout
		return createMatchIDFunc(values, addSearch)
		return createMatchIDFunc(values)
	case meta.TypeIDSet:
		return createMatchIDSetFunc(values, addSearch)
		return createMatchIDSetFunc(values)
	case meta.TypeTagSet:
		return createMatchTagSetFunc(values, addSearch)
		return createMatchTagSetFunc(values)
	case meta.TypeWord:
		return createMatchWordFunc(values, addSearch)
		return createMatchWordFunc(values)
	case meta.TypeWordSet:
		return createMatchWordSetFunc(values, addSearch)
		return createMatchWordSetFunc(values)
	}
	return createMatchStringFunc(values, addSearch)
	return createMatchStringFunc(values)
}

type boolPredicate func(bool) bool

func boolSame(value bool) bool   { return value }
func boolNegate(value bool) bool { return value }

func createMatchBoolFunc(values []opValue) matchValueFunc {
	preds := make([]boolPredicate, len(values))
	for i, v := range values {
		positiveTest := false
		switch v.op {
		case cmpDefault, cmpEqual, cmpPrefix, cmpSuffix, cmpContains:
			positiveTest = true
		case cmpNotDefault, cmpNotEqual, cmpNoPrefix, cmpNoSuffix, cmpNotContains:
			// positiveTest = false
		default:
			panic(fmt.Sprintf("Unknown compare operation %d", v.op))
		}
		bValue := meta.BoolValue(v.value)
		if positiveTest == bValue {
			preds[i] = boolSame
		} else {
			preds[i] = boolNegate
		}
	}
	return func(value string) bool {
		bValue := meta.BoolValue(value)
		for _, pred := range preds {
			if !pred(bValue) {
				return false
			}
		}
		return true
	}
}

func createMatchIDFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(values, cmpPrefix, addSearch)
func createMatchIDFunc(values []opValue) matchValueFunc {
	preds := valuesToStringPredicates(values, cmpPrefix)
	return func(value string) bool {
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func createMatchIDSetFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	predList := valuesToStringSetPredicates(preprocessSet(values), cmpPrefix, addSearch)
func createMatchIDSetFunc(values []opValue) matchValueFunc {
	predList := valuesToStringSetPredicates(preprocessSet(values), cmpPrefix)
	return func(value string) bool {
		ids := meta.ListFromValue(value)
		for _, preds := range predList {
			for _, pred := range preds {
				if !pred(ids) {
					return false
				}
			}
		}
		return true
	}
}

func createMatchTagSetFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	predList := valuesToStringSetPredicates(processTagSet(preprocessSet(sliceToLower(values))), cmpEqual, addSearch)
func createMatchTagSetFunc(values []opValue) matchValueFunc {
	predList := valuesToStringSetPredicates(processTagSet(preprocessSet(sliceToLower(values))), cmpEqual)
	return func(value string) bool {
		tags := meta.ListFromValue(value)
		// Remove leading '#' from each tag
		for i, tag := range tags {
			tags[i] = meta.CleanTag(tag)
		}
		for _, preds := range predList {
183
184
185
186
187
188
189
190
191


192
193
194
195
196
197
198
199
200
201
202
203
204


205
206
207
208
209
210
211
212
213
214
215
216
217
218
219


220
221
222
223
224
225
226
216
217
218
219
220
221
222


223
224
225
226
227
228
229
230
231
232
233
234
235


236
237
238
239
240
241
242
243
244
245
246
247
248
249
250


251
252
253
254
255
256
257
258
259







-
-
+
+











-
-
+
+













-
-
+
+







			}
		}
		result[i] = tags
	}
	return result
}

func createMatchWordFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpEqual, addSearch)
func createMatchWordFunc(values []opValue) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpEqual)
	return func(value string) bool {
		value = strings.ToLower(value)
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func createMatchWordSetFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	predsList := valuesToStringSetPredicates(preprocessSet(sliceToLower(values)), cmpEqual, addSearch)
func createMatchWordSetFunc(values []opValue) matchValueFunc {
	predsList := valuesToStringSetPredicates(preprocessSet(sliceToLower(values)), cmpEqual)
	return func(value string) bool {
		words := meta.ListFromValue(value)
		for _, preds := range predsList {
			for _, pred := range preds {
				if !pred(words) {
					return false
				}
			}
		}
		return true
	}
}

func createMatchStringFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpContains, addSearch)
func createMatchStringFunc(values []opValue) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpContains)
	return func(value string) bool {
		value = strings.ToLower(value)
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
255
256
257
258
259
260
261
262

263
264
265
266

267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297

298
299
300
301
302
303

304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
288
289
290
291
292
293
294

295
296
297
298

299

300

301
302
303
304

305
306
307
308

309
310
311
312

313
314
315
316
317
318
319
320
321
322
323
324

325
326
327
328
329
330

331

332

333
334
335
336

337
338
339
340

341
342
343
344

345
346
347
348
349
350
351







-
+



-
+
-

-




-




-




-












-
+





-
+
-

-




-




-




-







		}
	}
	return result
}

type stringPredicate func(string) bool

func valuesToStringPredicates(values []opValue, defOp compareOp, addSearch addSearchFunc) []stringPredicate {
func valuesToStringPredicates(values []opValue, defOp compareOp) []stringPredicate {
	result := make([]stringPredicate, len(values))
	for i, v := range values {
		opVal := v.value // loop variable is used in closure --> save needed value
		op := resolveDefaultOp(v.op, defOp)
		switch op := resolveDefaultOp(v.op, defOp); op {
		switch op {
		case cmpEqual:
			addSearch(opVal, op) // addSearch only for positive selections
			result[i] = func(metaVal string) bool { return metaVal == opVal }
		case cmpNotEqual:
			result[i] = func(metaVal string) bool { return metaVal != opVal }
		case cmpPrefix:
			addSearch(opVal, op)
			result[i] = func(metaVal string) bool { return strings.HasPrefix(metaVal, opVal) }
		case cmpNoPrefix:
			result[i] = func(metaVal string) bool { return !strings.HasPrefix(metaVal, opVal) }
		case cmpSuffix:
			addSearch(opVal, op)
			result[i] = func(metaVal string) bool { return strings.HasSuffix(metaVal, opVal) }
		case cmpNoSuffix:
			result[i] = func(metaVal string) bool { return !strings.HasSuffix(metaVal, opVal) }
		case cmpContains:
			addSearch(opVal, op)
			result[i] = func(metaVal string) bool { return strings.Contains(metaVal, opVal) }
		case cmpNotContains:
			result[i] = func(metaVal string) bool { return !strings.Contains(metaVal, opVal) }
		default:
			panic(fmt.Sprintf("Unknown compare operation %d/%d with value %q", op, v.op, opVal))
		}
	}
	return result
}

type stringSetPredicate func(value []string) bool

func valuesToStringSetPredicates(values [][]opValue, defOp compareOp, addSearch addSearchFunc) [][]stringSetPredicate {
func valuesToStringSetPredicates(values [][]opValue, defOp compareOp) [][]stringSetPredicate {
	result := make([][]stringSetPredicate, len(values))
	for i, val := range values {
		elemPreds := make([]stringSetPredicate, len(val))
		for j, v := range val {
			opVal := v.value // loop variable is used in closure --> save needed value
			op := resolveDefaultOp(v.op, defOp)
			switch op := resolveDefaultOp(v.op, defOp); op {
			switch op {
			case cmpEqual:
				addSearch(opVal, op) // addSearch only for positive selections
				elemPreds[j] = makeStringSetPredicate(opVal, stringEqual, true)
			case cmpNotEqual:
				elemPreds[j] = makeStringSetPredicate(opVal, stringEqual, false)
			case cmpPrefix:
				addSearch(opVal, op)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasPrefix, true)
			case cmpNoPrefix:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasPrefix, false)
			case cmpSuffix:
				addSearch(opVal, op)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasSuffix, true)
			case cmpNoSuffix:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasSuffix, false)
			case cmpContains:
				addSearch(opVal, op)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.Contains, true)
			case cmpNotContains:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.Contains, false)
			default:
				panic(fmt.Sprintf("Unknown compare operation %d/%d with value %q", op, v.op, opVal))
			}
		}

Changes to search/sorter.go.

24
25
26
27
28
29
30



31
32
33
34
35



















36
37
38
39
40
41
42
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64







+
+
+





+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







	keyType := meta.Type(key)
	if key == api.KeyID || keyType == meta.TypeCredential {
		if descending {
			return func(i, j int) bool { return ml[i].Zid > ml[j].Zid }
		}
		return func(i, j int) bool { return ml[i].Zid < ml[j].Zid }
	}
	if keyType == meta.TypeBool {
		return createSortBoolFunc(ml, key, descending)
	}
	if keyType == meta.TypeNumber {
		return createSortNumberFunc(ml, key, descending)
	}
	return createSortStringFunc(ml, key, descending)
}

func createSortBoolFunc(ml []*meta.Meta, key string, descending bool) sortFunc {
	if descending {
		return func(i, j int) bool {
			left := ml[i].GetBool(key)
			if left == ml[j].GetBool(key) {
				return i > j
			}
			return left
		}
	}
	return func(i, j int) bool {
		right := ml[j].GetBool(key)
		if ml[i].GetBool(key) == right {
			return i < j
		}
		return right
	}
}

func createSortNumberFunc(ml []*meta.Meta, key string, descending bool) sortFunc {
	if descending {
		return func(i, j int) bool {
			iVal, iOk := getNum(ml[i], key)
			jVal, jOk := getNum(ml[j], key)
			return (iOk && (!jOk || iVal > jVal)) || !jOk

Changes to strfun/escape.go.

1
2
3
4
5
6
7
8
9
10

11
12
13
14














15






















16
17


18
19
20


21
22

23
























24
25
26
27
28
29
30
1
2
3
4
5
6
7
8
9
10
11
12
13


14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50


51
52
53


54
55
56

57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89










+


-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+

-
-
+
+

-
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package strfun provides some string functions.
package strfun

import "io"

import (
	"io"
	"strings"
)

const (
	htmlQuot     = "&quot;" // longer than "&#34;", but often requested in standards
	htmlAmp      = "&amp;"
	htmlLt       = "&lt;"
	htmlGt       = "&gt;"
	htmlNull     = "\uFFFD"
	htmlVisSpace = "\u2423"
)

var (
	htmlEscapes = []string{`&`, htmlAmp,
		`<`, htmlLt,
		`>`, htmlGt,
		`"`, htmlQuot,
		"\000", htmlNull,
	}
	htmlEscaper    = strings.NewReplacer(htmlEscapes...)
	htmlVisEscapes = append(htmlEscapes,
		" ", htmlVisSpace,
		"\u00a0", htmlVisSpace,
	)
	htmlVisEscaper = strings.NewReplacer(htmlVisEscapes...)
)

// HTMLEscape writes to w the escaped HTML equivalent of the given string.
func HTMLEscape(w io.Writer, s string) (int, error) { return htmlEscaper.WriteString(w, s) }

// HTMLEscapeVisible writes to w the escaped HTML equivalent of the given string.
// Each space is written as U-2423.
func HTMLEscapeVisible(w io.Writer, s string) (int, error) { return htmlVisEscaper.WriteString(w, s) }

var (
	escQuot = []byte("&quot;") // longer than "&#34;", but often requested in standards
	escAmp  = []byte("&amp;")
	escQuot = []byte(htmlQuot) // longer than "&#34;", but often requested in standards
	escAmp  = []byte(htmlAmp)
	escApos = []byte("apos;") // longer than "&#39", but sometimes requested in tests
	escLt   = []byte("&lt;")
	escGt   = []byte("&gt;")
	escLt   = []byte(htmlLt)
	escGt   = []byte(htmlGt)
	escTab  = []byte("&#9;")
	escNull = []byte("\uFFFD")
	escNull = []byte(htmlNull)
)

// HTMLAttrEscape writes to w the escaped HTML equivalent of the given string to be used
// in attributes.
func HTMLAttrEscape(w io.Writer, s string) {
	last := 0
	var html []byte
	lenS := len(s)
	for i := 0; i < lenS; i++ {
		switch s[i] {
		case '\000':
			html = escNull
		case '"':
			html = escQuot
		case '&':
			html = escAmp
		default:
			continue
		}
		io.WriteString(w, s[last:i])
		w.Write(html)
		last = i + 1
	}
	io.WriteString(w, s[last:])
}

// XMLEscape writes the string to the given writer, where every rune that has a special
// meaning in XML is escaped.
func XMLEscape(w io.Writer, s string) {
	var esc []byte
	last := 0
	for i, ch := range s {

Changes to template/mustache.go.

24
25
26
27
28
29
30
31

32
33
34
35
36
37
38
24
25
26
27
28
29
30

31
32
33
34
35
36
37
38







-
+







import (
	"fmt"
	"io"
	"reflect"
	"regexp"
	"strings"

	"zettelstore.de/c/html"
	"zettelstore.de/z/strfun"
)

// Node represents a node in the parse tree.
// It is either a Tag or a textNode.
type node interface {
	node()
}
597
598
599
600
601
602
603
604

605
606
607
608
609
610
611
597
598
599
600
601
602
603

604
605
606
607
608
609
610
611







-
+







		if err != nil {
			return err
		}
		if val.IsValid() {
			if n.raw {
				fmt.Fprint(w, val.Interface())
			} else {
				html.Escape(w, fmt.Sprint(val.Interface()))
				strfun.HTMLEscape(w, fmt.Sprint(val.Interface()))
			}
		}
	case *sectionNode:
		if err := tmpl.renderSection(w, n, stack); err != nil {
			return err
		}
	case *partialNode:

Changes to testdata/testbox/00000000000100.zettel.

1
2
3
4
5
6


7
8
1
2
3
4
5

6
7
8
9





-
+
+


id: 00000000000100
title: Zettelstore Runtime Configuration
role: configuration
syntax: none
expert-mode: true
modified: 20220215171142
modified: 20210629174242
no-index: true
visibility: owner

Changes to tests/client/client_test.go.

18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
18
19
20
21
22
23
24

25
26
27
28
29
30
31







-







	"net/http"
	"net/url"
	"strconv"
	"testing"

	"zettelstore.de/c/api"
	"zettelstore.de/c/client"
	"zettelstore.de/z/kernel"
)

func nextZid(zid api.ZettelID) api.ZettelID {
	numVal, err := strconv.ParseUint(string(zid), 10, 64)
	if err != nil {
		panic(err)
	}
161
162
163
164
165
166
167
168

169
170
171
172
173
174
175
176
177
178
179
180
181
182

183
184
185
186
187
188
189
160
161
162
163
164
165
166

167
168
169
170
171
172
173
174
175
176
177
178
179
180

181
182
183
184
185
186
187
188







-
+













-
+







}

func TestGetParsedEvaluatedZettel(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	encodings := []api.EncodingEnum{
		api.EncoderZJSON,
		api.EncoderDJSON,
		api.EncoderHTML,
		api.EncoderNative,
		api.EncoderText,
	}
	for _, enc := range encodings {
		content, err := c.GetParsedZettel(context.Background(), api.ZidDefaultHome, enc)
		if err != nil {
			t.Error(err)
			continue
		}
		if len(content) == 0 {
			t.Errorf("Empty content for parsed encoding %v", enc)
		}
		content, err = c.GetEvaluatedZettel(context.Background(), api.ZidDefaultHome, enc, true)
		content, err = c.GetEvaluatedZettel(context.Background(), api.ZidDefaultHome, enc)
		if err != nil {
			t.Error(err)
			continue
		}
		if len(content) == 0 {
			t.Errorf("Empty content for evaluated encoding %v", enc)
		}
269
270
271
272
273
274
275























276
277
278
279
280
281
282
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







	l = rl.List
	if got := len(l); got != 1 {
		t.Errorf("Expected list of length 1, got %d", got)
		return
	}
	checkListZid(t, l, 0, allUserZid)
}

func TestGetZettelLinks(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	zl, err := c.GetZettelLinks(context.Background(), api.ZidDefaultHome)
	if err != nil {
		t.Error(err)
		return
	}
	if !checkZid(t, api.ZidDefaultHome, zl.ID) {
		return
	}
	if got := len(zl.Linked.Outgoing); got != 4 {
		t.Errorf("Expected 4 outgoing links, got %d", got)
	}
	if got := len(zl.Linked.Local); got != 1 {
		t.Errorf("Expected 1 local link, got %d", got)
	}
	if got := len(zl.Linked.External); got != 4 {
		t.Errorf("Expected 4 external link, got %d", got)
	}
}

func TestGetUnlinkedReferences(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	zl, err := c.GetUnlinkedReferences(context.Background(), api.ZidDefaultHome, nil)
	if err != nil {
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405

406
407
408
409
410
411
412
413
414
415
416
417
418
419
401
402
403
404
405
406
407













408
409
410
411
412
413

414






415
416
417
418
419
420
421
422







-
-
-
-
-
-
-
-
-
-
-
-
-






-
+
-
-
-
-
-
-








	for i, id := range exp {
		if id != rl[i] {
			t.Errorf("Role list pos %d: expected %q, got %q", i, id, rl[i])
		}
	}
}

func TestVersion(t *testing.T) {
	t.Parallel()
	c := getClient()
	ver, err := c.GetVersionJSON(context.Background())
	if err != nil {
		t.Error(err)
		return
	}
	if ver.Major != -1 || ver.Minor != -1 || ver.Patch != -1 || ver.Info != kernel.CoreDefaultVersion || ver.Hash != "" {
		t.Error(ver)
	}
}

var baseURL string

func init() {
	flag.StringVar(&baseURL, "base-url", "", "Base URL")
}

func getClient() *client.Client {
func getClient() *client.Client { return client.NewClient(baseURL) }
	u, err := url.Parse(baseURL)
	if err != nil {
		panic(err)
	}
	return client.NewClient(u)
}

// TestMain controls whether client API tests should run or not.
func TestMain(m *testing.M) {
	flag.Parse()
	if baseURL != "" {
		m.Run()
	}
}

Changes to tests/client/embed_test.go.

38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
38
39
40
41
42
43
44

45
46
47
48
49
50
51
52







-
+







	content, err := c.GetZettel(context.Background(), abcZid, api.PartContent)
	if err != nil {
		t.Error(err)
		return
	}
	baseContent := string(content)
	for zid, siz := range contentMap {
		content, err = c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML, true)
		content, err = c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML)
		if err != nil {
			t.Error(err)
			continue
		}
		sContent := string(content)
		prefix := "<p>"
		if !strings.HasPrefix(sContent, prefix) {
60
61
62
63
64
65
66
67

68
69
70
71
72
73
74
60
61
62
63
64
65
66

67
68
69
70
71
72
73
74







-
+







		}
		got := sContent[len(prefix) : len(content)-len(suffix)]
		if expect := strings.Repeat(baseContent, siz); expect != got {
			t.Errorf("Unexpected content for zettel %q\nExpect: %q\nGot:    %q", zid, expect, got)
		}
	}

	content, err = c.GetEvaluatedZettel(context.Background(), abc10000Zid, api.EncoderHTML, true)
	content, err = c.GetEvaluatedZettel(context.Background(), abc10000Zid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	checkContentContains(t, abc10000Zid, string(content), "Too many transclusions")
}

83
84
85
86
87
88
89
90

91
92
93
94
95
96
97
83
84
85
86
87
88
89

90
91
92
93
94
95
96
97







-
+







		return
	}
	expectedEnc := "base64"
	if got := zettelData.Encoding; expectedEnc != got {
		t.Errorf("Zettel %q: encoding %q expected, but got %q", abcZid, expectedEnc, got)
	}

	content, err := c.GetEvaluatedZettel(context.Background(), abc10Zid, api.EncoderHTML, true)
	content, err := c.GetEvaluatedZettel(context.Background(), abc10Zid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	expectedContent := "<img src=\"data:image/gif;" + expectedEnc + "," + zettelData.Content
	checkContentContains(t, abc10Zid, string(content), expectedContent)
}
124
125
126
127
128
129
130
131

132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150

151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166

167
168
169
170
171
172
173
174
175
176
177
178
179
124
125
126
127
128
129
130

131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149

150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165

166
167
168
169
170
171
172
173
174
175
176
177
178
179







-
+


















-
+















-
+













	)
	recursiveZettel := map[api.ZettelID]api.ZettelID{
		selfRecursiveZid:      selfRecursiveZid,
		indirectRecursive1Zid: indirectRecursive2Zid,
		indirectRecursive2Zid: indirectRecursive1Zid,
	}
	for zid, errZid := range recursiveZettel {
		content, err := c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML, true)
		content, err := c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML)
		if err != nil {
			t.Error(err)
			continue
		}
		sContent := string(content)
		checkContentContains(t, zid, sContent, "Recursive transclusion")
		checkContentContains(t, zid, sContent, string(errZid))
	}
}
func TestNothingToTransclude(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")

	const (
		transZid = api.ZettelID("20211020184342")
		emptyZid = api.ZettelID("20211020184300")
	)
	content, err := c.GetEvaluatedZettel(context.Background(), transZid, api.EncoderHTML, true)
	content, err := c.GetEvaluatedZettel(context.Background(), transZid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	sContent := string(content)
	checkContentContains(t, transZid, sContent, "<!-- Nothing to transclude")
	checkContentContains(t, transZid, sContent, string(emptyZid))
}

func TestSelfEmbedRef(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")

	const selfEmbedZid = api.ZettelID("20211020185400")
	content, err := c.GetEvaluatedZettel(context.Background(), selfEmbedZid, api.EncoderHTML, true)
	content, err := c.GetEvaluatedZettel(context.Background(), selfEmbedZid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	checkContentContains(t, selfEmbedZid, string(content), "Self embed reference")
}

func checkContentContains(t *testing.T, zid api.ZettelID, content, expected string) {
	if !strings.Contains(content, expected) {
		t.Helper()
		t.Errorf("Zettel %q should contain %q, but does not: %q", zid, expected, content)
	}
}

Changes to tests/markdown_test.go.

1
2

3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
1

2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

28
29
30
31
32
33
34

-
+

-
+



















+



-







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package tests provides some higher-level tests.
package tests

import (
	"bytes"
	"encoding/json"
	"fmt"
	"os"
	"testing"

	"zettelstore.de/c/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	_ "zettelstore.de/z/encoder/djsonenc"
	_ "zettelstore.de/z/encoder/htmlenc"
	_ "zettelstore.de/z/encoder/nativeenc"
	_ "zettelstore.de/z/encoder/textenc"
	_ "zettelstore.de/z/encoder/zjsonenc"
	_ "zettelstore.de/z/encoder/zmkenc"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	_ "zettelstore.de/z/parser/markdown"
	_ "zettelstore.de/z/parser/zettelmark"
)

65
66
67
68
69
70
71
72
73


74
75
76
77

78
79
80
81
82
83
84
85
86
87
88

89
90
91
92
93
94
95
96
97
98
99
100

101
102
103
104
105
106
107
108
109
110

111
112
113
114
115
116
117
65
66
67
68
69
70
71


72
73
74
75
76

77
78
79
80
81
82
83
84
85
86
87

88
89
90
91
92
93
94
95
96
97
98
99

100
101
102
103
104
105
106
107
108
109

110
111
112
113
114
115
116
117







-
-
+
+



-
+










-
+











-
+









-
+







	var testcases []markdownTestCase
	if err = json.Unmarshal(content, &testcases); err != nil {
		panic(err)
	}

	for _, tc := range testcases {
		ast := parser.ParseBlocks(input.NewInput([]byte(tc.Markdown)), nil, "markdown")
		testAllEncodings(t, tc, &ast)
		testZmkEncoding(t, tc, &ast)
		testAllEncodings(t, tc, ast)
		testZmkEncoding(t, tc, ast)
	}
}

func testAllEncodings(t *testing.T, tc markdownTestCase, ast *ast.BlockSlice) {
func testAllEncodings(t *testing.T, tc markdownTestCase, ast *ast.BlockListNode) {
	var buf bytes.Buffer
	testID := tc.Example*100 + 1
	for _, enc := range encodings {
		t.Run(fmt.Sprintf("Encode %v %v", enc, testID), func(st *testing.T) {
			encoder.Create(enc, nil).WriteBlocks(&buf, ast)
			buf.Reset()
		})
	}
}

func testZmkEncoding(t *testing.T, tc markdownTestCase, ast *ast.BlockSlice) {
func testZmkEncoding(t *testing.T, tc markdownTestCase, ast *ast.BlockListNode) {
	zmkEncoder := encoder.Create(api.EncoderZmk, nil)
	var buf bytes.Buffer
	testID := tc.Example*100 + 1
	t.Run(fmt.Sprintf("Encode zmk %14d", testID), func(st *testing.T) {
		buf.Reset()
		zmkEncoder.WriteBlocks(&buf, ast)
		// gotFirst := buf.String()

		testID = tc.Example*100 + 2
		secondAst := parser.ParseBlocks(input.NewInput(buf.Bytes()), nil, api.ValueSyntaxZmk)
		buf.Reset()
		zmkEncoder.WriteBlocks(&buf, &secondAst)
		zmkEncoder.WriteBlocks(&buf, secondAst)
		gotSecond := buf.String()

		// if gotFirst != gotSecond {
		// 	st.Errorf("\nCMD: %q\n1st: %q\n2nd: %q", tc.Markdown, gotFirst, gotSecond)
		// }

		testID = tc.Example*100 + 3
		thirdAst := parser.ParseBlocks(input.NewInput(buf.Bytes()), nil, api.ValueSyntaxZmk)
		buf.Reset()
		zmkEncoder.WriteBlocks(&buf, &thirdAst)
		zmkEncoder.WriteBlocks(&buf, thirdAst)
		gotThird := buf.String()

		if gotSecond != gotThird {
			st.Errorf("\n1st: %q\n2nd: %q", gotSecond, gotThird)
		}
	})
}

Changes to tests/regression_test.go.

32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46







-
+







	"zettelstore.de/z/parser"

	_ "zettelstore.de/z/box/dirbox"
)

var encodings = []api.EncodingEnum{
	api.EncoderHTML,
	api.EncoderZJSON,
	api.EncoderDJSON,
	api.EncoderNative,
	api.EncoderText,
}

func getFileBoxes(wd, kind string) (root string, boxes []box.ManagedBox) {
	root = filepath.Clean(filepath.Join(wd, "..", "testdata", kind))
	entries, err := os.ReadDir(root)

Added tests/result/meta/copyright/20200310125800.djson.


1
+
{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","copyright":"(c) 2020 Detlef Stern","license":"CC BY-SA 4.0"}

Deleted tests/result/meta/copyright/20200310125800.zjson.

1

-
{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","copyright":"(c) 2020 Detlef Stern","license":"CC BY-SA 4.0"}

Added tests/result/meta/header/20200310125800.djson.


1
+
{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","x-no":"00000000000000"}

Deleted tests/result/meta/header/20200310125800.zjson.

1

-
{"title":[{"t":"Text","s":"Header"},{"t":"Space"},{"t":"Text","s":"Test"}],"role":"zettel","syntax":"zmk","x-no":"00000000000000"}

Added tests/result/meta/title/20200310110300.djson.


1
+
{"title":[{"t":"Text","s":"A"},{"t":"Space"},{"t":"Quote","i":[{"t":"Text","s":"Title"}]},{"t":"Space"},{"t":"Text","s":"with"},{"t":"Space"},{"t":"Emph","i":[{"t":"Text","s":"Markup"}]},{"t":"Text","s":","},{"t":"Space"},{"t":"Code","a":{"":"zmk"},"s":"Zettelmarkup"}],"role":"zettel","syntax":"zmk"}

Deleted tests/result/meta/title/20200310110300.zjson.

1

-
{"title":[{"t":"Text","s":"A"},{"t":"Space"},{"t":"Quote","i":[{"t":"Text","s":"Title"}]},{"t":"Space"},{"t":"Text","s":"with"},{"t":"Space"},{"t":"Emph","i":[{"t":"Text","s":"Markup"}]},{"t":"Text","s":","},{"t":"Space"},{"t":"Code","a":{"":"zmk"},"s":"Zettelmarkup"}],"role":"zettel","syntax":"zmk"}

Changes to tools/build.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
// Copyright (c) 2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package main provides a command to build and run the software.
103
104
105
106
107
108
109
110

111
112
113
114
115
116
117
118
119



120
121

122
123
124
125
126
127
128
103
104
105
106
107
108
109

110
111
112
113
114
115
116
117


118
119
120
121

122
123
124
125
126
127
128
129







-
+







-
-
+
+
+

-
+







						return hash + suffix, nil
					}
					break
				}
			}
		}
	}
	return hash + suffix, nil
	return hash, nil
}

func getVersionData() (string, string) {
	base, err := readVersionFile()
	if err != nil {
		base = "dev"
	}
	if fossil, err2 := readFossilVersion(); err2 == nil {
		return base, fossil
	fossil, err := readFossilVersion()
	if err != nil {
		return base, ""
	}
	return base, ""
	return base, fossil
}

func calcVersion(base, vcs string) string { return base + "+" + vcs }

func getVersion() string {
	base, vcs := getVersionData()
	return calcVersion(base, vcs)
402
403
404
405
406
407
408

409
410
411
412
413
414
415
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417







+







func getReleaseVersionData() (string, string) {
	base, fossil := getVersionData()
	if strings.HasSuffix(base, "dev") {
		base = base[:len(base)-3] + "preview-" + time.Now().Format("20060102")
	}
	if strings.HasSuffix(fossil, dirtySuffix) {
		fmt.Fprintf(os.Stderr, "Warning: releasing a dirty version %v\n", fossil)
		base = base + dirtySuffix
	}
	return base, fossil
}

func cmdRelease() error {
	if err := cmdCheck(true); err != nil {
		return err

Added usecase/copy_zettel.go.









































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase

import (
	"zettelstore.de/c/api"
	"zettelstore.de/z/domain"
)

// CopyZettel is the data for this use case.
type CopyZettel struct{}

// NewCopyZettel creates a new use case.
func NewCopyZettel() CopyZettel {
	return CopyZettel{}
}

// Run executes the use case.
func (CopyZettel) Run(origZettel domain.Zettel) domain.Zettel {
	m := origZettel.Meta.Clone()
	if title, ok := m.Get(api.KeyTitle); ok {
		if len(title) > 0 {
			title = "Copy of " + title
		} else {
			title = "Copy"
		}
		m.Set(api.KeyTitle, title)
	}
	content := origZettel.Content
	content.TrimSpace()
	return domain.Zettel{Meta: m, Content: content}
}

Changes to usecase/create_zettel.go.

1
2

3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
1

2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

20
21
22
23
24
25
26

-
+

-
+















-







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase

import (
	"context"

	"zettelstore.de/c/api"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/logger"
)

// CreateZettelPort is the interface used by this use case.
type CreateZettelPort interface {
	// CreateZettel creates a new zettel.
	CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error)
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
38
39
40
41
42
43
44
































































45
46
47
48
49
50
51







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







	return CreateZettel{
		log:      log,
		rtConfig: rtConfig,
		port:     port,
	}
}

// PrepareCopy the zettel for further modification.
func (*CreateZettel) PrepareCopy(origZettel domain.Zettel) domain.Zettel {
	m := origZettel.Meta.Clone()
	if title, ok := m.Get(api.KeyTitle); ok {
		m.Set(api.KeyTitle, prependTitle(title, "Copy", "Copy of "))
	}
	if readonly, ok := m.Get(api.KeyReadOnly); ok {
		m.Set(api.KeyReadOnly, copyReadonly(readonly))
	}
	content := origZettel.Content
	content.TrimSpace()
	return domain.Zettel{Meta: m, Content: content}
}

// PrepareFolge the zettel for further modification.
func (uc *CreateZettel) PrepareFolge(origZettel domain.Zettel) domain.Zettel {
	origMeta := origZettel.Meta
	m := meta.New(id.Invalid)
	if title, ok := origMeta.Get(api.KeyTitle); ok {
		m.Set(api.KeyTitle, prependTitle(title, "Folge", "Folge of "))
	}
	m.Set(api.KeyRole, config.GetRole(origMeta, uc.rtConfig))
	m.Set(api.KeyTags, origMeta.GetDefault(api.KeyTags, ""))
	m.Set(api.KeySyntax, uc.rtConfig.GetDefaultSyntax())
	m.Set(api.KeyPrecursor, origMeta.Zid.String())
	return domain.Zettel{Meta: m, Content: domain.NewContent(nil)}
}

// PrepareNew the zettel for further modification.
func (*CreateZettel) PrepareNew(origZettel domain.Zettel) domain.Zettel {
	m := meta.New(id.Invalid)
	om := origZettel.Meta
	m.Set(api.KeyTitle, om.GetDefault(api.KeyTitle, ""))
	m.Set(api.KeyRole, om.GetDefault(api.KeyRole, ""))
	m.Set(api.KeyTags, om.GetDefault(api.KeyTags, ""))
	m.Set(api.KeySyntax, om.GetDefault(api.KeySyntax, ""))

	const prefixLen = len(meta.NewPrefix)
	for _, pair := range om.PairsRest() {
		if key := pair.Key; len(key) > prefixLen && key[0:prefixLen] == meta.NewPrefix {
			m.Set(key[prefixLen:], pair.Value)
		}
	}
	content := origZettel.Content
	content.TrimSpace()
	return domain.Zettel{Meta: m, Content: content}
}

func prependTitle(title, s0, s1 string) string {
	if len(title) > 0 {
		return s1 + title
	}
	return s0
}

func copyReadonly(string) string {
	// Currently, "false" is a safe value.
	//
	// If the current user and its role is known, a mor elaborative calculation
	// could be done: set it to a value, so that the current user will be able
	// to modify it later.
	return api.ValueFalse
}

// Run executes the use case.
func (uc *CreateZettel) Run(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	m := zettel.Meta
	if m.Zid.IsValid() {
		return m.Zid, nil // TODO: new error: already exists
	}
	if title, ok := m.Get(api.KeyTitle); !ok || title == "" {

Changes to usecase/evaluate.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
// Copyright (c) 2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase
50
51
52
53
54
55
56
57
58
59
60




61
62
63
64
65
66
67
68
69
70
71
50
51
52
53
54
55
56




57
58
59
60
61
62
63
64
65
66
67
68
69
70
71







-
-
-
-
+
+
+
+











	}

	evaluator.EvaluateZettel(ctx, uc, env, uc.rtConfig, zn)
	return zn, nil
}

// RunMetadata executes the use case for a metadata value.
func (uc *Evaluate) RunMetadata(ctx context.Context, value string, env *evaluator.Environment) ast.InlineSlice {
	is := parser.ParseMetadata(value)
	evaluator.EvaluateInline(ctx, uc, env, uc.rtConfig, &is)
	return is
func (uc *Evaluate) RunMetadata(ctx context.Context, value string, env *evaluator.Environment) *ast.InlineListNode {
	iln := parser.ParseMetadata(value)
	evaluator.EvaluateInline(ctx, uc, env, uc.rtConfig, iln)
	return iln
}

// GetMeta retrieves the metadata of a given zettel identifier.
func (uc *Evaluate) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	return uc.getMeta.Run(ctx, zid)
}

// GetZettel retrieves the full zettel of a given zettel identifier.
func (uc *Evaluate) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	return uc.getZettel.Run(ctx, zid)
}

Added usecase/folge_zettel.go.

















































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase

import (
	"zettelstore.de/c/api"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// FolgeZettel is the data for this use case.
type FolgeZettel struct {
	rtConfig config.Config
}

// NewFolgeZettel creates a new use case.
func NewFolgeZettel(rtConfig config.Config) FolgeZettel {
	return FolgeZettel{rtConfig}
}

// Run executes the use case.
func (uc FolgeZettel) Run(origZettel domain.Zettel) domain.Zettel {
	origMeta := origZettel.Meta
	m := meta.New(id.Invalid)
	if title, ok := origMeta.Get(api.KeyTitle); ok {
		if len(title) > 0 {
			title = "Folge of " + title
		} else {
			title = "Folge"
		}
		m.Set(api.KeyTitle, title)
	}
	m.Set(api.KeyRole, config.GetRole(origMeta, uc.rtConfig))
	m.Set(api.KeyTags, origMeta.GetDefault(api.KeyTags, ""))
	m.Set(api.KeySyntax, uc.rtConfig.GetDefaultSyntax())
	m.Set(api.KeyPrecursor, origMeta.Zid.String())
	return domain.Zettel{Meta: m, Content: domain.NewContent(nil)}
}

Added usecase/new_zettel.go.















































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase

import (
	"zettelstore.de/c/api"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// NewZettel is the data for this use case.
type NewZettel struct{}

// NewNewZettel creates a new use case.
func NewNewZettel() NewZettel {
	return NewZettel{}
}

// Run executes the use case.
func (NewZettel) Run(origZettel domain.Zettel) domain.Zettel {
	m := meta.New(id.Invalid)
	om := origZettel.Meta
	m.Set(api.KeyTitle, om.GetDefault(api.KeyTitle, ""))
	m.Set(api.KeyRole, om.GetDefault(api.KeyRole, ""))
	m.Set(api.KeyTags, om.GetDefault(api.KeyTags, ""))
	m.Set(api.KeySyntax, om.GetDefault(api.KeySyntax, ""))

	const prefixLen = len(meta.NewPrefix)
	for _, pair := range om.PairsRest() {
		if key := pair.Key; len(key) > prefixLen && key[0:prefixLen] == meta.NewPrefix {
			m.Set(key[prefixLen:], pair.Value)
		}
	}
	content := origZettel.Content
	content.TrimSpace()
	return domain.Zettel{Meta: m, Content: content}
}

Changes to usecase/unlinked_refs.go.

92
93
94
95
96
97
98
99
100
101



102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117

118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137

138
139
140
141
142
143
144
145
146
147
148
149


150
151
152

153
154
155
156
157
158
159

160
161
162

163
164
165
166
167
168
169
92
93
94
95
96
97
98



99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116

117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136

137
138
139
140
141
142
143
144
145
146
147


148
149
150
151

152
153
154
155
156
157
158

159
160
161

162
163
164
165
166
167
168
169







-
-
-
+
+
+















-
+



















-
+










-
-
+
+


-
+






-
+


-
+







		}
		v.text = v.joinWords(words)

		for _, pair := range zettel.Meta.Pairs() {
			if meta.Type(pair.Key) != meta.TypeZettelmarkup {
				continue
			}
			is := parser.ParseMetadata(pair.Value)
			evaluator.EvaluateInline(ctx, uc.port, nil, uc.rtConfig, &is)
			ast.Walk(&v, &is)
			iln := parser.ParseMetadata(pair.Value)
			evaluator.EvaluateInline(ctx, uc.port, nil, uc.rtConfig, iln)
			ast.Walk(&v, iln)
			if v.found {
				result = append(result, cand)
				continue candLoop
			}
		}

		syntax := zettel.Meta.GetDefault(api.KeySyntax, "")
		if !parser.IsTextParser(syntax) {
			continue
		}
		zn, err := parser.ParseZettel(zettel, syntax, nil), nil
		if err != nil {
			continue
		}
		evaluator.EvaluateZettel(ctx, uc.port, nil, uc.rtConfig, zn)
		ast.Walk(&v, &zn.Ast)
		ast.Walk(&v, zn.Ast)
		if v.found {
			result = append(result, cand)
		}
	}
	return result
}

func (*unlinkedVisitor) joinWords(words []string) string {
	return " " + strings.ToLower(strings.Join(words, " ")) + " "
}

type unlinkedVisitor struct {
	words []string
	text  string
	found bool
}

func (v *unlinkedVisitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.InlineSlice:
	case *ast.InlineListNode:
		v.checkWords(n)
		return nil
	case *ast.HeadingNode:
		return nil
	case *ast.LinkNode, *ast.EmbedRefNode, *ast.EmbedBLOBNode, *ast.CiteNode:
		return nil
	}
	return v
}

func (v *unlinkedVisitor) checkWords(is *ast.InlineSlice) {
	if len(*is) < 2*len(v.words)-1 {
func (v *unlinkedVisitor) checkWords(iln *ast.InlineListNode) {
	if len(iln.List) < 2*len(v.words)-1 {
		return
	}
	for _, text := range v.splitInlineTextList(is) {
	for _, text := range v.splitInlineTextList(iln) {
		if strings.Contains(text, v.text) {
			v.found = true
		}
	}
}

func (v *unlinkedVisitor) splitInlineTextList(is *ast.InlineSlice) []string {
func (v *unlinkedVisitor) splitInlineTextList(iln *ast.InlineListNode) []string {
	var result []string
	var curList []string
	for _, in := range *is {
	for _, in := range iln.List {
		switch n := in.(type) {
		case *ast.TextNode:
			curList = append(curList, makeWords(n.Text)...)
		case *ast.SpaceNode:
		default:
			if curList != nil {
				result = append(result, v.joinWords(curList))

Deleted usecase/version.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76












































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase

import (
	"regexp"
	"strconv"

	"zettelstore.de/z/kernel"
)

// Version is the data for this use case.
type Version struct {
	vr VersionResult
}

// NewVersion creates a new use case.
func NewVersion(version string) Version {
	return Version{calculateVersionResult(version)}
}

// VersionResult is the data structure returned by this usecase.
type VersionResult struct {
	Major int
	Minor int
	Patch int
	Info  string
	Hash  string
}

var invalidVersion = VersionResult{
	Major: -1,
	Minor: -1,
	Patch: -1,
	Info:  kernel.CoreDefaultVersion,
	Hash:  "",
}

var reVersion = regexp.MustCompile(`^(\d+)\.(\d+)(\.(\d+))?(-(([[:alnum:]]|-)+))?(\+(([[:alnum:]])+(-[[:alnum:]]+)?))?`)

func calculateVersionResult(version string) VersionResult {
	match := reVersion.FindStringSubmatch(version)
	if len(match) < 12 {
		return invalidVersion
	}
	major, err := strconv.Atoi(match[1])
	if err != nil {
		return invalidVersion
	}
	minor, err := strconv.Atoi(match[2])
	if err != nil {
		return invalidVersion
	}
	patch, err := strconv.Atoi(match[4])
	if err != nil {
		patch = 0
	}
	return VersionResult{
		Major: major,
		Minor: minor,
		Patch: patch,
		Info:  match[6],
		Hash:  match[9],
	}
}

// Run executes the use case.
func (uc Version) Run() VersionResult { return uc.vr }

Changes to web/adapter/api/content_type.go.

19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
19
20
21
22
23
24
25

26
27
28
29
30
31
32
33







-
+







	ctPlainText = "text/plain; charset=utf-8"
	ctSVG       = "image/svg+xml"
)

var mapEncoding2CT = map[api.EncodingEnum]string{
	api.EncoderHTML:   ctHTML,
	api.EncoderNative: ctPlainText,
	api.EncoderZJSON:  ctJSON,
	api.EncoderDJSON:  ctJSON,
	api.EncoderText:   ctPlainText,
	api.EncoderZmk:    ctPlainText,
}

func encoding2ContentType(enc api.EncodingEnum) string {
	if ct, ok := mapEncoding2CT[enc]; ok {
		return ct

Added web/adapter/api/encode_inlines.go.




































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"bytes"
	"encoding/json"
	"net/http"

	"zettelstore.de/c/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/usecase"
)

// MakePostEncodeInlinesHandler creates a new HTTP handler to encode given
// Zettelmarkup inline material
func (a *API) MakePostEncodeInlinesHandler(evaluate usecase.Evaluate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		dec := json.NewDecoder(r.Body)
		var reqJSON api.EncodeInlineReqJSON
		if err := dec.Decode(&reqJSON); err != nil {
			http.Error(w, "Unable to read request", http.StatusBadRequest)
			return
		}
		envEnc := &encoder.Environment{
			Lang:        reqJSON.Lang,
			Interactive: reqJSON.NoLinks,
		}
		if envEnc.Lang == "" {
			envEnc.Lang = a.rtConfig.GetDefaultLang()
		}
		htmlEnc := encoder.Create(api.EncoderHTML, envEnc)
		ctx := r.Context()
		envEval := evaluator.Environment{}
		var respJSON api.EncodedInlineRespJSON
		if iln := evaluate.RunMetadata(ctx, reqJSON.FirstZmk, &envEval); iln != nil {
			s, err := encodeInlines(htmlEnc, iln)
			if err != nil {
				http.Error(w, "Unable to encode first as HTML", http.StatusBadRequest)
				return
			}
			respJSON.FirstHTML = s

			s, err = encodeInlines(encoder.Create(api.EncoderText, nil), iln)
			if err != nil {
				http.Error(w, "Unable to encode first as Text", http.StatusBadRequest)
				return
			}
			respJSON.FirstText = s
		}

		if reqLen := len(reqJSON.OtherZmk); reqLen > 0 {
			respJSON.OtherHTML = make([]string, reqLen)
			for i, zmk := range reqJSON.OtherZmk {
				iln := evaluate.RunMetadata(ctx, zmk, &envEval)
				if iln == nil {
					continue
				}
				s, err := encodeInlines(htmlEnc, iln)
				if err != nil {
					http.Error(w, "Unable to encode other as HTML", http.StatusBadRequest)
					return
				}
				respJSON.OtherHTML[i] = s
			}
		}

		var buf bytes.Buffer
		err := encodeJSONData(&buf, respJSON)
		if err != nil {
			a.log.Fatal().Err(err).Msg("Unable to store inlines in buffer")
			http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
			return
		}

		err = writeBuffer(w, &buf, ctJSON)
		a.log.IfErr(err).Msg("Write JSON Inlines")
	}
}

func encodeInlines(encdr encoder.Encoder, inl *ast.InlineListNode) (string, error) {
	var buf bytes.Buffer
	_, err := encdr.WriteInlines(&buf, inl)
	if err != nil {
		return "", err
	}
	return buf.String(), nil
}

Deleted web/adapter/api/get_data.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43











































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package api

import (
	"bytes"
	"net/http"

	"zettelstore.de/c/api"
	"zettelstore.de/z/usecase"
)

// MakeGetDataHandler creates a new HTTP handler to return zettelstore data.
func (a *API) MakeGetDataHandler(ucVersion usecase.Version) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		version := ucVersion.Run()
		result := api.VersionJSON{
			Major: version.Major,
			Minor: version.Minor,
			Patch: version.Patch,
			Info:  version.Info,
			Hash:  version.Hash,
		}
		var buf bytes.Buffer
		err := encodeJSONData(&buf, result)
		if err != nil {
			a.log.Fatal().Err(err).Msg("Unable to version info in buffer")
			http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
			return
		}

		err = writeBuffer(w, &buf, ctJSON)
		a.log.IfErr(err).Msg("Write Version Info")
	}
}

Changes to web/adapter/api/get_eval_zettel.go.

33
34
35
36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51
52
53

54
55
56
57
58
33
34
35
36
37
38
39

40
41
42
43
44
45
46
47
48
49
50
51
52

53
54
55
56
57
58







-
+












-
+





		}

		ctx := r.Context()
		q := r.URL.Query()
		enc, encStr := adapter.GetEncoding(r, q, encoder.GetDefaultEncoding())
		part := getPart(q, partContent)
		var env evaluator.Environment
		if q.Has(api.QueryKeyEmbed) {
		if enc == api.EncoderHTML {
			env.GetImageMaterial = func(zettel domain.Zettel, syntax string) ast.InlineEmbedNode {
				return &ast.EmbedBLOBNode{
					Blob:   zettel.Content.AsBytes(),
					Syntax: syntax,
				}
			}
		}
		zn, err := evaluate.Run(ctx, zid, q.Get(api.KeySyntax), &env)
		if err != nil {
			a.reportUsecaseError(w, err)
			return
		}
		evalMeta := func(value string) ast.InlineSlice {
		evalMeta := func(value string) *ast.InlineListNode {
			return evaluate.RunMetadata(ctx, value, &env)
		}
		a.writeEncodedZettelPart(w, zn, evalMeta, enc, encStr, part)
	}
}

Added web/adapter/api/get_links.go.










































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"bytes"
	"net/http"

	"zettelstore.de/c/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/collect"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/usecase"
)

// MakeGetLinksHandler creates a new API handler to return links to other material.
func (a *API) MakeGetLinksHandler(evaluate usecase.Evaluate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		ctx := r.Context()
		q := r.URL.Query()
		zn, err := evaluate.Run(ctx, zid, q.Get(api.KeySyntax), nil)
		if err != nil {
			a.reportUsecaseError(w, err)
			return
		}
		summary := collect.References(zn)

		outData := api.ZettelLinksJSON{ID: api.ZettelID(zid.String())}
		zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Links)
		outData.Linked.Outgoing = idRefs(zetRefs)
		outData.Linked.Local = stringRefs(locRefs)
		outData.Linked.External = stringRefs(extRefs)
		for _, p := range zn.Meta.PairsRest() {
			if meta.Type(p.Key) == meta.TypeURL {
				outData.Linked.Meta = append(outData.Linked.Meta, p.Value)
			}
		}

		zetRefs, locRefs, extRefs = collect.DivideReferences(summary.Embeds)
		outData.Embedded.Outgoing = idRefs(zetRefs)
		outData.Embedded.Local = stringRefs(locRefs)
		outData.Embedded.External = stringRefs(extRefs)

		outData.Cites = stringCites(summary.Cites)

		var buf bytes.Buffer
		err = encodeJSONData(&buf, outData)
		if err != nil {
			a.log.Fatal().Err(err).Zid(zid).Msg("Unable to store links in buffer")
			http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
			return
		}

		err = writeBuffer(w, &buf, ctJSON)
		a.log.IfErr(err).Zid(zid).Msg("Write Zettel Links")
	}
}

func idRefs(refs []*ast.Reference) []string {
	result := make([]string, len(refs))
	for i, ref := range refs {
		path := ref.URL.Path
		if fragment := ref.URL.Fragment; len(fragment) > 0 {
			path = path + "#" + fragment
		}
		result[i] = path
	}
	return result
}

func stringRefs(refs []*ast.Reference) []string {
	result := make([]string, 0, len(refs))
	for _, ref := range refs {
		result = append(result, ref.String())
	}
	return result
}

func stringCites(cites []*ast.CiteNode) []string {
	mapKey := make(strfun.Set, len(cites))
	result := make([]string, 0, len(cites))
	for _, cn := range cites {
		if !mapKey.Has(cn.Key) {
			mapKey.Set(cn.Key)
			result = append(result, cn.Key)
		}
	}
	return result
}

Changes to web/adapter/api/get_unlinked_refs.go.

41
42
43
44
45
46
47
48

49
50
51

52
53
54
55
56
57
58
41
42
43
44
45
46
47

48
49
50

51
52
53
54
55
56
57
58







-
+


-
+







			return
		}

		q := r.URL.Query()
		phrase := q.Get(api.QueryKeyPhrase)
		if phrase == "" {
			zmkTitle := zm.GetDefault(api.KeyTitle, "")
			isTitle := evaluate.RunMetadata(ctx, zmkTitle, nil)
			ilnTitle := evaluate.RunMetadata(ctx, zmkTitle, nil)
			encdr := encoder.Create(api.EncoderText, nil)
			var b strings.Builder
			_, err = encdr.WriteInlines(&b, &isTitle)
			_, err = encdr.WriteInlines(&b, ilnTitle)
			if err == nil {
				phrase = b.String()
			}
		}

		metaList, err := unlinkedRefs.Run(
			ctx, phrase, adapter.AddUnlinkedRefsToSearch(adapter.GetSearch(q), zm))

Changes to web/adapter/response.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package adapter provides handlers for web requests.
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
62
63
64
65
66
67
68



69
70
71
72
73
74
75







-
-
-







	}
	if errors.Is(err, box.ErrStopped) {
		return http.StatusInternalServerError, fmt.Sprintf("Zettelstore not operational: %v", err)
	}
	if errors.Is(err, box.ErrConflict) {
		return http.StatusConflict, "Zettelstore operations conflicted"
	}
	if errors.Is(err, box.ErrCapacity) {
		return http.StatusInsufficientStorage, "Zettelstore reached one of its storage limits"
	}
	return http.StatusInternalServerError, err.Error()
}

// CreateTagReference builds a reference to list all tags.
func CreateTagReference(b server.Builder, key byte, enc, s string) *ast.Reference {
	u := b.NewURLBuilder(key).AppendQuery(api.QueryKeyEncoding, enc).AppendQuery(api.KeyAllTags, s)
	ref := ast.ParseReference(u.String())

Deleted web/adapter/webui/const.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44












































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package webui

// WebUI related constants.

const queryKeyAction = "action"

// Values for queryKeyAction
const (
	valueActionCopy  = "copy"
	valueActionFolge = "folge"
	valueActionNew   = "new"
)

// Enumeration for queryKeyAction
type createAction uint8

const (
	actionCopy createAction = iota
	actionFolge
	actionNew
)

func getCreateAction(s string) createAction {
	switch s {
	case valueActionCopy:
		return actionCopy
	case valueActionFolge:
		return actionFolge
	case valueActionNew:
		return actionNew
	default:
		return actionCopy
	}
}

Changes to web/adapter/webui/create_zettel.go.

1
2
3
4

5
6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30



31
32
33
34


35
36

37
38
39



40







41
42

43
44









45

46
47

48
49
50

51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74



























75
76
77
78









79
80
81
82
83
84
85
1
2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28



29
30
31
32
33


34
35


36

37
38
39
40
41

42
43
44
45
46
47
48
49

50
51
52
53
54
55
56
57
58
59
60
61

62
63

64
65


66
























67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93




94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109



-
+









+














-
-
-
+
+
+


-
-
+
+
-
-
+
-


+
+
+
-
+
+
+
+
+
+
+

-
+


+
+
+
+
+
+
+
+
+
-
+

-
+

-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
+
+
+
+
+
+
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package webui

import (
	"context"
	"fmt"
	"net/http"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetCreateZettelHandler creates a new HTTP handler to display the
// HTML edit view for the various zettel creation methods.
func (wui *WebUI) MakeGetCreateZettelHandler(getZettel usecase.GetZettel, createZettel *usecase.CreateZettel) http.HandlerFunc {
// MakeGetCopyZettelHandler creates a new HTTP handler to display the
// HTML edit view of a copied zettel.
func (wui *WebUI) MakeGetCopyZettelHandler(getZettel usecase.GetZettel, copyZettel usecase.CopyZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		q := r.URL.Query()
		op := getCreateAction(q.Get(queryKeyAction))
		origZettel, err := getOrigZettel(ctx, r, getZettel, "Copy")
		if err != nil {
		if enc, encText := adapter.GetEncoding(r, q, api.EncoderHTML); enc != api.EncoderHTML {
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
			wui.reportError(ctx, w, err)
				fmt.Sprintf("%v zettel not possible in encoding %q", mapActionOp[op], encText)))
			return
		}
		wui.renderZettelForm(w, r, copyZettel.Run(origZettel), "Copy Zettel", "Copy Zettel")
	}
}
		zid, err := id.Parse(r.URL.Path[1:])

// MakeGetFolgeZettelHandler creates a new HTTP handler to display the
// HTML edit view of a follow-up zettel.
func (wui *WebUI) MakeGetFolgeZettelHandler(getZettel usecase.GetZettel, folgeZettel usecase.FolgeZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getOrigZettel(ctx, r, getZettel, "Folge")
		if err != nil {
			wui.reportError(ctx, w, box.ErrNotFound)
			wui.reportError(ctx, w, err)
			return
		}
		wui.renderZettelForm(w, r, folgeZettel.Run(origZettel), "Folge Zettel", "Folgezettel")
	}
}

// MakeGetNewZettelHandler creates a new HTTP handler to display the
// HTML edit view of a zettel.
func (wui *WebUI) MakeGetNewZettelHandler(getZettel usecase.GetZettel, newZettel usecase.NewZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getZettel.Run(box.NoEnrichContext(ctx), zid)
		origZettel, err := getOrigZettel(ctx, r, getZettel, "New")
		if err != nil {
			wui.reportError(ctx, w, box.ErrNotFound)
			wui.reportError(ctx, w, err)
			return
		}

		}
		switch op {
		case actionCopy:
			wui.renderZettelForm(w, r, createZettel.PrepareCopy(origZettel), "Copy Zettel", "Copy Zettel")
		case actionFolge:
			wui.renderZettelForm(w, r, createZettel.PrepareFolge(origZettel), "Folge Zettel", "Folgezettel")
		case actionNew:
			m := origZettel.Meta
			title := parser.ParseMetadata(config.GetTitle(m, wui.rtConfig))
			textTitle, err2 := encodeInlines(&title, api.EncoderText, nil)
			if err2 != nil {
				wui.reportError(ctx, w, err2)
				return
			}
			env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)}
			htmlTitle, err2 := encodeInlines(&title, api.EncoderHTML, &env)
			if err2 != nil {
				wui.reportError(ctx, w, err2)
				return
			}
			wui.renderZettelForm(w, r, createZettel.PrepareNew(origZettel), textTitle, htmlTitle)
		}
	}
}

		m := origZettel.Meta
		title := parser.ParseMetadata(config.GetTitle(m, wui.rtConfig))
		textTitle, err := encodeInlines(title, api.EncoderText, nil)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)}
		htmlTitle, err := encodeInlines(title, api.EncoderHTML, &env)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		wui.renderZettelForm(w, r, newZettel.Run(origZettel), textTitle, htmlTitle)
	}
}

func getOrigZettel(
	ctx context.Context,
	r *http.Request,
	getZettel usecase.GetZettel,
	op string,
) (domain.Zettel, error) {
	if enc, encText := adapter.GetEncoding(r, r.URL.Query(), api.EncoderHTML); enc != api.EncoderHTML {
		return domain.Zettel{}, adapter.NewErrBadRequest(
			fmt.Sprintf("%v zettel not possible in encoding %q", op, encText))
	}
var mapActionOp = map[createAction]string{
	actionCopy:  "Copy",
	actionFolge: "Folge",
	actionNew:   "New",
	zid, err := id.Parse(r.URL.Path[1:])
	if err != nil {
		return domain.Zettel{}, box.ErrNotFound
	}
	origZettel, err := getZettel.Run(box.NoEnrichContext(ctx), zid)
	if err != nil {
		return domain.Zettel{}, box.ErrNotFound
	}
	return origZettel, nil
}

func (wui *WebUI) renderZettelForm(
	w http.ResponseWriter,
	r *http.Request,
	zettel domain.Zettel,
	title, heading string,

Changes to web/adapter/webui/forms.go.

1
2
3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
1
2
3

4
5
6
7
8
9
10
11
12
13
14
15

16
17
18
19
20
21
22



-
+











-







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package webui

import (
	"bytes"
	"net/http"
	"regexp"
	"strings"

	"zettelstore.de/c/api"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
43
44
45
46
47
48
49
50

51
52
53
54
55
56
57
42
43
44
45
46
47
48

49
50
51
52
53
54
55
56







-
+







	err := r.ParseForm()
	if err != nil {
		return domain.Zettel{}, false, err
	}

	var m *meta.Meta
	if postMeta, ok := trimmedFormValue(r, "meta"); ok {
		m = meta.NewFromInput(zid, input.NewInput(removeEmptyLines([]byte(postMeta))))
		m = meta.NewFromInput(zid, input.NewInput([]byte(postMeta)))
		m.Sanitize()
	} else {
		m = meta.New(zid)
	}
	if postTitle, ok := trimmedFormValue(r, "title"); ok {
		m.Set(api.KeyTitle, meta.RemoveNonGraphic(postTitle))
	}
83
84
85
86
87
88
89
90
91
92
93
94
95
96
82
83
84
85
86
87
88














-
-
-
-
-
-
-
		value := strings.TrimSpace(values[0])
		if len(value) > 0 {
			return value, true
		}
	}
	return "", false
}

var reEmptyLines = regexp.MustCompile(`(\n|\r)+\s*(\n|\r)+`)

func removeEmptyLines(s []byte) []byte {
	b := bytes.TrimSpace(s)
	return reEmptyLines.ReplaceAllLiteral(b, []byte{'\n'})
}

Deleted web/adapter/webui/forms_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35



































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package webui

import "testing"

func TestRemoveEmptyLines(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		in  string
		exp string
	}{
		{"", ""},
		{"a", "a"},
		{"\na", "a"},
		{"a\n", "a"},
		{"a\nb", "a\nb"},
		{"a\n\nb", "a\nb"},
		{"a\n \nb", "a\nb"},
	}
	for i, tc := range testcases {
		got := string(removeEmptyLines([]byte(tc.in)))
		if got != tc.exp {
			t.Errorf("%d/%q: expected=%q, got=%q", i, tc.in, tc.exp, got)
		}
	}
}

Changes to web/adapter/webui/get_info.go.

90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
90
91
92
93
94
95
96

97
98
99
100
101
102
103
104







-
+







		metaData := make([]metaDataInfo, len(pairs))
		getTextTitle := wui.makeGetTextTitle(ctx, getMeta, evaluate)
		for i, p := range pairs {
			var buf bytes.Buffer
			wui.writeHTMLMetaValue(
				&buf, p.Key, p.Value,
				getTextTitle,
				func(val string) ast.InlineSlice {
				func(val string) *ast.InlineListNode {
					return evaluate.RunMetadata(ctx, val, &envEval)
				},
				&envHTML)
			metaData[i] = metaDataInfo{p.Key, buf.String()}
		}
		summary := collect.References(zn)
		locLinks, extLinks := splitLocExtLinks(append(summary.Links, summary.Embeds...))
114
115
116
117
118
119
120
121

122
123
124
125
126
127
128
114
115
116
117
118
119
120

121
122
123
124
125
126
127
128







-
+







		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		unLinks := wui.buildHTMLMetaList(ctx, unlinkedMeta, evaluate)

		shadowLinks := getShadowLinks(ctx, zid, getAllMeta)
		endnotes, err := encodeBlocks(&ast.BlockSlice{}, api.EncoderHTML, &envHTML)
		endnotes, err := encodeBlocks(&ast.BlockListNode{}, api.EncoderHTML, &envHTML)
		if err != nil {
			endnotes = ""
		}

		user := wui.getUser(ctx)
		canCreate := wui.canCreate(ctx, user)
		apiZid := api.ZettelID(zid.String())
159
160
161
162
163
164
165
166

167
168

169
170
171
172
173
174
175
159
160
161
162
163
164
165

166
167

168
169
170
171
172
173
174
175







-
+

-
+







		}{
			Zid:            zid.String(),
			WebURL:         wui.NewURLBuilder('h').SetZid(apiZid).String(),
			ContextURL:     wui.NewURLBuilder('k').SetZid(apiZid).String(),
			CanWrite:       wui.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:        wui.NewURLBuilder('e').SetZid(apiZid).String(),
			CanFolge:       canCreate,
			FolgeURL:       wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionFolge).String(),
			FolgeURL:       wui.NewURLBuilder('f').SetZid(apiZid).String(),
			CanCopy:        canCreate && !zn.Content.IsBinary(),
			CopyURL:        wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionCopy).String(),
			CopyURL:        wui.NewURLBuilder('c').SetZid(apiZid).String(),
			CanRename:      wui.canRename(ctx, user, zn.Meta),
			RenameURL:      wui.NewURLBuilder('b').SetZid(apiZid).String(),
			CanDelete:      wui.canDelete(ctx, user, zn.Meta),
			DeleteURL:      wui.NewURLBuilder('d').SetZid(apiZid).String(),
			MetaData:       metaData,
			HasLocLinks:    len(locLinks) > 0,
			LocLinks:       locLinks,

Changes to web/adapter/webui/get_zettel.go.

57
58
59
60
61
62
63
64

65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

83
84
85
86
87
88
89
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

82
83
84
85
86
87
88
89







-
+

















-
+







		zn, err := evaluate.Run(ctx, zid, q.Get(api.KeySyntax), &env)

		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}

		evalMeta := func(value string) ast.InlineSlice {
		evalMeta := func(value string) *ast.InlineListNode {
			return evaluate.RunMetadata(ctx, value, &env)
		}
		lang := config.GetLang(zn.InhMeta, wui.rtConfig)
		envHTML := encoder.Environment{
			Lang:           lang,
			Xhtml:          false,
			MarkerExternal: wui.rtConfig.GetMarkerExternal(),
			NewWindow:      true,
			IgnoreMeta:     strfun.NewSet(api.KeyTitle, api.KeyLang),
		}
		metaHeader, err := encodeMeta(zn.InhMeta, evalMeta, api.EncoderHTML, &envHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		textTitle := wui.encodeTitleAsText(ctx, zn.InhMeta, evaluate)
		htmlTitle := wui.encodeTitleAsHTML(ctx, zn.InhMeta, evaluate, &env, &envHTML)
		htmlContent, err := encodeBlocks(&zn.Ast, api.EncoderHTML, &envHTML)
		htmlContent, err := encodeBlocks(zn.Ast, api.EncoderHTML, &envHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		user := wui.getUser(ctx)
		roleText := zn.Meta.GetDefault(api.KeyRole, "*")
		tags := wui.buildTagInfos(zn.Meta)
126
127
128
129
130
131
132
133

134
135

136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153

154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170

171
172
173
174
175
176
177

178
179
180
181
182
183
184
126
127
128
129
130
131
132

133
134

135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152

153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169

170
171
172
173
174
175
176

177
178
179
180
181
182
183
184







-
+

-
+

















-
+
















-
+






-
+







			Zid:           zid.String(),
			InfoURL:       wui.NewURLBuilder('i').SetZid(apiZid).String(),
			RoleText:      roleText,
			RoleURL:       wui.NewURLBuilder('h').AppendQuery("role", roleText).String(),
			HasTags:       len(tags) > 0,
			Tags:          tags,
			CanCopy:       canCreate && !zn.Content.IsBinary(),
			CopyURL:       wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionCopy).String(),
			CopyURL:       wui.NewURLBuilder('c').SetZid(apiZid).String(),
			CanFolge:      canCreate,
			FolgeURL:      wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionFolge).String(),
			FolgeURL:      wui.NewURLBuilder('f').SetZid(apiZid).String(),
			PrecursorRefs: wui.encodeIdentifierSet(zn.InhMeta, api.KeyPrecursor, getTextTitle),
			ExtURL:        extURL,
			HasExtURL:     hasExtURL,
			ExtNewWindow:  htmlAttrNewWindow(envHTML.NewWindow && hasExtURL),
			Content:       htmlContent,
			HasFolgeLinks: len(folgeLinks) > 0,
			FolgeLinks:    folgeLinks,
			HasBackLinks:  len(backLinks) > 0,
			BackLinks:     backLinks,
		})
	}
}

// errNoSuchEncoding signals an unsupported encoding encoding
var errNoSuchEncoding = errors.New("no such encoding")

// encodeInlines returns a string representation of the inline slice.
func encodeInlines(is *ast.InlineSlice, enc api.EncodingEnum, env *encoder.Environment) (string, error) {
func encodeInlines(is *ast.InlineListNode, enc api.EncodingEnum, env *encoder.Environment) (string, error) {
	if is == nil {
		return "", nil
	}
	encdr := encoder.Create(enc, env)
	if encdr == nil {
		return "", errNoSuchEncoding
	}

	var buf bytes.Buffer
	_, err := encdr.WriteInlines(&buf, is)
	if err != nil {
		return "", err
	}
	return buf.String(), nil
}

func encodeBlocks(bs *ast.BlockSlice, enc api.EncodingEnum, env *encoder.Environment) (string, error) {
func encodeBlocks(bln *ast.BlockListNode, enc api.EncodingEnum, env *encoder.Environment) (string, error) {
	encdr := encoder.Create(enc, env)
	if encdr == nil {
		return "", errNoSuchEncoding
	}

	var buf bytes.Buffer
	_, err := encdr.WriteBlocks(&buf, bs)
	_, err := encdr.WriteBlocks(&buf, bln)
	if err != nil {
		return "", err
	}
	return buf.String(), nil
}

func encodeMeta(

Changes to web/adapter/webui/htmlmeta.go.

1
2
3
4

5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

30
31
32
33
34
35

36
37
38
39
40
41
42
43
44


45
46
47
48
49
50
51
1
2
3

4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30
31
32
33
34

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53



-
+

















-







+





-
+









+
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package webui

import (
	"context"
	"errors"
	"fmt"
	"io"
	"net/url"
	"time"

	"zettelstore.de/c/api"
	"zettelstore.de/c/html"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/usecase"
)

var space = []byte{' '}

type evalMetadataFunc = func(string) ast.InlineSlice
type evalMetadataFunc = func(string) *ast.InlineListNode

func (wui *WebUI) writeHTMLMetaValue(
	w io.Writer,
	key, value string,
	getTextTitle getTextTitleFunc,
	evalMetadata evalMetadataFunc,
	envEnc *encoder.Environment,
) {
	switch kt := meta.Type(key); kt {
	case meta.TypeBool:
		wui.writeHTMLBool(w, key, value)
	case meta.TypeCredential:
		writeCredential(w, value)
	case meta.TypeEmpty:
		writeEmpty(w, value)
	case meta.TypeID:
		wui.writeIdentifier(w, value, getTextTitle)
	case meta.TypeIDSet:
65
66
67
68
69
70
71
72

73
74
75
76








77
78







79
80
81
82
83

84
85
86
87
88
89
90
67
68
69
70
71
72
73

74
75
76
77
78
79
80
81
82
83
84
85
86


87
88
89
90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
105







-
+




+
+
+
+
+
+
+
+
-
-
+
+
+
+
+
+
+




-
+







	case meta.TypeWord:
		wui.writeWord(w, key, value)
	case meta.TypeWordSet:
		wui.writeWordSet(w, key, meta.ListFromValue(value))
	case meta.TypeZettelmarkup:
		io.WriteString(w, encodeZmkMetadata(value, evalMetadata, api.EncoderHTML, envEnc))
	default:
		html.Escape(w, value)
		strfun.HTMLEscape(w, value)
		fmt.Fprintf(w, " <b>(Unhandled type: %v, key: %v)</b>", kt, key)
	}
}

func (wui *WebUI) writeHTMLBool(w io.Writer, key, value string) {
	if meta.BoolValue(value) {
		wui.writeLink(w, key, "true", "True")
	} else {
		wui.writeLink(w, key, "false", "False")
	}
}

func writeCredential(w io.Writer, val string) { html.Escape(w, val) }
func writeEmpty(w io.Writer, val string)      { html.Escape(w, val) }
func writeCredential(w io.Writer, val string) {
	strfun.HTMLEscape(w, val)
}

func writeEmpty(w io.Writer, val string) {
	strfun.HTMLEscape(w, val)
}

func (wui *WebUI) writeIdentifier(w io.Writer, val string, getTextTitle getTextTitleFunc) {
	zid, err := id.Parse(val)
	if err != nil {
		html.Escape(w, val)
		strfun.HTMLEscape(w, val)
		return
	}
	title, found := getTextTitle(zid)
	switch {
	case found > 0:
		ub := wui.NewURLBuilder('h').SetZid(api.ZettelID(zid.String()))
		if title == "" {
108
109
110
111
112
113
114
115



116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133

134
135
136
137

138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156

157
158
159
160
161
162
163
123
124
125
126
127
128
129

130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149

150
151
152
153

154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172

173
174
175
176
177
178
179
180







-
+
+
+

















-
+



-
+


















-
+







	}
}

func (wui *WebUI) writeNumber(w io.Writer, key, val string) {
	wui.writeLink(w, key, val, val)
}

func writeString(w io.Writer, val string) { html.Escape(w, val) }
func writeString(w io.Writer, val string) {
	strfun.HTMLEscape(w, val)
}

func (wui *WebUI) writeTagSet(w io.Writer, key string, tags []string) {
	for i, tag := range tags {
		if i > 0 {
			w.Write(space)
		}
		wui.writeLink(w, key, tag, tag)
	}
}

func writeTimestamp(w io.Writer, ts time.Time) {
	io.WriteString(w, ts.Format("2006-01-02&nbsp;15:04:05"))
}

func writeURL(w io.Writer, val string) {
	u, err := url.Parse(val)
	if err != nil {
		html.Escape(w, val)
		strfun.HTMLEscape(w, val)
		return
	}
	fmt.Fprintf(w, "<a href=\"%v\"%v>", u, htmlAttrNewWindow(true))
	html.Escape(w, val)
	strfun.HTMLEscape(w, val)
	io.WriteString(w, "</a>")
}

func (wui *WebUI) writeWord(w io.Writer, key, word string) {
	wui.writeLink(w, key, word, word)
}

func (wui *WebUI) writeWordSet(w io.Writer, key string, words []string) {
	for i, word := range words {
		if i > 0 {
			w.Write(space)
		}
		wui.writeWord(w, key, word)
	}
}

func (wui *WebUI) writeLink(w io.Writer, key, value, text string) {
	fmt.Fprintf(w, "<a href=\"%v?%v=%v\">", wui.NewURLBuilder('h'), url.QueryEscape(key), url.QueryEscape(value))
	html.Escape(w, text)
	strfun.HTMLEscape(w, text)
	io.WriteString(w, "</a>")
}

type getTextTitleFunc func(id.Zid) (string, int)

func (wui *WebUI) makeGetTextTitle(
	ctx context.Context,
179
180
181
182
183
184
185
186

187
188
189
190
191
192
193
194
195
196
197
198

199
200
201
202
203
204
205
206
207
208
209


210
211
212

213
214
215
216
217
196
197
198
199
200
201
202

203
204
205
206
207
208
209
210
211
212
213
214

215
216
217
218
219
220
221
222
223
224


225
226
227
228

229
230
231
232
233
234







-
+











-
+









-
-
+
+


-
+





	ctx context.Context, m *meta.Meta,
	evaluate *usecase.Evaluate, envEval *evaluator.Environment,
	envHTML *encoder.Environment,
) string {
	plainTitle := config.GetTitle(m, wui.rtConfig)
	return encodeZmkMetadata(
		plainTitle,
		func(val string) ast.InlineSlice {
		func(val string) *ast.InlineListNode {
			return evaluate.RunMetadata(ctx, plainTitle, envEval)
		},
		api.EncoderHTML, envHTML)
}

func (wui *WebUI) encodeTitleAsText(
	ctx context.Context, m *meta.Meta, evaluate *usecase.Evaluate,
) string {
	plainTitle := config.GetTitle(m, wui.rtConfig)
	return encodeZmkMetadata(
		plainTitle,
		func(val string) ast.InlineSlice {
		func(val string) *ast.InlineListNode {
			return evaluate.RunMetadata(ctx, plainTitle, nil)
		},
		api.EncoderText, nil)
}

func encodeZmkMetadata(
	value string, evalMetadata evalMetadataFunc,
	enc api.EncodingEnum, envHTML *encoder.Environment,
) string {
	is := evalMetadata(value)
	if len(is) == 0 {
	iln := evalMetadata(value)
	if iln.IsEmpty() {
		return ""
	}
	result, err := encodeInlines(&is, enc, envHTML)
	result, err := encodeInlines(iln, enc, envHTML)
	if err != nil {
		return err.Error()
	}
	return result
}

Changes to web/adapter/webui/webui.go.

1
2
3
4

5
6
7
8
9
10
11
1
2
3

4
5
6
7
8
9
10
11



-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package webui provides web-UI handlers for web requests.
265
266
267
268
269
270
271
272

273
274

275
276
277
278
279
280
281

282
283
284
285
286
287
288
289
265
266
267
268
269
270
271

272
273

274
275
276
277
278
279
280

281

282
283
284
285
286
287
288







-
+

-
+






-
+
-







		}
		if !wui.policy.CanRead(user, m) {
			continue
		}
		title := config.GetTitle(m, wui.rtConfig)
		astTitle := parser.ParseMetadata(title)
		env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)}
		menuTitle, err2 := encodeInlines(&astTitle, api.EncoderHTML, &env)
		menuTitle, err2 := encodeInlines(astTitle, api.EncoderHTML, &env)
		if err2 != nil {
			menuTitle, err2 = encodeInlines(&astTitle, api.EncoderText, nil)
			menuTitle, err2 = encodeInlines(astTitle, api.EncoderText, nil)
			if err2 != nil {
				menuTitle = title
			}
		}
		result = append(result, simpleLink{
			Text: menuTitle,
			URL: wui.NewURLBuilder('c').SetZid(api.ZettelID(m.Zid.String())).
			URL:  wui.NewURLBuilder('g').SetZid(api.ZettelID(m.Zid.String())).String(),
				AppendQuery(queryKeyAction, valueActionNew).String(),
		})
	}
	return result
}

func (wui *WebUI) renderTemplate(
	ctx context.Context,

Changes to web/server/impl/impl.go.

1
2

3
4

5
6
7
8
9
10
11
1

2
3

4
5
6
7
8
9
10
11

-
+

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
// Copyright (c) 2021 Detlef Stern
//
// This file is part of Zettelstore.
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the Zettelstore web service.
73
74
75
76
77
78
79
80

81
82
83
84
85
86
87

88
89
90
91
92
93
94
95
96
97
73
74
75
76
77
78
79

80
81
82
83
84
85


86



87
88
89
90
91
92
93







-
+





-
-
+
-
-
-







func (srv *myServer) SetToken(w http.ResponseWriter, token []byte, d time.Duration) {
	cookie := http.Cookie{
		Name:     sessionName,
		Value:    string(token),
		Path:     srv.GetURLPrefix(),
		Secure:   srv.secureCookie,
		HttpOnly: true,
		SameSite: http.SameSiteLaxMode,
		SameSite: http.SameSiteStrictMode,
	}
	if srv.persistentCookie && d > 0 {
		cookie.Expires = time.Now().Add(d).Add(30 * time.Second).UTC()
	}
	srv.log.Debug().Bytes("token", token).Msg("SetToken")
	if v := cookie.String(); v != "" {
		w.Header().Add("Set-Cookie", v)
	http.SetCookie(w, &cookie)
		w.Header().Add("Cache-Control", `no-cache="Set-Cookie"`)
		w.Header().Add("Vary", "Cookie")
	}
}

// ClearToken invalidates the session cookie by sending an empty one.
func (srv *myServer) ClearToken(ctx context.Context, w http.ResponseWriter) context.Context {
	if authData := srv.GetAuthData(ctx); authData == nil {
		// No authentication data stored in session, nothing to do.
		return ctx

Changes to www/changes.wiki.

1
2
3
4
5
6
7

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
1
2



3

4





















































5
6
7
8
9
10
11


-
-
-

-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







<title>Change Log</title>

<a name="0_5"></a>
<h2>Changes for Version 0.5 (pending)</h2>

<a name="0_4"></a>
<h2>Changes for Version 0.4 (2022-03-08)</h2>
<h2>Changes for Version 0.4 (pending)</h2>
  *  Encoding &ldquo;djson&rdquo; renamed to &ldquo;zjson&rdquo; (<em>zettel
     json</em>).
     (breaking: api; minor: webui)
  *  Remove inline quotation syntax <tt>&lt;&lt;...&lt;&lt;</tt>. Now,
     <tt>&quot;&quot;...&quot;&quot;</tt> generates the equivalent code.
     Typographical quotes are generated by the browser, not by Zettelstore.
     (breaking: Zettelmarkup)
  *  Remove inline formatting for monospace. Its syntax is now used by the
     similar syntax element of literal computer input. Monospace was just
     a visual element with no semantic association. Now, the syntax
     <kbd>++...++</kbd> is obsolete.
     (breaking: Zettelmarkup).
  *  Remove API call to parse Zettelmarkup texts and encode it as text and
     HTML. Was call &ldquo;POST /v&rdquo;. It was needed to separately encode
     the titles of zettel. The same effect can be achieved by fetching the
     ZJSON representation and encode it using the function in the Zettelstore
     client software.
     (breaking: api)
  *  Remove API call to retrieve all links of an zettel. This can be done more
     easily on the client side by traversing the ZJSON encoding of a zettel.
     (breaking: api)
  *  ZJSON will encode metadata value as pairs of a metadata type and metadata
     value. This allows a client to decode the associated value more easily.
     (minor: api)
  *  A sequence of inline-structured elements can be marked, not just a point
     in the Zettelmarkup text.
     (minor: zettelmarkup)
  *  Metadata keys with suffix <kbd>-title</kbd> force their value to be
     interpreted as Zettelmarkup. Similar, the suffix <kbd>-set</kbd> denotes
     a set/list of words and the suffix <kbd>-zids</kbd> a set/list of zettel
     identifier.
     (minor: api, webui)
  *  Change generated URLs for zettel-creation forms. If you have bookmarked
     them, e.g. to create a new zettel, you should update.
     (minor: webui)
  *  Remove support for metadata key <tt>no-index</tt> to suppress indexing
     selected zettel. It was introduced in <a href="#0_0_11">v0.0.11</a>, but
     disallows some future optimizations for searching zettel.
     (minor: api, webui)
  *  Make some metadata-based searches a little bit faster by executing
     a (in-memory-based) full-text search first. Now only those zettel are
     loaded from file that contain the metdata value.
     (minor: api, webui)
  *  Add an API call to retrieve the version of the Zettelstore.
     (minor: api)
  *  Limit the amount of zettel and bytes to be stored in a memory box. Allows
     to use it with public access.
     (minor: box)
  *  Disallow to cache the authentication cookie. Will remove most unexpected
     log-outs when using a mobile device.
     (minor: webui)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_3"></a>
<h2>Changes for Version 0.3 (2022-02-09)</h2>
  *  Zettel files with extension <tt>.meta</tt> are now treated as content
     files. Previoulsy, they were interpreted as metadata files. The
     interpretation as metadata files was deprecated in version 0.2.
     (breaking: directory and file/zip box)

Changes to www/download.wiki.

1
2
3
4
5
6
7
8
9
10
11
12

13
14
15
16
17
18





19
20
21
22
23
24

25
26
1
2
3
4
5
6
7
8
9
10
11

12
13





14
15
16
17
18
19
20
21
22
23

24
25
26











-
+

-
-
-
-
-
+
+
+
+
+





-
+


<title>Download</title>
<h1>Download of Zettelstore Software</h1>
<h2>Foreword</h2>
  *  Zettelstore is free/libre open source software, licensed under EUPL-1.2-or-later.
  *  The software is provided as-is.
  *  There is no guarantee that it will not damage your system.
  *  However, it is in use by the main developer since March 2020 without any damage.
  *  It may be useful for you. It is useful for me.
  *  Take a look at the [https://zettelstore.de/manual/|manual] to know how to start and use it.

<h2>ZIP-ped Executables</h2>
Build: <code>v0.4</code> (2022-03-08).
Build: <code>v0.3</code> (2022-02-09).

  *  [/uv/zettelstore-0.4-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.4-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.4-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.4-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.4-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)
  *  [/uv/zettelstore-0.3-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.3-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.3-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.3-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.3-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)

Unzip the appropriate file, install and execute Zettelstore according to the manual.

<h2>Zettel for the manual</h2>
As a starter, you can download the zettel for the manual
[/uv/manual-0.4.zip|here].
[/uv/manual-0.3.zip|here].
Just unzip the contained files and put them into your zettel folder or
configure a file box to read the zettel directly from the ZIP file.

Changes to www/index.wiki.

20
21
22
23
24
25
26
27

28
29
30
31
32
33





34
35
36
37
38
39
40
41
42
43
44
20
21
22
23
24
25
26

27
28





29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44







-
+

-
-
-
-
-
+
+
+
+
+











access Zettelstore via its API more easily,
[https://zettelstore.de/contrib|Zettelstore Contrib] contains contributed
software, which often connects to Zettelstore via its API. Some of the software
packages may be experimental.

[https://twitter.com/zettelstore|Stay tuned]&hellip;
<hr>
<h3>Latest Release: 0.4 (2022-03-08)</h3>
<h3>Latest Release: 0.3 (2022-02-09)</h3>
  *  [./download.wiki|Download]
  *  [./changes.wiki#0_4|Change summary]
  *  [/timeline?p=v0.4&bt=v0.3&y=ci|Check-ins for version 0.4],
     [/vdiff?to=v0.4&from=v0.3|content diff]
  *  [/timeline?df=v0.4&y=ci|Check-ins derived from the 0.4 release],
     [/vdiff?from=v0.4&to=trunk|content diff]
  *  [./changes.wiki#0_3|Change summary]
  *  [/timeline?p=v0.3&bt=v0.2&y=ci|Check-ins for version 0.3],
     [/vdiff?to=v0.3&from=v0.2|content diff]
  *  [/timeline?df=v0.3&y=ci|Check-ins derived from the 0.3 release],
     [/vdiff?from=v0.3&to=trunk|content diff]
  *  [./plan.wiki|Limitations and planned improvements]
  *  [/timeline?t=release|Timeline of all past releases]

<hr>
<h2>Build instructions</h2>
Just install [https://golang.org/dl/|Go] and some Go-based tools. Please read
the [./build.md|instructions] for details.

  *  [/dir?ci=trunk|Source code]
  *  [/download|Download the source code] as a tarball or a ZIP file
     (you must [/login|login] as user &quot;anonymous&quot;).