Zettelstore

Check-in Differences
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Difference From version-0.0.14 To version-0.0.15

2021-09-23
18:03
Increase version to 0.0.16-dev to begin next development cycle ... (check-in: 9748c52fdb user: stern tags: trunk)
2021-09-17
14:55
Version 0.0.15 ... (check-in: 66498fc30a user: stern tags: trunk, release, version-0.0.15, v0.0.15)
2021-09-16
18:02
Initial development zettel ... (check-in: 64e566f5db user: stern tags: trunk)
2021-07-23
16:43
Increase version to 0.0.15-dev to begin next development cycle ... (check-in: ba65777850 user: stern tags: trunk)
11:02
Version 0.0.14 ... (check-in: 6fe53d5db2 user: stern tags: trunk, release, version-0.0.14)
2021-07-22
13:50
Add client API for retrieving zettel links ... (check-in: e43ff68174 user: stern tags: trunk)

Changes to VERSION.

1


1
-
+
0.0.14
0.0.15

Changes to api/api.go.

16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

45
46
47
48
49
50
51
52
53
54
55
56
57












58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

80
81
82
83
84
85
86
87
88
89
90
16
17
18
19
20
21
22

23

24
25
26
27
28

29
30
31
32
33
34

35
36
37
38
39
40

41













42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

67
68
69
70
71
72
73

74
75
76
77
78
79
80
81
82
83
84
85







-
+
-





-






-






-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+













-







-
+











	Token   string `json:"token"`
	Type    string `json:"token_type"`
	Expires int    `json:"expires_in"`
}

// ZidJSON contains the identifier data of a zettel.
type ZidJSON struct {
	ID  string `json:"id"`
	ID string `json:"id"`
	URL string `json:"url"`
}

// ZidMetaJSON contains the identifier and the metadata of a zettel.
type ZidMetaJSON struct {
	ID   string            `json:"id"`
	URL  string            `json:"url"`
	Meta map[string]string `json:"meta"`
}

// ZidMetaRelatedList contains identifier/metadata of a zettel and the same for related zettel
type ZidMetaRelatedList struct {
	ID   string            `json:"id"`
	URL  string            `json:"url"`
	Meta map[string]string `json:"meta"`
	List []ZidMetaJSON     `json:"list"`
}

// ZettelLinksJSON store all links / connections from one zettel to other.
type ZettelLinksJSON struct {
	ID    string `json:"id"`
	ID     string `json:"id"`
	URL   string `json:"url"`
	Links struct {
		Incoming []ZidJSON `json:"incoming,omitempty"`
		Outgoing []ZidJSON `json:"outgoing,omitempty"`
		Local    []string  `json:"local,omitempty"`
		External []string  `json:"external,omitempty"`
		Meta     []string  `json:"meta,omitempty"`
	} `json:"links"`
	Images struct {
		Outgoing []ZidJSON `json:"outgoing,omitempty"`
		Local    []string  `json:"local,omitempty"`
		External []string  `json:"external,omitempty"`
	} `json:"images,omitempty"`
	Linked struct {
		Incoming []string `json:"incoming,omitempty"`
		Outgoing []string `json:"outgoing,omitempty"`
		Local    []string `json:"local,omitempty"`
		External []string `json:"external,omitempty"`
		Meta     []string `json:"meta,omitempty"`
	} `json:"linked"`
	Embedded struct {
		Outgoing []string `json:"outgoing,omitempty"`
		Local    []string `json:"local,omitempty"`
		External []string `json:"external,omitempty"`
	} `json:"embedded,omitempty"`
	Cites []string `json:"cites,omitempty"`
}

// ZettelDataJSON contains all data for a zettel.
type ZettelDataJSON struct {
	Meta     map[string]string `json:"meta"`
	Encoding string            `json:"encoding"`
	Content  string            `json:"content"`
}

// ZettelJSON contains all data for a zettel, the identifier, the metadata, and the content.
type ZettelJSON struct {
	ID       string            `json:"id"`
	URL      string            `json:"url"`
	Meta     map[string]string `json:"meta"`
	Encoding string            `json:"encoding"`
	Content  string            `json:"content"`
}

// ZettelListJSON contains data for a zettel list.
type ZettelListJSON struct {
	List []ZettelJSON `json:"list"`
	List []ZidMetaJSON `json:"list"`
}

// TagListJSON specifies the list/map of tags
type TagListJSON struct {
	Tags map[string][]string `json:"tags"`
}

// RoleListJSON specifies the list of roles.
type RoleListJSON struct {
	Roles []string `json:"role-list"`
}

Changes to api/const.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34










35
36
37
38
39
40
41
42
43

44
45
46


47
48

49
50
51


52
53
54
55
56



57
58

59
60
61


62
63

64
65
66
67


68
69
70
71
72
73



74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

97
98
99
100
101
102
103
104
105
106
107
108
1
2
3
4
5
6
7
8
9
10
11
12
13

14


15
16
17
18
19
20
21
22
23
24
25
26
27





28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

46
47


48
49


50



51
52
53
54



55
56
57


58



59
60
61

62
63
64


65
66
67
68
69



70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

86

87
88
89
90
91
92

93
94
95
96
97
98
99
100

101
102
103
104













-
+
-
-













-
-
-
-
-
+
+
+
+
+
+
+
+
+
+








-
+

-
-
+
+
-
-
+
-
-
-
+
+


-
-
-
+
+
+
-
-
+
-
-
-
+
+

-
+


-
-
+
+



-
-
-
+
+
+













-

-






-
+







-




//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package api contains common definition used for client and server.
package api

import (
import "fmt"
	"fmt"
)

// Additional HTTP constants used.
const (
	MethodMove = "MOVE" // HTTP method for renaming a zettel

	HeaderAccept      = "Accept"
	HeaderContentType = "Content-Type"
	HeaderDestination = "Destination"
	HeaderLocation    = "Location"
)

// Values for HTTP query parameter.
const (
	QueryKeyDepth  = "depth"
	QueryKeyDir    = "dir"
	QueryKeyFormat = "_format"
	QueryKeyLimit  = "limit"
	QueryKeyPart   = "_part"
	QueryKeyDepth    = "depth"
	QueryKeyDir      = "dir"
	QueryKeyEncoding = "_enc"
	QueryKeyLimit    = "_limit"
	QueryKeyNegate   = "_negate"
	QueryKeyOffset   = "_offset"
	QueryKeyOrder    = "_order"
	QueryKeyPart     = "_part"
	QueryKeySearch   = "_s"
	QueryKeySort     = "_sort"
)

// Supported dir values.
const (
	DirBackward = "backward"
	DirForward  = "forward"
)

// Supported format values.
// Supported encoding values.
const (
	FormatDJSON  = "djson"
	FormatHTML   = "html"
	EncodingDJSON  = "djson"
	EncodingHTML   = "html"
	FormatJSON   = "json"
	FormatNative = "native"
	EncodingNative = "native"
	FormatRaw    = "raw"
	FormatText   = "text"
	FormatZMK    = "zmk"
	EncodingText   = "text"
	EncodingZMK    = "zmk"
)

var formatEncoder = map[string]EncodingEnum{
	FormatDJSON:  EncoderDJSON,
	FormatHTML:   EncoderHTML,
var mapEncodingEnum = map[string]EncodingEnum{
	EncodingDJSON:  EncoderDJSON,
	EncodingHTML:   EncoderHTML,
	FormatJSON:   EncoderJSON,
	FormatNative: EncoderNative,
	EncodingNative: EncoderNative,
	FormatRaw:    EncoderRaw,
	FormatText:   EncoderText,
	FormatZMK:    EncoderZmk,
	EncodingText:   EncoderText,
	EncodingZMK:    EncoderZmk,
}
var encoderFormat = map[EncodingEnum]string{}
var mapEnumEncoding = map[EncodingEnum]string{}

func init() {
	for k, v := range formatEncoder {
		encoderFormat[v] = k
	for k, v := range mapEncodingEnum {
		mapEnumEncoding[v] = k
	}
}

// Encoder returns the internal encoder code for the given format string.
func Encoder(format string) EncodingEnum {
	if e, ok := formatEncoder[format]; ok {
// Encoder returns the internal encoder code for the given encoding string.
func Encoder(encoding string) EncodingEnum {
	if e, ok := mapEncodingEnum[encoding]; ok {
		return e
	}
	return EncoderUnknown
}

// EncodingEnum lists all valid encoder keys.
type EncodingEnum uint8

// Values for EncoderEnum
const (
	EncoderUnknown EncodingEnum = iota
	EncoderDJSON
	EncoderHTML
	EncoderJSON
	EncoderNative
	EncoderRaw
	EncoderText
	EncoderZmk
)

// String representation of an encoder key.
func (e EncodingEnum) String() string {
	if f, ok := encoderFormat[e]; ok {
	if f, ok := mapEnumEncoding[e]; ok {
		return f
	}
	return fmt.Sprintf("*Unknown*(%d)", e)
}

// Supported part values.
const (
	PartID      = "id"
	PartMeta    = "meta"
	PartContent = "content"
	PartZettel  = "zettel"
)

Changes to ast/ast.go.

18
19
20
21
22
23
24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54












55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
18
19
20
21
22
23
24

25
26
27
28

29
30
31
32
33
34
35
36
37
38
39
40
41
42



43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78



79
80
81
82
83
84
85







-




-
+













-
-
-








+
+
+
+
+
+
+
+
+
+
+
+
















-
-
-







	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// ZettelNode is the root node of the abstract syntax tree.
// It is *not* part of the visitor pattern.
type ZettelNode struct {
	// Zettel  domain.Zettel
	Meta    *meta.Meta     // Original metadata
	Content domain.Content // Original content
	Zid     id.Zid         // Zettel identification.
	InhMeta *meta.Meta     // Metadata of the zettel, with inherited values.
	Ast     BlockSlice     // Zettel abstract syntax tree is a sequence of block nodes.
	Ast     *BlockListNode // Zettel abstract syntax tree is a sequence of block nodes.
}

// Node is the interface, all nodes must implement.
type Node interface {
	WalkChildren(v Visitor)
}

// BlockNode is the interface that all block nodes must implement.
type BlockNode interface {
	Node
	blockNode()
}

// BlockSlice is a slice of BlockNodes.
type BlockSlice []BlockNode

// ItemNode is a node that can occur as a list item.
type ItemNode interface {
	BlockNode
	itemNode()
}

// ItemSlice is a slice of ItemNodes.
type ItemSlice []ItemNode

// ItemListNode is a list of BlockNodes.
type ItemListNode struct {
	List []ItemNode
}

// WalkChildren walks down to the descriptions.
func (iln *ItemListNode) WalkChildren(v Visitor) {
	for _, bn := range iln.List {
		Walk(v, bn)
	}
}

// DescriptionNode is a node that contains just textual description.
type DescriptionNode interface {
	ItemNode
	descriptionNode()
}

// DescriptionSlice is a slice of DescriptionNodes.
type DescriptionSlice []DescriptionNode

// InlineNode is the interface that all inline nodes must implement.
type InlineNode interface {
	Node
	inlineNode()
}

// InlineSlice is a slice of InlineNodes.
type InlineSlice []InlineNode

// Reference is a reference to external or internal material.
type Reference struct {
	URL   *url.URL
	Value string
	State RefState
}

Changes to ast/attr.go.

16
17
18
19
20
21
22



23
24
25
26
27
28
29
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32







+
+
+







)

// Attributes store additional information about some node types.
type Attributes struct {
	Attrs map[string]string
}

// IsEmpty returns true if there are no attributes.
func (a *Attributes) IsEmpty() bool { return a == nil || len(a.Attrs) == 0 }

// HasDefault returns true, if the default attribute "-" has been set.
func (a *Attributes) HasDefault() bool {
	if a != nil {
		_, ok := a.Attrs["-"]
		return ok
	}
	return false
75
76
77
78
79
80
81



82
83
84
85
86
87
88
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94







+
+
+







}

// AddClass adds a value to the class attribute.
func (a *Attributes) AddClass(class string) *Attributes {
	if a == nil {
		return &Attributes{map[string]string{"class": class}}
	}
	if a.Attrs == nil {
		a.Attrs = make(map[string]string)
	}
	classes := a.GetClasses()
	for _, cls := range classes {
		if cls == class {
			return a
		}
	}
	classes = append(classes, class)

Changes to ast/block.go.

9
10
11
12
13
14
15














16
17
18
19

20
21
22
23
24






25
26
27
28

29
30
31
32
33
34
35
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

33
34
35



36
37
38
39
40
41
42
43
44

45
46
47
48
49
50
51
52







+
+
+
+
+
+
+
+
+
+
+
+
+
+



-
+


-
-
-
+
+
+
+
+
+



-
+







//-----------------------------------------------------------------------------

// Package ast provides the abstract syntax tree.
package ast

// Definition of Block nodes.

// BlockListNode is a list of BlockNodes.
type BlockListNode struct {
	List []BlockNode
}

// WalkChildren walks down to the descriptions.
func (bln *BlockListNode) WalkChildren(v Visitor) {
	for _, bn := range bln.List {
		Walk(v, bn)
	}
}

//--------------------------------------------------------------------------

// ParaNode contains just a sequence of inline elements.
// Another name is "paragraph".
type ParaNode struct {
	Inlines InlineSlice
	Inlines *InlineListNode
}

func (pn *ParaNode) blockNode()       { /* Just a marker */ }
func (pn *ParaNode) itemNode()        { /* Just a marker */ }
func (pn *ParaNode) descriptionNode() { /* Just a marker */ }
func (*ParaNode) blockNode()       { /* Just a marker */ }
func (*ParaNode) itemNode()        { /* Just a marker */ }
func (*ParaNode) descriptionNode() { /* Just a marker */ }

// NewParaNode creates an empty ParaNode.
func NewParaNode() *ParaNode { return &ParaNode{Inlines: &InlineListNode{}} }

// WalkChildren walks down the inline elements.
func (pn *ParaNode) WalkChildren(v Visitor) {
	WalkInlineSlice(v, pn.Inlines)
	Walk(v, pn.Inlines)
}

//--------------------------------------------------------------------------

// VerbatimNode contains lines of uninterpreted text
type VerbatimNode struct {
	Kind  VerbatimKind
44
45
46
47
48
49
50
51
52


53
54
55

56
57
58
59
60
61
62
63
64


65
66
67
68
69
70
71
72
73
74
75
76
77
78
79


80
81
82
83
84




85
86
87
88
89
90
91
92
93
94





95
96
97
98


99
100
101
102

103
104
105
106
107
108
109
110
111
112
113


114
115
116

117
118
119
120
121
122
123
61
62
63
64
65
66
67


68
69
70
71

72
73
74
75
76
77
78
79


80
81
82
83
84
85
86
87
88
89
90
91
92
93
94


95
96
97
98
99


100
101
102
103
104
105
106
107
108
109




110
111
112
113
114
115
116


117
118
119
120
121

122
123
124
125
126
127
128
129
130
131


132
133
134
135

136
137
138
139
140
141
142
143







-
-
+
+


-
+







-
-
+
+













-
-
+
+



-
-
+
+
+
+






-
-
-
-
+
+
+
+
+


-
-
+
+



-
+









-
-
+
+


-
+







const (
	_               VerbatimKind = iota
	VerbatimProg                 // Program code.
	VerbatimComment              // Block comment
	VerbatimHTML                 // Block HTML, e.g. for Markdown
)

func (vn *VerbatimNode) blockNode() { /* Just a marker */ }
func (vn *VerbatimNode) itemNode()  { /* Just a marker */ }
func (*VerbatimNode) blockNode() { /* Just a marker */ }
func (*VerbatimNode) itemNode()  { /* Just a marker */ }

// WalkChildren does nothing.
func (vn *VerbatimNode) WalkChildren(v Visitor) { /* No children*/ }
func (*VerbatimNode) WalkChildren(Visitor) { /* No children*/ }

//--------------------------------------------------------------------------

// RegionNode encapsulates a region of block nodes.
type RegionNode struct {
	Kind    RegionKind
	Attrs   *Attributes
	Blocks  BlockSlice
	Inlines InlineSlice // Additional text at the end of the region
	Blocks  *BlockListNode
	Inlines *InlineListNode // Optional text at the end of the region
}

// RegionKind specifies the actual region type.
type RegionKind uint8

// Values for RegionCode
const (
	_           RegionKind = iota
	RegionSpan             // Just a span of blocks
	RegionQuote            // A longer quotation
	RegionVerse            // Line breaks matter
)

func (rn *RegionNode) blockNode() { /* Just a marker */ }
func (rn *RegionNode) itemNode()  { /* Just a marker */ }
func (*RegionNode) blockNode() { /* Just a marker */ }
func (*RegionNode) itemNode()  { /* Just a marker */ }

// WalkChildren walks down the blocks and the text.
func (rn *RegionNode) WalkChildren(v Visitor) {
	WalkBlockSlice(v, rn.Blocks)
	WalkInlineSlice(v, rn.Inlines)
	Walk(v, rn.Blocks)
	if iln := rn.Inlines; iln != nil {
		Walk(v, iln)
	}
}

//--------------------------------------------------------------------------

// HeadingNode stores the heading text and level.
type HeadingNode struct {
	Level   int
	Inlines InlineSlice // Heading text, possibly formatted
	Slug    string      // Heading text, suitable to be used as an URL fragment
	Attrs   *Attributes
	Level    int
	Inlines  *InlineListNode // Heading text, possibly formatted
	Slug     string          // Heading text, normalized
	Fragment string          // Heading text, suitable to be used as an unique URL fragment
	Attrs    *Attributes
}

func (hn *HeadingNode) blockNode() { /* Just a marker */ }
func (hn *HeadingNode) itemNode()  { /* Just a marker */ }
func (*HeadingNode) blockNode() { /* Just a marker */ }
func (*HeadingNode) itemNode()  { /* Just a marker */ }

// WalkChildren walks the heading text.
func (hn *HeadingNode) WalkChildren(v Visitor) {
	WalkInlineSlice(v, hn.Inlines)
	Walk(v, hn.Inlines)
}

//--------------------------------------------------------------------------

// HRuleNode specifies a horizontal rule.
type HRuleNode struct {
	Attrs *Attributes
}

func (hn *HRuleNode) blockNode() { /* Just a marker */ }
func (hn *HRuleNode) itemNode()  { /* Just a marker */ }
func (*HRuleNode) blockNode() { /* Just a marker */ }
func (*HRuleNode) itemNode()  { /* Just a marker */ }

// WalkChildren does nothing.
func (hn *HRuleNode) WalkChildren(v Visitor) { /* No children*/ }
func (*HRuleNode) WalkChildren(Visitor) { /* No children*/ }

//--------------------------------------------------------------------------

// NestedListNode specifies a nestable list, either ordered or unordered.
type NestedListNode struct {
	Kind  NestedListKind
	Items []ItemSlice
131
132
133
134
135
136
137
138
139


140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157

158
159
160
161

162
163
164
165
166

167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185


186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204

205
206
207
208
209

210
211
212
213

214
215
216
217
218
219
220
221
222
223
224
225
226
227
228

229
230
231

151
152
153
154
155
156
157


158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176

177
178
179
180

181
182
183
184
185

186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203


204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223

224
225
226
227
228

229
230
231
232

233
234
235
236
237
238
239
240
241
242
243
244
245
246
247

248
249
250

251







-
-
+
+

















-
+



-
+




-
+

















-
-
+
+


















-
+




-
+



-
+














-
+


-
+
const (
	_                   NestedListKind = iota
	NestedListOrdered                  // Ordered list.
	NestedListUnordered                // Unordered list.
	NestedListQuote                    // Quote list.
)

func (ln *NestedListNode) blockNode() { /* Just a marker */ }
func (ln *NestedListNode) itemNode()  { /* Just a marker */ }
func (*NestedListNode) blockNode() { /* Just a marker */ }
func (*NestedListNode) itemNode()  { /* Just a marker */ }

// WalkChildren walks down the items.
func (ln *NestedListNode) WalkChildren(v Visitor) {
	for _, item := range ln.Items {
		WalkItemSlice(v, item)
	}
}

//--------------------------------------------------------------------------

// DescriptionListNode specifies a description list.
type DescriptionListNode struct {
	Descriptions []Description
}

// Description is one element of a description list.
type Description struct {
	Term         InlineSlice
	Term         *InlineListNode
	Descriptions []DescriptionSlice
}

func (dn *DescriptionListNode) blockNode() {}
func (*DescriptionListNode) blockNode() { /* Just a marker */ }

// WalkChildren walks down to the descriptions.
func (dn *DescriptionListNode) WalkChildren(v Visitor) {
	for _, desc := range dn.Descriptions {
		WalkInlineSlice(v, desc.Term)
		Walk(v, desc.Term)
		for _, dns := range desc.Descriptions {
			WalkDescriptionSlice(v, dns)
		}
	}
}

//--------------------------------------------------------------------------

// TableNode specifies a full table
type TableNode struct {
	Header TableRow    // The header row
	Align  []Alignment // Default column alignment
	Rows   []TableRow  // The slice of cell rows
}

// TableCell contains the data for one table cell
type TableCell struct {
	Align   Alignment   // Cell alignment
	Inlines InlineSlice // Cell content
	Align   Alignment       // Cell alignment
	Inlines *InlineListNode // Cell content
}

// TableRow is a slice of cells.
type TableRow []*TableCell

// Alignment specifies text alignment.
// Currently only for tables.
type Alignment int

// Constants for Alignment.
const (
	_            Alignment = iota
	AlignDefault           // Default alignment, inherited
	AlignLeft              // Left alignment
	AlignCenter            // Center the content
	AlignRight             // Right alignment
)

func (tn *TableNode) blockNode() { /* Just a marker */ }
func (*TableNode) blockNode() { /* Just a marker */ }

// WalkChildren walks down to the cells.
func (tn *TableNode) WalkChildren(v Visitor) {
	for _, cell := range tn.Header {
		WalkInlineSlice(v, cell.Inlines)
		Walk(v, cell.Inlines)
	}
	for _, row := range tn.Rows {
		for _, cell := range row {
			WalkInlineSlice(v, cell.Inlines)
			Walk(v, cell.Inlines)
		}
	}
}

//--------------------------------------------------------------------------

// BLOBNode contains just binary data that must be interpreted according to
// a syntax.
type BLOBNode struct {
	Title  string
	Syntax string
	Blob   []byte
}

func (bn *BLOBNode) blockNode() { /* Just a marker */ }
func (*BLOBNode) blockNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (bn *BLOBNode) WalkChildren(v Visitor) { /* No children*/ }
func (*BLOBNode) WalkChildren(Visitor) { /* No children*/ }

Changes to ast/inline.go.

9
10
11
12
13
14
15










































16
17
18
19
20
21

22
23
24

25
26
27
28
29
30
31
32
33

34
35
36

37
38
39
40
41
42
43
44
45

46
47
48

49
50
51
52
53
54
55
56
57

58
59
60

61
62
63
64
65
66
67
68
69



70
71
72

73
74
75
76



77
78
79
80
81
82
83



84
85
86
87


88
89
90

91
92
93
94





95
96
97
98
99
100
101
102
103



104
105
106

107
108
109
110



111
112
113
114
115
116
117
118
119



120
121
122

123
124
125

126
127
128
129
130
131
132


133
134
135

136
137
138
139

140
141
142
143
144
145
146
147
148

149
150
151
152
153
154
155
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

63
64
65

66
67
68
69
70
71
72
73
74

75
76
77

78
79
80
81
82
83
84
85
86

87
88
89

90
91
92
93
94
95
96
97
98

99
100
101

102
103
104
105
106
107
108



109
110
111
112
113

114
115
116
117

118
119
120
121
122
123
124



125
126
127




128
129
130
131

132
133



134
135
136
137
138
139
140
141
142
143
144



145
146
147
148
149

150
151
152
153

154
155
156
157
158
159
160
161
162
163
164

165
166
167
168
169

170
171
172

173
174
175
176
177
178


179
180
181
182

183
184
185
186

187
188
189
190
191
192
193
194
195

196
197
198
199
200
201
202
203







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+





-
+


-
+








-
+


-
+








-
+


-
+








-
+


-
+






-
-
-
+
+
+


-
+



-
+
+
+




-
-
-
+
+
+
-
-
-
-
+
+


-
+

-
-
-
+
+
+
+
+






-
-
-
+
+
+


-
+



-
+
+
+








-
+
+
+


-
+


-
+





-
-
+
+


-
+



-
+








-
+







//-----------------------------------------------------------------------------

// Package ast provides the abstract syntax tree.
package ast

// Definitions of inline nodes.

// InlineListNode is a list of BlockNodes.
type InlineListNode struct {
	List []InlineNode
}

func (*InlineListNode) inlineNode() { /* Just a marker */ }

// CreateInlineListNode make a new inline list node from nodes
func CreateInlineListNode(nodes ...InlineNode) *InlineListNode {
	return &InlineListNode{List: nodes}
}

// CreateInlineListNodeFromWords makes a new inline list from words,
// that will be space-separated.
func CreateInlineListNodeFromWords(words ...string) *InlineListNode {
	inl := make([]InlineNode, 0, 2*len(words)-1)
	for i, word := range words {
		if i > 0 {
			inl = append(inl, &SpaceNode{Lexeme: " "})
		}
		inl = append(inl, &TextNode{Text: word})
	}
	return &InlineListNode{List: inl}
}

// WalkChildren walks down to the descriptions.
func (iln *InlineListNode) WalkChildren(v Visitor) {
	for _, bn := range iln.List {
		Walk(v, bn)
	}
}

// IsEmpty returns true if the list has no elements.
func (iln *InlineListNode) IsEmpty() bool { return iln == nil || len(iln.List) == 0 }

// Append inline node(s) to the list.
func (iln *InlineListNode) Append(in ...InlineNode) {
	iln.List = append(iln.List, in...)
}

// --------------------------------------------------------------------------

// TextNode just contains some text.
type TextNode struct {
	Text string // The text itself.
}

func (tn *TextNode) inlineNode() { /* Just a marker */ }
func (*TextNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (tn *TextNode) WalkChildren(v Visitor) { /* No children*/ }
func (*TextNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// TagNode contains a tag.
type TagNode struct {
	Tag string // The text itself.
}

func (tn *TagNode) inlineNode() { /* Just a marker */ }
func (*TagNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (tn *TagNode) WalkChildren(v Visitor) { /* No children*/ }
func (*TagNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// SpaceNode tracks inter-word space characters.
type SpaceNode struct {
	Lexeme string
}

func (sn *SpaceNode) inlineNode() { /* Just a marker */ }
func (*SpaceNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (sn *SpaceNode) WalkChildren(v Visitor) { /* No children*/ }
func (*SpaceNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// BreakNode signals a new line that must / should be interpreted as a new line break.
type BreakNode struct {
	Hard bool // Hard line break?
}

func (bn *BreakNode) inlineNode() { /* Just a marker */ }
func (*BreakNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (bn *BreakNode) WalkChildren(v Visitor) { /* No children*/ }
func (*BreakNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// LinkNode contains the specified link.
type LinkNode struct {
	Ref     *Reference
	Inlines InlineSlice // The text associated with the link.
	OnlyRef bool        // True if no text was specified.
	Attrs   *Attributes // Optional attributes
	Inlines *InlineListNode // The text associated with the link.
	OnlyRef bool            // True if no text was specified.
	Attrs   *Attributes     // Optional attributes
}

func (ln *LinkNode) inlineNode() { /* Just a marker */ }
func (*LinkNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the link text.
func (ln *LinkNode) WalkChildren(v Visitor) {
	WalkInlineSlice(v, ln.Inlines)
	if iln := ln.Inlines; iln != nil {
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// ImageNode contains the specified image reference.
type ImageNode struct {
	Ref     *Reference  // Reference to image
// EmbedNode contains the specified embedded material.
type EmbedNode struct {
	Material MaterialNode    // The material to be embedded
	Blob    []byte      // BLOB data of the image, as an alternative to Ref.
	Syntax  string      // Syntax of Blob
	Inlines InlineSlice // The text associated with the image.
	Attrs   *Attributes // Optional attributes
	Inlines  *InlineListNode // Optional text associated with the image.
	Attrs    *Attributes     // Optional attributes
}

func (in *ImageNode) inlineNode() { /* Just a marker */ }
func (*EmbedNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the image text.
func (in *ImageNode) WalkChildren(v Visitor) {
	WalkInlineSlice(v, in.Inlines)
// WalkChildren walks to the text that describes the embedded material.
func (en *EmbedNode) WalkChildren(v Visitor) {
	if iln := en.Inlines; iln != nil {
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// CiteNode contains the specified citation.
type CiteNode struct {
	Key     string      // The citation key
	Inlines InlineSlice // The text associated with the citation.
	Attrs   *Attributes // Optional attributes
	Key     string          // The citation key
	Inlines *InlineListNode // Optional text associated with the citation.
	Attrs   *Attributes     // Optional attributes
}

func (cn *CiteNode) inlineNode() { /* Just a marker */ }
func (*CiteNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the cite text.
func (cn *CiteNode) WalkChildren(v Visitor) {
	WalkInlineSlice(v, cn.Inlines)
	if iln := cn.Inlines; iln != nil {
		Walk(v, iln)
	}
}

// --------------------------------------------------------------------------

// MarkNode contains the specified merked position.
// It is a BlockNode too, because although it is typically parsed during inline
// mode, it is moved into block mode afterwards.
type MarkNode struct {
	Text string
	Text     string
	Slug     string // Slugified form of Text
	Fragment string // Unique form of Slug
}

func (mn *MarkNode) inlineNode() { /* Just a marker */ }
func (*MarkNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (mn *MarkNode) WalkChildren(v Visitor) { /* No children*/ }
func (*MarkNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// FootnoteNode contains the specified footnote.
type FootnoteNode struct {
	Inlines InlineSlice // The footnote text.
	Attrs   *Attributes // Optional attributes
	Inlines *InlineListNode // The footnote text.
	Attrs   *Attributes     // Optional attributes
}

func (fn *FootnoteNode) inlineNode() { /* Just a marker */ }
func (*FootnoteNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the footnote text.
func (fn *FootnoteNode) WalkChildren(v Visitor) {
	WalkInlineSlice(v, fn.Inlines)
	Walk(v, fn.Inlines)
}

// --------------------------------------------------------------------------

// FormatNode specifies some inline formatting.
type FormatNode struct {
	Kind    FormatKind
	Attrs   *Attributes // Optional attributes.
	Inlines InlineSlice
	Inlines *InlineListNode
}

// FormatKind specifies the format that is applied to the inline nodes.
type FormatKind uint8

// Constants for FormatCode
const (
167
168
169
170
171
172
173
174

175
176
177
178

179
180
181
182
183
184
185
215
216
217
218
219
220
221

222
223
224
225

226
227
228
229
230
231
232
233







-
+



-
+







	FormatQuote                // Quoted text.
	FormatQuotation            // Quotation text.
	FormatSmall                // Smaller text.
	FormatSpan                 // Generic inline container.
	FormatMonospace            // Monospaced text.
)

func (fn *FormatNode) inlineNode() { /* Just a marker */ }
func (*FormatNode) inlineNode() { /* Just a marker */ }

// WalkChildren walks to the formatted text.
func (fn *FormatNode) WalkChildren(v Visitor) {
	WalkInlineSlice(v, fn.Inlines)
	Walk(v, fn.Inlines)
}

// --------------------------------------------------------------------------

// LiteralNode specifies some uninterpreted text.
type LiteralNode struct {
	Kind  LiteralKind
196
197
198
199
200
201
202
203

204
205
206

244
245
246
247
248
249
250

251
252
253

254







-
+


-
+
	LiteralProg                // Inline program code.
	LiteralKeyb                // Keyboard strokes.
	LiteralOutput              // Sample output.
	LiteralComment             // Inline comment
	LiteralHTML                // Inline HTML, e.g. for Markdown
)

func (ln *LiteralNode) inlineNode() { /* Just a marker */ }
func (*LiteralNode) inlineNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (ln *LiteralNode) WalkChildren(v Visitor) { /* No children*/ }
func (*LiteralNode) WalkChildren(Visitor) { /* No children*/ }

Added ast/material.go.












































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package ast provides the abstract syntax tree.
package ast

// MaterialNode references the various types of zettel material.
type MaterialNode interface {
	Node
	materialNode()
}

// --------------------------------------------------------------------------

// ReferenceMaterialNode is material that can be retrieved by using a reference.
type ReferenceMaterialNode struct {
	Ref *Reference
}

func (*ReferenceMaterialNode) materialNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*ReferenceMaterialNode) WalkChildren(Visitor) { /* No children*/ }

// --------------------------------------------------------------------------

// BLOBMaterialNode represents itself.
type BLOBMaterialNode struct {
	Blob   []byte // BLOB data itself.
	Syntax string // Syntax of Blob
}

func (*BLOBMaterialNode) materialNode() { /* Just a marker */ }

// WalkChildren does nothing.
func (*BLOBMaterialNode) WalkChildren(Visitor) { /* No children*/ }

Changes to ast/walk.go.

21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
21
22
23
24
25
26
27














28
29
30
31
32
33
34
35
36
37
38
39
40







-
-
-
-
-
-
-
-
-
-
-
-
-
-













	if v = v.Visit(node); v == nil {
		return
	}
	node.WalkChildren(v)
	v.Visit(nil)
}

// WalkBlockSlice traverse a block slice.
func WalkBlockSlice(v Visitor, bns BlockSlice) {
	for _, bn := range bns {
		Walk(v, bn)
	}
}

// WalkInlineSlice traverses an inline slice.
func WalkInlineSlice(v Visitor, ins InlineSlice) {
	for _, in := range ins {
		Walk(v, in)
	}
}

// WalkItemSlice traverses an item slice.
func WalkItemSlice(v Visitor, ins ItemSlice) {
	for _, in := range ins {
		Walk(v, in)
	}
}

// WalkDescriptionSlice traverses an item slice.
func WalkDescriptionSlice(v Visitor, dns DescriptionSlice) {
	for _, dn := range dns {
		Walk(v, dn)
	}
}

Changes to auth/policy/policy_test.go.

41
42
43
44
45
46
47
48
49


50
51
52
53
54
55
56
41
42
43
44
45
46
47


48
49
50
51
52
53
54
55
56







-
-
+
+







			readOnly: ts.readonly,
			withAuth: ts.withAuth,
		}
		pol := newPolicy(authzManager, &authConfig{ts.expert})
		name := fmt.Sprintf("readonly=%v/withauth=%v/expert=%v",
			ts.readonly, ts.withAuth, ts.expert)
		t.Run(name, func(tt *testing.T) {
			testCreate(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testRead(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testCreate(tt, pol, ts.withAuth, ts.readonly)
			testRead(tt, pol, ts.withAuth, ts.expert)
			testWrite(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testRename(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testDelete(tt, pol, ts.withAuth, ts.readonly, ts.expert)
		})
	}
}

90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
90
91
92
93
94
95
96

97
98
99
100
101
102
103
104







-
+







func (ac *authConfig) GetVisibility(m *meta.Meta) meta.Visibility {
	if vis, ok := m.Get(meta.KeyVisibility); ok {
		return meta.GetVisibility(vis)
	}
	return meta.VisibilityLogin
}

func testCreate(t *testing.T, pol auth.Policy, withAuth, readonly, isExpert bool) {
func testCreate(t *testing.T, pol auth.Policy, withAuth, readonly bool) {
	t.Helper()
	anonUser := newAnon()
	creator := newCreator()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()
137
138
139
140
141
142
143
144

145
146
147
148
149
150
151
137
138
139
140
141
142
143

144
145
146
147
148
149
150
151







-
+







			if tc.exp != got {
				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testRead(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {
func testRead(t *testing.T, pol auth.Policy, withAuth, expert bool) {
	t.Helper()
	anonUser := newAnon()
	creator := newCreator()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()

Changes to box/box.go.

39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66






67
68
69
70
71


72
73



74
75
76
77
78
79
80
39
40
41
42
43
44
45



46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76


77
78
79
80
81
82
83
84
85
86







-
-
-


















+
+
+
+
+
+





+
+
-
-
+
+
+








	// GetZettel retrieves a specific zettel.
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)

	// GetMeta retrieves just the meta data of a specific zettel.
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)

	// FetchZids returns the set of all zettel identifer managed by the box.
	FetchZids(ctx context.Context) (id.Set, error)

	// CanUpdateZettel returns true, if box could possibly update the given zettel.
	CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool

	// UpdateZettel updates an existing zettel.
	UpdateZettel(ctx context.Context, zettel domain.Zettel) error

	// AllowRenameZettel returns true, if box will not disallow renaming the zettel.
	AllowRenameZettel(ctx context.Context, zid id.Zid) bool

	// RenameZettel changes the current Zid to a new Zid.
	RenameZettel(ctx context.Context, curZid, newZid id.Zid) error

	// CanDeleteZettel returns true, if box could possibly delete the given zettel.
	CanDeleteZettel(ctx context.Context, zid id.Zid) bool

	// DeleteZettel removes the zettel from the box.
	DeleteZettel(ctx context.Context, zid id.Zid) error
}

// ZidFunc is a function that processes identifier of a zettel.
type ZidFunc func(id.Zid)

// MetaFunc is a function that processes metadata of a zettel.
type MetaFunc func(*meta.Meta)

// ManagedBox is the interface of managed boxes.
type ManagedBox interface {
	BaseBox

	// Apply identifier of every zettel to the given function.
	ApplyZid(context.Context, ZidFunc) error
	// SelectMeta returns all zettel meta data that match the selection criteria.
	SelectMeta(ctx context.Context, match search.MetaMatchFunc) ([]*meta.Meta, error)

	// Apply metadata of every zettel to the given function.
	ApplyMeta(context.Context, MetaFunc) error

	// ReadStats populates st with box statistics
	ReadStats(st *ManagedBoxStats)
}

// ManagedBoxStats records statistics about the box.
type ManagedBoxStats struct {
94
95
96
97
98
99
100



101
102
103
104
105
106
107
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116







+
+
+







	// Stop the started box. Now only the Start() function is allowed.
	Stop(ctx context.Context) error
}

// Box is to be used outside the box package and its descendants.
type Box interface {
	BaseBox

	// FetchZids returns the set of all zettel identifer managed by the box.
	FetchZids(ctx context.Context) (id.Set, error)

	// SelectMeta returns a list of metadata that comply to the given selection criteria.
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)

	// GetAllZettel retrieves a specific zettel from all managed boxes.
	GetAllZettel(ctx context.Context, zid id.Zid) ([]domain.Zettel, error)

Changes to box/compbox/compbox.go.

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
16
17
18
19
20
21
22

23
24
25
26
27
28
29







-







	"net/url"

	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register(
		" comp",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return getCompBox(cdata.Number, cdata.Enricher), nil
53
54
55
56
57
58
59
60

61
62

63
64
65

66
67
68
69

70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

86
87
88
89
90
91
92
93
94
95
96
97

98
99
100
101
102

103
104
105
106

107
108
109

110
111
112
113
114
115

116
117
118
119
120
121

122
123
124

125
126

127
128
129
130


131
132

133
134
135
136
137

138
139
140
141
142
143
144

145
146

147
148
149
150
151
152
153

154
155
156
157
158
159
160
52
53
54
55
56
57
58

59
60

61
62


63
64
65
66

67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

83
84
85
86
87
88
89
90
91
92
93
94

95

96
97
98

99
100
101
102

103
104
105

106
107
108
109
110
111

112


113
114
115

116
117
118

119


120




121
122


123
124
125
126
127

128
129
130
131
132
133
134

135
136

137
138
139
140
141
142
143

144
145
146
147
148
149
150
151







-
+

-
+

-
-
+



-
+















-
+











-
+
-



-
+



-
+


-
+





-
+
-
-



-
+


-
+
-
-
+
-
-
-
-
+
+
-
-
+




-
+






-
+

-
+






-
+







func getCompBox(boxNumber int, mf box.Enricher) box.ManagedBox {
	return &compBox{number: boxNumber, enricher: mf}
}

// Setup remembers important values.
func Setup(cfg *meta.Meta) { myConfig = cfg.Clone() }

func (pp *compBox) Location() string { return "" }
func (*compBox) Location() string { return "" }

func (pp *compBox) CanCreateZettel(ctx context.Context) bool { return false }
func (*compBox) CanCreateZettel(context.Context) bool { return false }

func (pp *compBox) CreateZettel(
	ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
func (*compBox) CreateZettel(context.Context, domain.Zettel) (id.Zid, error) {
	return id.Invalid, box.ErrReadOnly
}

func (pp *compBox) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
func (*compBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	if gen, ok := myZettel[zid]; ok && gen.meta != nil {
		if m := gen.meta(zid); m != nil {
			updateMeta(m)
			if genContent := gen.content; genContent != nil {
				return domain.Zettel{
					Meta:    m,
					Content: domain.NewContent(genContent(m)),
				}, nil
			}
			return domain.Zettel{Meta: m}, nil
		}
	}
	return domain.Zettel{}, box.ErrNotFound
}

func (pp *compBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
func (*compBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	if gen, ok := myZettel[zid]; ok {
		if genMeta := gen.meta; genMeta != nil {
			if m := genMeta(zid); m != nil {
				updateMeta(m)
				return m, nil
			}
		}
	}
	return nil, box.ErrNotFound
}

func (pp *compBox) FetchZids(ctx context.Context) (id.Set, error) {
func (*compBox) ApplyZid(_ context.Context, handle box.ZidFunc) error {
	result := id.NewSetCap(len(myZettel))
	for zid, gen := range myZettel {
		if genMeta := gen.meta; genMeta != nil {
			if genMeta(zid) != nil {
				result[zid] = true
				handle(zid)
			}
		}
	}
	return result, nil
	return nil
}

func (pp *compBox) SelectMeta(ctx context.Context, match search.MetaMatchFunc) (res []*meta.Meta, err error) {
func (pp *compBox) ApplyMeta(ctx context.Context, handle box.MetaFunc) error {
	for zid, gen := range myZettel {
		if genMeta := gen.meta; genMeta != nil {
			if m := genMeta(zid); m != nil {
				updateMeta(m)
				pp.enricher.Enrich(ctx, m, pp.number)
				if match(m) {
				handle(m)
					res = append(res, m)
				}
			}
		}
	}
	return res, nil
	return nil
}

func (pp *compBox) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
func (*compBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return false }
	return false
}


func (pp *compBox) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	return box.ErrReadOnly
}
func (*compBox) UpdateZettel(context.Context, domain.Zettel) error { return box.ErrReadOnly }


func (pp *compBox) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
func (*compBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	_, ok := myZettel[zid]
	return !ok
}

func (pp *compBox) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
func (*compBox) RenameZettel(_ context.Context, curZid, _ id.Zid) error {
	if _, ok := myZettel[curZid]; ok {
		return box.ErrReadOnly
	}
	return box.ErrNotFound
}

func (pp *compBox) CanDeleteZettel(ctx context.Context, zid id.Zid) bool { return false }
func (*compBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (pp *compBox) DeleteZettel(ctx context.Context, zid id.Zid) error {
func (*compBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	if _, ok := myZettel[zid]; ok {
		return box.ErrReadOnly
	}
	return box.ErrNotFound
}

func (pp *compBox) ReadStats(st *box.ManagedBoxStats) {
func (*compBox) ReadStats(st *box.ManagedBoxStats) {
	st.ReadOnly = true
	st.Zettel = len(myZettel)
}

func updateMeta(m *meta.Meta) {
	m.Set(meta.KeyNoIndex, meta.ValueTrue)
	m.Set(meta.KeySyntax, meta.ValueSyntaxZmk)

Changes to box/constbox/base.css.

252
253
254
255
256
257
258



259
260
261
262
263
264
265
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268







+
+
+







  }
  .zs-meta a {
    color:#444;
  }
  h1+.zs-meta {
    margin-top:-1rem;
  }
  nav > details {
    margin-top:1rem;
  }
  details > summary {
    width: 100%;
    background-color: #eee;
    font-family:sans-serif;
  }
  details > ul {
    margin-top:0;

Changes to box/constbox/base.mustache.

22
23
24
25
26
27
28
29

30
31
32
33
34
35
36
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36







-
+







{{#UserIsValid}}
<a href="{{{UserZettelURL}}}">{{UserIdent}}</a>
{{/UserIsValid}}
{{^UserIsValid}}
<a href="{{{LoginURL}}}">Login</a>
{{/UserIsValid}}
{{#UserIsValid}}
<a href="{{{UserLogoutURL}}}">Logout</a>
<a href="{{{LogoutURL}}}">Logout</a>
{{/UserIsValid}}
{{/WithAuth}}
</nav>
</div>
{{/WithUser}}
<div class="zs-dropdown">
<button>Lists</button>
47
48
49
50
51
52
53
54

55
56
57
58
59
60
61
62
63
64
65
66
47
48
49
50
51
52
53

54
55
56
57
58
59
60
61
62
63
64
65
66







-
+












{{#NewZettelLinks}}
<a href="{{{URL}}}">{{Text}}</a>
{{/NewZettelLinks}}
</nav>
</div>
{{/HasNewZettelLinks}}
<form action="{{{SearchURL}}}">
<input type="text" placeholder="Search.." name="s">
<input type="text" placeholder="Search.." name="{{QueryKeySearch}}">
</form>
</nav>
<main class="content">
{{{Content}}}
</main>
{{#FooterHTML}}
<footer>
{{{FooterHTML}}}
</footer>
{{/FooterHTML}}
</body>
</html>

Changes to box/constbox/constbox.go.

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

61
62

63
64

65
66

67
68
69
70

71
72

73
74
75
76
77

78
79

80
81
82
83
84

85
86
87

88
89

90
91
92

93
94

95
96

97
98

99
100

101
102
103

104
105

106
107
108
109


110
111

112
113
114
115
116

117
118
119
120
121
122

123
124

125
126
127
128
129
130
131
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39








40
41
42
43
44
45
46
47
48
49
50

51


52


53
54

55
56
57
58

59
60

61
62
63
64
65

66
67

68
69
70
71
72

73

74

75
76

77
78
79

80
81

82
83

84


85


86
87
88

89


90




91
92


93
94
95
96
97

98
99
100
101
102
103

104
105

106
107
108
109
110
111
112
113







-
















-
-
-
-
-
-
-
-











-
+
-
-
+
-
-
+

-
+



-
+

-
+




-
+

-
+




-
+
-

-
+

-
+


-
+

-
+

-
+
-
-
+
-
-
+


-
+
-
-
+
-
-
-
-
+
+
-
-
+




-
+





-
+

-
+







	"net/url"

	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register(
		" const",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return &constBox{
				number:   cdata.Number,
				zettel:   constZettelMap,
				enricher: cdata.Enricher,
			}, nil
		})
}

type constHeader map[string]string

func makeMeta(zid id.Zid, h constHeader) *meta.Meta {
	m := meta.New(zid)
	for k, v := range h {
		m.Set(k, v)
	}
	return m
}

type constZettel struct {
	header  constHeader
	content domain.Content
}

type constBox struct {
	number   int
	zettel   map[id.Zid]constZettel
	enricher box.Enricher
}

func (cp *constBox) Location() string {
func (*constBox) Location() string { return "const:" }
	return "const:"
}


func (cp *constBox) CanCreateZettel(ctx context.Context) bool { return false }
func (*constBox) CanCreateZettel(context.Context) bool { return false }

func (cp *constBox) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
func (*constBox) CreateZettel(context.Context, domain.Zettel) (id.Zid, error) {
	return id.Invalid, box.ErrReadOnly
}

func (cp *constBox) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
func (cp *constBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	if z, ok := cp.zettel[zid]; ok {
		return domain.Zettel{Meta: makeMeta(zid, z.header), Content: z.content}, nil
		return domain.Zettel{Meta: meta.NewWithData(zid, z.header), Content: z.content}, nil
	}
	return domain.Zettel{}, box.ErrNotFound
}

func (cp *constBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
func (cp *constBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	if z, ok := cp.zettel[zid]; ok {
		return makeMeta(zid, z.header), nil
		return meta.NewWithData(zid, z.header), nil
	}
	return nil, box.ErrNotFound
}

func (cp *constBox) FetchZids(ctx context.Context) (id.Set, error) {
func (cp *constBox) ApplyZid(_ context.Context, handle box.ZidFunc) error {
	result := id.NewSetCap(len(cp.zettel))
	for zid := range cp.zettel {
		result[zid] = true
		handle(zid)
	}
	return result, nil
	return nil
}

func (cp *constBox) SelectMeta(ctx context.Context, match search.MetaMatchFunc) (res []*meta.Meta, err error) {
func (cp *constBox) ApplyMeta(ctx context.Context, handle box.MetaFunc) error {
	for zid, zettel := range cp.zettel {
		m := makeMeta(zid, zettel.header)
		m := meta.NewWithData(zid, zettel.header)
		cp.enricher.Enrich(ctx, m, cp.number)
		if match(m) {
		handle(m)
			res = append(res, m)
		}
	}
	}
	return res, nil
	return nil
}

func (cp *constBox) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
func (*constBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return false }
	return false
}


func (cp *constBox) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	return box.ErrReadOnly
}
func (*constBox) UpdateZettel(context.Context, domain.Zettel) error { return box.ErrReadOnly }


func (cp *constBox) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
func (cp *constBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	_, ok := cp.zettel[zid]
	return !ok
}

func (cp *constBox) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
func (cp *constBox) RenameZettel(_ context.Context, curZid, _ id.Zid) error {
	if _, ok := cp.zettel[curZid]; ok {
		return box.ErrReadOnly
	}
	return box.ErrNotFound
}
func (cp *constBox) CanDeleteZettel(ctx context.Context, zid id.Zid) bool { return false }
func (*constBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (cp *constBox) DeleteZettel(ctx context.Context, zid id.Zid) error {
func (cp *constBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	if _, ok := cp.zettel[zid]; ok {
		return box.ErrReadOnly
	}
	return box.ErrNotFound
}

func (cp *constBox) ReadStats(st *box.ManagedBoxStats) {

Changes to box/constbox/home.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

16
17
18
19
20
21
22
1
2
3
4
5
6
7

8
9
10
11
12
13

14
15
16
17
18
19
20
21







-






-
+







=== Thank you for using Zettelstore!

You will find the lastest information about Zettelstore at [[https://zettelstore.de]].
Check that website regulary for [[upgrades|https://zettelstore.de/home/doc/trunk/www/download.wiki]] to the latest version.
You should consult the [[change log|https://zettelstore.de/home/doc/trunk/www/changes.wiki]] before upgrading.
Sometimes, you have to edit some of your Zettelstore-related zettel before upgrading.
Since Zettelstore is currently in a development state, every upgrade might fix some of your problems.
To check for versions, there is a zettel with the [[current version|00000000000001]] of your Zettelstore.

If you have problems concerning Zettelstore,
do not hesitate to get in [[contact with the main developer|mailto:ds@zettelstore.de]].

=== Reporting errors
If you have encountered an error, please include the content of the following zettel in your mail (if possible):
* [[Zettelstore Version|00000000000001]]
* [[Zettelstore Version|00000000000001]]: {{00000000000001}}
* [[Zettelstore Operating System|00000000000003]]
* [[Zettelstore Startup Configuration|00000000000096]]
* [[Zettelstore Runtime Configuration|00000000000100]]

Additionally, you have to describe, what you have done before that error occurs
and what you have expected instead.
Please do not forget to include the error message, if there is one.

Changes to box/constbox/info.mustache.

27
28
29
30
31
32
33
34

35
36

37

38

39
40
41











42
43
44
45
46
47
48
27
28
29
30
31
32
33

34
35

36
37
38

39
40
41

42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59







-
+

-
+

+
-
+


-
+
+
+
+
+
+
+
+
+
+
+







<ul>
{{#ExtLinks}}
<li><a href="{{{.}}}"{{{ExtNewWindow}}}>{{.}}</a></li>
{{/ExtLinks}}
</ul>
{{/HasExtLinks}}
{{/HasLinks}}
<h2>Parts and format</h3>
<h2>Parts and encodings</h2>
<table>
{{#Matrix}}
{{#EvalMatrix}}
<tr>
<th>{{Header}}</th>
{{#Elements}}{{#HasURL}}<td><a href="{{{URL}}}">{{Text}}</td>{{/HasURL}}{{^HasURL}}<th>{{Text}}</th>{{/HasURL}}
{{#Elements}}<td><a href="{{{URL}}}">{{Text}}</td>
{{/Elements}}
</tr>
{{/Matrix}}
{{/EvalMatrix}}
</table>
<h3>Parsed (not evaluated)</h3>
<table>
{{#ParseMatrix}}
<tr>
<th>{{Header}}</th>
{{#Elements}}<td><a href="{{{URL}}}">{{Text}}</td>
{{/Elements}}
</tr>
{{/ParseMatrix}}
</table>
{{#HasShadowLinks}}
<h2>Shadowed Boxes</h2>
<ul>{{#ShadowLinks}}<li>{{.}}</li>{{/ShadowLinks}}</ul>
{{/HasShadowLinks}}
{{#Endnotes}}{{{Endnotes}}}{{/Endnotes}}
</article>

Changes to box/constbox/login.mustache.

1
2
3
4
5
6
7
8

9
10
11
12
13
14
15
16
17
18
19
1
2
3
4
5
6
7

8
9
10
11
12
13
14
15
16
17
18
19







-
+











<article>
<header>
<h1>{{Title}}</h1>
</header>
{{#Retry}}
<div class="zs-indication zs-error">Wrong user name / password. Try again.</div>
{{/Retry}}
<form method="POST" action="?_format=html">
<form method="POST" action="">
<div>
<label for="username">User name</label>
<input class="zs-input" type="text" id="username" name="username" placeholder="Your user name.." autofocus>
</div>
<div>
<label for="password">Password</label>
<input class="zs-input" type="password" id="password" name="password" placeholder="Your password..">
</div>
<input class="zs-button" type="submit" value="Login">
</form>
</article>

Changes to box/constbox/zettel.mustache.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17













18

19
20
21
22
23
24
25
26

27
28
1
2
3
4
5
6
7
8
9
10
11

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41












-





+
+
+
+
+
+
+
+
+
+
+
+
+

+








+

-
<article>
<header>
<h1>{{{HTMLTitle}}}</h1>
<div class="zs-meta">
{{#CanWrite}}<a href="{{{EditURL}}}">Edit</a> &#183;{{/CanWrite}}
{{Zid}} &#183;
<a href="{{{InfoURL}}}">Info</a> &#183;
(<a href="{{{RoleURL}}}">{{RoleText}}</a>)
{{#HasTags}}&#183; {{#Tags}} <a href="{{{URL}}}">{{Text}}</a>{{/Tags}}{{/HasTags}}
{{#CanCopy}}&#183; <a href="{{{CopyURL}}}">Copy</a>{{/CanCopy}}
{{#CanFolge}}&#183; <a href="{{{FolgeURL}}}">Folge</a>{{/CanFolge}}
{{#FolgeRefs}}<br>Folge: {{{FolgeRefs}}}{{/FolgeRefs}}
{{#PrecursorRefs}}<br>Precursor: {{{PrecursorRefs}}}{{/PrecursorRefs}}
{{#HasExtURL}}<br>URL: <a href="{{{ExtURL}}}"{{{ExtNewWindow}}}>{{ExtURL}}</a>{{/HasExtURL}}
</div>
</header>
{{{Content}}}
</article>
{{#HasFolgeLinks}}
<nav>
<details open>
<summary>Folgezettel</summary>
<ul>
{{#FolgeLinks}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/FolgeLinks}}
</ul>
</details>
</nav>
{{/HasFolgeLinks}}
{{#HasBackLinks}}
<nav>
<details>
<summary>Additional links to this zettel</summary>
<ul>
{{#BackLinks}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/BackLinks}}
</ul>
</details>
</nav>
{{/HasBackLinks}}
</article>

Changes to box/dirbox/dirbox.go.

25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
25
26
27
28
29
30
31

32
33
34
35
36
37
38







-







	"zettelstore.de/z/box"
	"zettelstore.de/z/box/dirbox/directory"
	"zettelstore.de/z/box/filebox"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register("dir", func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
		path := getDirPath(u)
		if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {
			return nil, err
108
109
110
111
112
113
114
115

116
117
118
119
120

121
122
123
124
125
126
127
128
129
130
131

132
133
134
135
136
137
138
107
108
109
110
111
112
113

114
115
116
117
118

119
120
121
122
123
124
125
126
127
128
129

130
131
132
133
134
135
136
137







-
+




-
+










-
+







	mxCmds     sync.RWMutex
}

func (dp *dirBox) Location() string {
	return dp.location
}

func (dp *dirBox) Start(ctx context.Context) error {
func (dp *dirBox) Start(context.Context) error {
	dp.mxCmds.Lock()
	dp.fCmds = make([]chan fileCmd, 0, dp.fSrvs)
	for i := uint32(0); i < dp.fSrvs; i++ {
		cc := make(chan fileCmd)
		go fileService(i, cc)
		go fileService(cc)
		dp.fCmds = append(dp.fCmds, cc)
	}
	dp.setupDirService()
	dp.mxCmds.Unlock()
	if dp.dirSrv == nil {
		panic("No directory service")
	}
	return dp.dirSrv.Start()
}

func (dp *dirBox) Stop(ctx context.Context) error {
func (dp *dirBox) Stop(_ context.Context) error {
	dirSrv := dp.dirSrv
	dp.dirSrv = nil
	err := dirSrv.Stop()
	for _, c := range dp.fCmds {
		close(c)
	}
	return err
154
155
156
157
158
159
160
161

162
163
164
165

166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186

187
188
189
190
191
192
193
194
195

196
197
198
199
200

201
202
203
204
205
206
207
208
209

210
211
212
213

214
215
216

217
218
219
220

221
222

223
224
225

226
227
228

229
230
231
232
233
234
235


236
237
238

239
240

241
242
243

244
245
246
247


248
249

250
251

252
253
254
255

256
257
258
259
260
261
262
153
154
155
156
157
158
159

160
161
162
163

164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184

185
186
187
188
189
190
191
192
193

194
195
196
197
198

199
200
201
202
203
204
205
206
207

208
209
210
211

212
213
214

215
216

217

218
219

220
221
222

223
224
225

226
227

228
229
230


231
232

233

234
235

236



237




238
239


240


241
242
243
244

245
246
247
248
249
250
251
252







-
+



-
+




















-
+








-
+




-
+








-
+



-
+


-
+

-

-
+

-
+


-
+


-
+

-



-
-
+
+
-

-
+

-
+
-
-
-
+
-
-
-
-
+
+
-
-
+
-
-
+



-
+







	sum *= 16777619

	dp.mxCmds.RLock()
	defer dp.mxCmds.RUnlock()
	return dp.fCmds[sum%dp.fSrvs]
}

func (dp *dirBox) CanCreateZettel(ctx context.Context) bool {
func (dp *dirBox) CanCreateZettel(_ context.Context) bool {
	return !dp.readonly
}

func (dp *dirBox) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
func (dp *dirBox) CreateZettel(_ context.Context, zettel domain.Zettel) (id.Zid, error) {
	if dp.readonly {
		return id.Invalid, box.ErrReadOnly
	}

	entry, err := dp.dirSrv.GetNew()
	if err != nil {
		return id.Invalid, err
	}
	meta := zettel.Meta
	meta.Zid = entry.Zid
	dp.updateEntryFromMeta(entry, meta)

	err = setZettel(dp, entry, zettel)
	if err == nil {
		dp.dirSrv.UpdateEntry(entry)
	}
	dp.notifyChanged(box.OnUpdate, meta.Zid)
	return meta.Zid, err
}

func (dp *dirBox) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
func (dp *dirBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	entry, err := dp.dirSrv.GetEntry(zid)
	if err != nil || !entry.IsValid() {
		return domain.Zettel{}, box.ErrNotFound
	}
	m, c, err := getMetaContent(dp, entry, zid)
	if err != nil {
		return domain.Zettel{}, err
	}
	dp.cleanupMeta(ctx, m)
	dp.cleanupMeta(m)
	zettel := domain.Zettel{Meta: m, Content: domain.NewContent(c)}
	return zettel, nil
}

func (dp *dirBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
func (dp *dirBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	entry, err := dp.dirSrv.GetEntry(zid)
	if err != nil || !entry.IsValid() {
		return nil, box.ErrNotFound
	}
	m, err := getMeta(dp, entry, zid)
	if err != nil {
		return nil, err
	}
	dp.cleanupMeta(ctx, m)
	dp.cleanupMeta(m)
	return m, nil
}

func (dp *dirBox) FetchZids(ctx context.Context) (id.Set, error) {
func (dp *dirBox) ApplyZid(_ context.Context, handle box.ZidFunc) error {
	entries, err := dp.dirSrv.GetEntries()
	if err != nil {
		return nil, err
		return err
	}
	result := id.NewSetCap(len(entries))
	for _, entry := range entries {
		result[entry.Zid] = true
		handle(entry.Zid)
	}
	return result, nil
	return nil
}

func (dp *dirBox) SelectMeta(ctx context.Context, match search.MetaMatchFunc) (res []*meta.Meta, err error) {
func (dp *dirBox) ApplyMeta(ctx context.Context, handle box.MetaFunc) error {
	entries, err := dp.dirSrv.GetEntries()
	if err != nil {
		return nil, err
		return err
	}
	res = make([]*meta.Meta, 0, len(entries))
	// The following loop could be parallelized if needed for performance.
	for _, entry := range entries {
		m, err1 := getMeta(dp, entry, entry.Zid)
		err = err1
		if err != nil {
		if err1 != nil {
			return err1
			continue
		}
		dp.cleanupMeta(ctx, m)
		dp.cleanupMeta(m)
		dp.cdata.Enricher.Enrich(ctx, m, dp.number)

		handle(m)
		if match(m) {
			res = append(res, m)
		}
	}
	}
	if err != nil {
		return nil, err
	}
	return nil
}
	return res, nil
}


func (dp *dirBox) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
func (dp *dirBox) CanUpdateZettel(context.Context, domain.Zettel) bool {
	return !dp.readonly
}

func (dp *dirBox) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
func (dp *dirBox) UpdateZettel(_ context.Context, zettel domain.Zettel) error {
	if dp.readonly {
		return box.ErrReadOnly
	}

	meta := zettel.Meta
	if !meta.Zid.IsValid() {
		return &box.ErrInvalidID{Zid: meta.Zid}
315
316
317
318
319
320
321
322

323
324
325
326
327
328
329
305
306
307
308
309
310
311

312
313
314
315
316
317
318
319







-
+







		if s == syntax {
			return directory.MetaSpecHeader, "zettel"
		}
	}
	return directory.MetaSpecFile, syntax
}

func (dp *dirBox) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
func (dp *dirBox) AllowRenameZettel(context.Context, id.Zid) bool {
	return !dp.readonly
}

func (dp *dirBox) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	if curZid == newZid {
		return nil
	}
367
368
369
370
371
372
373
374

375
376
377
378
379
380
381
382

383
384
385
386
387
388
389
357
358
359
360
361
362
363

364
365
366
367
368
369
370
371

372
373
374
375
376
377
378
379







-
+







-
+







	if err == nil {
		dp.notifyChanged(box.OnDelete, curZid)
		dp.notifyChanged(box.OnUpdate, newZid)
	}
	return err
}

func (dp *dirBox) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
func (dp *dirBox) CanDeleteZettel(_ context.Context, zid id.Zid) bool {
	if dp.readonly {
		return false
	}
	entry, err := dp.dirSrv.GetEntry(zid)
	return err == nil && entry.IsValid()
}

func (dp *dirBox) DeleteZettel(ctx context.Context, zid id.Zid) error {
func (dp *dirBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	if dp.readonly {
		return box.ErrReadOnly
	}

	entry, err := dp.dirSrv.GetEntry(zid)
	if err != nil || !entry.IsValid() {
		return box.ErrNotFound
397
398
399
400
401
402
403
404

405
406
407
408
409
410
411
387
388
389
390
391
392
393

394
395
396
397
398
399
400
401







-
+







}

func (dp *dirBox) ReadStats(st *box.ManagedBoxStats) {
	st.ReadOnly = dp.readonly
	st.Zettel, _ = dp.dirSrv.NumEntries()
}

func (dp *dirBox) cleanupMeta(ctx context.Context, m *meta.Meta) {
func (dp *dirBox) cleanupMeta(m *meta.Meta) {
	if role, ok := m.Get(meta.KeyRole); !ok || role == "" {
		m.Set(meta.KeyRole, dp.cdata.Config.GetDefaultRole())
	}
	if syntax, ok := m.Get(meta.KeySyntax); !ok || syntax == "" {
		m.Set(meta.KeySyntax, dp.cdata.Config.GetDefaultSyntax())
	}
}

Changes to box/dirbox/service.go.

18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32







-
+







	"zettelstore.de/z/box/filebox"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
)

func fileService(num uint32, cmds <-chan fileCmd) {
func fileService(cmds <-chan fileCmd) {
	for cmd := range cmds {
		cmd.run()
	}
}

type fileCmd interface {
	run()

Changes to box/filebox/zipbox.go.

19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
19
20
21
22
23
24
25

26
27
28
29
30
31
32







-







	"strings"

	"zettelstore.de/z/box"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/search"
)

var validFileName = regexp.MustCompile(`^(\d{14}).*(\.(.+))$`)

func matchValidFileName(name string) []string {
	return validFileName.FindStringSubmatch(name)
}
49
50
51
52
53
54
55
56

57
58
59
60
61
62
63
48
49
50
51
52
53
54

55
56
57
58
59
60
61
62







-
+







func (zp *zipBox) Location() string {
	if strings.HasPrefix(zp.name, "/") {
		return "file://" + zp.name
	}
	return "file:" + zp.name
}

func (zp *zipBox) Start(ctx context.Context) error {
func (zp *zipBox) Start(context.Context) error {
	reader, err := zip.OpenReader(zp.name)
	if err != nil {
		return err
	}
	defer reader.Close()
	zp.zettel = make(map[id.Zid]*zipEntry)
	for _, f := range reader.File {
94
95
96
97
98
99
100
101

102
103
104
105
106

107
108

109
110
111
112

113
114
115
116
117
118
119
93
94
95
96
97
98
99

100
101
102
103
104

105
106

107
108
109
110

111
112
113
114
115
116
117
118







-
+




-
+

-
+



-
+







		if entry.contentExt == "" {
			entry.contentExt = ext
			entry.contentName = name
		}
	}
}

func (zp *zipBox) Stop(ctx context.Context) error {
func (zp *zipBox) Stop(context.Context) error {
	zp.zettel = nil
	return nil
}

func (zp *zipBox) CanCreateZettel(ctx context.Context) bool { return false }
func (*zipBox) CanCreateZettel(context.Context) bool { return false }

func (zp *zipBox) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
func (*zipBox) CreateZettel(context.Context, domain.Zettel) (id.Zid, error) {
	return id.Invalid, box.ErrReadOnly
}

func (zp *zipBox) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
func (zp *zipBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	entry, ok := zp.zettel[zid]
	if !ok {
		return domain.Zettel{}, box.ErrNotFound
	}
	reader, err := zip.OpenReader(zp.name)
	if err != nil {
		return domain.Zettel{}, err
144
145
146
147
148
149
150
151

152
153
154
155
156
157
158
159
160
161
162
163
164

165
166
167

168
169

170
171
172

173
174
175

176
177
178
179
180
181
182
183
184

185
186

187
188

189
190
191

192
193

194
195
196
197


198
199

200
201
202
203
204

205
206
207
208
209
210
211

212
213

214
215
216
217
218
219
220
143
144
145
146
147
148
149

150
151
152
153
154
155
156
157
158
159
160
161
162

163

164

165
166

167
168
169

170
171
172

173
174
175
176
177
178
179
180
181

182


183


184
185
186

187


188




189
190


191
192
193
194
195

196
197
198
199
200
201
202

203
204

205
206
207
208
209
210
211
212







-
+












-
+
-

-
+

-
+


-
+


-
+








-
+
-
-
+
-
-
+


-
+
-
-
+
-
-
-
-
+
+
-
-
+




-
+






-
+

-
+







	} else {
		m = CalcDefaultMeta(zid, entry.contentExt)
	}
	CleanupMeta(m, zid, entry.contentExt, inMeta, false)
	return domain.Zettel{Meta: m, Content: domain.NewContent(src)}, nil
}

func (zp *zipBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
func (zp *zipBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	entry, ok := zp.zettel[zid]
	if !ok {
		return nil, box.ErrNotFound
	}
	reader, err := zip.OpenReader(zp.name)
	if err != nil {
		return nil, err
	}
	defer reader.Close()
	return readZipMeta(reader, zid, entry)
}

func (zp *zipBox) FetchZids(ctx context.Context) (id.Set, error) {
func (zp *zipBox) ApplyZid(_ context.Context, handle box.ZidFunc) error {
	result := id.NewSetCap(len(zp.zettel))
	for zid := range zp.zettel {
		result[zid] = true
		handle(zid)
	}
	return result, nil
	return nil
}

func (zp *zipBox) SelectMeta(ctx context.Context, match search.MetaMatchFunc) (res []*meta.Meta, err error) {
func (zp *zipBox) ApplyMeta(ctx context.Context, handle box.MetaFunc) error {
	reader, err := zip.OpenReader(zp.name)
	if err != nil {
		return nil, err
		return err
	}
	defer reader.Close()
	for zid, entry := range zp.zettel {
		m, err := readZipMeta(reader, zid, entry)
		if err != nil {
			continue
		}
		zp.enricher.Enrich(ctx, m, zp.number)
		if match(m) {
		handle(m)
			res = append(res, m)
		}
	}
	}
	return res, nil
	return nil
}

func (zp *zipBox) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
func (*zipBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return false }
	return false
}


func (zp *zipBox) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	return box.ErrReadOnly
}
func (*zipBox) UpdateZettel(context.Context, domain.Zettel) error { return box.ErrReadOnly }


func (zp *zipBox) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
func (zp *zipBox) AllowRenameZettel(_ context.Context, zid id.Zid) bool {
	_, ok := zp.zettel[zid]
	return !ok
}

func (zp *zipBox) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
func (zp *zipBox) RenameZettel(_ context.Context, curZid, _ id.Zid) error {
	if _, ok := zp.zettel[curZid]; ok {
		return box.ErrReadOnly
	}
	return box.ErrNotFound
}

func (zp *zipBox) CanDeleteZettel(ctx context.Context, zid id.Zid) bool { return false }
func (*zipBox) CanDeleteZettel(context.Context, id.Zid) bool { return false }

func (zp *zipBox) DeleteZettel(ctx context.Context, zid id.Zid) error {
func (zp *zipBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	if _, ok := zp.zettel[zid]; ok {
		return box.ErrReadOnly
	}
	return box.ErrNotFound
}

func (zp *zipBox) ReadStats(st *box.ManagedBoxStats) {

Changes to box/manager/box.go.

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
10
11
12
13
14
15
16

17
18
19
20
21
22
23







-








// Package manager coordinates the various boxes and indexes of a Zettelstore.
package manager

import (
	"context"
	"errors"
	"sort"
	"strings"

	"zettelstore.de/z/box"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
125
126
127
128
129
130
131
132

133
134
135
136
137

138
139

140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167

168
169
170
171
172




173
174
175
176





177
178

179
180


181
182






183
184
185
186
187
188
189
124
125
126
127
128
129
130

131
132
133
134
135
136
137
138

139
140
141
142












143
144
145
146
147
148
149
150
151
152
153
154

155
156




157
158
159
160
161



162
163
164
165
166
167

168
169
170
171
172


173
174
175
176
177
178
179
180
181
182
183
184
185







-
+





+

-
+



-
-
-
-
-
-
-
-
-
-
-
-












-
+

-
-
-
-
+
+
+
+

-
-
-
+
+
+
+
+

-
+


+
+
-
-
+
+
+
+
+
+







			result = append(result, m)
		}
	}
	return result, nil
}

// FetchZids returns the set of all zettel identifer managed by the box.
func (mgr *Manager) FetchZids(ctx context.Context) (result id.Set, err error) {
func (mgr *Manager) FetchZids(ctx context.Context) (id.Set, error) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return nil, box.ErrStopped
	}
	result := id.Set{}
	for _, p := range mgr.boxes {
		zids, err := p.FetchZids(ctx)
		err := p.ApplyZid(ctx, func(zid id.Zid) { result[zid] = true })
		if err != nil {
			return nil, err
		}
		if result == nil {
			result = zids
		} else if len(result) <= len(zids) {
			for zid := range result {
				zids[zid] = true
			}
			result = zids
		} else {
			for zid := range zids {
				result[zid] = true
			}
		}
	}
	return result, nil
}

// SelectMeta returns all zettel meta data that match the selection
// criteria. The result is ordered by descending zettel id.
func (mgr *Manager) SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return nil, box.ErrStopped
	}
	var result []*meta.Meta
	selected, rejected := map[id.Zid]*meta.Meta{}, id.Set{}
	match := s.CompileMatch(mgr)
	for _, p := range mgr.boxes {
		selected, err := p.SelectMeta(ctx, match)
		if err != nil {
			return nil, err
	handleMeta := func(m *meta.Meta) {
		zid := m.Zid
		if rejected[zid] {
			return
		}
		sort.Slice(selected, func(i, j int) bool { return selected[i].Zid > selected[j].Zid })
		if len(result) == 0 {
			result = selected
		if _, ok := selected[zid]; ok {
			return
		}
		if match(m) {
			selected[zid] = m
		} else {
			result = box.MergeSorted(result, selected)
			rejected[zid] = true
		}
	}
	for _, p := range mgr.boxes {
		if err := p.ApplyMeta(ctx, handleMeta); err != nil {
	if s == nil {
		return result, nil
			return nil, err
		}
	}
	result := make([]*meta.Meta, 0, len(selected))
	for _, m := range selected {
		result = append(result, m)
	}
	return s.Sort(result), nil
}

// CanUpdateZettel returns true, if box could possibly update the given zettel.
func (mgr *Manager) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
	mgr.mgrMx.RLock()

Changes to box/manager/collect.go.

20
21
22
23
24
25
26

27
28
29
30
31
32

33
34
35
36

37
38
39
40


41
42
43
44
45
46
47
48
49
50
51
52

53
54
55
56




57
58
59
60
61
62
63
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

38
39
40


41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57


58
59
60
61
62
63
64
65
66
67
68







+






+



-
+


-
-
+
+












+


-
-
+
+
+
+







	"zettelstore.de/z/strfun"
)

type collectData struct {
	refs  id.Set
	words store.WordSet
	urls  store.WordSet
	itags store.WordSet
}

func (data *collectData) initialize() {
	data.refs = id.NewSet()
	data.words = store.NewWordSet()
	data.urls = store.NewWordSet()
	data.itags = store.NewWordSet()
}

func collectZettelIndexData(zn *ast.ZettelNode, data *collectData) {
	ast.WalkBlockSlice(data, zn.Ast)
	ast.Walk(data, zn.Ast)
}

func collectInlineIndexData(ins ast.InlineSlice, data *collectData) {
	ast.WalkInlineSlice(data, ins)
func collectInlineIndexData(iln *ast.InlineListNode, data *collectData) {
	ast.Walk(data, iln)
}

func (data *collectData) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.VerbatimNode:
		for _, line := range n.Lines {
			data.addText(line)
		}
	case *ast.TextNode:
		data.addText(n.Text)
	case *ast.TagNode:
		data.addText(n.Tag)
		data.itags.Add("#" + strings.ToLower(n.Tag))
	case *ast.LinkNode:
		data.addRef(n.Ref)
	case *ast.ImageNode:
		data.addRef(n.Ref)
	case *ast.EmbedNode:
		if m, ok := n.Material.(*ast.ReferenceMaterialNode); ok {
			data.addRef(m.Ref)
		}
	case *ast.LiteralNode:
		data.addText(n.Text)
	}
	return data
}

func (data *collectData) addText(s string) {

Changes to box/manager/indexer.go.

193
194
195
196
197
198
199

200
201
202
203
204
205
206
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207







+







			zi.AddBackRef(ref)
		} else {
			zi.AddDeadRef(ref)
		}
	}
	zi.SetWords(cData.words)
	zi.SetUrls(cData.urls)
	zi.SetITags(cData.itags)
}

func (mgr *Manager) idxUpdateValue(ctx context.Context, inverseKey, value string, zi *store.ZettelIndex) {
	zid, err := id.Parse(value)
	if err != nil {
		return
	}

Changes to box/manager/memstore/memstore.go.

32
33
34
35
36
37
38

39
40
41
42
43
44
45

46
47
48
49
50
51
52
32
33
34
35
36
37
38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53







+






-
+







type zettelIndex struct {
	dead     id.Slice
	forward  id.Slice
	backward id.Slice
	meta     map[string]metaRefs
	words    []string
	urls     []string
	itags    string // Inline tags
}

func (zi *zettelIndex) isEmpty() bool {
	if len(zi.forward) > 0 || len(zi.backward) > 0 || len(zi.dead) > 0 || len(zi.words) > 0 {
		return false
	}
	return zi.meta == nil || len(zi.meta) == 0
	return len(zi.meta) == 0
}

type stringRefs map[string]id.Slice

type memStore struct {
	mx    sync.RWMutex
	idx   map[id.Zid]*zettelIndex
64
65
66
67
68
69
70
71
72


73
74
75
76
77
78
79

80
81
82
83
84
85
86
65
66
67
68
69
70
71


72
73
74
75
76
77
78
79

80
81
82
83
84
85
86
87







-
-
+
+






-
+







		idx:   make(map[id.Zid]*zettelIndex),
		dead:  make(map[id.Zid]id.Slice),
		words: make(stringRefs),
		urls:  make(stringRefs),
	}
}

func (ms *memStore) Enrich(ctx context.Context, m *meta.Meta) {
	if ms.doEnrich(ctx, m) {
func (ms *memStore) Enrich(_ context.Context, m *meta.Meta) {
	if ms.doEnrich(m) {
		ms.mx.Lock()
		ms.updates++
		ms.mx.Unlock()
	}
}

func (ms *memStore) doEnrich(ctx context.Context, m *meta.Meta) bool {
func (ms *memStore) doEnrich(m *meta.Meta) bool {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	zi, ok := ms.idx[m.Zid]
	if !ok {
		return false
	}
	var updated bool
106
107
108
109
110
111
112











113
114
115
116
117
118
119
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131







+
+
+
+
+
+
+
+
+
+
+







				updated = true
			}
		}
	}
	if len(back) > 0 {
		m.Set(meta.KeyBack, back.String())
		updated = true
	}
	if zi.itags != "" {
		if tags, ok := m.Get(meta.KeyTags); ok {
			m.Set(meta.KeyAllTags, tags+" "+zi.itags)
		} else {
			m.Set(meta.KeyAllTags, zi.itags)
		}
		updated = true
	} else if tags, ok := m.Get(meta.KeyTags); ok {
		m.Set(meta.KeyAllTags, tags)
		updated = true
	}
	return updated
}

// SearchEqual returns all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SearchEqual(word string) id.Set {
252
253
254
255
256
257
258
259

260
261
262
263
264
265
266
264
265
266
267
268
269
270

271
272
273
274
275
276
277
278







-
+







				}
			}
		}
	}
	return back
}

func (ms *memStore) UpdateReferences(ctx context.Context, zidx *store.ZettelIndex) id.Set {
func (ms *memStore) UpdateReferences(_ context.Context, zidx *store.ZettelIndex) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()
	zi, ziExist := ms.idx[zidx.Zid]
	if !ziExist || zi == nil {
		zi = &zettelIndex{}
		ziExist = false
	}
274
275
276
277
278
279
280

281
282
283
284
285
286
287
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300







+







	}

	ms.updateDeadReferences(zidx, zi)
	ms.updateForwardBackwardReferences(zidx, zi)
	ms.updateMetadataReferences(zidx, zi)
	zi.words = updateWordSet(zidx.Zid, ms.words, zi.words, zidx.GetWords())
	zi.urls = updateWordSet(zidx.Zid, ms.urls, zi.urls, zidx.GetUrls())
	zi.itags = setITags(zidx.GetITags())

	// Check if zi must be inserted into ms.idx
	if !ziExist && !zi.isEmpty() {
		ms.idx[zidx.Zid] = zi
	}

	return toCheck
366
367
368
369
370
371
372









373
374
375
376
377
378
379
380
381
382
383
384

385
386
387
388
389
390
391
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405

406
407
408
409
410
411
412
413







+
+
+
+
+
+
+
+
+











-
+







			delete(srefs, word)
			continue
		}
		srefs[word] = refs2
	}
	return next.Words()
}

func setITags(next store.WordSet) string {
	itags := next.Words()
	if len(itags) == 0 {
		return ""
	}
	sort.Strings(itags)
	return strings.Join(itags, " ")
}

func (ms *memStore) getEntry(zid id.Zid) *zettelIndex {
	// Must only be called if ms.mx is write-locked!
	if zi, ok := ms.idx[zid]; ok {
		return zi
	}
	zi := &zettelIndex{}
	ms.idx[zid] = zi
	return zi
}

func (ms *memStore) DeleteZettel(ctx context.Context, zid id.Zid) id.Set {
func (ms *memStore) DeleteZettel(_ context.Context, zid id.Zid) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()

	zi, ok := ms.idx[zid]
	if !ok {
		return nil
	}

Changes to box/manager/store/zettel.go.

17
18
19
20
21
22
23

24
25
26
27
28
29
30
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31







+







type ZettelIndex struct {
	Zid      id.Zid            // zid of the indexed zettel
	backrefs id.Set            // set of back references
	metarefs map[string]id.Set // references to inverse keys
	deadrefs id.Set            // set of dead references
	words    WordSet
	urls     WordSet
	itags    WordSet
}

// NewZettelIndex creates a new zettel index.
func NewZettelIndex(zid id.Zid) *ZettelIndex {
	return &ZettelIndex{
		Zid:      zid,
		backrefs: id.NewSet(),
56
57
58
59
60
61
62



63
64
65
66
67
68
69
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73







+
+
+








// SetWords sets the words to the given value.
func (zi *ZettelIndex) SetWords(words WordSet) { zi.words = words }

// SetUrls sets the words to the given value.
func (zi *ZettelIndex) SetUrls(urls WordSet) { zi.urls = urls }

// SetITags sets the words to the given value.
func (zi *ZettelIndex) SetITags(itags WordSet) { zi.itags = itags }

// GetDeadRefs returns all dead references as a sorted list.
func (zi *ZettelIndex) GetDeadRefs() id.Slice {
	return zi.deadrefs.Sorted()
}

// GetBackRefs returns all back references as a sorted list.
func (zi *ZettelIndex) GetBackRefs() id.Slice {
83
84
85
86
87
88
89



87
88
89
90
91
92
93
94
95
96







+
+
+
}

// GetWords returns a reference to the set of words. It must not be modified.
func (zi *ZettelIndex) GetWords() WordSet { return zi.words }

// GetUrls returns a reference to the set of URLs. It must not be modified.
func (zi *ZettelIndex) GetUrls() WordSet { return zi.urls }

// GetITags returns a reference to the set of internal tags. It must not be modified.
func (zi *ZettelIndex) GetITags() WordSet { return zi.itags }

Changes to box/membox/membox.go.

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
17
18
19
20
21
22
23

24
25
26
27
28
29
30







-







	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register(
		"mem",
		func(u *url.URL, cdata *manager.ConnectData) (box.ManagedBox, error) {
			return &memBox{u: u, cdata: *cdata}, nil
45
46
47
48
49
50
51
52

53
54
55
56
57
58
59

60
61
62
63
64
65
66

67
68

69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

88
89
90
91
92
93
94
95
96
97
98

99
100
101
102
103
104
105
106
107
108

109
110

111
112

113
114
115

116
117
118

119
120

121
122
123
124

125
126

127
128
129

130
131
132

133
134

135
136

137
138
139
140
141
142
143
144
145
146
147
148
149

150
151

152
153
154
155
156
157
158
44
45
46
47
48
49
50

51
52
53
54
55
56
57

58
59
60
61
62
63
64

65
66

67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

86
87
88
89
90
91
92
93
94
95
96

97
98
99
100
101
102
103
104
105
106

107
108

109
110

111
112


113
114
115

116

117
118
119
120
121

122


123



124
125
126

127


128


129
130
131
132
133
134
135
136
137
138
139
140
141

142
143

144
145
146
147
148
149
150
151







-
+






-
+






-
+

-
+


















-
+










-
+









-
+

-
+

-
+

-
-
+


-
+
-

+



-
+
-
-
+
-
-
-
+


-
+
-
-
+
-
-
+












-
+

-
+







	}
}

func (mp *memBox) Location() string {
	return mp.u.String()
}

func (mp *memBox) Start(ctx context.Context) error {
func (mp *memBox) Start(context.Context) error {
	mp.mx.Lock()
	mp.zettel = make(map[id.Zid]domain.Zettel)
	mp.mx.Unlock()
	return nil
}

func (mp *memBox) Stop(ctx context.Context) error {
func (mp *memBox) Stop(context.Context) error {
	mp.mx.Lock()
	mp.zettel = nil
	mp.mx.Unlock()
	return nil
}

func (mp *memBox) CanCreateZettel(ctx context.Context) bool { return true }
func (*memBox) CanCreateZettel(context.Context) bool { return true }

func (mp *memBox) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
func (mp *memBox) CreateZettel(_ context.Context, zettel domain.Zettel) (id.Zid, error) {
	mp.mx.Lock()
	zid, err := box.GetNewZid(func(zid id.Zid) (bool, error) {
		_, ok := mp.zettel[zid]
		return !ok, nil
	})
	if err != nil {
		mp.mx.Unlock()
		return id.Invalid, err
	}
	meta := zettel.Meta.Clone()
	meta.Zid = zid
	zettel.Meta = meta
	mp.zettel[zid] = zettel
	mp.mx.Unlock()
	mp.notifyChanged(box.OnUpdate, zid)
	return zid, nil
}

func (mp *memBox) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
func (mp *memBox) GetZettel(_ context.Context, zid id.Zid) (domain.Zettel, error) {
	mp.mx.RLock()
	zettel, ok := mp.zettel[zid]
	mp.mx.RUnlock()
	if !ok {
		return domain.Zettel{}, box.ErrNotFound
	}
	zettel.Meta = zettel.Meta.Clone()
	return zettel, nil
}

func (mp *memBox) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
func (mp *memBox) GetMeta(_ context.Context, zid id.Zid) (*meta.Meta, error) {
	mp.mx.RLock()
	zettel, ok := mp.zettel[zid]
	mp.mx.RUnlock()
	if !ok {
		return nil, box.ErrNotFound
	}
	return zettel.Meta.Clone(), nil
}

func (mp *memBox) FetchZids(ctx context.Context) (id.Set, error) {
func (mp *memBox) ApplyZid(_ context.Context, handle box.ZidFunc) error {
	mp.mx.RLock()
	result := id.NewSetCap(len(mp.zettel))
	defer mp.mx.RUnlock()
	for zid := range mp.zettel {
		result[zid] = true
		handle(zid)
	}
	mp.mx.RUnlock()
	return result, nil
	return nil
}

func (mp *memBox) SelectMeta(ctx context.Context, match search.MetaMatchFunc) ([]*meta.Meta, error) {
func (mp *memBox) ApplyMeta(ctx context.Context, handle box.MetaFunc) error {
	result := make([]*meta.Meta, 0, len(mp.zettel))
	mp.mx.RLock()
	defer mp.mx.RUnlock()
	for _, zettel := range mp.zettel {
		m := zettel.Meta.Clone()
		mp.cdata.Enricher.Enrich(ctx, m, mp.cdata.Number)
		if match(m) {
		handle(m)
			result = append(result, m)
		}
	}
	}
	mp.mx.RUnlock()
	return result, nil
	return nil
}

func (mp *memBox) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
func (*memBox) CanUpdateZettel(context.Context, domain.Zettel) bool { return true }
	return true
}


func (mp *memBox) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
func (mp *memBox) UpdateZettel(_ context.Context, zettel domain.Zettel) error {
	mp.mx.Lock()
	meta := zettel.Meta.Clone()
	if !meta.Zid.IsValid() {
		return &box.ErrInvalidID{Zid: meta.Zid}
	}
	zettel.Meta = meta
	mp.zettel[meta.Zid] = zettel
	mp.mx.Unlock()
	mp.notifyChanged(box.OnUpdate, meta.Zid)
	return nil
}

func (mp *memBox) AllowRenameZettel(ctx context.Context, zid id.Zid) bool { return true }
func (*memBox) AllowRenameZettel(context.Context, id.Zid) bool { return true }

func (mp *memBox) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
func (mp *memBox) RenameZettel(_ context.Context, curZid, newZid id.Zid) error {
	mp.mx.Lock()
	zettel, ok := mp.zettel[curZid]
	if !ok {
		mp.mx.Unlock()
		return box.ErrNotFound
	}

169
170
171
172
173
174
175
176

177
178
179
180
181
182
183

184
185
186
187
188
189
190
162
163
164
165
166
167
168

169
170
171
172
173
174
175

176
177
178
179
180
181
182
183







-
+






-
+







	delete(mp.zettel, curZid)
	mp.mx.Unlock()
	mp.notifyChanged(box.OnDelete, curZid)
	mp.notifyChanged(box.OnUpdate, newZid)
	return nil
}

func (mp *memBox) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
func (mp *memBox) CanDeleteZettel(_ context.Context, zid id.Zid) bool {
	mp.mx.RLock()
	_, ok := mp.zettel[zid]
	mp.mx.RUnlock()
	return ok
}

func (mp *memBox) DeleteZettel(ctx context.Context, zid id.Zid) error {
func (mp *memBox) DeleteZettel(_ context.Context, zid id.Zid) error {
	mp.mx.Lock()
	if _, ok := mp.zettel[zid]; !ok {
		mp.mx.Unlock()
		return box.ErrNotFound
	}
	delete(mp.zettel, zid)
	mp.mx.Unlock()

Deleted box/merge.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46














































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package box provides a generic interface to zettel boxes.
package box

import "zettelstore.de/z/domain/meta"

// MergeSorted returns a merged sequence of metadata, sorted by Zid.
// The lists first and second must be sorted descending by Zid.
func MergeSorted(first, second []*meta.Meta) []*meta.Meta {
	lenFirst := len(first)
	lenSecond := len(second)
	result := make([]*meta.Meta, 0, lenFirst+lenSecond)
	iFirst := 0
	iSecond := 0
	for iFirst < lenFirst && iSecond < lenSecond {
		zidFirst := first[iFirst].Zid
		zidSecond := second[iSecond].Zid
		if zidFirst > zidSecond {
			result = append(result, first[iFirst])
			iFirst++
		} else if zidFirst < zidSecond {
			result = append(result, second[iSecond])
			iSecond++
		} else { // zidFirst == zidSecond
			result = append(result, first[iFirst])
			iFirst++
			iSecond++
		}
	}
	if iFirst < lenFirst {
		result = append(result, first[iFirst:]...)
	} else {
		result = append(result, second[iSecond:]...)
	}

	return result
}

Changes to client/client.go.

13
14
15
16
17
18
19

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45












46
47
48
49
50
51
52

53
54
55
56
57
58
59
60
61

62
63
64
65
66
67
68
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

65
66
67
68
69
70
71
72


73
74
75
76
77
78
79
80







+


















+







-
+
+
+
+
+
+
+
+
+
+
+
+






-
+







-
-
+








import (
	"bytes"
	"context"
	"encoding/json"
	"errors"
	"io"
	"net"
	"net/http"
	"net/url"
	"strconv"
	"strings"
	"time"

	"zettelstore.de/z/api"
	"zettelstore.de/z/domain/id"
)

// Client contains all data to execute requests.
type Client struct {
	baseURL   string
	username  string
	password  string
	token     string
	tokenType string
	expires   time.Time
	client    http.Client
}

// NewClient create a new client.
func NewClient(baseURL string) *Client {
	if !strings.HasSuffix(baseURL, "/") {
		baseURL += "/"
	}
	c := Client{baseURL: baseURL}
	c := Client{
		baseURL: baseURL,
		client: http.Client{
			Timeout: 10 * time.Second,
			Transport: &http.Transport{
				DialContext: (&net.Dialer{
					Timeout: 5 * time.Second, // TCP connect timeout
				}).DialContext,
				TLSHandshakeTimeout: 5 * time.Second,
			},
		},
	}
	return &c
}

func (c *Client) newURLBuilder(key byte) *api.URLBuilder {
	return api.NewURLBuilder(c.baseURL, key)
}
func (c *Client) newRequest(ctx context.Context, method string, ub *api.URLBuilder, body io.Reader) (*http.Request, error) {
func (*Client) newRequest(ctx context.Context, method string, ub *api.URLBuilder, body io.Reader) (*http.Request, error) {
	return http.NewRequestWithContext(ctx, method, ub.String(), body)
}

func (c *Client) executeRequest(req *http.Request) (*http.Response, error) {
	if c.token != "" {
		req.Header.Add("Authorization", c.tokenType+" "+c.token)
	}
	client := http.Client{}
	resp, err := client.Do(req)
	resp, err := c.client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
		}
		return nil, err
	}
	return resp, err
123
124
125
126
127
128
129
130

131
132
133
134
135
136
137
138
139
140

141
142
143
144
145
146
147
148























149
150
151
152
153

154
155
156
157
158
159
160
135
136
137
138
139
140
141

142
143
144
145
146
147
148
149
150
151

152
153
154
155
156
157
158
159

160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186

187
188
189
190
191
192
193
194







-
+









-
+







-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+




-
+







	}
	return c.RefreshToken(ctx)
}

// Authenticate sets a new token by sending user name and password.
func (c *Client) Authenticate(ctx context.Context) error {
	authData := url.Values{"username": {c.username}, "password": {c.password}}
	req, err := c.newRequest(ctx, http.MethodPost, c.newURLBuilder('v'), strings.NewReader(authData.Encode()))
	req, err := c.newRequest(ctx, http.MethodPost, c.newURLBuilder('a'), strings.NewReader(authData.Encode()))
	if err != nil {
		return err
	}
	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
	return c.executeAuthRequest(req)
}

// RefreshToken updates the access token
func (c *Client) RefreshToken(ctx context.Context) error {
	req, err := c.newRequest(ctx, http.MethodPut, c.newURLBuilder('v'), nil)
	req, err := c.newRequest(ctx, http.MethodPut, c.newURLBuilder('a'), nil)
	if err != nil {
		return err
	}
	return c.executeAuthRequest(req)
}

// CreateZettel creates a new zettel and returns its URL.
func (c *Client) CreateZettel(ctx context.Context, data *api.ZettelDataJSON) (id.Zid, error) {
func (c *Client) CreateZettel(ctx context.Context, data string) (id.Zid, error) {
	ub := c.jsonZettelURLBuilder('z', nil)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, strings.NewReader(data), nil)
	if err != nil {
		return id.Invalid, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusCreated {
		return id.Invalid, errors.New(resp.Status)
	}
	b, err := io.ReadAll(resp.Body)
	if err != nil {
		return id.Invalid, err
	}
	zid, err := id.Parse(string(b))
	if err != nil {
		return id.Invalid, err
	}
	return zid, nil
}

// CreateZettelJSON creates a new zettel and returns its URL.
func (c *Client) CreateZettelJSON(ctx context.Context, data *api.ZettelDataJSON) (id.Zid, error) {
	var buf bytes.Buffer
	if err := encodeZettelData(&buf, data); err != nil {
		return id.Invalid, err
	}
	ub := c.jsonZettelURLBuilder(nil)
	ub := c.jsonZettelURLBuilder('j', nil)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, &buf, nil)
	if err != nil {
		return id.Invalid, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusCreated {
		return id.Invalid, errors.New(resp.Status)
175
176
177
178
179
180
181
182
183




















184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200





















201
202
203


204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221






222




223
224


225
226
227
228
229
230
231
209
210
211
212
213
214
215


216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274


275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293

294
295
296
297
298
299
300
301
302
303
304


305
306
307
308
309
310
311
312
313







-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

















+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
-
+
+

















-
+
+
+
+
+
+

+
+
+
+
-
-
+
+







func encodeZettelData(buf *bytes.Buffer, data *api.ZettelDataJSON) error {
	enc := json.NewEncoder(buf)
	enc.SetEscapeHTML(false)
	return enc.Encode(&data)
}

// ListZettel returns a list of all Zettel.
func (c *Client) ListZettel(ctx context.Context, query url.Values) ([]api.ZettelJSON, error) {
	ub := c.jsonZettelURLBuilder(query)
func (c *Client) ListZettel(ctx context.Context, query url.Values) (string, error) {
	ub := c.jsonZettelURLBuilder('z', query)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return "", err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return "", errors.New(resp.Status)
	}
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", err
	}
	return string(data), nil
}

// ListZettelJSON returns a list of all Zettel.
func (c *Client) ListZettelJSON(ctx context.Context, query url.Values) ([]api.ZidMetaJSON, error) {
	ub := c.jsonZettelURLBuilder('j', query)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, errors.New(resp.Status)
	}
	dec := json.NewDecoder(resp.Body)
	var zl api.ZettelListJSON
	err = dec.Decode(&zl)
	if err != nil {
		return nil, err
	}
	return zl.List, nil
}

// GetZettel returns a zettel as a string.
func (c *Client) GetZettel(ctx context.Context, zid id.Zid, part string) (string, error) {
	ub := c.jsonZettelURLBuilder('z', nil).SetZid(zid)
	if part != "" && part != api.PartContent {
		ub.AppendQuery(api.QueryKeyPart, part)
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return "", err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return "", errors.New(resp.Status)
	}
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", err
	}
	return string(data), nil
}

// GetZettelJSON returns a zettel as a JSON struct.
func (c *Client) GetZettelJSON(ctx context.Context, zid id.Zid, query url.Values) (*api.ZettelDataJSON, error) {
	ub := c.jsonZettelURLBuilder(query).SetZid(zid)
func (c *Client) GetZettelJSON(ctx context.Context, zid id.Zid) (*api.ZettelDataJSON, error) {
	ub := c.jsonZettelURLBuilder('j', nil).SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, errors.New(resp.Status)
	}
	dec := json.NewDecoder(resp.Body)
	var out api.ZettelDataJSON
	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}
	return &out, nil
}

// GetEvaluatedZettel return a zettel in a defined encoding.
// GetParsedZettel return a parsed zettel in a defined encoding.
func (c *Client) GetParsedZettel(ctx context.Context, zid id.Zid, enc api.EncodingEnum) (string, error) {
	return c.getZettelString(ctx, 'p', zid, enc)
}

// GetEvaluatedZettel return an evaluated zettel in a defined encoding.
func (c *Client) GetEvaluatedZettel(ctx context.Context, zid id.Zid, enc api.EncodingEnum) (string, error) {
	return c.getZettelString(ctx, 'v', zid, enc)
}

func (c *Client) getZettelString(ctx context.Context, key byte, zid id.Zid, enc api.EncodingEnum) (string, error) {
	ub := c.jsonZettelURLBuilder(nil).SetZid(zid)
	ub.AppendQuery(api.QueryKeyFormat, enc.String())
	ub := c.jsonZettelURLBuilder(key, nil).SetZid(zid)
	ub.AppendQuery(api.QueryKeyEncoding, enc.String())
	ub.AppendQuery(api.QueryKeyPart, api.PartContent)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return "", err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
302
303
304
305
306
307
308
309

310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330















331
332
333
334
335

336
337
338
339
340
341
342
343
344
345
346
347
348
349

350
351

352
353
354
355
356
357
358
359
360
361
362
363
364
365
366

367
368
369
370
371
372
373
374
375
376
377
378
379


380
381

382
383
384
385
386
387
388
384
385
386
387
388
389
390

391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411

412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430

431
432
433
434
435
436
437
438
439
440
441
442
443
444

445
446

447
448
449
450
451
452
453
454
455
456
457
458
459
460
461

462
463
464
465
466
467
468
469
470
471
472
473


474
475
476

477
478
479
480
481
482
483
484







-
+




















-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+




-
+













-
+

-
+














-
+











-
-
+
+

-
+







	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}
	return &out, nil
}

// GetZettelLinks returns connections to ohter zettel, images, externals URLs.
// GetZettelLinks returns connections to other zettel, embedded material, externals URLs.
func (c *Client) GetZettelLinks(ctx context.Context, zid id.Zid) (*api.ZettelLinksJSON, error) {
	ub := c.newURLBuilder('l').SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, errors.New(resp.Status)
	}
	dec := json.NewDecoder(resp.Body)
	var out api.ZettelLinksJSON
	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}
	return &out, nil
}

// UpdateZettel updates an existing zettel.
func (c *Client) UpdateZettel(ctx context.Context, zid id.Zid, data *api.ZettelDataJSON) error {
func (c *Client) UpdateZettel(ctx context.Context, zid id.Zid, data string) error {
	ub := c.jsonZettelURLBuilder('z', nil).SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPut, ub, strings.NewReader(data), nil)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusNoContent {
		return errors.New(resp.Status)
	}
	return nil
}

// UpdateZettelJSON updates an existing zettel.
func (c *Client) UpdateZettelJSON(ctx context.Context, zid id.Zid, data *api.ZettelDataJSON) error {
	var buf bytes.Buffer
	if err := encodeZettelData(&buf, data); err != nil {
		return err
	}
	ub := c.jsonZettelURLBuilder(nil).SetZid(zid)
	ub := c.jsonZettelURLBuilder('j', nil).SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPut, ub, &buf, nil)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusNoContent {
		return errors.New(resp.Status)
	}
	return nil
}

// RenameZettel renames a zettel.
func (c *Client) RenameZettel(ctx context.Context, oldZid, newZid id.Zid) error {
	ub := c.jsonZettelURLBuilder(nil).SetZid(oldZid)
	ub := c.jsonZettelURLBuilder('z', nil).SetZid(oldZid)
	h := http.Header{
		api.HeaderDestination: {c.jsonZettelURLBuilder(nil).SetZid(newZid).String()},
		api.HeaderDestination: {c.jsonZettelURLBuilder('z', nil).SetZid(newZid).String()},
	}
	resp, err := c.buildAndExecuteRequest(ctx, api.MethodMove, ub, nil, h)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusNoContent {
		return errors.New(resp.Status)
	}
	return nil
}

// DeleteZettel deletes a zettel with the given identifier.
func (c *Client) DeleteZettel(ctx context.Context, zid id.Zid) error {
	ub := c.jsonZettelURLBuilder(nil).SetZid(zid)
	ub := c.jsonZettelURLBuilder('z', nil).SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodDelete, ub, nil, nil)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusNoContent {
		return errors.New(resp.Status)
	}
	return nil
}

func (c *Client) jsonZettelURLBuilder(query url.Values) *api.URLBuilder {
	ub := c.newURLBuilder('z')
func (c *Client) jsonZettelURLBuilder(key byte, query url.Values) *api.URLBuilder {
	ub := c.newURLBuilder(key)
	for key, values := range query {
		if key == api.QueryKeyFormat {
		if key == api.QueryKeyEncoding {
			continue
		}
		for _, val := range values {
			ub.AppendQuery(key, val)
		}
	}
	return ub

Changes to client/client_test.go.

12
13
14
15
16
17
18

19
20
21
22
23
24
25
26










































27

28
29
30
31

32
33
34
35
36
37
38
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

70
71
72
73

74
75
76
77
78
79
80
81







+








+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+



-
+







package client_test

import (
	"context"
	"flag"
	"fmt"
	"net/url"
	"strings"
	"testing"

	"zettelstore.de/z/api"
	"zettelstore.de/z/client"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

func TestCreateGetRenameDeleteZettel(t *testing.T) {
	// Is not to be allowed to run in parallel with other tests.
	zettel := `title: A Test

Example content.`
	c := getClient()
	c.SetAuth("owner", "owner")
	zid, err := c.CreateZettel(context.Background(), zettel)
	if err != nil {
		t.Error("Cannot create zettel:", err)
		return
	}
	if !zid.IsValid() {
		t.Error("Invalid zettel ID", zid)
		return
	}
	data, err := c.GetZettel(context.Background(), zid, api.PartZettel)
	if err != nil {
		t.Error("Cannot read zettel", zid, err)
		return
	}
	exp := `title: A Test
role: zettel
syntax: zmk

Example content.`
	if data != exp {
		t.Errorf("Expected zettel data: %q, but got %q", exp, data)
	}
	newZid := zid + 1
	err = c.RenameZettel(context.Background(), zid, newZid)
	if err != nil {
		t.Error("Cannot rename", zid, ":", err)
		newZid = zid
	}
	err = c.DeleteZettel(context.Background(), newZid)
	if err != nil {
		t.Error("Cannot delete", zid, ":", err)
		return
	}
}

func TestCreateRenameDeleteZettel(t *testing.T) {
func TestCreateRenameDeleteZettelJSON(t *testing.T) {
	// Is not to be allowed to run in parallel with other tests.
	c := getClient()
	c.SetAuth("creator", "creator")
	zid, err := c.CreateZettel(context.Background(), &api.ZettelDataJSON{
	zid, err := c.CreateZettelJSON(context.Background(), &api.ZettelDataJSON{
		Meta:     nil,
		Encoding: "",
		Content:  "Example",
	})
	if err != nil {
		t.Error("Cannot create zettel:", err)
		return
53
54
55
56
57
58
59
60







































61
62

63
64
65
66
67
68
69
70
71
72
73

74
75
76
77
78

79
80
81
82
83
84
85

86
87
88

89
90
91
92
93
94
95
96
97
98
99
100
101
102

103
104
105
106

107
108
109
110
111
112
113
114
115
116
117

118
119
120
121
122
123
124
125
126
127






















128
129
130
131

132
133
134
135
136
137


138
139
140
141
142
143
144

145
146
147
148
149
150
151
152
153
154








155

156
157
158
159
160
161

162
163
164

















165
166
167
168
169
170
171
172
173
174
175
176

177
178
179
180
181
182
183
184
185

186
187
188

189
190
191
192
193
194
195
196
197
198
199
200
201
202

203
204
205
206
207
208
209
210

211
212
213
214

215
216
217

218
219
220
221
222
223
224
225
226

227
228
229
230
231
232
233
234

235
236
237
238
239
240
241
242
243
244
245
246
247
248
249

250
251
252
253


254
255

256
257
258

259
260
261

262
263
264
265
266
267
268
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143

144
145
146
147
148
149
150
151
152
153
154

155
156
157
158
159

160
161
162
163
164
165
166
167
168
169
170

171
172
173
174
175
176
177
178
179
180
181
182
183
184

185
186
187
188

189
190
191
192
193
194
195
196
197
198
199

200
201
202
203
204
205
206
207
208


209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233

234
235
236
237
238


239
240
241
242
243
244
245
246

247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265

266
267
268
269
270
271

272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302


303
304
305
306
307
308
309
310


311



312

313
314
315
316
317
318
319
320
321
322
323


324
325
326
327
328
329
330
331

332




333



334

335
336
337
338
339
340


341
342
343
344
345
346
347
348

349


350
351
352
353
354
355
356
357
358
359
360


361
362
363


364
365
366

367
368
369

370
371
372

373
374
375
376
377
378
379
380








+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
+










-
+




-
+







+


-
+













-
+



-
+










-
+








-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+



-
+




-
-
+
+






-
+










+
+
+
+
+
+
+
+
-
+





-
+



+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+










-
-
+







-
-
+
-
-
-
+
-











-
-
+







-
+
-
-
-
-
+
-
-
-
+
-






-
-
+







-
+
-
-











-
-
+


-
-
+
+

-
+


-
+


-
+







		t.Error("Cannot delete", zid, ":", err)
		return
	}
}

func TestUpdateZettel(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	z, err := c.GetZettel(context.Background(), id.DefaultHomeZid, api.PartZettel)
	if err != nil {
		t.Error(err)
		return
	}
	if !strings.HasPrefix(z, "title: Home\n") {
		t.Error("Got unexpected zettel", z)
		return
	}
	newZettel := `title: New Home
role: zettel
syntax: zmk

Empty`
	err = c.UpdateZettel(context.Background(), id.DefaultHomeZid, newZettel)
	if err != nil {
		t.Error(err)
		return
	}
	zt, err := c.GetZettel(context.Background(), id.DefaultHomeZid, api.PartZettel)
	if err != nil {
		t.Error(err)
		return
	}
	if zt != newZettel {
		t.Errorf("Expected zettel %q, got %q", newZettel, zt)
	}
	// Must delete to clean up for next tests
	err = c.DeleteZettel(context.Background(), id.DefaultHomeZid)
	if err != nil {
		t.Error("Cannot delete", id.DefaultHomeZid, ":", err)
		return
	}
}

func TestUpdateZettelJSON(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("writer", "writer")
	z, err := c.GetZettelJSON(context.Background(), id.DefaultHomeZid, nil)
	z, err := c.GetZettelJSON(context.Background(), id.DefaultHomeZid)
	if err != nil {
		t.Error(err)
		return
	}
	if got := z.Meta[meta.KeyTitle]; got != "Home" {
		t.Errorf("Title of zettel is not \"Home\", but %q", got)
		return
	}
	newTitle := "New Home"
	z.Meta[meta.KeyTitle] = newTitle
	err = c.UpdateZettel(context.Background(), id.DefaultHomeZid, z)
	err = c.UpdateZettelJSON(context.Background(), id.DefaultHomeZid, z)
	if err != nil {
		t.Error(err)
		return
	}
	zt, err := c.GetZettelJSON(context.Background(), id.DefaultHomeZid, nil)
	zt, err := c.GetZettelJSON(context.Background(), id.DefaultHomeZid)
	if err != nil {
		t.Error(err)
		return
	}
	if got := zt.Meta[meta.KeyTitle]; got != newTitle {
		t.Errorf("Title of zettel is not %q, but %q", newTitle, got)
	}
	// No need to clean up, because we just changed the title.
}

func TestList(t *testing.T) {
func TestListZettel(t *testing.T) {
	testdata := []struct {
		user string
		exp  int
	}{
		{"", 7},
		{"creator", 10},
		{"reader", 12},
		{"writer", 12},
		{"owner", 34},
	}

	t.Parallel()
	c := getClient()
	query := url.Values{api.QueryKeyFormat: {"html"}} // Client must remove "html"
	query := url.Values{api.QueryKeyEncoding: {api.EncodingHTML}} // Client must remove "html"
	for i, tc := range testdata {
		t.Run(fmt.Sprintf("User %d/%q", i, tc.user), func(tt *testing.T) {
			c.SetAuth(tc.user, tc.user)
			l, err := c.ListZettel(context.Background(), query)
			l, err := c.ListZettelJSON(context.Background(), query)
			if err != nil {
				tt.Error(err)
				return
			}
			got := len(l)
			if got != tc.exp {
				tt.Errorf("List of length %d expected, but got %d\n%v", tc.exp, got, l)
			}
		})
	}
	l, err := c.ListZettel(context.Background(), url.Values{meta.KeyRole: {meta.ValueRoleConfiguration}})
	l, err := c.ListZettelJSON(context.Background(), url.Values{meta.KeyRole: {meta.ValueRoleConfiguration}})
	if err != nil {
		t.Error(err)
		return
	}
	got := len(l)
	if got != 27 {
		t.Errorf("List of length %d expected, but got %d\n%v", 27, got, l)
	}
}
func TestGetZettel(t *testing.T) {

	pl, err := c.ListZettel(context.Background(), url.Values{meta.KeyRole: {meta.ValueRoleConfiguration}})
	if err != nil {
		t.Error(err)
		return
	}
	lines := strings.Split(pl, "\n")
	if lines[len(lines)-1] == "" {
		lines = lines[:len(lines)-1]
	}
	if len(lines) != len(l) {
		t.Errorf("Different list lenght: Plain=%d, JSON=%d", len(lines), len(l))
	} else {
		for i, line := range lines {
			if got := line[:14]; got != l[i].ID {
				t.Errorf("%d: JSON=%q, got=%q", i, l[i].ID, got)
			}
		}
	}
}

func TestGetZettelJSON(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	z, err := c.GetZettelJSON(context.Background(), id.DefaultHomeZid, url.Values{api.QueryKeyPart: {api.PartContent}})
	z, err := c.GetZettelJSON(context.Background(), id.DefaultHomeZid)
	if err != nil {
		t.Error(err)
		return
	}
	if m := z.Meta; len(m) > 0 {
		t.Errorf("Exptected empty meta, but got %v", z.Meta)
	if m := z.Meta; len(m) == 0 {
		t.Errorf("Exptected non-empty meta, but got %v", z.Meta)
	}
	if z.Content == "" || z.Encoding != "" {
		t.Errorf("Expect non-empty content, but empty encoding (got %q)", z.Encoding)
	}
}

func TestGetEvaluatedZettel(t *testing.T) {
func TestGetParsedEvaluatedZettel(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	encodings := []api.EncodingEnum{
		api.EncoderDJSON,
		api.EncoderHTML,
		api.EncoderNative,
		api.EncoderText,
	}
	for _, enc := range encodings {
		content, err := c.GetParsedZettel(context.Background(), id.DefaultHomeZid, enc)
		if err != nil {
			t.Error(err)
			continue
		}
		if len(content) == 0 {
			t.Errorf("Empty content for parsed encoding %v", enc)
		}
		content, err := c.GetEvaluatedZettel(context.Background(), id.DefaultHomeZid, enc)
		content, err = c.GetEvaluatedZettel(context.Background(), id.DefaultHomeZid, enc)
		if err != nil {
			t.Error(err)
			continue
		}
		if len(content) == 0 {
			t.Errorf("Empty content for encoding %v", enc)
			t.Errorf("Empty content for evaluated encoding %v", enc)
		}
	}
}

func checkZid(t *testing.T, expected id.Zid, got string) bool {
	t.Helper()
	if exp := expected.String(); exp != got {
		t.Errorf("Expected a Zid %q, but got %q", exp, got)
		return false
	}
	return true
}

func checkListZid(t *testing.T, l []api.ZidMetaJSON, pos int, expected id.Zid) {
	t.Helper()
	exp := expected.String()
	if got := l[pos].ID; got != exp {
		t.Errorf("Expected result[%d]=%v, but got %v", pos, exp, got)
	}
}

func TestGetZettelOrder(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	rl, err := c.GetZettelOrder(context.Background(), id.TOCNewTemplateZid)
	if err != nil {
		t.Error(err)
		return
	}
	if rl.ID != id.TOCNewTemplateZid.String() {
		t.Errorf("Expected an Zid %v, but got %v", id.TOCNewTemplateZid, rl.ID)
	if !checkZid(t, id.TOCNewTemplateZid, rl.ID) {
		return
	}
	l := rl.List
	if got := len(l); got != 2 {
		t.Errorf("Expected list fo length 2, got %d", got)
		return
	}
	if got := l[0].ID; got != id.TemplateNewZettelZid.String() {
		t.Errorf("Expected result[0]=%v, but got %v", id.TemplateNewZettelZid, got)
	checkListZid(t, l, 0, id.TemplateNewZettelZid)
	}
	if got := l[1].ID; got != id.TemplateNewUserZid.String() {
		t.Errorf("Expected result[1]=%v, but got %v", id.TemplateNewUserZid, got)
	checkListZid(t, l, 1, id.TemplateNewUserZid)
	}
}

func TestGetZettelContext(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	rl, err := c.GetZettelContext(context.Background(), id.VersionZid, client.DirBoth, 0, 3)
	if err != nil {
		t.Error(err)
		return
	}
	if rl.ID != id.VersionZid.String() {
		t.Errorf("Expected an Zid %v, but got %v", id.VersionZid, rl.ID)
	if !checkZid(t, id.VersionZid, rl.ID) {
		return
	}
	l := rl.List
	if got := len(l); got != 3 {
		t.Errorf("Expected list fo length 3, got %d", got)
		return
	}
	if got := l[0].ID; got != id.DefaultHomeZid.String() {
	checkListZid(t, l, 0, id.DefaultHomeZid)
		t.Errorf("Expected result[0]=%v, but got %v", id.DefaultHomeZid, got)
	}
	if got := l[1].ID; got != id.OperatingSystemZid.String() {
		t.Errorf("Expected result[1]=%v, but got %v", id.OperatingSystemZid, got)
	checkListZid(t, l, 1, id.OperatingSystemZid)
	}
	if got := l[2].ID; got != id.StartupConfigurationZid.String() {
		t.Errorf("Expected result[2]=%v, but got %v", id.StartupConfigurationZid, got)
	checkListZid(t, l, 2, id.StartupConfigurationZid)
	}

	rl, err = c.GetZettelContext(context.Background(), id.VersionZid, client.DirBackward, 0, 0)
	if err != nil {
		t.Error(err)
		return
	}
	if rl.ID != id.VersionZid.String() {
		t.Errorf("Expected an Zid %v, but got %v", id.VersionZid, rl.ID)
	if !checkZid(t, id.VersionZid, rl.ID) {
		return
	}
	l = rl.List
	if got := len(l); got != 1 {
		t.Errorf("Expected list fo length 1, got %d", got)
		return
	}
	if got := l[0].ID; got != id.DefaultHomeZid.String() {
	checkListZid(t, l, 0, id.DefaultHomeZid)
		t.Errorf("Expected result[0]=%v, but got %v", id.DefaultHomeZid, got)
	}
}

func TestGetZettelLinks(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
	zl, err := c.GetZettelLinks(context.Background(), id.DefaultHomeZid)
	if err != nil {
		t.Error(err)
		return
	}
	if zl.ID != id.DefaultHomeZid.String() {
		t.Errorf("Expected an Zid %v, but got %v", id.DefaultHomeZid, zl.ID)
	if !checkZid(t, id.DefaultHomeZid, zl.ID) {
		return
	}
	if len(zl.Links.Incoming) != 0 {
		t.Error("No incomings expected", zl.Links.Incoming)
	if len(zl.Linked.Incoming) != 0 {
		t.Error("No incomings expected", zl.Linked.Incoming)
	}
	if got := len(zl.Links.Outgoing); got != 4 {
	if got := len(zl.Linked.Outgoing); got != 4 {
		t.Errorf("Expected 4 outgoing links, got %d", got)
	}
	if got := len(zl.Links.Local); got != 1 {
	if got := len(zl.Linked.Local); got != 1 {
		t.Errorf("Expected 1 local link, got %d", got)
	}
	if got := len(zl.Links.External); got != 4 {
	if got := len(zl.Linked.External); got != 4 {
		t.Errorf("Expected 4 external link, got %d", got)
	}
}

func TestListTags(t *testing.T) {
	t.Parallel()
	c := getClient()

Changes to cmd/cmd_file.go.

23
24
25
26
27
28
29
30
31


32
33
34
35
36
37
38
39
40
41
42
43

44
45
46




47
48
49

50
51
52
53
54
55
56
23
24
25
26
27
28
29


30
31
32
33
34
35
36
37
38
39
40
41
42
43
44



45
46
47
48
49
50

51
52
53
54
55
56
57
58







-
-
+
+












+
-
-
-
+
+
+
+


-
+







	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

// ---------- Subcommand: file -----------------------------------------------

func cmdFile(fs *flag.FlagSet, cfg *meta.Meta) (int, error) {
	format := fs.Lookup("t").Value.String()
func cmdFile(fs *flag.FlagSet, _ *meta.Meta) (int, error) {
	enc := fs.Lookup("t").Value.String()
	m, inp, err := getInput(fs.Args())
	if m == nil {
		return 2, err
	}
	z := parser.ParseZettel(
		domain.Zettel{
			Meta:    m,
			Content: domain.NewContent(inp.Src[inp.Pos:]),
		},
		m.GetDefault(meta.KeySyntax, meta.ValueSyntaxZmk),
		nil,
	)
	encdr := encoder.Create(api.Encoder(enc), &encoder.Environment{
	enc := encoder.Create(api.Encoder(format), &encoder.Environment{Lang: m.GetDefault(meta.KeyLang, meta.ValueLangEN)})
	if enc == nil {
		fmt.Fprintf(os.Stderr, "Unknown format %q\n", format)
		Lang: m.GetDefault(meta.KeyLang, meta.ValueLangEN),
	})
	if encdr == nil {
		fmt.Fprintf(os.Stderr, "Unknown format %q\n", enc)
		return 2, nil
	}
	_, err = enc.WriteZettel(os.Stdout, z, format != "raw")
	_, err = encdr.WriteZettel(os.Stdout, z, parser.ParseMetadata)
	if err != nil {
		return 2, err
	}
	fmt.Println()

	return 0, nil
}

Changes to cmd/cmd_run.go.

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
8
9
10
11
12
13
14

15

16
17
18
19
20
21
22







-

-







// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter/api"
39
40
41
42
43
44
45
46

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71

72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89



90
91
92
93
94
95
96






97
98
99


100
101

102
103
104
105
106
107
108
109
110
111













112
113




114
115
116
117
118






119
120
121
122



123
124

125
126
127
128
129
130








131
132
133
134
135
136
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81



82



83
84
85
86






87
88
89
90
91
92
93


94
95
96

97
98









99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117





118
119
120
121
122
123




124
125
126


127

128




129
130
131
132
133
134
135
136
137
138
139
140
141
142







-
+
















-
+








+











-
-
-

-
-
-
+
+
+

-
-
-
-
-
-
+
+
+
+
+
+

-
-
+
+

-
+

-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+


+
+
+
+
-
-
-
-
-
+
+
+
+
+
+
-
-
-
-
+
+
+
-
-
+
-

-
-
-
-
+
+
+
+
+
+
+
+






}

func withDebug(fs *flag.FlagSet) bool {
	dbg := fs.Lookup("debug")
	return dbg != nil && dbg.Value.String() == "true"
}

func runFunc(fs *flag.FlagSet, cfg *meta.Meta) (int, error) {
func runFunc(fs *flag.FlagSet, _ *meta.Meta) (int, error) {
	exitCode, err := doRun(withDebug(fs))
	kernel.Main.WaitForShutdown()
	return exitCode, err
}

func doRun(debug bool) (int, error) {
	kern := kernel.Main
	kern.SetDebug(debug)
	if err := kern.StartService(kernel.WebService); err != nil {
		return 1, err
	}
	return 0, nil
}

func setupRouting(webSrv server.Server, boxManager box.Manager, authManager auth.Manager, rtConfig config.Config) {
	protectedBoxManager, authPolicy := authManager.BoxWithPolicy(webSrv, boxManager, rtConfig)
	api := api.New(webSrv, authManager, authManager, webSrv, rtConfig)
	a := api.New(webSrv, authManager, authManager, webSrv, rtConfig)
	wui := webui.New(webSrv, authManager, rtConfig, authManager, boxManager, authPolicy)

	ucAuthenticate := usecase.NewAuthenticate(authManager, authManager, boxManager)
	ucCreateZettel := usecase.NewCreateZettel(rtConfig, protectedBoxManager)
	ucGetMeta := usecase.NewGetMeta(protectedBoxManager)
	ucGetAllMeta := usecase.NewGetAllMeta(protectedBoxManager)
	ucGetZettel := usecase.NewGetZettel(protectedBoxManager)
	ucParseZettel := usecase.NewParseZettel(rtConfig, ucGetZettel)
	ucEvaluate := usecase.NewEvaluate(rtConfig, ucGetZettel, ucGetMeta)
	ucListMeta := usecase.NewListMeta(protectedBoxManager)
	ucListRoles := usecase.NewListRole(protectedBoxManager)
	ucListTags := usecase.NewListTags(protectedBoxManager)
	ucZettelContext := usecase.NewZettelContext(protectedBoxManager)
	ucDelete := usecase.NewDeleteZettel(protectedBoxManager)
	ucUpdate := usecase.NewUpdateZettel(protectedBoxManager)
	ucRename := usecase.NewRenameZettel(protectedBoxManager)

	webSrv.Handle("/", wui.MakeGetRootHandler(protectedBoxManager))

	// Web user interface
	webSrv.AddListRoute('a', http.MethodGet, wui.MakeGetLoginHandler())
	webSrv.AddListRoute('a', http.MethodPost, wui.MakePostLoginHandler(ucAuthenticate))
	webSrv.AddZettelRoute('a', http.MethodGet, wui.MakeGetLogoutHandler())
	if !authManager.IsReadonly() {
		webSrv.AddZettelRoute('b', http.MethodGet, wui.MakeGetRenameZettelHandler(ucGetMeta))
		webSrv.AddZettelRoute('b', http.MethodPost, wui.MakePostRenameZettelHandler(ucRename))
		webSrv.AddZettelRoute('c', http.MethodGet, wui.MakeGetCopyZettelHandler(
		webSrv.AddZettelRoute('b', server.MethodGet, wui.MakeGetRenameZettelHandler(ucGetMeta))
		webSrv.AddZettelRoute('b', server.MethodPost, wui.MakePostRenameZettelHandler(ucRename))
		webSrv.AddZettelRoute('c', server.MethodGet, wui.MakeGetCopyZettelHandler(
			ucGetZettel, usecase.NewCopyZettel()))
		webSrv.AddZettelRoute('c', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('d', http.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetZettel))
		webSrv.AddZettelRoute('d', http.MethodPost, wui.MakePostDeleteZettelHandler(ucDelete))
		webSrv.AddZettelRoute('e', http.MethodGet, wui.MakeEditGetZettelHandler(ucGetZettel))
		webSrv.AddZettelRoute('e', http.MethodPost, wui.MakeEditSetZettelHandler(ucUpdate))
		webSrv.AddZettelRoute('f', http.MethodGet, wui.MakeGetFolgeZettelHandler(
		webSrv.AddZettelRoute('c', server.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('d', server.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetZettel))
		webSrv.AddZettelRoute('d', server.MethodPost, wui.MakePostDeleteZettelHandler(ucDelete))
		webSrv.AddZettelRoute('e', server.MethodGet, wui.MakeEditGetZettelHandler(ucGetZettel))
		webSrv.AddZettelRoute('e', server.MethodPost, wui.MakeEditSetZettelHandler(ucUpdate))
		webSrv.AddZettelRoute('f', server.MethodGet, wui.MakeGetFolgeZettelHandler(
			ucGetZettel, usecase.NewFolgeZettel(rtConfig)))
		webSrv.AddZettelRoute('f', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('g', http.MethodGet, wui.MakeGetNewZettelHandler(
		webSrv.AddZettelRoute('f', server.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('g', server.MethodGet, wui.MakeGetNewZettelHandler(
			ucGetZettel, usecase.NewNewZettel()))
		webSrv.AddZettelRoute('g', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('g', server.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
	}
	webSrv.AddListRoute('f', http.MethodGet, wui.MakeSearchHandler(
		usecase.NewSearch(protectedBoxManager), ucGetMeta, ucGetZettel))
	webSrv.AddListRoute('h', http.MethodGet, wui.MakeListHTMLMetaHandler(
		ucListMeta, ucListRoles, ucListTags))
	webSrv.AddZettelRoute('h', http.MethodGet, wui.MakeGetHTMLZettelHandler(
		ucParseZettel, ucGetMeta))
	webSrv.AddZettelRoute('i', http.MethodGet, wui.MakeGetInfoHandler(
		ucParseZettel, ucGetMeta, ucGetAllMeta))
	webSrv.AddZettelRoute('j', http.MethodGet, wui.MakeZettelContextHandler(ucZettelContext))
	webSrv.AddListRoute('f', server.MethodGet, wui.MakeSearchHandler(
		usecase.NewSearch(protectedBoxManager), &ucEvaluate))
	webSrv.AddListRoute('h', server.MethodGet, wui.MakeListHTMLMetaHandler(
		ucListMeta, ucListRoles, ucListTags, &ucEvaluate))
	webSrv.AddZettelRoute('h', server.MethodGet, wui.MakeGetHTMLZettelHandler(
		&ucEvaluate, ucGetMeta))
	webSrv.AddListRoute('i', server.MethodGet, wui.MakeGetLoginOutHandler())
	webSrv.AddListRoute('i', server.MethodPost, wui.MakePostLoginHandler(ucAuthenticate))
	webSrv.AddZettelRoute('i', server.MethodGet, wui.MakeGetInfoHandler(
		ucParseZettel, &ucEvaluate, ucGetMeta, ucGetAllMeta))
	webSrv.AddListRoute('i', server.MethodGet, wui.MakeGetLoginOutHandler())
	webSrv.AddZettelRoute('k', server.MethodGet, wui.MakeZettelContextHandler(
		ucZettelContext, &ucEvaluate))

	// API
	webSrv.AddListRoute('a', server.MethodPost, a.MakePostLoginHandler(ucAuthenticate))
	webSrv.AddListRoute('a', server.MethodPut, a.MakeRenewAuthHandler())
	webSrv.AddListRoute('j', server.MethodGet, api.MakeListMetaHandler(ucListMeta))
	webSrv.AddZettelRoute('j', server.MethodGet, api.MakeGetZettelHandler(ucGetZettel))
	webSrv.AddZettelRoute('l', http.MethodGet, api.MakeGetLinksHandler(ucParseZettel))
	webSrv.AddZettelRoute('o', http.MethodGet, api.MakeGetOrderHandler(
		usecase.NewZettelOrder(protectedBoxManager, ucParseZettel)))
	webSrv.AddListRoute('r', http.MethodGet, api.MakeListRoleHandler(ucListRoles))
	webSrv.AddListRoute('t', http.MethodGet, api.MakeListTagsHandler(ucListTags))
	webSrv.AddZettelRoute('l', server.MethodGet, api.MakeGetLinksHandler(ucEvaluate))
	webSrv.AddZettelRoute('o', server.MethodGet, api.MakeGetOrderHandler(
		usecase.NewZettelOrder(protectedBoxManager, ucEvaluate)))
	webSrv.AddZettelRoute('p', server.MethodGet, a.MakeGetParsedZettelHandler(ucParseZettel))
	webSrv.AddListRoute('r', server.MethodGet, api.MakeListRoleHandler(ucListRoles))
	webSrv.AddListRoute('t', server.MethodGet, api.MakeListTagsHandler(ucListTags))
	webSrv.AddListRoute('v', http.MethodPost, api.MakePostLoginHandler(ucAuthenticate))
	webSrv.AddListRoute('v', http.MethodPut, api.MakeRenewAuthHandler())
	webSrv.AddZettelRoute('x', http.MethodGet, api.MakeZettelContextHandler(ucZettelContext))
	webSrv.AddListRoute('z', http.MethodGet, api.MakeListMetaHandler(
	webSrv.AddZettelRoute('v', server.MethodGet, a.MakeGetEvalZettelHandler(ucEvaluate))
	webSrv.AddZettelRoute('x', server.MethodGet, api.MakeZettelContextHandler(ucZettelContext))
	webSrv.AddListRoute('z', server.MethodGet, a.MakeListPlainHandler(ucListMeta))
		usecase.NewListMeta(protectedBoxManager), ucGetMeta, ucParseZettel))
	webSrv.AddZettelRoute('z', http.MethodGet, api.MakeGetZettelHandler(
	webSrv.AddZettelRoute('z', server.MethodGet, a.MakeGetPlainZettelHandler(ucGetZettel))
		ucParseZettel, ucGetMeta))
	if !authManager.IsReadonly() {
		webSrv.AddListRoute('z', http.MethodPost, api.MakePostCreateZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('z', http.MethodDelete, api.MakeDeleteZettelHandler(ucDelete))
		webSrv.AddZettelRoute('z', http.MethodPut, api.MakeUpdateZettelHandler(ucUpdate))
		webSrv.AddZettelRoute('z', zsapi.MethodMove, api.MakeRenameZettelHandler(ucRename))
		webSrv.AddListRoute('j', server.MethodPost, a.MakePostCreateZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('j', server.MethodPut, api.MakeUpdateZettelHandler(ucUpdate))
		webSrv.AddZettelRoute('j', server.MethodDelete, api.MakeDeleteZettelHandler(ucDelete))
		webSrv.AddZettelRoute('j', server.MethodMove, api.MakeRenameZettelHandler(ucRename))
		webSrv.AddListRoute('z', server.MethodPost, a.MakePostCreatePlainZettelHandler(ucCreateZettel))
		webSrv.AddZettelRoute('z', server.MethodPut, api.MakeUpdatePlainZettelHandler(ucUpdate))
		webSrv.AddZettelRoute('z', server.MethodDelete, api.MakeDeleteZettelHandler(ucDelete))
		webSrv.AddZettelRoute('z', server.MethodMove, api.MakeRenameZettelHandler(ucRename))
	}

	if authManager.WithAuth() {
		webSrv.SetUserRetriever(usecase.NewGetUserByZid(boxManager))
	}
}

Changes to cmd/fd_limit.go.

1
2
3
4
5
6
7
8
9
10

11
12
13
14
15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16










+





//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//go:build !darwin
// +build !darwin

package cmd

func raiseFdLimit() error { return nil }

Changes to cmd/fd_limit_raise.go.

1
2
3
4
5
6
7
8
9
10

11
12
13
14
15
16
17
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18










+







//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//go:build darwin
// +build darwin

package cmd

import (
	"log"
	"syscall"

Changes to cmd/main.go.

16
17
18
19
20
21
22

23
24
25
26
27
28
29
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30







+







	"fmt"
	"net"
	"net/url"
	"os"
	"strconv"
	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/impl"
	"zettelstore.de/z/box"
	"zettelstore.de/z/box/compbox"
	"zettelstore.de/z/box/manager"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
68
69
70
71
72
73
74
75

76
77
78
79
80
81
82
69
70
71
72
73
74
75

76
77
78
79
80
81
82
83







-
+







		Header: true,
		Flags:  flgSimpleRun,
	})
	RegisterCommand(Command{
		Name: "file",
		Func: cmdFile,
		Flags: func(fs *flag.FlagSet) {
			fs.String("t", "html", "target output format")
			fs.String("t", api.EncoderHTML.String(), "target output encoding")
		},
	})
	RegisterCommand(Command{
		Name: "password",
		Func: cmdPassword,
	})
}

Changes to cmd/register.go.

14
15
16
17
18
19
20

21
22
23
24
25
26
27
28
29
30
31
32
33
14
15
16
17
18
19
20
21
22

23

24
25
26
27
28
29
30
31
32







+

-

-









// Mention all needed encoders, parsers and stores to have them registered.
import (
	_ "zettelstore.de/z/box/compbox"       // Allow to use computed box.
	_ "zettelstore.de/z/box/constbox"      // Allow to use global internal box.
	_ "zettelstore.de/z/box/dirbox"        // Allow to use directory box.
	_ "zettelstore.de/z/box/filebox"       // Allow to use file box.
	_ "zettelstore.de/z/box/membox"        // Allow to use in-memory box.
	_ "zettelstore.de/z/encoder/djsonenc"  // Allow to use DJSON encoder.
	_ "zettelstore.de/z/encoder/htmlenc"   // Allow to use HTML encoder.
	_ "zettelstore.de/z/encoder/jsonenc"   // Allow to use JSON encoder.
	_ "zettelstore.de/z/encoder/nativeenc" // Allow to use native encoder.
	_ "zettelstore.de/z/encoder/rawenc"    // Allow to use raw encoder.
	_ "zettelstore.de/z/encoder/textenc"   // Allow to use text encoder.
	_ "zettelstore.de/z/encoder/zmkenc"    // Allow to use zmk encoder.
	_ "zettelstore.de/z/kernel/impl"       // Allow kernel implementation to create itself
	_ "zettelstore.de/z/parser/blob"       // Allow to use BLOB parser.
	_ "zettelstore.de/z/parser/markdown"   // Allow to use markdown parser.
	_ "zettelstore.de/z/parser/none"       // Allow to use none parser.
	_ "zettelstore.de/z/parser/plain"      // Allow to use plain parser.
	_ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser.
)

Changes to collect/collect.go.

11
12
13
14
15
16
17
18
19


20
21
22
23
24
25

26


27
28
29
30
31
32
33
34
35
36
37



38
39
40
41
42
43
11
12
13
14
15
16
17


18
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35
36



37
38
39
40
41
42
43
44
45







-
-
+
+






+
-
+
+








-
-
-
+
+
+






// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"

// Summary stores the relevant parts of the syntax tree
type Summary struct {
	Links  []*ast.Reference // list of all referenced links
	Images []*ast.Reference // list of all referenced images
	Links  []*ast.Reference // list of all linked material
	Embeds []*ast.Reference // list of all embedded material
	Cites  []*ast.CiteNode  // list of all referenced citations
}

// References returns all references mentioned in the given zettel. This also
// includes references to images.
func References(zn *ast.ZettelNode) (s Summary) {
	if zn.Ast != nil {
	ast.WalkBlockSlice(&s, zn.Ast)
		ast.Walk(&s, zn.Ast)
	}
	return s
}

// Visit all node to collect data for the summary.
func (s *Summary) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.LinkNode:
		s.Links = append(s.Links, n.Ref)
	case *ast.ImageNode:
		if n.Ref != nil {
			s.Images = append(s.Images, n.Ref)
	case *ast.EmbedNode:
		if m, ok := n.Material.(*ast.ReferenceMaterialNode); ok {
			s.Embeds = append(s.Embeds, m.Ref)
		}
	case *ast.CiteNode:
		s.Cites = append(s.Cites, n)
	}
	return s
}

Changes to collect/collect_test.go.

26
27
28
29
30
31
32
33
34


35
36
37
38
39

40
41
42

43
44

45
46
47


48
49
50

51
52
53
54
55
56
57

58
59
60

61
62
63
64
65





66
67
68
69
70


71
72
26
27
28
29
30
31
32


33
34
35
36
37
38

39
40
41

42
43

44
45


46
47
48
49

50
51
52
53
54
55
56

57
58
59

60
61




62
63
64
65
66

67
68


69
70
71
72







-
-
+
+




-
+


-
+

-
+

-
-
+
+


-
+






-
+


-
+

-
-
-
-
+
+
+
+
+
-


-
-
+
+


	return r
}

func TestLinks(t *testing.T) {
	t.Parallel()
	zn := &ast.ZettelNode{}
	summary := collect.References(zn)
	if summary.Links != nil || summary.Images != nil {
		t.Error("No links/images expected, but got:", summary.Links, "and", summary.Images)
	if summary.Links != nil || summary.Embeds != nil {
		t.Error("No links/images expected, but got:", summary.Links, "and", summary.Embeds)
	}

	intNode := &ast.LinkNode{Ref: parseRef("01234567890123")}
	para := &ast.ParaNode{
		Inlines: ast.InlineSlice{
		Inlines: ast.CreateInlineListNode(
			intNode,
			&ast.LinkNode{Ref: parseRef("https://zettelstore.de/z")},
		},
		),
	}
	zn.Ast = ast.BlockSlice{para}
	zn.Ast = &ast.BlockListNode{List: []ast.BlockNode{para}}
	summary = collect.References(zn)
	if summary.Links == nil || summary.Images != nil {
		t.Error("Links expected, and no images, but got:", summary.Links, "and", summary.Images)
	if summary.Links == nil || summary.Embeds != nil {
		t.Error("Links expected, and no images, but got:", summary.Links, "and", summary.Embeds)
	}

	para.Inlines = append(para.Inlines, intNode)
	para.Inlines.Append(intNode)
	summary = collect.References(zn)
	if cnt := len(summary.Links); cnt != 3 {
		t.Error("Link count does not work. Expected: 3, got", summary.Links)
	}
}

func TestImage(t *testing.T) {
func TestEmbed(t *testing.T) {
	t.Parallel()
	zn := &ast.ZettelNode{
		Ast: ast.BlockSlice{
		Ast: &ast.BlockListNode{List: []ast.BlockNode{
			&ast.ParaNode{
				Inlines: ast.InlineSlice{
					&ast.ImageNode{Ref: parseRef("12345678901234")},
				},
			},
				Inlines: ast.CreateInlineListNode(
					&ast.EmbedNode{Material: &ast.ReferenceMaterialNode{Ref: parseRef("12345678901234")}},
				),
			},
		}},
		},
	}
	summary := collect.References(zn)
	if summary.Images == nil {
		t.Error("Only image expected, but got: ", summary.Images)
	if summary.Embeds == nil {
		t.Error("Only image expected, but got: ", summary.Embeds)
	}
}

Changes to collect/order.go.

11
12
13
14
15
16
17



18
19
20
21
22
23
24










25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45





46
47
48
49
50
51
52

53
54
55
56
57
58
59
11
12
13
14
15
16
17
18
19
20







21
22
23
24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48


49
50
51
52
53
54
55
56
57
58
59

60
61
62
63
64
65
66
67







+
+
+
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
-


















-
-
+
+
+
+
+






-
+







// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"

// Order of internal reference within the given zettel.
func Order(zn *ast.ZettelNode) (result []*ast.Reference) {
	if zn.Ast == nil {
		return nil
	}
	for _, bn := range zn.Ast {
		if ln, ok := bn.(*ast.NestedListNode); ok {
			switch ln.Kind {
			case ast.NestedListOrdered, ast.NestedListUnordered:
				for _, is := range ln.Items {
					if ref := firstItemZettelReference(is); ref != nil {
						result = append(result, ref)
	for _, bn := range zn.Ast.List {
		ln, ok := bn.(*ast.NestedListNode)
		if !ok {
			continue
		}
		switch ln.Kind {
		case ast.NestedListOrdered, ast.NestedListUnordered:
			for _, is := range ln.Items {
				if ref := firstItemZettelReference(is); ref != nil {
					result = append(result, ref)
					}
				}
			}
		}
	}
	return result
}

func firstItemZettelReference(is ast.ItemSlice) *ast.Reference {
	for _, in := range is {
		if pn, ok := in.(*ast.ParaNode); ok {
			if ref := firstInlineZettelReference(pn.Inlines); ref != nil {
				return ref
			}
		}
	}
	return nil
}

func firstInlineZettelReference(ins ast.InlineSlice) (result *ast.Reference) {
	for _, inl := range ins {
func firstInlineZettelReference(iln *ast.InlineListNode) (result *ast.Reference) {
	if iln == nil {
		return nil
	}
	for _, inl := range iln.List {
		switch in := inl.(type) {
		case *ast.LinkNode:
			if ref := in.Ref; ref.IsZettel() {
				return ref
			}
			result = firstInlineZettelReference(in.Inlines)
		case *ast.ImageNode:
		case *ast.EmbedNode:
			result = firstInlineZettelReference(in.Inlines)
		case *ast.CiteNode:
			result = firstInlineZettelReference(in.Inlines)
		case *ast.FootnoteNode:
			// Ignore references in footnotes
			continue
		case *ast.FormatNode:

Changes to config/config.go.

39
40
41
42
43
44
45



46
47
48
49
50
51
52
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55







+
+
+







	GetSiteName() string

	// GetHomeZettel returns the value of the "home-zettel" key.
	GetHomeZettel() id.Zid

	// GetDefaultVisibility returns the default value for zettel visibility.
	GetDefaultVisibility() meta.Visibility

	// GetMaxTransclusions return the maximum number of indirect transclusions.
	GetMaxTransclusions() int

	// GetYAMLHeader returns the current value of the "yaml-header" key.
	GetYAMLHeader() bool

	// GetZettelFileSyntax returns the current value of the "zettel-file-syntax" key.
	GetZettelFileSyntax() []string

69
70
71
72
73
74
75

76



77
78
79
80
81
82
83
72
73
74
75
76
77
78
79

80
81
82
83
84
85
86
87
88
89







+
-
+
+
+








// GetTitle returns the value of the "title" key of the given meta. If there
// is no such value, GetDefaultTitle is returned.
func GetTitle(m *meta.Meta, cfg Config) string {
	if val, ok := m.Get(meta.KeyTitle); ok {
		return val
	}
	if cfg != nil {
	return cfg.GetDefaultTitle()
		return cfg.GetDefaultTitle()
	}
	return "Untitled"
}

// GetRole returns the value of the "role" key of the given meta. If there
// is no such value, GetDefaultRole is returned.
func GetRole(m *meta.Meta, cfg Config) string {
	if val, ok := m.Get(meta.KeyRole); ok {
		return val

Added docs/development/00010000000000.zettel.









1
2
3
4
5
6
7
8
+
+
+
+
+
+
+
+
id: 00010000000000
title: Developments Notes
role: zettel
syntax: zmk
modified: 20210916194954

* [[Required Software|20210916193200]]
* [[Checklist for Release|20210916194900]]

Added docs/development/20210916193200.zettel.





















1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 20210916193200
title: Required Software
role: zettel
syntax: zmk
modified: 20210916194748

The following software must be installed:

* A current, supported [[release of Go|https://golang.org/doc/devel/release.html]],
* [[golint|https://github.com/golang/lint]],
* [[shadow|https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/shadow]] via ``go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow@latest``,
* [[staticcheck|https://staticcheck.io/]] via ``go get honnef.co/go/tools/cmd/staticcheck``,
* [[unparam|mvdan.cc/unparam]][^[[GitHub|https://github.com/mvdan/unparam]]] via ``go install mvdan.cc/unparam@latest``

Make sure that the software is in your path, e.g. via:

```sh
export PATH=$PATH:/usr/local/go/bin
export PATH=$PATH:$(go env GOPATH)/bin
```

Added docs/development/20210916194900.zettel.





















































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 20210916194900
title: Checklist for Release
role: zettel
syntax: zmk
modified: 20210917165448

# Check for unused parameters:
#* ``unparam ./...``
#* ``unparam -exported -tests ./...``
# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# All internal tests must succeed:
#* ``go run tools/build.go check`` (alternatively: ``make check``).
# The API tests must succeed on every development platform:
#* ``go run tools/build.go testapi`` (alternatively: ``make api``).
# Run [[linkchecker|https://linkchecker.github.io/linkchecker/]] with the manual:
#* ``go run -race cmd/zettelstore/main.go run -d docs/manual``
#* ``linkchecker  http://127.0.0.1:23123 2>&1 | tee lc.txt``
#* Check all ""Error: 404 Not Found""
#* Check all ""Error: 403 Forbidden"": allowed for endpoint ''/p'' with encoding ''html'' for those zettel that are accessible only in ''expert-mode''.
#* Try to resolve other error messages and warnings
#* Warnings about empty content can be ignored
# On every development platform, the box with 10.000 zettel must run, with ''-race'' enabled:
#* ``go run -race cmd/zettelstore/main.go run -d DIR``.
# Create a development release:
#* ``go run tools/build.go release`` (alternatively: ``make release``).
# On every platform (esp. macOS), the box with 10.000 zettel must run properly:
#* ``./zettelstore -d DIR``
# Update files in directory ''www''
#* index.wiki
#* download.wiki
#* changes.wiki
#* plan.wiki
# Set file ''VERSION'' to the new release version
# Disable Fossil autosync mode:
#* ``fossil setting autosync off``
# Commit the new release version:
#* ``fossil commit --tag release --tag version-VERSION -m "Version VERSION"``
# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# Create the release:
#* ``go run tools/build.go release`` (alternatively: ``make release``).
# Remove previous executables:
#* ``fossil uv remove --glob '*-PREVVERSION*'``
# Add executables for release:
#* ``cd release``
#* ``fossil uv add *.zip``
#* ``cd ..``
#* Synchronize with main repository:
#* ``fossil sync -u``
# Enable autosync:
#* ``fossil setting autosync on``

Changes to docs/manual/00001004020000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
1
2
3
4
5

6
7
8
9
10
11
12
13





-
+







id: 00001004020000
title: Configure the running Zettelstore
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20210611213730
modified: 20210810103936

You can configure a running Zettelstore by modifying the special zettel with the ID [[00000000000100]].
This zettel is called ""configuration zettel"".
The following metadata keys change the appearance / behavior of Zettelstore:

; [!default-copyright]''default-copyright''
: Copyright value to be used when rendering content.
51
52
53
54
55
56
57




58
59
60
61
62
63
64
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68







+
+
+
+







  Default: (the empty string).
; [!home-zettel]''home-zettel''
: Specifies the identifier of the zettel, that should be presented for the default view / home view.
  If not given or if the identifier does not identify a zettel, the zettel with the identifier ''00010000000000'' is shown.
; [!marker-external]''marker-external''
: Some HTML code that is displayed after a reference to external material.
  Default: ''&\#10138;'', to display a ""&#10138;"" sign.
; [!max-transclusions]''max-transclusions''
: Maximum number of indirect transclusion.
  This is used to avoid an exploding ""transclusion bomb"", a form of a [[billion laughs attack|https://en.wikipedia.org/wiki/Billion_laughs_attack]].
  Default: 1024.
; [!site-name]''site-name''
: Name of the Zettelstore instance.
  Will be used when displaying some lists.
  Default: ''Zettelstore''.
; [!yaml-header]''yaml-header''
: If true, metadata and content will be separated by ''-\--\\n'' instead of an empty line (''\\n\\n'').
  Default: ''false''.

Changes to docs/manual/00001004051200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18

19

20
21
22
23
24
25
26
27
28





-
+












-

-









id: 00001004051200
title: The ''file'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20210712234222
modified: 20210727120507

Reads zettel data from a file (or from standard input / stdin) and renders it to standard output / stdout.
This allows Zettelstore to render files manually.
```
zettelstore file [-t FORMAT] [file-1 [file-2]]
```

; ''-t FORMAT''
: Specifies the output format.
  Supported values are:
  [[''html''|00001012920510]] (default),
  [[''djson''|00001012920503]],
  [[''json''|00001012920501]],
  [[''native''|00001012920513]],
  [[''raw''|00001012920516]],
  [[''text''|00001012920519]],
  and [[''zmk''|00001012920522]].
; ''file-1''
: Specifies the file name, where at least metadata is read.
  If ''file-2'' is not given, the zettel content is also read from here.
; ''file-2''
: File name where the zettel content is stored.

If neither ''file-1'' nor ''file-2'' are given, metadata and zettel content are read from standard input / stdin.

Changes to docs/manual/00001006000000.zettel.

1
2

3
4
5

6
7

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

1
2
3
4
5

6
7

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

29


+


-
+

-
+




















-
+
id: 00001006000000
title: Layout of a Zettel
role: manual
tags: #design #manual #zettelstore
syntax: zmk
role: manual
modified: 20210903210655

A zettel consists of two part: the metadata and the zettel content.
A zettel consists of two parts: the metadata and the zettel content.
Metadata gives some information mostly about the zettel content, how it should be interpreted, how it is sorted within Zettelstore.
The zettel content is, well, the actual content.
In many cases, the content is in plain text form.
Plain text is long-lasting.
However, content in binary format is also possible.

Metadata has to conform to a [[special syntax|00001006010000]].
It is effectively a collection of key/value pairs.
Some keys have a [[special meaning|00001006020000]] and most of the predefined keys need values of a specific [[type|00001006030000]].

Each zettel is given a unique [[identifier|00001006050000]].
To some degree, the zettel identifier is part of the metadata..

The zettel content is your valuable content.
Zettelstore contains some predefined parsers that interpret the zettel content to the syntax of the zettel.
This includes markup languages, like [[Zettelmarkup|00001007000000]] and [[CommonMark|https://commonmark.org/]].
Other text formats are also supported, like CSS and HTML templates.
Plain text content is always Unicode, encoded as UTF-8.
Other character encodings are not supported and will never be[^This is not a real problem, since every modern software should support UTF-8 as an encoding.].
There is support for a graphical format with a text represenation: SVG.
And the is support for some binary image formats, like GIF, PNG, and JPEG.
And there is support for some binary image formats, like GIF, PNG, and JPEG.

Changes to docs/manual/00001006020000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12


13
14
15
16
17
18
19
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21





-
+






+
+







id: 00001006020000
title: Supported Metadata Keys
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20210709162756
modified: 20210822234119

Although you are free to define your own metadata, by using any key (according to the [[syntax|00001006010000]]), some keys have a special meaning that is enforced by Zettelstore.
See the [[computed list of supported metadata keys|00000000000090]] for details.

Most keys conform to a [[type|00001006030000]].

; [!all-tags]''all-tags''
: A property (a computed values that is not stored) that contains both the value of [[''tags''|#tags]] together with all [[tags|00001007040000#tag]] that are specified within the content.
; [!back]''back''
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata, that are not referenced by this zettel.
  Basically, it is the value of [[''backward''|#bachward]], but without any zettel identifier that is contained in [[''forward''|#forward]].
; [!backward]''backward''
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata.
  References within inversable values are not included here, e.g. [[''precursor''|#precursor]].
; [!copyright]''copyright''

Changes to docs/manual/00001006020100.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
1
2
3
4
5
6
7
8
9
10
11
12
13





+







id: 00001006020100
title: Supported Zettel Roles
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20210727120817

The [[''role'' key|00001006020000#role]] defines what kind of zettel you are writing.
The following values are used internally by Zettelstore and must exist:

; [!user]''user''
: If you want to use [[authentication|00001010000000]], all zettel that identify users of the zettel store must have this role.

31
32
33
34
35
36
37
38

32
33
34
35
36
37
38

39







-
+
  Think of them as Post-it notes.
; [!literature]''literature''
: Contains some remarks about a book, a paper, a web page, etc.
  You should add a citation key for citing it.
; [!zettel]''zettel''
: A real zettel that contains your own thoughts.

However, you are free to define additional roles, e.g. ''material'' for literature that is web-based only, ''slide'' for presentation slides, ''paper'' for the raw text of a scientific paper, ''project'' to define a project, ...
However, you are free to define additional roles, e.g. ''material'' for literature that is web-based only, ''slide'' for presentation slides, ''paper'' for the text of a scientific paper, ''project'' to define a project, ...

Changes to docs/manual/00001006030000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15
16
17
18
19
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20





-
+






+







id: 00001006030000
title: Supported Key Types
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20210627170437
modified: 20210830132855

All [[supported metadata keys|00001006020000]] conform to a type.

User-defined metadata keys conform also to a type, based on the suffix of the key.
|=Suffix|Type
| ''-number'' | [[Number|00001006033000]]
| ''-role'' | [[Word|00001006035500]]
| ''-url'' | [[URL|00001006035000]]
| ''-zid''  | [[Identifier|00001006032000]]
| any other suffix | [[EString|00001006031500]]

The name of the metadata key is bound to the key type

Every key type has an associated validation rule to check values of the given type.

Changes to docs/manual/00001006034000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14


15
16
17
18
19
20
21
22

23
24
25
26
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25

26
27
28





+









+
+







-
+
-



id: 00001006034000
title: TagSet Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20210817201410

Values of this type denote a (sorted) set of tags.

A set is different to a list, as no duplicates are allowed.

=== Allowed values
Every tag must must begin with the number sign character (""''#''"", ''U+0023''), followed by at least one printable character.
Tags are separated by space characters.

All characters are mapped to their lower case values.

=== Match operator
It depends of the first character of a search string how it is matched against a tag set value:

* If the first character of the search string is a number sign character,
  it must exactly match one of the values of a tag.
* In other cases, the search string must be the prefix of at least one tag.

Conpectually, all number sign characters are removed at the beginning of the search string
Conceptually, all number sign characters are removed at the beginning of the search string and of all tags.
and of all tags.

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

Changes to docs/manual/00001006035500.zettel.

1
2
3
4
5

6
7
8
9
10
11


12
13

14
15
16
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

16
17
18
19





+






+
+

-
+



id: 00001006035500
title: Word Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20210817201304

Values of this type denote a single word.

=== Allowed values
Must be a non-empty sequence of characters, but without the space character.

All characters are mapped to their lower case values.

=== Match operator
A value matches a word value, if both value are character-wise equal.
A value matches a word value, if both value are character-wise equal, ignoring upper / lower case.

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

Changes to docs/manual/00001006055000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
1
2
3
4
5

6
7
8
9
10
11
12
13





-
+







id: 00001006055000
title: Reserved zettel identifier
role: manual
tags: #design #manual #zettelstore
syntax: zmk
modified: 20210721125518
modified: 20210917154229

[[Zettel identifier|00001006050000]] are typically created by examine the current date and time.
By renaming a zettel, you are able to provide any sequence of 14 digits.
If no other zettel has the same identifier, you are allowed to rename a zettel.

To make things easier, you normally should not use zettel identifier that begin with four zeroes (''0000'').

36
37
38
39
40
41
42
43

36
37
38
39
40
41
42

43







-
+
| 00000200000000 | 0000899999999 | Reserved for future use
| 00009000000000 | 0000999999999 | Reserved for applications

This list may change in the future.

==== External Applications
|= From | To | Description
| 00009000001000 | 00009000001000 | ZS Slides, an application to display zettel as a HTML-based slideshow
| 00009000001000 | 00009000001999 | ZS Slides, an application to display zettel as a HTML-based slideshow

Changes to docs/manual/00001007040000.zettel.

1
2

3
4
5

6
7
8
9
10
11
12
13
14
15




16
17
18
19



20
21
22
23
24
25
26
27







28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
1
2
3
4
5

6
7
8
9
10
11
12




13
14
15
16
17



18
19
20
21







22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51


+


-
+






-
-
-
-
+
+
+
+

-
-
-
+
+
+

-
-
-
-
-
-
-
+
+
+
+
+
+
+















-
+







id: 00001007040000
title: Zettelmarkup: Inline-Structured Elements
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
role: manual
modified: 20210822234205

Most characters you type is concerned with inline-structured elements.
The content of a zettel contains is many cases just ordinary text, lightly formatted.
Inline-structured elements allow to format your text and add some helpful links or images.
Sometimes, you want to enter characters that have no representation on your keyboard.

=== Text formatting
Every [[text formatting|00001007040100]] element begins with two same characters at the beginning.
It lasts until the same two characters occurred the second time.
Some of these elements explicitly support [[attributes|00001007050000]].
; Text formatting
: Every [[text formatting|00001007040100]] element begins with two same characters at the beginning.
  It lasts until the same two characters occurred the second time.
  Some of these elements explicitly support [[attributes|00001007050000]].

=== Literal-like formatting
Sometime you want to render the text as it is.
This is the core motivation of [[literal-like formatting|00001007040200]].
; Literal-like formatting
: Sometime you want to render the text as it is.
: This is the core motivation of [[literal-like formatting|00001007040200]].

=== Reference-like text
You can reference other zettel and (external) material within one zettel.
This kind of reference may be a link, or an images that is display inline when the zettel is rendered.
Footnotes sometimes factor out some useful text that hinders the flow of reading text.
Internal marks allow to reference something within a zettel.
An important aspect of all knowledge work is to reference others work, e.g. with citation keys.
All these elements can be subsumed under [[reference-like text|00001007040300]].
; Reference-like text
: You can reference other zettel and (external) material within one zettel.
  This kind of reference may be a link, or an images that is display inline when the zettel is rendered.
  Footnotes sometimes factor out some useful text that hinders the flow of reading text.
  Internal marks allow to reference something within a zettel.
  An important aspect of all knowledge work is to reference others work, e.g. with citation keys.
  All these elements can be subsumed under [[reference-like text|00001007040300]].

=== Other inline elements
==== Comments
A comment begins with two consecutive percent sign characters (""''%''"", ''U+0025'').
It ends at the end of the line where it begins.

==== Backslash
The backslash character (""''\\''"", ''U+005C'') gives the next character another meaning.
* If a space character follows, it is converted in a non-breaking space (''U+00A0'').
* If a line ending follows the backslash character, the line break is converted from a //soft break// into a //hard break//.
* Every other character is taken as itself, but without the interpretation of a Zettelmarkup element.
  For example, if you want to enter a ""'']''"" into a footnote text, you should escape it with a backslash.

==== Tag
Any text that begins with a number sign character (""''#''"", ''U+0023''), followed by a non-empty sequence of Unicode letters, Unicode digits, the hyphen-minus character (""''-''"", ''U+002D''), or the low line character (""''_''"", ''U+005F'') is interpreted as an //inline tag//.
They will be considered equivalent to tags in metadata.
They are be considered equivalent to tags in metadata.

==== Entities & more
Sometimes it is not easy to enter special characters.
If you know the Unicode code point of that character, or its name according to the [[HTML standard|https://html.spec.whatwg.org/multipage/named-characters.html]], you can enter it by number or by name.

Regardless which method you use, an entity always begins with an ampersand character (""''&''"", ''U+0026'') and ends with a semicolon character (""'';''"", ''U+003B'').
If you know the HTML name of the character you want to enter, put it between these two character.

Changes to docs/manual/00001007040300.zettel.

1
2
3
4
5

6
7
8
9
10
11


12
13
14
15
16



17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
1
2
3
4
5
6
7
8
9
10


11
12





13
14
15























































































+




-
-
+
+
-
-
-
-
-
+
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001007040300
title: Zettelmarkup: Reference-like text
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20210810172531

An important aspect of knowledge work is to interconnect your zettel as well as provide links to (external) material.

There are several kinds of references that are allowed in Zettelmarkup:
* Links to other zettel.
* Links to (external material).
* [[Links to other zettel or to (external) material|00001007040310]]
* [[Embedded zettel or (external) material|00001007040320]] (""inline transclusion"")
* Embed images that are stored within your Zettelstore.
* Embed external images.
* Reference via a footnote.
* Reference via a citation key.
* Put a mark within your zettel that you can reference later with a link.
* [[Reference via a footnote|00001007040330]]
* [[Reference via a citation key|00001007040340]]
* Put a [[mark|00001007040350]] within your zettel that you can reference later with a link.

=== Links
There are two kinds of links, regardless of links to (internal) other zettel or to (external) material.
Both kinds begin with two consecutive left square bracket characters (""''[''"", ''U+005B'') and ends with two consecutive right square bracket characters (""'']''"", ''U+005D'').

The first form provides some text plus the link specification, delimited by a vertical bar character (""''|''"", ''U+007C''): ``[[text|linkspecification]]``.

The second form just provides a link specification between the square brackets.
Its text is derived from the link specification, e.g. by interpreting the link specification as text: ``[[linkspecification]]``.

The link specification for another zettel within the same Zettelstore is just the [[zettel identifier|00001006050000]].
To reference some content within a zettel, you can append a number sign character (""''#''"", ''U+0023'') and the name of the mark to the zettel identifier.
The resulting reference is called ""zettel reference"".

To specify some material outside the Zettelstore, just use an normal Uniform Resource Identifier (URI) as defined by [[RFC\ 3986|https://tools.ietf.org/html/rfc3986]].
If the URL begins with the slash character (""/"", ''U+002F''), or if it begins with ""./"" or with ""../"", i.e. without scheme, user info, and host name, the reference will be treated as a ""local reference"", otherwise as an ""external reference"".
If the URL begins with two slash characters, it will be interpreted relative to the value of [[''url-prefix''|00001004010000#url-prefix]].

The text in the second form is just a sequence of inline elements.

=== Images
To some degree, an image specification is conceptually not too far away from a link specification.
Both contain a link specification and optionally some text.
In contrast to a link, the link specification of an image must resolve to actual graphical image data.
That data is read when rendered as HTML, and is embedded inside the zettel as an inline image.

An image specification begins with two consecutive left curly bracket characters (""''{''"", ''U+007B'') and ends with two consecutive right curly bracket characters (""''}''"", ''U+007D'').
The curly brackets delimits either a link specification or some text, a vertical bar character and the link specification, similar to a link.

One difference to a link: if the text was not given, an empty string is assumed.

The link specification must reference a graphical image representation if the image is about to be rendered.
Supported formats are:

* Portable Network Graphics (""PNG""), as defined by [[RFC\ 2083|https://tools.ietf.org/html/rfc2083]].
* Graphics Interchange Format (""GIF"), as defined by [[https://www.w3.org/Graphics/GIF/spec-gif89a.txt]].
* JPEG / JPG, defined by the //Joint Photographic Experts Group//.
* Scalable Vector Graphics (SVG), defined by [[https://www.w3.org/Graphics/SVG/]]

If the text is given, it will be interpreted as an alternative textual representation, to help persons with some visual disabilities.

[[Attributes|00001007050000]] are supported.
They must follow the last right curly bracket character immediately.
One prominent example is to specify an explicit title attribute that is shown on certain web browsers when the zettel is rendered in HTML:

Examples:
* ``{{Spinning Emoji|00000000040001}}{title=Emoji width=30}`` is rendered as ::{{Spinning Emoji|00000000040001}}{title=Emoji width=30}::{=example}.
* The above image is also a placeholder for a non-existent image:
** ``{{00000000000000}}`` will be rendered as ::{{00000000000000}}::{=example}.
** ``{{00000000009999}}`` will be rendered as ::{{00000000009999}}::{=example}.

=== Footnotes

A footnote begins with a left square bracket, followed by a circumflex accent (""''^''"", ''U+005E''), followed by some text, and ends with a right square bracket.

Example:

``Main text[^Footnote text.].`` is rendered in HTML as: ::Main text[^Footnote text.].::{=example}.

=== Citation key

A citation key references some external material that is part of a bibliografical collection.

Currently, Zettelstore implements this only partially, it is ""work in progress"".

However, the syntax is: beginning with a left square bracket and followed by an at sign character (""''@''"", ''U+0040''), a the citation key is given.
The key is typically a sequence of letters and digits.
If a comma character (""'',''"", ''U+002C'') or a vertical bar character is given, the following is interpreted as inline elements.
A right square bracket ends the text and the citation key element.

=== Mark
A mark allows to name a point within a zettel.
This is useful if you want to reference some content in a bigger-sized zettel[^Other uses of marks will be given, if Zettelmarkup is extended by a concept called //transclusion//.].

A mark begins with a left square bracket, followed by an exclamation mark character (""''!''"", ''U+0021'').
Now the optional mark name follows.
It is a (possibly empty) sequence of Unicode letters, Unicode digits, the hyphen-minus character (""''-''"", ''U+002D''), or the low-line character (""''_''"", ''U+005F'').
The mark element ends with a right square bracket.

Examples:
* ``[!]`` is a mark without a name, the empty mark.
* ``[!mark]`` is a mark with the name ""mark"".

Added docs/manual/00001007040310.zettel.
























1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001007040310
title: Zettelmarkup: Links
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

There are two kinds of links, regardless of links to (internal) other zettel or to (external) material.
Both kinds begin with two consecutive left square bracket characters (""''[''"", ''U+005B'') and ends with two consecutive right square bracket characters (""'']''"", ''U+005D'').

The first form provides some text plus the link specification, delimited by a vertical bar character (""''|''"", ''U+007C''): ``[[text|linkspecification]]``.

The second form just provides a link specification between the square brackets.
Its text is derived from the link specification, e.g. by interpreting the link specification as text: ``[[linkspecification]]``.

The link specification for another zettel within the same Zettelstore is just the [[zettel identifier|00001006050000]].
To reference some content within a zettel, you can append a number sign character (""''#''"", ''U+0023'') and the name of the mark to the zettel identifier.
The resulting reference is called ""zettel reference"".

To specify some material outside the Zettelstore, just use an normal Uniform Resource Identifier (URI) as defined by [[RFC\ 3986|https://tools.ietf.org/html/rfc3986]].
If the URL begins with the slash character (""/"", ''U+002F''), or if it begins with ""./"" or with ""../"", i.e. without scheme, user info, and host name, the reference will be treated as a ""local reference"", otherwise as an ""external reference"".
If the URL begins with two slash characters, it will be interpreted relative to the value of [[''url-prefix''|00001004010000#url-prefix]].

The text in the second form is just a sequence of inline elements.

Added docs/manual/00001007040320.zettel.


























1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001007040320
title: Zettelmarkup: Inline Embedding / Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20210811162201

To some degree, an specification for embedded material is conceptually not too far away from a specification for [[linked material|00001007040310]].
Both contain a reference specification and optionally some text.
In contrast to a link, the specification of embedded material must currently resolve to some kind of real content.
This content replaces the embed specification.

An image specification begins with two consecutive left curly bracket characters (""''{''"", ''U+007B'') and ends with two consecutive right curly bracket characters (""''}''"", ''U+007D'').
The curly brackets delimits either a reference specification or some text, a vertical bar character and the link specification, similar to a link.

One difference to a link: if the text was not given, an empty string is assumed.

The reference must point to some content, either zettel content or URL-referenced content.
If the referenced zettel does not exist, or is not readable, a spinning emoji is presented as a visual hint:
 
Example: ``{{00000000000000}}`` will be rendered as ::{{00000000000000}}::{=example}.

There are two kind of content:
# [[image content|00001007040322]],
# [[textual content|00001007040324]].

Added docs/manual/00001007040322.zettel.



























1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001007040322
title: Zettelmarkup: Image Embedding
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20210811165719

Image content is assumed, if an URL is used or if the referenced zettel contains an image.

Supported formats are:

* Portable Network Graphics (""PNG""), as defined by [[RFC\ 2083|https://tools.ietf.org/html/rfc2083]].
* Graphics Interchange Format (""GIF"), as defined by [[https://www.w3.org/Graphics/GIF/spec-gif89a.txt]].
* JPEG / JPG, defined by the //Joint Photographic Experts Group//.
* Scalable Vector Graphics (SVG), defined by [[https://www.w3.org/Graphics/SVG/]]

If the text is given, it will be interpreted as an alternative textual representation, to help persons with some visual disabilities.

[[Attributes|00001007050000]] are supported.
They must follow the last right curly bracket character immediately.
One prominent example is to specify an explicit title attribute that is shown on certain web browsers when the zettel is rendered in HTML:

Examples:
* [!spin] ``{{Spinning Emoji|00000000040001}}{title=Emoji width=30}`` is rendered as ::{{Spinning Emoji|00000000040001}}{title=Emoji width=30}::{=example}.
* The above image is also the placeholder for a non-existent zettel:
** ``{{00000000009999}}`` will be rendered as ::{{00000000009999}}::{=example}.

Added docs/manual/00001007040324.zettel.










































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001007040324
title: Zettelmarkup: Inline Transclusion of Text
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20210811172440

Inline transclusion applies to all zettel that are parsed in an non-trivial way.
For example, textual content is assumed if the [[syntax|00001006020000#syntax]] of a zettel is ''zmk'' ([[Zettelmarkup|00001007000000]]), or ''markdown'' / ''md'' ([[Markdown|00001008010000]]).

Since the transclusion is at the level of [[inline-structured elements|00001007040000]], the transclude specification must be replaced with some inline-structured elements.

First, the referenced zettel is read.
If it contains inline transclusions, these will be expanded, recursively.
When an endless recursion is detected, expansion does not take place.
Instead an error message replaces the transclude specification.

The result of this (indirect) transclusion is searched for inline-structured elements.

* If only the zettel identifier was specified, the first top-level [[paragraph|00001007030000#paragraphs]] is used.
  Since a paragraph is basically a sequence of inline-structured elements, these elements will replace the transclude specification.

  Example: ``{{00010000000000}}`` (see [[00010000000000]]) is rendered as ::{{00010000000000}}::{=example}.

* If a fragment identifier was additionally specified, the element with the given fragment is searched:
** If it specifies a [[heading|00001007030300]], the next top-level paragraph is used.

   Example: ``{{00010000000000#reporting-errors}}`` is rendered as ::{{00010000000000#reporting-errors}}::{=example}.

** In case the fragment names a [[mark|00001007040350]], the inline-structured elements after the mark are used.
   Initial spaces and line breaks are ignored in this case.

   Example: ``{{00001007040322#spin}}`` is rendered as ::{{00001007040322#spin}}::{=example}.

** Just specifying the fragment identifier will reference something in the current page.
   This is not allowed, to prevent a possible endless recursion.

If no inline-structured elements are found, the transclude specification is replaced by an error message.

To avoid an exploding ""transclusion bomb"", a form of a [[billion laughs attack|https://en.wikipedia.org/wiki/Billion_laughs_attack]] (also known as ""XML bomb""), the total number of transclusions / expansions is limited.
The limit can be controlled by setting the value [[''max-transclusions''|00001004020000#max-transclusions]] of the runtime configuration zettel.

Added docs/manual/00001007040330.zettel.












1
2
3
4
5
6
7
8
9
10
11
+
+
+
+
+
+
+
+
+
+
+
id: 00001007040330
title: Zettelmarkup: Footnotes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

A footnote begins with a left square bracket, followed by a circumflex accent (""''^''"", ''U+005E''), followed by some text, and ends with a right square bracket.

Example:

``Main text[^Footnote text.].`` is rendered in HTML as: ::Main text[^Footnote text.].::{=example}.

Added docs/manual/00001007040340.zettel.
















1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001007040340
title: Zettelmarkup: Citation Key
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20210810172339

A citation key references some external material that is part of a bibliografical collection.

Currently, Zettelstore implements this only partially, it is ""work in progress"".

However, the syntax is: beginning with a left square bracket and followed by an at sign character (""''@''"", ''U+0040''), a the citation key is given.
The key is typically a sequence of letters and digits.
If a comma character (""'',''"", ''U+002C'') or a vertical bar character is given, the following is interpreted as inline elements.
A right square bracket ends the text and the citation key element.

Added docs/manual/00001007040350.zettel.


















1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001007040350
title: Zettelmarkup: Mark
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

A mark allows to name a point within a zettel.
This is useful if you want to reference some content in a bigger-sized zettel, currently with a [[link|00001007040310]] only[^Other uses of marks will be given, if Zettelmarkup is extended by a concept called //transclusion//.].

A mark begins with a left square bracket, followed by an exclamation mark character (""''!''"", ''U+0021'').
Now the optional mark name follows.
It is a (possibly empty) sequence of Unicode letters, Unicode digits, the hyphen-minus character (""''-''"", ''U+002D''), or the low-line character (""''_''"", ''U+005F'').
The mark element ends with a right square bracket.

Examples:
* ``[!]`` is a mark without a name, the empty mark.
* ``[!mark]`` is a mark with the name ""mark"".

Changes to docs/manual/00001007050000.zettel.

1
2

3
4
5

6
7
8
9
10
11

12
13
14
15
16
17
18
1
2
3
4
5

6
7
8
9
10
11

12
13
14
15
16
17
18
19


+


-
+





-
+







id: 00001007050000
title: Zettelmarkup: Attributes
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
role: manual
modified: 20210830161438

Attributes allows to modify the way how material is presented.
Alternatively, they provide additional information to markup elements.
To some degree, attributes are similar to [[HTML attributes|https://html.spec.whatwg.org/multipage/dom.html#global-attributes]].

Typical use cases for attributes are to specify the (natural) [[language|00001007050100]] for a text region, to specify the [[programming language|00001007050200]] for highlighting program code, or to make white space visible in raw text.
Typical use cases for attributes are to specify the (natural) [[language|00001007050100]] for a text region, to specify the [[programming language|00001007050200]] for highlighting program code, or to make white space visible in plain text.

Attributes are specified within curly brackets ``{...}``.
Of course, more than one attribute can be specified.
Attributes are separated by a sequence of space characters or by a comma character.

An attribute normally consists of an optional key and an optional value.
The key is a sequence of letters, digits, a hyphen-minus (""''-''"", ''U+002D'', and a low line / underscore (""''_''"", ''U+005D'').
43
44
45
46
47
48
49
50

51
52
53
54
55
56
57
44
45
46
47
48
49
50

51
52
53
54
55
56
57
58







-
+








This is not true for the generic attribute.
In ``{=key1 =key2}``, the first key is ignored.
Therefore it is equivalent to ``{=key2}``.

The key ""''-''"" (just hyphen-minus) is special.
It is called //default attribute//{-} and has a markup specific meaning.
For example, when used for raw text, it replaces the non-visible space with a visible representation:
For example, when used for plain text, it replaces the non-visible space with a visible representation:

* ++``Hello, world``{-}++ produces ==Hello, world=={-}.
* ++``Hello, world``++ produces ==Hello, world==.

For some block elements, there is a syntax variant if you only want to specify a generic attribute.
For all line-range blocks you can specify the generic attributes directly in the first line, after the three (or more) block characters.

Changes to docs/manual/00001007060000.zettel.

1
2

3
4
5

6
7
8
9
10
11
12
1
2
3
4
5

6
7
8
9
10
11
12
13


+


-
+







id: 00001007060000
title: Zettelmarkup: Summary of Formatting Characters
role: manual
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
role: manual
modified: 20210728162830

The following table gives an overview about the use of all characters that begin a markup element.

|= Character :|= Blocks <|= Inlines <
| ''!''  | (free) | (free)
| ''"''  | [[Verse block|00001007030700]] | [[Typographic quotation mark|00001007040100]]
| ''#''  | [[Ordered list|00001007030200]] | [[Tag|00001007040000]]
25
26
27
28
29
30
31
32

33
34
35
36
37
38
39
40



41
26
27
28
29
30
31
32

33
34
35
36
37
38



39
40
41
42







-
+





-
-
-
+
+
+

| '':''  | [[Region block|00001007030800]] / [[description text|00001007030100]] | [[Inline region|00001007040100]]
| '';''  | [[Description term|00001007030100]] | [[Small text|00001007040100]]
| ''<''  | [[Quotation block|00001007030600]] | [[Short inline quote|00001007040100]]
| ''=''  | [[Headings|00001007030300]] | [[Computer output|00001007040200]]
| ''>''  | [[Quotation lists|00001007030200]] | (blocked: to remove anyambiguity with quotation lists)
| ''?''  | (free) | (free)
| ''@''  | (free) | (reserved)
| ''[''  | (reserved)  | [[Link|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]
| ''[''  | (reserved)  | [[Linked material|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]
| ''\\'' | (blocked by inline meaning) | [[Escape character|00001007040000]]
| '']''  | (reserved) | End of link, citation key, footnote, mark
| ''^''  | (free) | [[Superscripted text|00001007040100]]
| ''_''  | (free) | [[Underlined text|00001007040100]]
| ''`''  | [[Verbatim block|00001007030500]] | [[Literal text|00001007040200]]
| ''{''  | (reserved) | [[Image|00001007040300]], [[Attribute|00001007050000]]
| ''|''  | [[Table row / table cell|00001007031000]] | Separator within link and image formatting
| ''}''  | (reserved) | End of Image, End of Attribute
| ''{''  | (reserved) | [[Embedded material|00001007040300]], [[Attribute|00001007050000]]
| ''|''  | [[Table row / table cell|00001007031000]] | Separator within link and embed formatting
| ''}''  | (reserved) | End of embedded material, End of Attribute
| ''~''  | (free) | [[Strike-through text|00001007040100]]

Changes to docs/manual/00001008010000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
1
2
3
4
5

6
7
8
9
10
11
12
13





-
+







id: 00001008010000
title: Use Markdown within Zettelstore
role: manual
tags: #manual #markdown #zettelstore
syntax: zmk
modified: 20210518102155
modified: 20210726191610

If you are customized to use Markdown as your markup language, you can configure Zettelstore to support your decision.
Zettelstore supports [[CommonMark|https://commonmark.org/]], an [[attempt|https://xkcd.com/927/]] to unify all the different, divergent dialects of Markdown.

=== Use Markdown as the default markup language of Zettelstore

Add the key ''default-syntax'' with a value of ''md'' or ''markdown'' to the [[configuration zettel|00000000000100]].
38
39
40
41
42
43
44
45

46
47
38
39
40
41
42
43
44

45
46
47







-
+


An attacker might include malicious HTML code in your zettel.
For example, HTML allows to embed JavaScript, a full-sized programming language that drives many web sites.
When a zettel is displayed, JavaScript code might be executed, sometimes with harmful results.

Zettelstore mitigates this problem by ignoring suspicious text when it encodes a zettel as HTML.
Any HTML text that might contain the ``<script>`` tag or the ``<iframe>`` tag is ignored.
This may lead to unexpected results if you depend on these.
Other encoding [[formats|00001012920500]] may still contain the full HTML text.
Other [[encodings|00001012920500]] may still contain the full HTML text.

Any external client of Zettelstore, which does not use Zettelstore's HTML encoding, must be programmed to take care of malicious code.

Changes to docs/manual/00001010070200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
1
2
3
4
5

6
7
8
9
10
11
12
13





-
+







id: 00001010070200
title: Visibility rules for zettel
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
modified: 20210510194652
modified: 20210728162201

For every zettel you can specify under which condition the zettel is visible to others.
This is controlled with the metadata key [[''visibility''|00001006020000#visibility]].
The following values are supported:

; [!public]""public""
: The zettel is visible to everybody, even if the user is not authenticated.
26
27
28
29
30
31
32
33

34
35
36
37
38
39
26
27
28
29
30
31
32

33
34
35
36
37
38
39







-
+






  Computed zettel with internal runtime information are examples for such a zettel.

When you install a Zettelstore, only some zettel have visibility ""public"".
One is the zettel that contains CSS for displaying the web interface.
This is to ensure that the web interface looks nice even for not authenticated users.
Another is the zettel containing the [[version|00000000000001]] of the Zettelstore.
Yet other zettel lists the Zettelstore [[license|00000000000004]], its [[contributors|00000000000005]], and external, licensed [[dependencies|00000000000006]], such as program code written by others or graphics designed by others.
The [[default image|00000000040001]], used if an image reference is invalid,is also public visible.
The [[default image|00000000040001]], used if an image reference is invalid, is also public visible.

Please note: if authentication is not enabled, every user has the same rights as the owner of a Zettelstore.
This is also true, if the Zettelstore runs additionally in [[read-only mode|00001004010000#read-only-mode]].
In this case, the [[runtime configuration zettel|00001004020000]] is shown (its visibility is ""owner"").
The [[startup configuration|00001004010000]] is not shown, because the associated computed zettel with identifier ''00000000000096'' is stored with the visibility ""expert"".
If you want to show such a zettel, you must set ''expert-mode'' to true.

Changes to docs/manual/00001012000000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


42
43
44
45


46

47
48
49
50
51
1
2
3
4
5

6
7
8
9
10
11
12

13
14
15
16







17
18
19
20
21
22
23
24


25
26

27
28
29


30
31
32
33
34
35
36
37

38
39
40
41
42
43





-
+






-
+



-
-
-
-
-
-
-








-
-


-



-
-
+
+




+
+
-
+





id: 00001012000000
title: API
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210721120820
modified: 20210908220502

The API (short for ""**A**pplication **P**rogramming **I**nterface"") is the primary way to communicate with a running Zettelstore.
Most integration with other systems and services is done through the API.
The [[web user interface|00001014000000]] is just an alternative, secondary way of interacting with a Zettelstore.

=== Background
The API is HTTP-based and uses JSON as its main encoding format for exchanging messages between a Zettelstore and its client software.
The API is HTTP-based and uses plain text and JSON as its main encoding format for exchanging messages between a Zettelstore and its client software.

There is an [[overview zettel|00001012920000]] that shows the structure of the endpoints used by the API and gives an indication about its use.

While JSON is the main encoding format, it is possible to retrieve zettel representations in other formats.
If you want to create a new zettel or to change an existing one, you have to use JSON.
There is an [[overview zettel for encoding formats|00001012920500]] that describes the valid formats.

Various parts of a zettel can be retrieved.
There are the [[possible values to specify zettel parts|00001012920800]].

=== Authentication
If [[authentication is enabled|00001010040100]], most API calls must include an [[access token|00001010040700]] that proves the identity of the caller.
* [[Authenticate an user|00001012050200]] to obtain an access token
* [[Renew an access token|00001012050400]] without costly re-authentication
* [[Provide an access token|00001012050600]] when doing an API call

=== Zettel lists
* [[List metadata of all zettel|00001012051200]]
* [[List all zettel, but in different encoding formats|00001012051400]]
* [[List all zettel, but include different parts of a zettel|00001012051600]]
* [[Shape the list of zettel metadata|00001012051800]]
** [[Selection of zettel|00001012051810]]
** [[Zettel parts|00001012051820]]
** [[Limit the list length|00001012051830]]
** [[Content search|00001012051840]]
** [[Sort the list of zettel metadata|00001012052000]]
* [[List all tags|00001012052200]]
* [[List all roles|00001012052400]]
* [[List all tags|00001012052400]]
* [[List all roles|00001012052600]]

=== Working with zettel
* [[Create a new zettel|00001012053200]]
* [[Retrieve metadata and content of an existing zettel|00001012053400]]
* [[Retrieve evaluated metadata and content of an existing zettel in various encodings|00001012053500]]
* [[Retrieve parsed metadata and content of an existing zettel in various encodings|00001012053600]]
* [[Retrieve references of an existing zettel|00001012053600]]
* [[Retrieve references of an existing zettel|00001012053700]]
* [[Retrieve context of an existing zettel|00001012053800]]
* [[Retrieve zettel order within an existing zettel|00001012054000]]
* [[Update metadata and content of a zettel|00001012054200]]
* [[Rename a zettel|00001012054400]]
* [[Delete a zettel|00001012054600]]

Changes to docs/manual/00001012050200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14

15
16
17
18
19
20

21
22
23
24
25
26

27
28
29
30
31
32
33
1
2
3
4
5

6
7
8
9
10
11

12
13

14
15
16
17
18
19

20
21
22
23
24
25

26
27
28
29
30
31
32
33





-
+





-
+

-
+





-
+





-
+







id: 00001012050200
title: API: Authenticate a client
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210712221945
modified: 20210726123709

Authentication for future API calls is done by sending a [[user identification|00001010040200]] and a password to the Zettelstore to obtain an [[access token|00001010040700]].
This token has to be used for other API calls.
It is valid for a relatively short amount of time, as configured with the key ''token-timeout-api'' of the [[startup configuration|00001004010000]] (typically 10 minutes).

The simplest way is to send user identification (''IDENT'') and password (''PASSWORD'') via [[HTTP Basic Authentication|https://tools.ietf.org/html/rfc7617]] and send them to the [[endpoint|00001012920000]] ''/v''[^The endpoint ''/a'' is already taken for the [[web user interface|00001014000000]], ""v"" stands for the German word ""verbürgen""] with a POST request:
The simplest way is to send user identification (''IDENT'') and password (''PASSWORD'') via [[HTTP Basic Authentication|https://tools.ietf.org/html/rfc7617]] and send them to the [[endpoint|00001012920000]] ''/a'' with a POST request:
```sh
# curl -X POST -u IDENT:PASSWORD http://127.0.0.1:23123/v
# curl -X POST -u IDENT:PASSWORD http://127.0.0.1:23123/a
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":600}
```

Some tools, like [[curl|https://curl.haxx.se/]], also allow to specify user identification and password as part of the URL:
```sh
# curl -X POST http://IDENT:PASSWORD@127.0.0.1:23123/v
# curl -X POST http://IDENT:PASSWORD@127.0.0.1:23123/a
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":600}
```

If you do not want to use Basic Authentication, you can also send user identification and password as HTML form data:
```sh
# curl -X POST -d 'username=IDENT&password=PASSWORD' http://127.0.0.1:23123/v
# curl -X POST -d 'username=IDENT&password=PASSWORD' http://127.0.0.1:23123/a
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":600}
```

In all cases, you will receive an JSON object will all [[relevant data|00001012921000]] to be used for further API calls.

**Important:** obtaining a token is a time-intensive process.
Zettelstore will delay every request to obtain a token for a certain amount of time.

Changes to docs/manual/00001012050400.zettel.

1
2
3
4
5
6

7
8
9
10
11

12
13
14

15
16
17
18
19
20
21
1
2
3
4
5

6
7
8
9
10

11
12
13

14
15
16
17
18
19
20
21





-
+




-
+


-
+







id: 00001012050400
title: API: Renew an access token
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210712222135
modified: 20210726123745

An access token is only valid for a certain duration.
Since the [[authentication process|00001012050200]] will need some processing time, there is a way to renew the token without providing full authentication data.

Send a HTTP PUT request to the [[endpoint|00001012920000]] ''/v'' and include the current access token in the ''Authorization'' header:
Send a HTTP PUT request to the [[endpoint|00001012920000]] ''/a'' and include the current access token in the ''Authorization'' header:

```sh
# curl -X PUT -H 'Authorization: Bearer TOKEN' http://127.0.0.1:23123/v
# curl -X PUT -H 'Authorization: Bearer TOKEN' http://127.0.0.1:23123/a
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":456}
```
You may receive a new access token, or the current one if it was obtained not a long time ago.
However, the lifetime of the returned [[access token|00001012921000]] is accurate.

=== HTTP Status codes
; ''200''

Changes to docs/manual/00001012050600.zettel.

1
2

3
4
5

6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
1
2
3
4
5

6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
21


+


-
+







-
+







id: 00001012050600
title: API: Provide an access token
role: manual
tags: #api #manual #zettelstore
syntax: zmk
role: manual
modified: 20210830161320

The [[authentication process|00001012050200]] provides you with an [[access token|00001012921000]].
Most API calls need such an access token, so that they know the identity of the caller.

You send the access token in the ""Authorization"" request header field, as described in [[RFC 6750, section 2.1|https://tools.ietf.org/html/rfc6750#section-2.1]].
You need to use the ""Bearer"" authentication scheme to transmit the access token.

For example (raw HTTP):
For example (in plain text HTTP):
```
GET /z HTTP/1.0
Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg
```
Note, that there is exactly one space character (''U+0020'') between the string ""Bearer"" and the access token: ``Authorization: Bearer eyJhbGciOiJIUzUxMiJ9.ey...``{-}.

If you use the [[curl|https://curl.haxx.se/]] tool, you can use the ++-H++ command line parameter to set this header field.

Changes to docs/manual/00001012051200.zettel.

1
2
3
4
5

6
7

8
9
10
11
12


13
14
15
16
17


18
19

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80




81


82
83






84
85
86
87
88
89
90
91
1
2
3
4
5
6
7

8
9
10
11


12
13
14
15
16


17
18
19

20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35

36
37
38
39
40
41
42
43
44

45
46
47
48
49
50
51
52
53

54
55
56
57
58
59
60
61
62

63
64
65
66
67
68
69
70
71
72
73
74


75
76
77
78
79
80
81


82
83
84
85
86
87
88
89
90
91
92
93
94
95





+

-
+



-
-
+
+



-
-
+
+

-
+






-









-









-









-









-












-
-
+
+
+
+

+
+
-
-
+
+
+
+
+
+








id: 00001012051200
title: API: List metadata of all zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210905203616

To list the metadata of all zettel just send a HTTP GET request to the [[endpoint|00001012920000]] ''/z''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
To list the metadata of all zettel just send a HTTP GET request to the [[endpoint|00001012920000]] ''/j''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
If successful, the output is a JSON object:

```sh
# curl http://127.0.0.1:23123/z
{"list":[{"id":"00001012051200","url":"/z/00001012051200","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012050600","url":"/z/00001012050600","meta":{"title":"API: Provide an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012050400","url":"/z/00001012050400","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012050200","url":"/z/00001012050200","meta":{"title":"API: Authenticate a client","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012000000","url":"/z/00001012000000","meta":{"title":"API","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}}]}
# curl http://127.0.0.1:23123/j
{"list":[{"id":"00001012051200","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012050600","meta":{"title":"API: Provide an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012050400","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012050200","meta":{"title":"API: Authenticate a client","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012000000","meta":{"title":"API","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"}}]}
```

The JSON object contains a key ''"list"'' where its value is a list of zettel JSON objects.
These zettel JSON objects themself contains the keys ''"id"'' (value is a string containing the zettel identifier), ''"url"'' (value is a string containing the URL of the zettel), and ''"meta"'' (value ss a JSON object).
The vlaue of key ''"meta"'' effectively contains all metadata of the identified zettel, where metadata keys are encoded as JSON object keys and metadata values encoded as JSON strings.
These zettel JSON objects themself contains the keys ''"id"'' (value is a string containing the zettel identifier), , and ''"meta"'' (value as a JSON object).
The value of key ''"meta"'' effectively contains all metadata of the identified zettel, where metadata keys are encoded as JSON object keys and metadata values encoded as JSON strings.

If you reformat the JSON output from the ''GET /z'' call, you'll see its structure better:
If you reformat the JSON output from the ''GET /j'' call, you'll see its structure better:

```json
{
  "list": [
    {
      "id": "00001012051200",
      "url": "/z/00001012051200",
      "meta": {
        "title": "API: List for all zettel some data",
        "tags": "#api #manual #zettelstore",
        "syntax": "zmk",
        "role": "manual"
      }
    },
    {
      "id": "00001012050600",
      "url": "/z/00001012050600",
      "meta": {
        "title": "API: Provide an access token",
        "tags": "#api #manual #zettelstore",
        "syntax": "zmk",
        "role": "manual"
      }
    },
    {
      "id": "00001012050400",
      "url": "/z/00001012050400",
      "meta": {
        "title": "API: Renew an access token",
        "tags": "#api #manual #zettelstore",
        "syntax": "zmk",
        "role": "manual"
      }
    },
    {
      "id": "00001012050200",
      "url": "/z/00001012050200",
      "meta": {
        "title": "API: Authenticate a client",
        "tags": "#api #manual #zettelstore",
        "syntax": "zmk",
        "role": "manual"
      }
    },
    {
      "id": "00001012000000",
      "url": "/z/00001012000000",
      "meta": {
        "title": "API",
        "tags": "#api #manual #zettelstore",
        "syntax": "zmk",
        "role": "manual"
      }
    }
  ]
}
```
In this special case, the metadata of each zettel just contains the four default keys ''title'', ''tags'', ''syntax'', and ''role''.

If you specify the request in above simple way, you will always get a JSON object that contains all zettel maintained by your Zettelstore and which contains just the metadata of each zettel.
There are several ways to change this behavior.
[!plain]Alternatively, you can retrieve the list of zettel in a simple, plain format using the [[endpoint|00001012920000]] ''/z''.
In this case, a plain text document is returned, with one line per zettel.
Each line contains in the first 14 characters the zettel identifier.
Separated by a space character, the title of the zettel follows:

```sh
# curl http://127.0.0.1:23123/z
* [[Specify a different encoding format|00001012051400]]
* [[Specify which detail information of a zettel should be included|00001012051600]]
00001012051200 API: Renew an access token
00001012050600 API: Provide an access token
00001012050400 API: Renew an access token
00001012050200 API: Authenticate a client
00001012000000 API
```

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an [[appropriate JSON object|00001012921000]].
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the access bearer token was not valid.

Deleted docs/manual/00001012051400.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82


















































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012051400
title: API: List all zettel, but in different encoding formats
tags: #api #manual #zettelstore
syntax: zmk
role: manual

You can add a query parameter ''_format=[[FORMAT|00001012920500]]'' to select the encoding format when [[retrieving all zettel|00001012051200]].
Probably some formats are not very useful and may not make sense.
Currently implemented are the formats [[''json''|00001012920501]] (default), [[''djson''|00001012920503]], and [[''html''|00001012920510]].

The format ''json'' will be selected, if no ''_format'' is specified.
''djson'' will return a JSON object with some more detailed information.
For example, the [[Zettelmarkup|00001007000000]] structure will be returned and tags are returned as a list of strings.
''html'' will return an HTML encoding of the zettel list, using a HTML unnumbered list.

```sh
# curl 'http://127.0.0.1:23123/z?_format=djson'
{"list":[{"id":"00001012051400","url":"/z/00001012051400?_format=djson","meta":{"title":[{"t":"Text","s":"API:"},{"t":"Space"},{"t":"Text","s":"List"},{"t":"Space"},{"t":"Text","s":"all"},{"t":"Space"},{"t":"Text","s":"zettel,"},{"t":"Space"},{"t":"Text","s":"but"},{"t":"Space"},{"t":"Text","s":"in"},{"t":"Space"},{"t":"Text","s":"different"},{"t":"Space"},{"t":"Text","s":"encoding"},{"t":"Space"},{"t":"Text","s":"formats"}],"tags":["#api","#manual","#zettelstore"],"syntax":"zmk","role":"manual"}}, ...
```
Or reformatted:
````json
{
  "list": [
    {
      "id": "00001012051400",
      "url": "/z/00001012051400?_format=djson",
      "meta": {
        "title": [
          {
            "t": "Text",
            "s": "API:"
          },
          {
            "t": "Space"
          },
          {
            "t": "Text",
            "s": "List"
          },
          {
            "t": "Space"
          },
          {
            "t": "Text",
            "s": "all"
          },
          {
            "t": "Space"
          },
          {
            "t": "Text",
            "s": "zettel,"
          },
...
````

```sh
# curl 'http://127.0.0.1:23123/z?_format=html'
<html lang="en">
<body>
<ul>
<li><a href="/z/00001012051400?_format=html">API: List all zettel, but in different encoding formats</a></li>
<li><a href="/z/00001012051200?_format=html">API: List metadata of all zettel</a></li>
<li><a href="/z/00001012050600?_format=html">API: Provide an access token</a></li>
<li><a href="/z/00001012050400?_format=html">API: Renew an access token</a></li>
<li><a href="/z/00001012050200?_format=html">API: Authenticate a client</a></li>
<li><a href="/z/00001012000000?_format=html">API</a></li>
</ul>
</body>
</html>```

If you request a JSON-based encoding format, you can request additional [[parts of a zettel|00001012051600]].

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an [[appropriate JSON object|00001012921000]].
; ''400''
: Request was not valid.
  There are several reasons for this.
  Maybe you provided a wrong value for ''_format''.
; ''501''
: Encoding format is not yet implemented, but will be in the future.

Deleted docs/manual/00001012051600.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72








































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012051600
title: API: List all zettel, but include different parts of a zettel
tags: #api #manual #zettelstore
syntax: zmk
role: manual

For JSON-based formats[^[[''json''|00001012920501]] and [[''djson''|00001012920503]]] you can add a query parameter ''_part=[[PART|00001012920800]]'' to select which parts of a zettel must be encoded.

All allowed parts are implemented.
The JSON keys ''"id"'' and ''"url"'' are always included.
The default value is ''_part=meta''.

If you just want to know the zettel identifier and the zettel url, specify ''_part=id'':

```sh
# curl 'http://127.0.0.1:23123/z?_format=djson&_part=id'
{"list":[{"id":"00001012051600","url":"/z/00001012051600?_format=djson"},{"id":"00001012051400","url":"/z/00001012051400?_format=djson"},{"id":"00001012051200","url":"/z/00001012051200?_format=djson"},{"id":"00001012050600","url":"/z/00001012050600?_format=djson"},{"id":"00001012050400","url":"/z/00001012050400?_format=djson"},{"id":"00001012050200","url":"/z/00001012050200?_format=djson"},{"id":"00001012000000","url":"/z/00001012000000?_format=djson"}]}
```
Or reformatted:
````json
{
  "list": [
    {
      "id": "00001012051600",
      "url": "/z/00001012051600?_format=djson"
    },
    {
      "id": "00001012051400",
      "url": "/z/00001012051400?_format=djson"
    },
    {
      "id": "00001012051200",
      "url": "/z/00001012051200?_format=djson"
    },
    {
      "id": "00001012050600",
      "url": "/z/00001012050600?_format=djson"
    },
    {
      "id": "00001012050400",
      "url": "/z/00001012050400?_format=djson"
    },
    {
      "id": "00001012050200",
      "url": "/z/00001012050200?_format=djson"
    },
    {
      "id": "00001012000000",
      "url": "/z/00001012000000?_format=djson"
    }
  ]
}
````

A value ''_part=content'' will include the zettel content.
If ''_format=json'', the content is a string value.
For ''_format=djson'', a detailed JSON value will be returned.
```sh
# curl 'http://127.0.0.1:23123/z?_part=content'
{"list":[{"id":"00001012051600","url":"/z/00001012051600","content":"For JSON-based formats[^[[''json''|00001012920501]] and [[''djson''|00001012920503]]] you can add a query parameter ''_part=[[PART|00001012920800]]'' to select the encoding format when [[retrieving all zettel|00001012051200]].\n\nAll given parts are implemented.\nThe JSON keys ''\"id\"'' ...
```

The value ''_part=zettel'' will include both the metadata of a zettel //and// its zettel content.
Metadata will additionally include all autogenerated default values, such as the key ''"copyright"'' and ''"license"''.

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an [[appropriate JSON object|00001012921000]].
; ''400''
: Request was not valid.
  There are several reasons for this.
  Maybe you provided a wrong value for ''_part''.

Changes to docs/manual/00001012051800.zettel.

1
2
3
4
5
6

7
8
9
10

11
12
13
14
15
16
1
2
3
4
5

6
7
8
9

10
11
12

13
14
15





-
+



-
+


-



id: 00001012051800
title: API: Shape the list of zettel metadata
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210721120658
modified: 20210905203719

In most cases, it is not essential to list //all// zettel.
Typically, you are interested only in a subset of the zettel maintained by your Zettelstore.
This is done by adding some query parameters to the general ''GET /z'' request.
This is done by adding some query parameters to the general ''GET /j'' request.

* [[Select|00001012051810]] just some zettel, based on metadata.
* You can specify which [[parts of a zettel|00001012051820]] must be returned.
* Only a specific amount of zettel will be selected by specifying [[a length and/or an offset|00001012051830]].
* [[Searching for specific content|00001012051840]], not just the metadata, is another way of selecting some zettel.
* The resulting list can be [[sorted|00001012052000]] according to various criteria.

Changes to docs/manual/00001012051810.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16


17
18
19
20
21
22
23


24
25
26
27
28
29
30

31
32
33
34
35

36
37
38
39
40

41
42










1
2
3
4
5

6
7
8
9
10
11
12
13
14


15
16
17
18
19
20
21


22
23
24
25
26
27
28
29

30
31
32
33
34

35
36
37
38
39

40
41
42
43
44
45
46
47
48
49
50
51
52





-
+








-
-
+
+





-
-
+
+






-
+




-
+




-
+


+
+
+
+
+
+
+
+
+
+
id: 00001012051810
title: API: Select zettel based on their metadata
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210721121038
modified: 20210905203929

Every query parameter that does //not// begin with the low line character (""_"", ''U+005F'') is treated as the name of a [[metadata|00001006010000]] key.
According to the [[type|00001006030000]] of a metadata key, zettel are possibly selected.
All [[supported|00001006020000]] metadata keys have a well-defined type.
User-defined keys have the type ''e'' (string, possibly empty).

For example, if you want to retrieve all zettel that contain the string ""API"" in its title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/z?title=API'
{"list":[{"id":"00001012921000","url":"/z/00001012921000","meta":{"title":"API: JSON structure of an access token","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920500","url":"/z/00001012920500","meta":{"title":"Formats available by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920000","url":"/z/00001012920000","meta":{"title":"Endpoints used by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}}, ...
# curl 'http://127.0.0.1:23123/j?title=API'
{"list":[{"id":"00001012921000","meta":{"title":"API: JSON structure of an access token","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920500","meta":{"title":"Formats available by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920000","meta":{"title":"Endpoints used by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}}, ...
```

However, if you want all zettel that does //not// match a given value, you must prefix the value with the exclamation mark character (""!"", ''U+0021'').
For example, if you want to retrieve all zettel that do not contain the string ""API"" in their title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/z?title=!API'
{"list":[{"id":"00010000000000","url":"/z/00010000000000","meta":{"back":"00001003000000 00001005090000","backward":"00001003000000 00001005090000","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00000000000001 00000000000003 00000000000096 00000000000100","lang":"en","license":"EUPL-1.2-or-later","role":"zettel","syntax":"zmk","title":"Home"}},{"id":"00001014000000","url":"/z/00001014000000","meta":{"back":"00001000000000 00001004020000 00001012920510","backward":"00001000000000 00001004020000 00001012000000 00001012920510","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00001012000000","lang":"en","license":"EUPL-1.2-or-later","published":"00001014000000","role":"manual","syntax":"zmk","tags":"#manual #webui #zettelstore","title":"Web user interface"}},
# curl 'http://127.0.0.1:23123/j?title=!API'
{"list":[{"id":"00010000000000","meta":{"back":"00001003000000 00001005090000","backward":"00001003000000 00001005090000","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00000000000001 00000000000003 00000000000096 00000000000100","lang":"en","license":"EUPL-1.2-or-later","role":"zettel","syntax":"zmk","title":"Home"}},{"id":"00001014000000","meta":{"back":"00001000000000 00001004020000 00001012920510","backward":"00001000000000 00001004020000 00001012000000 00001012920510","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00001012000000","lang":"en","license":"EUPL-1.2-or-later","published":"00001014000000","role":"manual","syntax":"zmk","tags":"#manual #webui #zettelstore","title":"Web user interface"}},
...
```

In both cases, an implicit precondition is that the zettel must contain the given metadata key.
For a metadata key like [[''title''|00001006020000#title]], which has a default value, this precondition should always be true.
But the situation is different for a key like [[''url''|00001006020000#url]].
Both ``curl 'http://localhost:23123/z?url='`` and ``curl 'http://localhost:23123/z?url=!'`` may result in an empty list.
Both ``curl 'http://localhost:23123/j?url='`` and ``curl 'http://localhost:23123/j?url=!'`` may result in an empty list.

The empty query parameter values matches all zettel that contain the given metadata key.
Similar, if you specify just the exclamation mark character as a query parameter value, only those zettel match that does //not// contain the given metadata key.
This is in contrast to above rule that the metadata value must exist before a match is done.
For example ``curl 'http://localhost:23123/z?back=!&backward='`` returns all zettel that are reachable via other zettel, but also references these zettel.
For example ``curl 'http://localhost:23123/j?back=!&backward='`` returns all zettel that are reachable via other zettel, but also references these zettel.

Above example shows that all sub-expressions of a select specification must be true so that no zettel is rejected from the final list.

If you specify the query parameter ''_negate'', either with or without a value, the whole selection will be negated.
Because of the precondition described above, ``curl 'http://127.0.0.1:23123/z?url=!com'`` and ``curl 'http://127.0.0.1:23123/z?url=com&_negate'`` may produce different lists.
Because of the precondition described above, ``curl 'http://127.0.0.1:23123/j?url=!com'`` and ``curl 'http://127.0.0.1:23123/j?url=com&_negate'`` may produce different lists.
The first query produces a zettel list, where each zettel does have a ''url'' metadata value, which does not contain the characters ""com"".
The second query produces a zettel list, that excludes any zettel containing a ''url'' metadata value that contains the characters ""com""; this also includes all zettel that do not contain the metadata key ''url''.

Alternatively, you also can use the [[endpoint|00001012920000]] ''/z'' for a simpler result format.
The first example translates to:
```sh
# curl 'http://127.0.0.1:23123/z?title=API'
00001012921000 API: JSON structure of an access token
00001012920500 Formats available by the API
00001012920000 Endpoints used by the API
...
```

Deleted docs/manual/00001012051820.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

























-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012051820
title: API: Shape the list of zettel metadata by returning specific parts of a zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210721120743

=== Basic usage
If you are just interested in the zettel identifier, you should add the [[''_part''|00001012920800]] query parameter:
```sh
# curl 'http://127.0.0.1:23123/z?title=API&_part=id'
{"list":[{"id":"00001012921000","url":"/z/00001012921000"},{"id":"00001012920500","url":"/z/00001012920500"},{"id":"00001012920000","url":"/z/00001012920000"},{"id":"00001012051800","url":"/z/00001012051800"},{"id":"00001012051600","url":"/z/00001012051600"},{"id":"00001012051400","url":"/z/00001012051400"},{"id":"00001012051200","url":"/z/00001012051200"},{"id":"00001012050600","url":"/z/00001012050600"},{"id":"00001012050400","url":"/z/00001012050400"},{"id":"00001012050200","url":"/z/00001012050200"},{"id":"00001012000000","url":"/z/00001012000000"}]}
```

=== Combined usage with select options
If you want only those zettel that additionally must contain the string ""JSON"", you have to add an additional query parameter:
```sh
# curl 'http://127.0.0.1:23123/z?title=API&_part=id&title=JSON'
{"list":[{"id":"00001012921000","url":"/z/00001012921000"}]}
```
Similarly, if you add another query parameter, the intersection of both results is returned:
```sh
# curl 'http://127.0.0.1:23123/z?title=API&_part=id&id=00001012050'
{"list":[{"id":"00001012050600","url":"/z/00001012050600"},{"id":"00001012050400","url":"/z/00001012050400"},{"id":"00001012050200","url":"/z/00001012050200"}]}
```

Changes to docs/manual/00001012051830.zettel.

1
2
3
4
5

6
7
8
9
10
11








12
13
14
15
16
17
18








19
1
2
3
4
5
6
7
8
9
10


11
12
13
14
15
16
17
18
19
20
21
22
23


24
25
26
27
28
29
30
31
32





+




-
-
+
+
+
+
+
+
+
+





-
-
+
+
+
+
+
+
+
+

id: 00001012051830
title: API: Shape the list of zettel metadata by limiting its length
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210905204006

=== Limit
By using the query parameter ""''_limit''"", which must have an integer value, you specifying an upper limit of list elements:
```sh
# curl 'http://192.168.17.7:23121/z?title=API&_part=id&_sort=id&_limit=2'
{"list":[{"id":"00001012000000","url":"/z/00001012000000"},{"id":"00001012050200","url":"/z/00001012050200"}]}
# curl 'http://127.0.0.1:23123/j?title=API&_sort=id&_limit=2'
{"list":[{"id":"00001012000000","meta":{"all-tags":"#api #manual #zettelstore","back":"00001000000000 00001004020000","backward":"00001000000000 00001004020000 00001012053200 00001012054000 00001014000000","box-number":"1","forward":"00001010040100 00001010040700 00001012050200 00001012050400 00001012050600 00001012051200 00001012051800 00001012051810 00001012051830 00001012051840 00001012052000 00001012052200 00001012052400 00001012052600 00001012053200 00001012053400 00001012053500 00001012053600 00001012053700 00001012053800 00001012054000 00001012054200 00001012054400 00001012054600 00001012920000 00001014000000","modified":"20210817160844","published":"20210817160844","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API"}},{"id":"00001012050200","meta":{"all-tags":"#api #manual #zettelstore","back":"00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600","backward":"00001010040700 00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600 00001012920000 00001012921000","box-number":"1","forward":"00001004010000 00001010040200 00001010040700 00001012920000 00001012921000","modified":"20210726123709","published":"20210726123709","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Authenticate a client"}}]}
```

```sh
# curl 'http://127.0.0.1:23123/z?title=API&_sort=id&_limit=2'
00001012000000 API
00001012050200 API: Authenticate a client
```

=== Offset
The query parameter ""''_offset''"" allows to list not only the first elements, but to begin at a specific element:
```sh
# curl 'http://192.168.17.7:23121/z?title=API&_part=id&_sort=id&_limit=2&_offset=1'
{"list":[{"id":"00001012050200","url":"/z/00001012050200"},{"id":"00001012050400","url":"/z/00001012050400"}]}
# curl 'http://127.0.0.1:23123/j?title=API&_sort=id&_limit=2&_offset=1'
{"list":[{"id":"00001012050200","meta":{"all-tags":"#api #manual #zettelstore","back":"00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600","backward":"00001010040700 00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600 00001012920000 00001012921000","box-number":"1","forward":"00001004010000 00001010040200 00001010040700 00001012920000 00001012921000","modified":"20210726123709","published":"20210726123709","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Authenticate a client"}},{"id":"00001012050400","meta":{"all-tags":"#api #manual #zettelstore","back":"00001010040700 00001012000000","backward":"00001010040700 00001012000000 00001012920000 00001012921000","box-number":"1","forward":"00001010040100 00001012050200 00001012920000 00001012921000","modified":"20210726123745","published":"20210726123745","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Renew an access token"}}]}
```

```sh
# curl 'http://127.0.0.1:23123/z?title=API&_sort=id&_limit=2&_offset=1'
00001012050200 API: Authenticate a client
00001012050400 API: Renew an access token
```

Deleted docs/manual/00001012052200.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18


















-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012052200
title: API: List all tags
role: manual
tags: #api #manual #zettelstore
syntax: zmk

To list all [[tags|00001006020000#tags]] used in the Zettelstore just send a HTTP GET request to the [[endpoint|00001012920000]] ''/t''.
If successful, the output is a JSON object:

```sh
# curl http://127.0.0.1:23123/t
{"tags":{"#api":[:["00001012921000","00001012920800","00001012920522",...],"#authorization":["00001010040700","00001010040400",...],...,"#zettelstore":["00010000000000","00001014000000",...,"00001001000000"]}}
```

The JSON object only contains the key ''"tags"'' with the value of another object.
This second object contains all tags as keys and the list of identifier of those zettel with this tag as a value.

Please note that this structure will likely change in the future to be more compliant with other API calls.

Changes to docs/manual/00001012052400.zettel.

1
2

3
4
5
6
7

8
9
10
11
12


13
14
15
16


17
18
1

2
3
4
5
6

7
8
9
10


11
12
13
14


15
16
17
18

-
+




-
+



-
-
+
+


-
-
+
+


id: 00001012052400
title: API: List all roles
title: API: List all tags
role: manual
tags: #api #manual #zettelstore
syntax: zmk

To list all [[roles|00001006020100]] used in the Zettelstore just send a HTTP GET request to the [[endpoint|00001012920000]] ''/r''.
To list all [[tags|00001006020000#tags]] used in the Zettelstore just send a HTTP GET request to the [[endpoint|00001012920000]] ''/t''.
If successful, the output is a JSON object:

```sh
# curl http://127.0.0.1:23123/r
{"role-list":["configuration","manual","user","zettel"]}
# curl http://127.0.0.1:23123/t
{"tags":{"#api":[:["00001012921000","00001012920800","00001012920522",...],"#authorization":["00001010040700","00001010040400",...],...,"#zettelstore":["00010000000000","00001014000000",...,"00001001000000"]}}
```

The JSON object only contains the key ''"role-list"'' with the value of a sorted string list.
Each string names one role.
The JSON object only contains the key ''"tags"'' with the value of another object.
This second object contains all tags as keys and the list of identifier of those zettel with this tag as a value.

Please note that this structure will likely change in the future to be more compliant with other API calls.

Added docs/manual/00001012052600.zettel.



















1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001012052600
title: API: List all roles
role: manual
tags: #api #manual #zettelstore
syntax: zmk

To list all [[roles|00001006020100]] used in the Zettelstore just send a HTTP GET request to the [[endpoint|00001012920000]] ''/r''.
If successful, the output is a JSON object:

```sh
# curl http://127.0.0.1:23123/r
{"role-list":["configuration","manual","user","zettel"]}
```

The JSON object only contains the key ''"role-list"'' with the value of a sorted string list.
Each string names one role.

Please note that this structure will likely change in the future to be more compliant with other API calls.

Changes to docs/manual/00001012053200.zettel.

1
2
3
4
5
6

7
8
9

10
11
12
13
14
15
16
1
2
3
4
5

6
7
8

9
10
11
12
13
14
15
16





-
+


-
+







id: 00001012053200
title: API: Create a new zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210713163927
modified: 20210905204325

A zettel is created by adding it to the [[list of zettel|00001012000000#zettel-lists]].
Therefore, the [[endpoint|00001012920000]] to create a new zettel is also ''/z'', but you must send the data of the new zettel via a HTTP POST request.
Therefore, the [[endpoint|00001012920000]] to create a new zettel is also ''/j'', but you must send the data of the new zettel via a HTTP POST request.

The body of the POST request must contain a JSON object that specifies metadata and content of the zettel to be created.
The following keys of the JSON object are used:
; ''"meta"''
: References an embedded JSON object with only string values.
  The name/value pairs of this objects are interpreted as the metadata of the new zettel.
  Please consider the [[list of supported metadata keys|00001006020000]] (and their value types).
26
27
28
29
30
31
32
33
34


35
36

37
38
39
40
41
42
43
44


45
46
47
48
49


50









51
52
53
54
55
56
57
58
59
26
27
28
29
30
31
32


33
34
35

36
37
38




39

40
41
42
43
44


45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65







-
-
+
+

-
+


-
-
-
-

-
+
+



-
-
+
+

+
+
+
+
+
+
+
+
+









Even these three keys are just optional.
The body of the HTTP POST request must not be empty and it must contain a JSON object.

Therefore, a body containing just ''{}'' is perfectly valid.
The new zettel will have no content, its title will be set to the value of [[''default-title''|00001004020000#default-title]] (default: ""Untitled""), its role is set to the value of [[''default-role''|00001004020000#default-role]] (default: ""zettel""), and its syntax is set to the value of [[''default-syntax''|00001004020000#default-syntax]] (default: ""zmk"").

```
# curl -X POST --data '{}' http://127.0.0.1:23123/z
{"id":"20210713161000","url":"/z/20210713161000"}
# curl -X POST --data '{}' http://127.0.0.1:23123/j
{"id":"20210713161000"}
```
If creating the zettel was successful, the HTTP response will contain a JSON object with two keys:
If creating the zettel was successful, the HTTP response will contain a JSON object with one key:
; ''"id"''
: Contains the zettel identifier of the created zettel for further usage.
; ''"url"''
: The URL for [[reading metadata and content|00001012053400]] of the new zettel.
  In most cases, the URL is a relative one.
  A client must prepend the HTTP protocol scheme, the host name, and (optional, but often needed) the post number to make it an absolute URL.

In addition, the HTTP response header contains a key ''Location'' with the same value of the relative URL.
In addition, the HTTP response header contains a key ''Location'' with a relative URL for the new zettel.
A client must prepend the HTTP protocol scheme, the host name, and (optional, but often needed) the post number to make it an absolute URL.

As an example, a zettel with title ""Note"" and content ""Important content."" can be created by issuing:
```
# curl -X POST --data '{"meta":{"title":"Note"},"content":"Important content."}' http://127.0.0.1:23123/z
{"id":"20210713163100","url":"/z/20210713163100"}
# curl -X POST --data '{"meta":{"title":"Note"},"content":"Important content."}' http://127.0.0.1:23123/j
{"id":"20210713163100"}
```

[!plain]Alternatively, you can use the [[endpoint|00001012920000]] ''/z'' to create a new zettel.
In this case, the zettel must be encoded in a [[plain|00001006000000]] format: first comes the [[metadata|00001006010000]] and the following content is separated by an empty line.
This is the same format as used by storing zettel within a [[directory box|00001006010000]].
```
# curl -X POST --data $'title: Note\n\nImportant content.' http://127.0.0.1:23123/z
20210903211500
```

=== HTTP Status codes
; ''201''
: Zettel creation was successful, the body contains a JSON object that contains its zettel identifier.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Most likely, the JSON was not formed according to above rules.
; ''403''
: You are not allowed to create a new zettel.

Changes to docs/manual/00001012053400.zettel.

1
2
3
4
5

6
7

8
9

10
11
12
13


14
15

16
17
18
19




20
21
22
23
24
25
26









27
28
29


30
31
32

33
34
35
36
37
38
39
40
41
42
43
44




























45
46
47
48
49
50
51
52
53

54
55
56
57
58
1
2
3
4
5
6
7

8
9

10
11
12


13
14
15

16




17
18
19
20







21
22
23
24
25
26
27
28
29



30
31



32












33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

61
62
63
64
65
66
67

68
69
70
71
72
73





+

-
+

-
+


-
-
+
+

-
+
-
-
-
-
+
+
+
+
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
-
-
-
+
+
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-







-
+





id: 00001012053400
title: API: Retrieve metadata and content of an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210905204428

The [[endpoint|00001012920000]] to work with metadata and content of a specific zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
The [[endpoint|00001012920000]] to work with metadata and content of a specific zettel is ''/j/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).

For example, to retrieve some data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/z/00001012053400''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
For example, to retrieve some data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/j/00001012053400''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
If successful, the output is a JSON object:
```sh
# curl http://127.0.0.1:23123/z/00001012053400
{"id":"00001012053400","url":"/z/00001012053400","meta":{"title":"API: Retrieve data for an exisiting zettel","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual","copyright":"(c) 2020 by Detlef Stern <ds@zettelstore.de>","lang":"en","license":"CC BY-SA 4.0"},"content":"The endpoint to work with a specific zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).\n\nFor example, ...
# curl http://127.0.0.1:23123/j/00001012053400
{"id":"00001012053400","meta":{"title":"API: Retrieve data for an exisiting zettel","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual","copyright":"(c) 2020 by Detlef Stern <ds@zettelstore.de>","lang":"en","license":"CC BY-SA 4.0"},"content":"The endpoint to work with a specific zettel is ''/j/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).\n\nFor example, ...
```
Similar to listing all or some zettel, you can provide a query parameter ''_format=[[FORMAT|00001012920500]]'' to select a different encoding format.

The default encoding format is ""[[json|00001012920501]]"".
Others are ""[[djson|00001012920503]]"", ""[[html|00001012920510]]"", and some more.
```sh
# curl 'http://127.0.0.1:23123/z/00001012053400?_format=html'
Pretty-printed, this results in:
```
{
  "id": "00001012053400",
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>API: Retrieve data for an exisiting zettel</title>
<meta name="keywords" content="api, manual, zettelstore">
<meta name="zs-syntax" content="zmk">
  "meta": {
    "back": "00001012000000 00001012053200 00001012054400",
    "backward": "00001012000000 00001012053200 00001012054400 00001012920000",
    "box-number": "1",
    "forward": "00001010040100 00001012050200 00001012920000 00001012920800",
    "modified": "20210726190012",
    "published": "20210726190012",
    "role": "manual",
    "syntax": "zmk",
<meta name="zs-role" content="manual">
<meta name="copyright" content="(c) 2020 by Detlef Stern <ds@zettelstore.de>">
<meta name="license" content="CC BY-SA 4.0">
    "tags": "#api #manual #zettelstore",
    "title": "API: Retrieve metadata and content of an existing zettel"
</head>
<body>
<p>The endpoint to work with a specific zettel is <span style="font-family:monospace">/z/{ID}</span>, where <span style="font-family:monospace">{ID}</span> is a placeholder for the zettel identifier (14 digits).</p>
  },
...
```

You also can use the query parameter ''_part=[[PART|00001012920800]]'' to specify which parts of a zettel must be encoded.
In this case, its default value is ''zettel''.
```sh
# curl 'http://192.168.17.7:23121/z/00001012053400?_format=html&_part=meta'
<meta name="zs-title" content="API: Retrieve data for an exisiting zettel">
<meta name="keywords" content="api, manual, zettelstore">
<meta name="zs-syntax" content="zmk">
<meta name="zs-role" content="manual">
```
  "encoding": "",
  "content": "The [[endpoint|00001012920000]] to work with metadata and content of a specific zettel is ''/j/{ID}'', where ''{ID}'' (...)
}
```

[!plain]Additionally, you can retrieve the plain zettel, without using JSON.
Just change the [[endpoint|00001012920000]] to ''/z/{ID}''
Optionally, you may provide which parts of the zettel you are requesting.
In this case, add an additional query parameter ''_part=[[PART|00001012920800]]''.
Valid values are ""zettel"", ""meta"", and ""content"" (the default value).

````sh
# curl 'http://127.0.0.1:23123/z/00001012053400'
The [[endpoint|00001012920000]] to work with metadata and content of a specific zettel is ''/j/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).

For example, to retrieve some data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/j/00001012053400''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
If successful, the output is a JSON object:
```sh
...
````

````sh
# curl 'http://127.0.0.1:23123/z/00001012053400?_part=meta'
title: API: Retrieve metadata and content of an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
````
(Note that metadata of the whole zettel contains some computed values, such as ''"copyright"'' and ''"license"'', while the metadata for ''_part=meta'' just contain the explicitly specified metadata.)

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the zettel identifier did not consist of exactly 14 digits or ''_format'' / ''_part'' contained illegal values.
  Maybe the zettel identifier did not consist of exactly 14 digits.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Added docs/manual/00001012053500.zettel.







































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001012053500
title: API: Retrieve evaluated metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210826224328

The [[endpoint|00001012920000]] to work with evaluated metadata and content of a specific zettel is ''/v/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).

For example, to retrieve some evaluated data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/v/00001012053500''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
If successful, the output is a JSON object:
```sh
# curl http://127.0.0.1:23123/v/00001012053500
{"meta":{"title":[{"t":"Text","s":"API:"},{"t":"Space"},{"t":"Text","s":"Retrieve"},{"t":"Space"},{"t":"Text","s":"evaluated"},{"t":"Space"},{"t":"Text","s":"metadata"},{"t":"Space"},{"t":"Text","s":"and"},{"t":"Space"},{"t":"Text","s":"content"},{"t":"Space"},{"t":"Text","s":"of"},{"t":"Space"},{"t":"Text","s":"an"},{"t":"Space"},{"t":"Text","s":"existing"},{"t":"Space"},{"t":"Text","s":"zettel"},{"t":"Space"},{"t":"Text","s":"in"},{"t":"Space"}, ...
```

To select another encoding, you can provide a query parameter ''_enc=[[ENCODING|00001012920500]]''.
The default encoding is ""[[djson|00001012920503]]"".
Others are ""[[html|00001012920510]]"", ""[[text|00001012920519]]"", and some more.
```sh
# curl 'http://127.0.0.1:23123/v/00001012053500?_enc=html'
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>API: Retrieve evaluated metadata and content of an existing zettel in various encodings</title>
<meta name="zs-role" content="manual">
<meta name="keywords" content="api, manual, zettelstore">
<meta name="zs-syntax" content="zmk">
<meta name="zs-back" content="00001012000000">
<meta name="zs-backward" content="00001012000000">
<meta name="zs-box-number" content="1">
<meta name="copyright" content="(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>">
<meta name="zs-forward" content="00001010040100 00001012050200 00001012920000 00001012920800">
<meta name="zs-published" content="00001012053500">
</head>
<body>
<p>The <a href="/v/00001012920000?_enc=html">endpoint</a> to work with metadata and content of a specific zettel is <span style="font-family:monospace">/v/{ID}</span>, where <span style="font-family:monospace">{ID}</span> is a placeholder for the zettel identifier (14 digits).</p>
...
```

You also can use the query parameter ''_part=[[PART|00001012920800]]'' to specify which parts of a zettel must be encoded.
In this case, its default value is ''content''.
```sh
# curl 'http://127.0.0.1:23123/v/00001012053500?_enc=html&_part=meta'
<meta name="zs-title" content="API: Retrieve evaluated metadata and content of an existing zettel in various encodings">
<meta name="zs-role" content="manual">
<meta name="keywords" content="api, manual, zettelstore">
<meta name="zs-syntax" content="zmk">
<meta name="zs-back" content="00001012000000">
<meta name="zs-backward" content="00001012000000">
<meta name="zs-box-number" content="1">
<meta name="copyright" content="(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>">
<meta name="zs-forward" content="00001010040100 00001012050200 00001012920000 00001012920800">
<meta name="zs-lang" content="en">
<meta name="zs-published" content="00001012053500">
```

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the zettel identifier did not consist of exactly 14 digits or ''_enc'' / ''_part'' contained illegal values.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Changes to docs/manual/00001012053600.zettel.

1
2

3
4
5
6

7
8
9
10
11

12
13
14
15
16
17
18
19


20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

50
51
52
53
54
55
56
57
58

59
60
61
62
63
64
65
66

67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

85
86
87



88
89
90


91
92
93
94
95
96
97
98
99
100

101
102
103
104
105
1

2
3
4
5

6
7




8








9
10






























11






12


13








14


















15



16
17
18
19


20
21

22

23
24
25
26
27
28

29
30
31
32
33
34

-
+



-
+

-
-
-
-
+
-
-
-
-
-
-
-
-
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-

-
-
+
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
+
+
+

-
-
+
+
-

-






-
+





id: 00001012053600
title: API: Retrieve references of an existing zettel
title: API: Retrieve parsed metadata and content of an existing zettel in various encodings
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210702183357
modified: 20210830165056

The web of zettel is one important value of a Zettelstore.
Many zettel references other zettel, images, external/local material or, via citations, external literature.
By using the [[endpoint|00001012920000]] ''/l/{ID}'' you are able to retrieve these references.

The [[endpoint|00001012920000]] to work with parsed metadata and content of a specific zettel is ''/p/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
````
# curl http://127.0.0.1:23123/l/00001012053600
{"id":"00001012053600","url":"/z/00001012053600","links":{"outgoing":[{"id":"00001012920000","url":"/z/00001012920000"},{"id":"00001007040300","url":"/z/00001007040300#links"},{"id":"00001007040300","url":"/z/00001007040300#images"},{"id":"00001007040300","url":"/z/00001007040300#citation-key"}]},"images":{}}
````
Formatted, this translates into:
````json
{
  "id": "00001012053600",

A //parsed// zettel is basically an [[unevaluated|00001012053500]] zettel: the zettel is read and analyzed, but its content is //not evaluated//.
  "url": "/z/00001012053600",
  "links": {
    "outgoing": [
      {
        "id": "00001012920000",
        "url": "/z/00001012920000"
      },
      {
        "id": "00001007040300",
        "url": "/z/00001007040300#links"
      },
      {
        "id": "00001007040300",
        "url": "/z/00001007040300#images"
      },
      {
        "id": "00001007040300",
        "url": "/z/00001007040300#citation-key"
      }
    ],
  },
  "images": {},
}
````
=== Kind
The following to-level JSON keys are returned:
; ''id''
: The zettel identifier for which the references were requested.
; ''url''
: The API endpoint to fetch more information about the zettel.
By using this endpoint, you are able to retrieve the structure of a zettel before it is evaluated.
; ''link''
: A JSON object that contains information about incoming and outgoing [[links|00001007040300#links]].
; ''image''
: A JSON object that contains information about referenced [[images|00001007040300#images]].
; ''cite''
: A JSON list of [[citation keys|00001007040300#citation-key]] (as JSON strings).

The query parameter ''kind'' controls which of these values is retrieved:

For example, to retrieve some data about this zettel you are currently viewing, just send a HTTP GET request to the endpoint ''/v/00001012053600''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
|= ''kind'' <| Number >| Returned values
| (nothing) | (14) |links, images, and citation keys
| ''link'' | 2|links 
| ''image'' | 4|images
| ''cite'' | 8|citation keys
| ''both'' | (6)|links and images
| ''all'' | (14)|links, images, and citation keys

If successful, the output is a JSON object:
The ""Number"" column gives an indication about an alternative way of specifying the kind.
Every kind is given a number, which you also can use to specify the requested kind of reference.
To get more than one kind, just add the numbers.
If you want to retrieve only images and citations, which was not given a ''kind'' value, just specify ''kind=12''.
=== Matter
If you request to retrieve referenced links and/or referenced images, you can further control, which matter of references should be retrieved.
This is controlled by the value of the query parameter ''matter'':
|= ''matter''| Number >| Returned reference list
| (nothing) | (30) | incoming, outgoing, local, and external references
| ''incoming'' | 2|incoming reference, not allowed for images (aka ""backlinks"", not yet implemented)
| ''outgoing'' | 4|outgoing references
| ''local'' | 8|local references, i.e. local, non-zettel material
| ''external'' |16| external references, i.e. on the web
| ''meta'' |32| external reference, stored in metadata
| ''zettel'' | (6)|incoming and outgoing references
| ''material'' |(56)| local and external references
| ''all'' | (62)| incoming, outgoing, local, and external references

```sh
Incoming and outgoing references are basically zettel.
Therefore the list elements are JSON objects with keys ''id'' and ''url''.
Local and external references are strings.
# curl http://127.0.0.1:23123/p/00001012053600
{"meta":{"title":[{"t":"Text","s":"Retrieve"},{"t":"Space"},{"t":"Text","s":"parsed"},{"t":"Space"},{"t":"Text","s":"metadata"},{"t":"Space"},{"t":"Text","s":"and"},{"t":"Space"},{"t":"Text","s":"content"},{"t":"Space"},{"t":"Text","s":"of"},{"t":"Space"},{"t":"Text","s":"an"},{"t":"Space"},{"t":"Text","s":"existing"},{"t":"Space"},{"t":"Text","s":"zettel"},{"t":"Space"},{"t":"Text","s":"in"},{"t":"Space"},{"t":"Text","s":"various"},{"t":"Space"},{"t":"Text","s":"encodings"}],"role":"manual","tags":["#api", ...
```

Similar to the ''kind'' query parameter, each matter is associated with a number.
To retrieve a combination of matter values that does not have a name, just add the numbers.
Similar to [[retrieving an encoded zettel|00001012053500]], you can specify an [[encoding|00001012920500]] and state which [[part|00001012920800]] of a zettel you are interested in.
The same default values applies to this endpoint.
For example, if you want to retrieve only the outgoing and the external references, specify ''matter=20''.

If a list is not going to retrieved, the associated value is ``null``{=json} instead of an empty list.
=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the zettel identifier did not consist of exactly 14 digits or ''_format'' / ''_part'' contained illegal values.
  Maybe the zettel identifier did not consist of exactly 14 digits or ''_enc'' / ''_part'' contained illegal values.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Added docs/manual/00001012053700.zettel.


























































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001012053700
title: API: Retrieve references of an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210825225643

The web of zettel is one important value of a Zettelstore.
Many zettel references other zettel, embedded material, external/local material or, via citations, external literature.
By using the [[endpoint|00001012920000]] ''/l/{ID}'' you are able to retrieve these references.

````
# curl http://127.0.0.1:23123/l/00001012053700
{"id":"00001012053700","linked":{"outgoing":["00001012920000","00001007040300#links","00001007040300#embedded-material","00001007040300#citation-key"]},"embedded":{}}
````
Formatted, this translates into:
````json
{
  "id": "00001012053700",
  "linked": {
    "outgoing": [
      "00001012920000",
      "00001007040300#links",
      "00001007040300#embedded-material",
      "00001007040300#citation-key"
    ]
  },
  "embedded": {}
}
````
=== Kind
The following top-level JSON keys are returned:
; ''id''
: The zettel identifier for which the references were requested.
; ''linked''
: A JSON object that contains information about incoming and outgoing [[links|00001007040300#links]].
; ''embedded''
: A JSON object that contains information about referenced [[embedded material|00001007040300#embedded-material]].
; ''cite''
: A JSON list of [[citation keys|00001007040300#citation-key]] (as JSON strings).

Incoming and outgoing references are basically zettel.
Therefore the list elements are JSON objects with key ''id'', optionally with an appended fragment.
Local and external references are strings.

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid. 
  There are several reasons for this.
  Maybe the zettel identifier did not consist of exactly 14 digits.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Changes to docs/manual/00001012053800.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
1
2
3
4
5

6
7
8
9
10
11
12
13





-
+







id: 00001012053800
title: API: Retrieve context of an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210712223623
modified: 20210825194347

The context of an origin zettel consists of those zettel that are somehow connected to the origin zettel.
Direct connections of an origin zettel to other zettel are visible via [[metadata values|00001006020000]], such as ''backward'', ''forward'' or other values with type [[identifier|00001006032000]] or [[set of identifier|00001006032500]].
Zettel are also connected by using same [[tags|00001006020000#tags]].

The context is defined by a //direction//, a //depth//, and a //limit//:
* Direction: connections are directed.
27
28
29
30
31
32
33
34

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

71
72
73
74
75
76
77
78
79
80
81
27
28
29
30
31
32
33

34
35
36
37
38
39

40
41
42
43

44
45
46
47

48
49
50
51

52
53
54
55
56
57
58
59
60


61
62
63

64
65
66
67
68
69
70
71
72
73
74
75







-
+





-




-




-




-









-
-



-
+











Currently, only some of the newest zettel with a given tag are considered a connection.[^The number of zettel is given by the value of parameter ''depth''.]
Otherwise the context would become too big and therefore unusable.

To retrieve the context of an existing zettel, use the [[endpoint|00001012920000]] ''/x/{ID}''[^Mnemonic: conte**X**t].

````
# curl 'http://127.0.0.1:23123/x/00001012053800?limit=3&dir=forward&depth=2'
{"id": "00001012053800","url": "/z/00001012053800","meta": {...},"list": [{"id": "00001012921000","url": "/z/00001012921000","meta": {...}},{"id": "00001012920800","url": "/z/00001012920800","meta": {...}},{"id": "00010000000000","url": "/z/00010000000000","meta": {...}}]}
{"id": "00001012053800","meta": {...},"list": [{"id": "00001012921000","meta": {...}},{"id": "00001012920800","meta": {...}},{"id": "00010000000000","meta": {...}}]}
````
Formatted, this translates into:[^Metadata (key ''meta'') are hidden to make the overall structure easier to read.]
````json
{
  "id": "00001012053800",
  "url": "/z/00001012053800",
  "meta": {...},
  "list": [
    {
      "id": "00001012921000",
      "url": "/z/00001012921000",
      "meta": {...}
    },
    {
      "id": "00001012920800",
      "url": "/z/00001012920800",
      "meta": {...}
    },
    {
      "id": "00010000000000",
      "url": "/z/00010000000000",
      "meta": {...}
    }
  ]
}
````
=== Keys
The following top-level JSON keys are returned:
; ''id''
: The zettel identifier for which the context was requested.
; ''url''
: The API endpoint to fetch more information about the zettel.
; ''meta'':
: The metadata of the zettel, encoded as a JSON object.
; ''list''
: A list of JSON objects with keys ''id'', ''url'' and ''meta'' that contains the zettel of the context.
: A list of JSON objects with keys ''id'' and ''meta'' that contains the zettel of the context.

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Changes to docs/manual/00001012054000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71
72

73
74
75
76
77
78
79
80
81
82
83
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

26
27
28
29
30
31

32
33
34

35
36
37
38

39
40
41
42

43
44
45
46

47
48
49
50
51

52
53
54
55
56

57
58
59
60


61
62
63

64
65
66
67
68
69
70
71
72
73
74
75





-
+



















-
+





-



-




-




-




-





-





-
+



-
-



-
+











id: 00001012054000
title: API: Retrieve zettel order within an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210721184434
modified: 20210825194515

Some zettel act as a ""table of contents"" for other zettel.
The [[initial zettel|00001000000000]] of this manual is one example, the [[general API description|00001012000000]] is another.
Every zettel with a certain internal structure can act as the ""table of contents"" for others.

What is a ""table of contents""?
Basically, it is just a list of references to other zettel.

To retrieve the ""table of contents"", the software looks at first level [[list items|00001007030200]].
If an item contains a valid reference to a zettel, this reference will be interpreted as an item in the table of contents.

This applies only to first level list items (ordered or unordered list), but not to deeper levels.
Only the first reference to a valid zettel is collected for the table of contents.
Following references to zettel within such an list item are ignored.

To retrieve the zettel order of an existing zettel, use the [[endpoint|00001012920000]] ''/o/{ID}''.

````
# curl http://127.0.0.1:23123/o/00001000000000
{"id":"00001000000000","url":"/z/00001000000000","meta":{...},"list":[{"id":"00001001000000","url":"/z/00001001000000","meta":{...}},{"id":"00001002000000","url":"/z/00001002000000","meta":{...}},{"id":"00001003000000","url":"/z/00001003000000","meta":{...}},{"id":"00001004000000","url":"/z/00001004000000","meta":{...}},...,{"id":"00001014000000","url":"/z/00001014000000","meta":{...}}]}
{"id":"00001000000000","meta":{...},"list":[{"id":"00001001000000","meta":{...}},{"id":"00001002000000","meta":{...}},{"id":"00001003000000","meta":{...}},{"id":"00001004000000","meta":{...}},...,{"id":"00001014000000","meta":{...}}]}
````
Formatted, this translates into:[^Metadata (key ''meta'') are hidden to make the overall structure easier to read.]
````json
{
  "id": "00001000000000",
  "url": "/z/00001000000000",
  "list": [
    {
      "id": "00001001000000",
      "url": "/z/00001001000000",
      "meta": {...}
    },
    {
      "id": "00001002000000",
      "url": "/z/00001002000000",
      "meta": {...}
    },
    {
      "id": "00001003000000",
      "url": "/z/00001003000000",
      "meta": {...}
    },
    {
      "id": "00001004000000",
      "url": "/z/00001004000000",
      "meta": {...}
    },
    ...
    {
      "id": "00001014000000",
      "url": "/z/00001014000000",
      "meta": {...}
    }
  ]
}
````
=== Kind

The following top-level JSON keys are returned:
; ''id''
: The zettel identifier for which the references were requested.
; ''url''
: The API endpoint to fetch more information about the zettel.
; ''meta'':
: The metadata of the zettel, encoded as a JSON object.
; ''list''
: A list of JSON objects with keys ''id'', ''url'', and ''meta'' that describe other zettel in the defined order.
: A list of JSON objects with keys ''id'' and ''meta'' that describe other zettel in the defined order.

=== HTTP Status codes
; ''200''
: Retrieval was successful, the body contains an appropriate JSON object.
; ''400''
: Request was not valid.
; ''403''
: You are not allowed to retrieve data of the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Changes to docs/manual/00001012054200.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15
16

17
18
19
20
21







22
23
24
25
26
27
28
29
30
31
32
33
1
2
3
4
5

6
7
8
9
10
11

12
13
14
15

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40





-
+





-
+



-
+





+
+
+
+
+
+
+












id: 00001012054200
title: API: Update a zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210713163606
modified: 20210905204628

Updating metadata and content of a zettel is technically quite similar to [[creating a new zettel|00001012053200]].
In both cases you must provide the data for the new or updated zettel in the body of the HTTP request.

One difference is the endpoint.
The [[endpoint|00001012920000]] to update a zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
The [[endpoint|00001012920000]] to update a zettel is ''/j/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
You must send a HTTP PUT request to that endpoint:

```
# curl -X PUT --data '{}' http://127.0.0.1:23123/z/00001012054200
# curl -X PUT --data '{}' http://127.0.0.1:23123/j/00001012054200
```
This will put some empty content and metadata to the zettel you are currently reading.
As usual, some metadata will be calculated if it is empty.

The body of the HTTP response is empty, if the request was successful.

[!plain]Alternatively, you can use the [[endpoint|00001012920000]] ''/z/{ID}'' to update a zettel.
In this case, the zettel must be encoded in a [[plain|00001006000000]] format: first comes the [[metadata|00001006010000]] and the following content is separated by an empty line.
This is the same format as used by storing zettel within a [[directory box|00001006010000]].
```
# curl -X POST --data $'title: Updated Note\n\nUpdated content.' http://127.0.0.1:23123/z/00001012054200
```

=== HTTP Status codes
; ''204''
: Update was successful, there is no body in the response.
; ''400''
: Request was not valid.
  For example, the request body was not valid.
; ''403''
: You are not allowed to delete the given zettel.
; ''404''
: Zettel not found.
  You probably used a zettel identifier that is not used in the Zettelstore.

Changes to docs/manual/00001012054400.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15

16
17
18

19
20
21
22
23
24



25
26
27
28
29
30
31
1
2
3
4
5

6
7
8
9
10
11
12
13
14

15
16
17

18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34





-
+








-
+


-
+






+
+
+







id: 00001012054400
title: API: Rename a zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210713163708
modified: 20210905204715

Renaming a zettel is effectively just specifying a new identifier for the zettel.
Since more than one [[box|00001004011200]] might contain a zettel with the old identifier, the rename operation must success in every relevant box to be overall successful.
If the rename operation fails in one box, Zettelstore tries to rollback previous successful operations.

As a consequence, you cannot rename a zettel when its identifier is used in a read-only box.
This applies to all predefined zettel, for example.

The [[endpoint|00001012920000]] to rename a zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
The [[endpoint|00001012920000]] to rename a zettel is ''/j/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
You must send a HTTP MOVE request to this endpoint, and you must specify the new zettel identifier as an URL, placed under the HTTP request header key ''Destination''.
```
# curl -X MOVE -H "Destination: 10000000000001" http://127.0.0.1:23123/z/00001000000000
# curl -X MOVE -H "Destination: 10000000000001" http://127.0.0.1:23123/j/00001000000000
```

Only the last 14 characters of the value of ''Destination'' are taken into account and those must form an unused zettel identifier.
If the value contains less than 14 characters that do not form an unused zettel identifier, the response will contain a HTTP status code ''400''.
All other characters, besides those 14 digits, are effectively ignored.
However, the value should form a valid URL that could be used later to [[read the content|00001012053400]] of the freshly renamed zettel.

[!plain]Alternatively, you can also use the [[endpoint|00001012920000]] ''/z/{ID}''.
Both endpoints behave identical.

=== HTTP Status codes
; ''204''
: Rename was successful, there is no body in the response.
; ''400''
: Request was not valid.
  For example, the HTTP header did not contain a valid ''Destination'' key, or the new identifier is already in use.

Changes to docs/manual/00001012054600.zettel.

1
2
3
4
5
6

7
8
9
10
11
12

13
14
15

16



17
18
19
20
21
22
23
24
25
26
1
2
3
4
5

6
7
8
9
10
11

12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29





-
+





-
+


-
+

+
+
+










id: 00001012054600
title: API: Delete a zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210713163722
modified: 20210905204749

Deleting a zettel within the Zettelstore is executed on the first [[box|00001004011200]] that contains that zettel.
Zettel with the same identifier, but in subsequent boxes remain.
If the first box containing the zettel is read-only, deleting that zettel will fail, as well for a Zettelstore in [[read-only mode|00001004010000#read-only-mode]] or if authentication is enabled and the user has no [[access right|00001010070600]] to do so.

The [[endpoint|00001012920000]] to delete a zettel is ''/z/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
The [[endpoint|00001012920000]] to delete a zettel is ''/j/{ID}'', where ''{ID}'' is a placeholder for the zettel identifier (14 digits).
You must send a HTTP DELETE request to this endpoint:
```
# curl -X DELETE http://127.0.0.1:23123/z/00001000000000
# curl -X DELETE http://127.0.0.1:23123/j/00001000000000
```

[!plain]Alternatively, you can also use the [[endpoint|00001012920000]] ''/z/{ID}''.
Both endpoints behave identical.

=== HTTP Status codes
; ''204''
: Delete was successful, there is no body in the response.
; ''403''
: You are not allowed to delete the given zettel.
  Maybe you do not have enough access rights, or either the box or Zettelstore itself operate in read-only mode.
; ''404''
: Zettel not found.
  You probably specified a zettel identifier that is not used in the Zettelstore.

Changes to docs/manual/00001012920000.zettel.

1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26

27
28
29











30
31
32
33
34
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18





19
20


21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40





-
+












-
-
-
-
-
+

-
-
+



+
+
+
+
+
+
+
+
+
+
+





id: 00001012920000
title: Endpoints used by the API
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20210712225257
modified: 20210908220438

All API endpoints conform to the pattern ''[PREFIX]LETTER[/ZETTEL-ID]'', where:
; ''PREFIX''
: is the URL prefix (default: ''/''), configured via the ''url-prefix'' [[startup configuration|00001004010000]],
; ''LETTER''
: is a single letter that specifies the ressource type,
; ''ZETTEL-ID''
: is an optional 14 digits string that uniquely [[identify a zettel|00001006050000]].

The following letters are currently in use:

|= Letter:| Without zettel identifier | With [[zettel identifier|00001006050000]] | Mnemonic
| ''l'' |  | GET: [[list references|00001012053600]] | **L**inks
| ''o'' |  | GET: [[list zettel order|00001012054000]] | **O**rder
| ''r'' | GET: [[list roles|00001012052400]] | | **R**oles
| ''t'' | GET: [[list tags|00001012052200]] || **T**ags
| ''v'' | POST: [[client authentication|00001012050200]] | | **V**erbürgen[^German translation for ""authentication"", since ''/a'' is already in use by the [[web user interface|00001014000000]].]
| ''a'' | POST: [[client authentication|00001012050200]] | | **A**uthenticate
|       | PUT: [[renew access token|00001012050400]] |
| ''x'' |  | GET: [[list zettel context|00001012053800]] | Conte**x**t
| ''z'' | GET: [[list zettel|00001012051200]] | GET: [[retrieve zettel|00001012053400]] | **Z**ettel
| ''j'' | GET: [[list zettel AS JSON|00001012051200]] | GET: [[retrieve zettel AS JSON|00001012053400]] | **J**SON
|       | POST: [[create new zettel|00001012053200]] | PUT: [[update a zettel|00001012054200]]
|       |  | DELETE: [[delete the zettel|00001012054600]]
|       |  | MOVE: [[rename the zettel|00001012054400]]
| ''l'' |  | GET: [[list references|00001012053700]] | **L**inks
| ''o'' |  | GET: [[list zettel order|00001012054000]] | **O**rder
| ''p'' |  | GET: [[retrieve parsed zettel|00001012053600]]| **P**arsed
| ''r'' | GET: [[list roles|00001012052600]] | | **R**oles
| ''t'' | GET: [[list tags|00001012052400]] || **T**ags
| ''v'' |  | GET: [[retrieve evaluated zettel|00001012053500]] | E**v**aluated
| ''x'' |  | GET: [[list zettel context|00001012053800]] | Conte**x**t
| ''z'' | GET: [[list zettel|00001012051200#plain]] | GET: [[retrieve zettel|00001012053400#plain]] | **Z**ettel
|       | POST: [[create new zettel|00001012053200#plain]] | PUT: [[update a zettel|00001012054200#plain]]
|       |  | DELETE: [[delete zettel|00001012054600#plain]]
|       |  | MOVE: [[rename zettel|00001012054400#plain]]

The full URL will contain either the ''http'' oder ''https'' scheme, a host name, and an optional port number.

The API examples will assume the ''http'' schema, the local host ''127.0.0.1'', the default port ''23123'', and the default empty ''PREFIX''.
Therefore, all URLs in the API documentation will begin with ''http://127.0.0.1:23123''.

Changes to docs/manual/00001012920500.zettel.

1
2


3
4
5

6
7
8
9
10

11
12
13
14
15
16
17
18
19
1

2
3
4
5

6
7
8
9


10
11
12

13
14





-
+
+


-
+



-
-
+


-


-
-
-
-
id: 00001012920500
title: Formats available by the API
title: Encodings available by the API
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
role: manual
modified: 20210727120214

A zettel representation can be encoded in various formats for further processing.

* [[json|00001012920501]] (default)
* [[djson|00001012920503]]
* [[djson|00001012920503]] (default)
* [[html|00001012920510]]
* [[native|00001012920513]]
* [[raw|00001012920516]]
* [[text|00001012920519]]
* [[zmk|00001012920522]]

Planned formats:
; ''pjson''
: Encoder that emits JSON data that can be fed into [[pandoc|https://pandoc.org]] for further processing.

Deleted docs/manual/00001012920501.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
















































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012920501
title: JSON Format
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk

This is the default representation of a zettel or a list of zettel.
Basically, user provided data is encoded as a string (zettel content and metadata values),
The metadata and some other structuring data is encoded a JSON object.

The JSON objects contains various name/value pairs, depending which content should be encoded:

* ''"id"'': the [[zettel identifier|00001006050000]], as a string.
* ''"url"'': the base URL to the zettel, as a string.
* ''"meta"'': the metadata of the zettel, as an JSON object. See below for details.
* ''"encoding"'' and ''"content"'': the actual content of the zettel. See below for details.

''"id"'' and ''"url"'' are always sent to the client.
It depends on the value of the required [[zettel part|00001012920800]], whether ''"meta"'' or ''"content"'' or both are sent.

For an example, take a look at the JSON encoding of this page, which is available via the ""Info"" sub-page of this zettel: 

* [[//z/00001012920501?_part=id]],
* [[//z/00001012920501?_part=zettel]],
* [[//z/00001012920501?_part=meta]],
* [[//z/00001012920501?_part=content]].

If transferred via HTTP, the content type will be ''application/json''.

=== Metadata
This ia a JSON object, that maps [[metadata keys|00001006010000]] to their values.
Their values are encoded as strings, even if they contain a number (or something else).

You can always expect the keys ''"title"'', ''"tags"'', ''"syntax"'', and ''"role"'', together with their values.
The Zettelstore provides default values for these values, if they are not set for a zettel.

There is a list of [[supported metadata keys|00001006020000]].

=== Content
When the content is text-only, it is encoded as a plain string with an ''"encoding"'' value of ''""'' (empty string).

If the content contains binary content:

* ''"encoding"'' specifies the string encoding of the content.
  Currently, only the value ''"base64"'' is supported (as described in [[RFC4648, section 4|https://tools.ietf.org/html/rfc4648#section-4]].
* ''"value"'' is a string that contains the encoded binary content.

For example, if the content just consists of three zero bytes, it will be encoded as ``(...),"encoding":"base64","value":"AAAA"``{=json}.

Changes to docs/manual/00001012920503.zettel.

1
2

3
4
5

6
7
8

9
10
11
12
13
14
15




16
17
18
19
1

2
3
4
5
6
7
8

9
10
11
12




13
14
15
16
17
18
19
20

-
+



+


-
+



-
-
-
-
+
+
+
+




id: 00001012920503
title: DJSON Format
title: DJSON Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20210727120128

A zettel representation that allows to process the syntactic structure of a zettel.
It is a JSON-based encoding format, but different to [[json|00001012920501]].
It is a JSON-based encoding format, but different to the structures returned by [[endpoint|00001012920000]] ''/j/{ID}''.

For an example, take a look at the JSON encoding of this page, which is available via the ""Info"" sub-page of this zettel: 

* [[../z/00001012920503?_format=djson&_part=id]],
* [[../z/00001012920503?_format=djson&_part=zettel]],
* [[../z/00001012920503?_format=djson&_part=meta]],
* [[../z/00001012920503?_format=djson&_part=content]].
* [[//v/00001012920503?_enc=djson&_part=id]],
* [[//v/00001012920503?_enc=djson&_part=zettel]],
* [[//v/00001012920503?_enc=djson&_part=meta]],
* [[//v/00001012920503?_enc=djson&_part=content]].

If transferred via HTTP, the content type will be ''application/json''.

TODO: detailed description.

Changes to docs/manual/00001012920510.zettel.

1
2


3
4
5

6
7
8
9
10
11
12
1

2
3
4
5

6
7
8
9
10
11
12
13

-
+
+


-
+







id: 00001012920510
title: HTML Format
title: HTML Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
role: manual
modified: 20210726193034

A zettel representation in HTML.
This representation is different form the [[web user interface|00001014000000]] as it contains the zettel representation only and no additional data such as the menu bar.

It is intended to be used by external clients.

If transferred via HTTP, the content type will be ''text/html''.

Changes to docs/manual/00001012920513.zettel.

1
2


3
4
5

6
7
8
9
10
11
12
1

2
3
4
5

6
7
8
9
10
11
12
13

-
+
+


-
+







id: 00001012920513
title: Native Format
title: Native Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
role: manual
modified: 20210726193049

A zettel representation shows the structure of a zettel in a more user-friendly way.
Mostly used for debugging.

If transferred via HTTP, the content type will be ''text/plain''.

TODO: formal description

Deleted docs/manual/00001012920516.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

















-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001012920516
title: Raw Format
tags: #api #manual #reference #zettelstore
syntax: zmk
role: manual

A zettel representation as it was loaded from the zettel content.

Often used to have access to the bytes of an image.

If transferred via HTTP, the content type will depend on various factors:

* If the whole zettel should be transferred and it contains textual data only, ''text/plain'' will be used.
Otherwise the content type of the whole zettel will be ''application/octet-stream''.
* If just metadata is transferred, ''text/plain'' will be used.
* If just the content has to be transferred, the content type depends on the actual dats.
  A PNG image will be transferred as ''image/png'', HTML content as ''text/html'', and so on.

Changes to docs/manual/00001012920519.zettel.

1
2


3
4
5

6
7
8
9
10
11
12
1

2
3
4
5

6
7
8
9
10
11
12
13

-
+
+


-
+







id: 00001012920519
title: Text Format
title: Text Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
role: manual
modified: 20210726193119

A zettel representation contains just all textual data of a zettel.
Could be used for creating a search index.

Every line may contain zero, one, or more words, spearated by space character.

If transferred via HTTP, the content type will be ''text/plain''.

Changes to docs/manual/00001012920522.zettel.

1
2


3
4
5

6
7
8
9
10
1

2
3
4
5

6
7
8
9
10
11

-
+
+


-
+





id: 00001012920522
title: Zmk Format
title: Zmk Encoding
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
role: manual
modified: 20210726193136

A zettel representation that tries to recreate a [[Zettelmarkup|00001007000000]] representation of the zettel.
Useful if you want to convert [[other markup languages|00001008000000]] to Zettelmarkup (e.g. Markdown).

If transferred via HTTP, the content type will be ''text/plain''.

Changes to docs/manual/00001012920800.zettel.

1
2

3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17





+


-
+











-
-
-
id: 00001012920800
title: Values to specify zettel parts
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
role: manual
modified: 20210727125306

When working with [[zettel|00001006000000]], you could work with the whole zettel, with its metadata, or with its content:
; [!zettel]''zettel''
: Specifies that you work with a zettel as a whole.
  Contains identifier, metadata, and content of a zettel.
; [!meta]''meta''
: Specifies that you only want to cope with the metadata of a zettel.
  Contains identifier and metadata of a zettel.
; [!content]''content''
: Specifies that you are only interested in the zettel content.
  Contains identifier and content of a zettel.
; [!id]''id''
: States that you just want the zettel identifier and the URL of the zettel.
  Only valid for JSON-based encoding formats[^[[json|00001012920501]] and [[djson|00001012920503]].].

Changes to domain/content.go.

10
11
12
13
14
15
16

17
18
19
20
21
22
23
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24







+








// Package domain provides domain specific types, constants, and functions.
package domain

import (
	"encoding/base64"
	"errors"
	"io"
	"unicode/utf8"

	"zettelstore.de/z/strfun"
)

// Content is just the content of a zettel.
type Content struct {
31
32
33
34
35
36
37





38
39
40
41
42
43
44
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50







+
+
+
+
+







}

// Set content to new string value.
func (zc *Content) Set(s string) {
	zc.data = s
	zc.isBinary = calcIsBinary(s)
}

// Write it to a Writer
func (zc *Content) Write(w io.Writer) (int, error) {
	return io.WriteString(w, zc.data)
}

// AsString returns the content itself is a string.
func (zc *Content) AsString() string { return zc.data }

// AsBytes returns the content itself is a byte slice.
func (zc *Content) AsBytes() []byte { return []byte(zc.data) }

Changes to domain/meta/meta.go.

109
110
111
112
113
114
115

116
117
118
119
120
121
122
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123







+







// Supported keys.
var (
	KeyID                = registerKey("id", TypeID, usageComputed, "")
	KeyTitle             = registerKey("title", TypeZettelmarkup, usageUser, "")
	KeyRole              = registerKey("role", TypeWord, usageUser, "")
	KeyTags              = registerKey("tags", TypeTagSet, usageUser, "")
	KeySyntax            = registerKey("syntax", TypeWord, usageUser, "")
	KeyAllTags           = registerKey("all-"+KeyTags, TypeTagSet, usageProperty, "")
	KeyBack              = registerKey("back", TypeIDSet, usageProperty, "")
	KeyBackward          = registerKey("backward", TypeIDSet, usageProperty, "")
	KeyBoxNumber         = registerKey("box-number", TypeNumber, usageComputed, "")
	KeyCopyright         = registerKey("copyright", TypeString, usageUser, "")
	KeyCredential        = registerKey("credential", TypeCredential, usageUser, "")
	KeyDead              = registerKey("dead", TypeIDSet, usageProperty, "")
	KeyDefaultCopyright  = registerKey("default-copyright", TypeString, usageUser, "")
131
132
133
134
135
136
137

138
139
140
141
142
143
144
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146







+







	KeyFolge             = registerKey("folge", TypeIDSet, usageProperty, "")
	KeyFooterHTML        = registerKey("footer-html", TypeString, usageUser, "")
	KeyForward           = registerKey("forward", TypeIDSet, usageProperty, "")
	KeyHomeZettel        = registerKey("home-zettel", TypeID, usageUser, "")
	KeyLang              = registerKey("lang", TypeWord, usageUser, "")
	KeyLicense           = registerKey("license", TypeEmpty, usageUser, "")
	KeyMarkerExternal    = registerKey("marker-external", TypeEmpty, usageUser, "")
	KeyMaxTransclusions  = registerKey("max-transclusions", TypeNumber, usageUser, "")
	KeyModified          = registerKey("modified", TypeTimestamp, usageComputed, "")
	KeyNoIndex           = registerKey("no-index", TypeBool, usageUser, "")
	KeyPrecursor         = registerKey("precursor", TypeIDSet, usageUser, KeyFolge)
	KeyPublished         = registerKey("published", TypeTimestamp, usageProperty, "")
	KeyReadOnly          = registerKey("read-only", TypeWord, usageUser, "")
	KeySiteName          = registerKey("site-name", TypeString, usageUser, "")
	KeyURL               = registerKey("url", TypeURL, usageUser, "")
178
179
180
181
182
183
184
185

186
187
188









189
190
191
192
193
194
195
180
181
182
183
184
185
186

187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206







-
+



+
+
+
+
+
+
+
+
+







// Meta contains all meta-data of a zettel.
type Meta struct {
	Zid     id.Zid
	pairs   map[string]string
	YamlSep bool
}

// New creates a new chunk for storing meta-data
// New creates a new chunk for storing metadata.
func New(zid id.Zid) *Meta {
	return &Meta{Zid: zid, pairs: make(map[string]string, 5)}
}

// NewWithData creates metadata object with given data.
func NewWithData(zid id.Zid, data map[string]string) *Meta {
	pairs := make(map[string]string, len(data))
	for k, v := range data {
		pairs[k] = v
	}
	return &Meta{Zid: zid, pairs: pairs}
}

// Clone returns a new copy of the metadata.
func (m *Meta) Clone() *Meta {
	return &Meta{
		Zid:     m.Zid,
		pairs:   m.Map(),
		YamlSep: m.YamlSep,

Changes to domain/meta/parse.go.

158
159
160
161
162
163
164
165

166
167
168
169
170
171
172
158
159
160
161
162
163
164

165
166
167
168
169
170
171
172







-
+








	switch Type(key) {
	case TypeString, TypeZettelmarkup:
		if v != "" {
			addData(m, key, v)
		}
	case TypeTagSet:
		addSet(m, key, v, func(s string) bool { return s[0] == '#' })
		addSet(m, key, strings.ToLower(v), func(s string) bool { return s[0] == '#' })
	case TypeWord:
		m.Set(key, strings.ToLower(v))
	case TypeWordSet:
		addSet(m, key, strings.ToLower(v), func(s string) bool { return true })
	case TypeID:
		if _, err := id.Parse(v); err == nil {
			m.Set(key, v)

Changes to domain/meta/type.go.

53
54
55
56
57
58
59
60

61
62
63
64
65
66
67
68

69
70
71
72




73
74


75
76
77
78
79
80
81
82
83
84
85
86
87

88
89


90
91
92
93




94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
53
54
55
56
57
58
59

60
61
62
63
64
65
66


67




68
69
70
71


72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87


88
89




90
91
92
93


94
95
96
97
98
99
100
101
102
103
104
105
106
107








108
109
110
111
112
113
114







-
+






-
-
+
-
-
-
-
+
+
+
+
-
-
+
+













+
-
-
+
+
-
-
-
-
+
+
+
+
-
-














-
-
-
-
-
-
-
-







	TypeWord         = registerType("Word", false)
	TypeWordSet      = registerType("WordSet", true)
	TypeZettelmarkup = registerType("Zettelmarkup", false)
)

// Type returns a type hint for the given key. If no type hint is specified,
// TypeUnknown is returned.
func (m *Meta) Type(key string) *DescriptionType {
func (*Meta) Type(key string) *DescriptionType {
	return Type(key)
}

var (
	cachedTypedKeys = make(map[string]*DescriptionType)
	mxTypedKey      sync.RWMutex
)

	suffixTypes     = map[string]*DescriptionType{
func typedKey(key string, t *DescriptionType) *DescriptionType {
	mxTypedKey.Lock()
	defer mxTypedKey.Unlock()
	cachedTypedKeys[key] = t
		"-number": TypeNumber,
		"-role":   TypeWord,
		"-url":    TypeURL,
		"-zid":    TypeID,
	return t
}
	}
)

// Type returns a type hint for the given key. If no type hint is specified,
// TypeUnknown is returned.
func Type(key string) *DescriptionType {
	if k, ok := registeredKeys[key]; ok {
		return k.Type
	}
	mxTypedKey.RLock()
	k, ok := cachedTypedKeys[key]
	mxTypedKey.RUnlock()
	if ok {
		return k
	}
	for suffix, t := range suffixTypes {
	if strings.HasSuffix(key, "-url") {
		return typedKey(key, TypeURL)
		if strings.HasSuffix(key, suffix) {
			mxTypedKey.Lock()
	}
	if strings.HasSuffix(key, "-number") {
		return typedKey(key, TypeNumber)
	}
			defer mxTypedKey.Unlock()
			cachedTypedKeys[key] = t
			return t
		}
	if strings.HasSuffix(key, "-zid") {
		return typedKey(key, TypeID)
	}
	return TypeEmpty
}

// SetList stores the given string list value under the given key.
func (m *Meta) SetList(key string, values []string) {
	if key != KeyID {
		for i, val := range values {
			values[i] = trimValue(val)
		}
		m.pairs[key] = strings.Join(values, " ")
	}
}

// CleanTag removes the number charachter ('#') from a tag value.
func CleanTag(tag string) string {
	if len(tag) > 1 && tag[0] == '#' {
		return tag[1:]
	}
	return tag
}

// SetNow stores the current timestamp under the given key.
func (m *Meta) SetNow(key string) {
	m.Set(key, time.Now().Format("20060102150405"))
}

// BoolValue returns the value interpreted as a bool.
func BoolValue(value string) bool {
169
170
171
172
173
174
175
176

177
178
179

180
181
182
183
184








185
186
187
188
189
190
191
159
160
161
162
163
164
165

166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190







-
+



+





+
+
+
+
+
+
+
+







	}
	return ListFromValue(value), true
}

// GetTags returns the list of tags as a string list. Each tag does not begin
// with the '#' character, in contrast to `GetList`.
func (m *Meta) GetTags(key string) ([]string, bool) {
	tags, ok := m.GetList(key)
	tagsValue, ok := m.Get(key)
	if !ok {
		return nil, false
	}
	tags := ListFromValue(strings.ToLower(tagsValue))
	for i, tag := range tags {
		tags[i] = CleanTag(tag)
	}
	return tags, len(tags) > 0
}

// CleanTag removes the number character ('#') from a tag value and lowercases it.
func CleanTag(tag string) string {
	if len(tag) > 1 && tag[0] == '#' {
		return tag[1:]
	}
	return tag
}

// GetListOrNil retrieves the string list value of a given key. If there was
// nothing stores, a nil list is returned.
func (m *Meta) GetListOrNil(key string) []string {
	if value, ok := m.GetList(key); ok {
		return value
	}

Added encoder/djsonenc/djsonenc.go.













































































































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package jsonenc encodes the abstract syntax tree into JSON.
package jsonenc

import (
	"fmt"
	"io"
	"sort"
	"strconv"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

func init() {
	encoder.Register(api.EncoderDJSON, encoder.Info{
		Create: func(env *encoder.Environment) encoder.Encoder { return &jsonDetailEncoder{env: env} },
	})
}

type jsonDetailEncoder struct {
	env *encoder.Environment
}

// WriteZettel writes the encoded zettel to the writer.
func (je *jsonDetailEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newDetailVisitor(w, je)
	v.b.WriteString("{\"meta\":{")
	v.writeMeta(zn.InhMeta, evalMeta)
	v.b.WriteByte('}')
	v.b.WriteString(",\"content\":")
	ast.Walk(v, zn.Ast)
	v.b.WriteByte('}')
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as JSON.
func (je *jsonDetailEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newDetailVisitor(w, je)
	v.b.WriteByte('{')
	v.writeMeta(m, evalMeta)
	v.b.WriteByte('}')
	length, err := v.b.Flush()
	return length, err
}

func (je *jsonDetailEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return je.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes a block slice to the writer
func (je *jsonDetailEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newDetailVisitor(w, je)
	ast.Walk(v, bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (je *jsonDetailEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newDetailVisitor(w, je)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b   encoder.BufWriter
	env *encoder.Environment
}

func newDetailVisitor(w io.Writer, je *jsonDetailEncoder) *visitor {
	return &visitor{b: encoder.NewBufWriter(w), env: je.env}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockListNode:
		v.visitBlockList(n)
		return nil
	case *ast.InlineListNode:
		v.walkInlineList(n)
		return nil
	case *ast.ParaNode:
		v.writeNodeStart("Para")
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.visitHeading(n)
	case *ast.HRuleNode:
		v.writeNodeStart("Hrule")
		v.visitAttributes(n.Attrs)
	case *ast.NestedListNode:
		v.visitNestedList(n)
	case *ast.DescriptionListNode:
		v.visitDescriptionList(n)
	case *ast.TableNode:
		v.visitTable(n)
	case *ast.BLOBNode:
		v.writeNodeStart("Blob")
		v.writeContentStart('q')
		writeEscaped(&v.b, n.Title)
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Syntax)
		v.writeContentStart('o')
		v.b.WriteBase64(n.Blob)
		v.b.WriteByte('"')
	case *ast.TextNode:
		v.writeNodeStart("Text")
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Text)
	case *ast.TagNode:
		v.writeNodeStart("Tag")
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Tag)
	case *ast.SpaceNode:
		v.writeNodeStart("Space")
		if l := len(n.Lexeme); l > 1 {
			v.writeContentStart('n')
			v.b.WriteString(strconv.Itoa(l))
		}
	case *ast.BreakNode:
		if n.Hard {
			v.writeNodeStart("Hard")
		} else {
			v.writeNodeStart("Soft")
		}
	case *ast.LinkNode:
		v.writeNodeStart("Link")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('q')
		writeEscaped(&v.b, mapRefState[n.Ref.State])
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Ref.String())
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.EmbedNode:
		v.visitEmbed(n)
	case *ast.CiteNode:
		v.writeNodeStart("Cite")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Key)
		if n.Inlines != nil {
			v.writeContentStart('i')
			ast.Walk(v, n.Inlines)
		}
	case *ast.FootnoteNode:
		v.writeNodeStart("Footnote")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.MarkNode:
		v.visitMark(n)
	case *ast.FormatNode:
		v.writeNodeStart(mapFormatKind[n.Kind])
		v.visitAttributes(n.Attrs)
		v.writeContentStart('i')
		ast.Walk(v, n.Inlines)
	case *ast.LiteralNode:
		kind, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown literal kind %v", n.Kind))
		}
		v.writeNodeStart(kind)
		v.visitAttributes(n.Attrs)
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Text)
	default:
		return v
	}
	v.b.WriteByte('}')
	return nil
}

var mapVerbatimKind = map[ast.VerbatimKind]string{
	ast.VerbatimProg:    "CodeBlock",
	ast.VerbatimComment: "CommentBlock",
	ast.VerbatimHTML:    "HTMLBlock",
}

func (v *visitor) visitVerbatim(vn *ast.VerbatimNode) {
	kind, ok := mapVerbatimKind[vn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown verbatim kind %v", vn.Kind))
	}
	v.writeNodeStart(kind)
	v.visitAttributes(vn.Attrs)
	v.writeContentStart('l')
	for i, line := range vn.Lines {
		v.writeComma(i)
		writeEscaped(&v.b, line)
	}
	v.b.WriteByte(']')
}

var mapRegionKind = map[ast.RegionKind]string{
	ast.RegionSpan:  "SpanBlock",
	ast.RegionQuote: "QuoteBlock",
	ast.RegionVerse: "VerseBlock",
}

func (v *visitor) visitRegion(rn *ast.RegionNode) {
	kind, ok := mapRegionKind[rn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown region kind %v", rn.Kind))
	}
	v.writeNodeStart(kind)
	v.visitAttributes(rn.Attrs)
	v.writeContentStart('b')
	ast.Walk(v, rn.Blocks)
	if rn.Inlines != nil {
		v.writeContentStart('i')
		ast.Walk(v, rn.Inlines)
	}
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	v.writeNodeStart("Heading")
	v.visitAttributes(hn.Attrs)
	v.writeContentStart('n')
	v.b.WriteString(strconv.Itoa(hn.Level))
	if fragment := hn.Fragment; fragment != "" {
		v.writeContentStart('s')
		v.b.WriteStrings("\"", fragment, "\"")
	}
	v.writeContentStart('i')
	ast.Walk(v, hn.Inlines)
}

var mapNestedListKind = map[ast.NestedListKind]string{
	ast.NestedListOrdered:   "OrderedList",
	ast.NestedListUnordered: "BulletList",
	ast.NestedListQuote:     "QuoteList",
}

func (v *visitor) visitNestedList(ln *ast.NestedListNode) {
	v.writeNodeStart(mapNestedListKind[ln.Kind])
	v.writeContentStart('c')
	for i, item := range ln.Items {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, in := range item {
			v.writeComma(j)
			ast.Walk(v, in)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.writeNodeStart("DescriptionList")
	v.writeContentStart('g')
	for i, def := range dn.Descriptions {
		v.writeComma(i)
		v.b.WriteByte('[')
		ast.Walk(v, def.Term)

		if len(def.Descriptions) > 0 {
			for _, b := range def.Descriptions {
				v.b.WriteString(",[")
				for j, dn := range b {
					v.writeComma(j)
					ast.Walk(v, dn)
				}
				v.b.WriteByte(']')
			}
		}
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitTable(tn *ast.TableNode) {
	v.writeNodeStart("Table")
	v.writeContentStart('p')

	// Table header
	v.b.WriteByte('[')
	for i, cell := range tn.Header {
		v.writeComma(i)
		v.writeCell(cell)
	}
	v.b.WriteString("],")

	// Table rows
	v.b.WriteByte('[')
	for i, row := range tn.Rows {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, cell := range row {
			v.writeComma(j)
			v.writeCell(cell)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteString("]]")
}

var alignmentCode = map[ast.Alignment]string{
	ast.AlignDefault: "[\"\",",
	ast.AlignLeft:    "[\"<\",",
	ast.AlignCenter:  "[\":\",",
	ast.AlignRight:   "[\">\",",
}

func (v *visitor) writeCell(cell *ast.TableCell) {
	v.b.WriteString(alignmentCode[cell.Align])
	ast.Walk(v, cell.Inlines)
	v.b.WriteByte(']')
}

var mapRefState = map[ast.RefState]string{
	ast.RefStateInvalid:  "invalid",
	ast.RefStateZettel:   "zettel",
	ast.RefStateSelf:     "self",
	ast.RefStateFound:    "zettel",
	ast.RefStateBroken:   "broken",
	ast.RefStateHosted:   "local",
	ast.RefStateBased:    "based",
	ast.RefStateExternal: "external",
}

func (v *visitor) visitEmbed(en *ast.EmbedNode) {
	v.writeNodeStart("Embed")
	v.visitAttributes(en.Attrs)
	switch m := en.Material.(type) {
	case *ast.ReferenceMaterialNode:
		v.writeContentStart('s')
		writeEscaped(&v.b, m.Ref.String())
	case *ast.BLOBMaterialNode:
		v.writeContentStart('j')
		v.b.WriteString("\"s\":")
		writeEscaped(&v.b, m.Syntax)
		switch m.Syntax {
		case "svg":
			v.writeContentStart('q')
			writeEscaped(&v.b, string(m.Blob))
		default:
			v.writeContentStart('o')
			v.b.WriteBase64(m.Blob)
			v.b.WriteByte('"')
		}
		v.b.WriteByte('}')
	default:
		panic(fmt.Sprintf("Unknown material type %t for %v", en.Material, en.Material))
	}

	if en.Inlines != nil {
		v.writeContentStart('i')
		ast.Walk(v, en.Inlines)
	}
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	v.writeNodeStart("Mark")
	if text := mn.Text; text != "" {
		v.writeContentStart('s')
		writeEscaped(&v.b, text)
	}
	if fragment := mn.Fragment; fragment != "" {
		v.writeContentStart('q')
		v.b.WriteByte('"')
		v.b.WriteString(fragment)
		v.b.WriteByte('"')
	}
}

var mapFormatKind = map[ast.FormatKind]string{
	ast.FormatItalic:    "Italic",
	ast.FormatEmph:      "Emph",
	ast.FormatBold:      "Bold",
	ast.FormatStrong:    "Strong",
	ast.FormatMonospace: "Mono",
	ast.FormatStrike:    "Strikethrough",
	ast.FormatDelete:    "Delete",
	ast.FormatUnder:     "Underline",
	ast.FormatInsert:    "Insert",
	ast.FormatSuper:     "Super",
	ast.FormatSub:       "Sub",
	ast.FormatQuote:     "Quote",
	ast.FormatQuotation: "Quotation",
	ast.FormatSmall:     "Small",
	ast.FormatSpan:      "Span",
}

var mapLiteralKind = map[ast.LiteralKind]string{
	ast.LiteralProg:    "Code",
	ast.LiteralKeyb:    "Input",
	ast.LiteralOutput:  "Output",
	ast.LiteralComment: "Comment",
	ast.LiteralHTML:    "HTML",
}

func (v *visitor) visitBlockList(bln *ast.BlockListNode) {
	v.b.WriteByte('[')
	for i, bn := range bln.List {
		v.writeComma(i)
		ast.Walk(v, bn)
	}
	v.b.WriteByte(']')
}

func (v *visitor) walkInlineList(iln *ast.InlineListNode) {
	v.b.WriteByte('[')
	for i, in := range iln.List {
		v.writeComma(i)
		ast.Walk(v, in)
	}
	v.b.WriteByte(']')
}

// visitAttributes write JSON attributes
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)
	}
	sort.Strings(keys)

	v.b.WriteString(",\"a\":{\"")
	for i, k := range keys {
		if i > 0 {
			v.b.WriteString("\",\"")
		}
		strfun.JSONEscape(&v.b, k)
		v.b.WriteString("\":\"")
		strfun.JSONEscape(&v.b, a.Attrs[k])
	}
	v.b.WriteString("\"}")
}

func (v *visitor) writeNodeStart(t string) {
	v.b.WriteStrings("{\"t\":\"", t, "\"")
}

var contentCode = map[rune][]byte{
	'b': []byte(",\"b\":"),   // List of blocks
	'c': []byte(",\"c\":["),  // List of list of blocks
	'g': []byte(",\"g\":["),  // General list
	'i': []byte(",\"i\":"),   // List of inlines
	'j': []byte(",\"j\":{"),  // Embedded JSON object
	'l': []byte(",\"l\":["),  // List of lines
	'n': []byte(",\"n\":"),   // Number
	'o': []byte(",\"o\":\""), // Byte object
	'p': []byte(",\"p\":["),  // Generic tuple
	'q': []byte(",\"q\":"),   // String, if 's' is also needed
	's': []byte(",\"s\":"),   // String
	't': []byte("Content code 't' is not allowed"),
	'y': []byte("Content code 'y' is not allowed"), // field after 'j'
}

func (v *visitor) writeContentStart(code rune) {
	if b, ok := contentCode[code]; ok {
		v.b.Write(b)
		return
	}
	panic("Unknown content code " + strconv.Itoa(int(code)))
}

func (v *visitor) writeMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) {
	for i, p := range m.Pairs(true) {
		if i > 0 {
			v.b.WriteByte(',')
		}
		v.b.WriteByte('"')
		key := p.Key
		strfun.JSONEscape(&v.b, key)
		v.b.WriteString("\":")
		t := m.Type(key)
		if t.IsSet {
			v.writeSetValue(p.Value)
			continue
		}
		if t == meta.TypeZettelmarkup {
			ast.Walk(v, evalMeta(p.Value))
			continue
		}
		writeEscaped(&v.b, p.Value)
	}
}

func (v *visitor) writeSetValue(value string) {
	v.b.WriteByte('[')
	for i, val := range meta.ListFromValue(value) {
		v.writeComma(i)
		writeEscaped(&v.b, val)
	}
	v.b.WriteByte(']')
}

func (v *visitor) writeComma(pos int) {
	if pos > 0 {
		v.b.WriteByte(',')
	}
}

func writeEscaped(b *encoder.BufWriter, s string) {
	b.WriteByte('"')
	strfun.JSONEscape(b, s)
	b.WriteByte('"')
}

Deleted encoder/encfun/encfun.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38






































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package encfun provides some helper function to work with encodings.
package encfun

import (
	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
)

// MetaAsInlineSlice returns the value of the given metadata key as an inlince slice.
func MetaAsInlineSlice(m *meta.Meta, key string) ast.InlineSlice {
	return parser.ParseMetadata(m.GetDefault(key, ""))
}

// MetaAsText returns the value of given metadata as text.
func MetaAsText(m *meta.Meta, key string) string {
	textEncoder := encoder.Create(api.EncoderText, nil)
	var sb strings.Builder
	_, err := textEncoder.WriteInlines(&sb, MetaAsInlineSlice(m, key))
	if err == nil {
		return sb.String()
	}
	return ""
}

Changes to encoder/encoder.go.

10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28


29
30
31


32




33
34
35
36
37
38
39
40
41
42
43
44
45


46
47
48
49
50
51
52
53
54
55
56
57
58

59
60
61
62
63



64
65
66
67


68
69

70
71

72
73
74
75


76
77
78


79
80
81
82
83
84
85
86




87
88
89


90
91

92
93
10
11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26


27
28
29


30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47


48
49
50
51
52
53
54
55
56
57
58
59
60
61

62
63
64



65
66
67
68
69


70
71
72

73
74

75
76
77


78
79
80


81
82
83
84
85
86




87
88
89
90
91


92
93
94

95

96







+

-








-
-
+
+

-
-
+
+

+
+
+
+











-
-
+
+












-
+


-
-
-
+
+
+


-
-
+
+

-
+

-
+


-
-
+
+

-
-
+
+




-
-
-
-
+
+
+
+

-
-
+
+

-
+
-


// Package encoder provides a generic interface to encode the abstract syntax
// tree into some text form.
package encoder

import (
	"errors"
	"fmt"
	"io"
	"log"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
)

// Encoder is an interface that allows to encode different parts of a zettel.
type Encoder interface {
	WriteZettel(io.Writer, *ast.ZettelNode, bool) (int, error)
	WriteMeta(io.Writer, *meta.Meta) (int, error)
	WriteZettel(io.Writer, *ast.ZettelNode, EvalMetaFunc) (int, error)
	WriteMeta(io.Writer, *meta.Meta, EvalMetaFunc) (int, error)
	WriteContent(io.Writer, *ast.ZettelNode) (int, error)
	WriteBlocks(io.Writer, ast.BlockSlice) (int, error)
	WriteInlines(io.Writer, ast.InlineSlice) (int, error)
	WriteBlocks(io.Writer, *ast.BlockListNode) (int, error)
	WriteInlines(io.Writer, *ast.InlineListNode) (int, error)
}

// EvalMetaFunc is a function that takes a string of metadata and returns
// a list of syntax elements.
type EvalMetaFunc func(string) *ast.InlineListNode

// Some errors to signal when encoder methods are not implemented.
var (
	ErrNoWriteZettel  = errors.New("method WriteZettel is not implemented")
	ErrNoWriteMeta    = errors.New("method WriteMeta is not implemented")
	ErrNoWriteContent = errors.New("method WriteContent is not implemented")
	ErrNoWriteBlocks  = errors.New("method WriteBlocks is not implemented")
	ErrNoWriteInlines = errors.New("method WriteInlines is not implemented")
)

// Create builds a new encoder with the given options.
func Create(format api.EncodingEnum, env *Environment) Encoder {
	if info, ok := registry[format]; ok {
func Create(enc api.EncodingEnum, env *Environment) Encoder {
	if info, ok := registry[enc]; ok {
		return info.Create(env)
	}
	return nil
}

// Info stores some data about an encoder.
type Info struct {
	Create  func(*Environment) Encoder
	Default bool
}

var registry = map[api.EncodingEnum]Info{}
var defFormat api.EncodingEnum
var defEncoding api.EncodingEnum

// Register the encoder for later retrieval.
func Register(format api.EncodingEnum, info Info) {
	if _, ok := registry[format]; ok {
		log.Fatalf("Writer with format %q already registered", format)
func Register(enc api.EncodingEnum, info Info) {
	if _, ok := registry[enc]; ok {
		panic(fmt.Sprintf("Encoder %q already registered", enc))
	}
	if info.Default {
		if defFormat != api.EncoderUnknown && defFormat != format {
			log.Fatalf("Default format already set: %q, new format: %q", defFormat, format)
		if defEncoding != api.EncoderUnknown && defEncoding != enc {
			panic(fmt.Sprintf("Default encoder already set: %q, new encoding: %q", defEncoding, enc))
		}
		defFormat = format
		defEncoding = enc
	}
	registry[format] = info
	registry[enc] = info
}

// GetFormats returns all registered formats, ordered by format code.
func GetFormats() []api.EncodingEnum {
// GetEncodings returns all registered encodings, ordered by encoding value.
func GetEncodings() []api.EncodingEnum {
	result := make([]api.EncodingEnum, 0, len(registry))
	for format := range registry {
		result = append(result, format)
	for enc := range registry {
		result = append(result, enc)
	}
	return result
}

// GetDefaultFormat returns the format that should be used as default.
func GetDefaultFormat() api.EncodingEnum {
	if defFormat != api.EncoderUnknown {
		return defFormat
// GetDefaultEncoding returns the encoding that should be used as default.
func GetDefaultEncoding() api.EncodingEnum {
	if defEncoding != api.EncoderUnknown {
		return defEncoding
	}
	if _, ok := registry[api.EncoderJSON]; ok {
		return api.EncoderJSON
	if _, ok := registry[api.EncoderDJSON]; ok {
		return api.EncoderDJSON
	}
	log.Fatalf("No default format given")
	panic("No default encoding given")
	return api.EncoderUnknown
}

Changes to encoder/env.go.

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
12
13
14
15
16
17
18





19
20
21
22
23
24
25
26
27
28













































29
30
31
32
33
34
35







-
-
-
-
-










-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







// tree into some text form.
package encoder

import "zettelstore.de/z/ast"

// Environment specifies all data and functions that affects encoding.
type Environment struct {
	// Important for many encoder.
	LinkAdapter  func(*ast.LinkNode) ast.InlineNode
	ImageAdapter func(*ast.ImageNode) ast.InlineNode
	CiteAdapter  func(*ast.CiteNode) ast.InlineNode

	// Important for HTML encoder
	Lang           string // default language
	Interactive    bool   // Encoded data will be placed in interactive content
	Xhtml          bool   // use XHTML syntax instead of HTML syntax
	MarkerExternal string // Marker after link to (external) material.
	NewWindow      bool   // open link in new window
	IgnoreMeta     map[string]bool
	footnotes      []*ast.FootnoteNode // Stores footnotes detected while encoding
}

// AdaptLink helps to call the link adapter.
func (env *Environment) AdaptLink(ln *ast.LinkNode) (*ast.LinkNode, ast.InlineNode) {
	if env == nil || env.LinkAdapter == nil {
		return ln, nil
	}
	n := env.LinkAdapter(ln)
	if n == nil {
		return ln, nil
	}
	if ln2, ok := n.(*ast.LinkNode); ok {
		return ln2, nil
	}
	return nil, n
}

// AdaptImage helps to call the link adapter.
func (env *Environment) AdaptImage(in *ast.ImageNode) (*ast.ImageNode, ast.InlineNode) {
	if env == nil || env.ImageAdapter == nil {
		return in, nil
	}
	n := env.ImageAdapter(in)
	if n == nil {
		return in, nil
	}
	if in2, ok := n.(*ast.ImageNode); ok {
		return in2, nil
	}
	return nil, n
}

// AdaptCite helps to call the link adapter.
func (env *Environment) AdaptCite(cn *ast.CiteNode) (*ast.CiteNode, ast.InlineNode) {
	if env == nil || env.CiteAdapter == nil {
		return cn, nil
	}
	n := env.CiteAdapter(cn)
	if n == nil {
		return cn, nil
	}
	if cn2, ok := n.(*ast.CiteNode); ok {
		return cn2, nil
	}
	return nil, n
}

// IsInteractive returns true, if Interactive is enabled and currently embedded
// interactive encoding will take place.
func (env *Environment) IsInteractive(inInteractive bool) bool {
	return inInteractive && env != nil && env.Interactive
}

// IsXHTML return true, if XHTML is enabled.

Changes to encoder/htmlenc/block.go.

114
115
116
117
118
119
120
121
122


123
124

125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144





145
146

147
148
149
150
151
152
153
114
115
116
117
118
119
120


121
122
123

124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141



142
143
144
145
146
147

148
149
150
151
152
153
154
155







-
-
+
+

-
+

















-
-
-
+
+
+
+
+

-
+








	v.lang.push(attrs)
	defer v.lang.pop()

	v.b.WriteStrings("<", code)
	v.visitAttributes(attrs)
	v.b.WriteString(">\n")
	ast.WalkBlockSlice(v, rn.Blocks)
	if len(rn.Inlines) > 0 {
	ast.Walk(v, rn.Blocks)
	if rn.Inlines != nil {
		v.b.WriteString("<cite>")
		ast.WalkInlineSlice(v, rn.Inlines)
		ast.Walk(v, rn.Inlines)
		v.b.WriteString("</cite>\n")
	}
	v.b.WriteStrings("</", code, ">\n")
	v.inVerse = oldVerse
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	v.lang.push(hn.Attrs)
	defer v.lang.pop()

	lvl := hn.Level
	if lvl > 6 {
		lvl = 6 // HTML has H1..H6
	}
	strLvl := strconv.Itoa(lvl)
	v.b.WriteStrings("<h", strLvl)
	v.visitAttributes(hn.Attrs)
	if slug := hn.Slug; len(slug) > 0 {
		v.b.WriteStrings(" id=\"", slug, "\"")
	}
	if _, ok := hn.Attrs.Get("id"); !ok {
		if fragment := hn.Fragment; fragment != "" {
			v.b.WriteStrings(" id=\"", fragment, "\"")
		}
	}
	v.b.WriteByte('>')
	ast.WalkInlineSlice(v, hn.Inlines)
	ast.Walk(v, hn.Inlines)
	v.b.WriteStrings("</h", strLvl, ">\n")
}

var mapNestedListKind = map[ast.NestedListKind]string{
	ast.NestedListOrdered:   "ol",
	ast.NestedListUnordered: "ul",
}
186
187
188
189
190
191
192
193

194
195
196
197
198
199
200
188
189
190
191
192
193
194

195
196
197
198
199
200
201
202







-
+







		if pn := getParaItem(item); pn != nil {
			if inPara {
				v.b.WriteByte('\n')
			} else {
				v.b.WriteString("<p>")
				inPara = true
			}
			ast.WalkInlineSlice(v, pn.Inlines)
			ast.Walk(v, pn.Inlines)
		} else {
			if inPara {
				v.writeEndPara()
				inPara = false
			}
			ast.WalkItemSlice(v, item)
		}
241
242
243
244
245
246
247
248

249
250
251
252
253
254
255
256
257
258

259
260
261
262
263
264
265
266
267
268
269

270
271
272
273
274
275
276
243
244
245
246
247
248
249

250
251
252
253
254
255
256
257
258
259

260
261
262
263
264
265
266
267
268
269
270

271
272
273
274
275
276
277
278







-
+









-
+










-
+








// writeItemSliceOrPara emits the content of a paragraph if the paragraph is
// the only element of the block slice and if compact mode is true. Otherwise,
// the item slice is emitted normally.
func (v *visitor) writeItemSliceOrPara(ins ast.ItemSlice, compact bool) {
	if compact && len(ins) == 1 {
		if para, ok := ins[0].(*ast.ParaNode); ok {
			ast.WalkInlineSlice(v, para.Inlines)
			ast.Walk(v, para.Inlines)
			return
		}
	}
	ast.WalkItemSlice(v, ins)
}

func (v *visitor) writeDescriptionsSlice(ds ast.DescriptionSlice) {
	if len(ds) == 1 {
		if para, ok := ds[0].(*ast.ParaNode); ok {
			ast.WalkInlineSlice(v, para.Inlines)
			ast.Walk(v, para.Inlines)
			return
		}
	}
	ast.WalkDescriptionSlice(v, ds)
}

func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.b.WriteString("<dl>\n")
	for _, descr := range dn.Descriptions {
		v.b.WriteString("<dt>")
		ast.WalkInlineSlice(v, descr.Term)
		ast.Walk(v, descr.Term)
		v.b.WriteString("</dt>\n")

		for _, b := range descr.Descriptions {
			v.b.WriteString("<dd>")
			v.writeDescriptionsSlice(b)
			v.b.WriteString("</dd>\n")
		}
302
303
304
305
306
307
308
309

310
311
312
313

314
315
316
317
318
319
320
304
305
306
307
308
309
310

311
312
313
314

315
316
317
318
319
320
321
322







-
+



-
+







	ast.AlignRight:   " style=\"text-align:right\">",
}

func (v *visitor) writeRow(row ast.TableRow, cellStart, cellEnd string) {
	v.b.WriteString("<tr>")
	for _, cell := range row {
		v.b.WriteString(cellStart)
		if len(cell.Inlines) == 0 {
		if cell.Inlines.IsEmpty() {
			v.b.WriteByte('>')
		} else {
			v.b.WriteString(alignStyle[cell.Align])
			ast.WalkInlineSlice(v, cell.Inlines)
			ast.Walk(v, cell.Inlines)
		}
		v.b.WriteString(cellEnd)
	}
	v.b.WriteString("</tr>\n")
}

func (v *visitor) visitBLOB(bn *ast.BLOBNode) {

Changes to encoder/htmlenc/htmlenc.go.

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53












54
55


56
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71
72

73
74
75
76
77

78
79
80
81
82
83
84
85
86
87

88
89

90
91
92
93
94
95
96

97
98
99
100
101

102
103
104
9
10
11
12
13
14
15

16
17
18
19
20


21
22
23
24
25
26
27
28
29
30
31
32
33

34
35
36
37
38
39
40
41
42
43
44






45
46
47
48
49
50
51
52
53
54
55
56


57
58
59
60
61
62
63
64
65

66
67
68
69
70



71

72
73
74
75
76

77
78
79
80
81
82
83
84
85
86

87
88

89
90
91
92
93
94
95

96
97
98
99
100

101
102
103
104







-





-
-













-
+










-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+







-
+




-
-
-

-
+




-
+









-
+

-
+






-
+




-
+



//-----------------------------------------------------------------------------

// Package htmlenc encodes the abstract syntax tree into HTML5.
package htmlenc

import (
	"io"
	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/encfun"
	"zettelstore.de/z/parser"
)

func init() {
	encoder.Register(api.EncoderHTML, encoder.Info{
		Create: func(env *encoder.Environment) encoder.Encoder { return &htmlEncoder{env: env} },
	})
}

type htmlEncoder struct {
	env *encoder.Environment
}

// WriteZettel encodes a full zettel as HTML5.
func (he *htmlEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, inhMeta bool) (int, error) {
func (he *htmlEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(he, w)
	if !he.env.IsXHTML() {
		v.b.WriteString("<!DOCTYPE html>\n")
	}
	if env := he.env; env != nil && env.Lang == "" {
		v.b.WriteStrings("<html>\n<head>")
	} else {
		v.b.WriteStrings("<html lang=\"", env.Lang, "\">")
	}
	v.b.WriteString("\n<head>\n<meta charset=\"utf-8\">\n")
	v.b.WriteStrings("<title>", encfun.MetaAsText(zn.InhMeta, meta.KeyTitle), "</title>")
	if inhMeta {
		v.acceptMeta(zn.InhMeta)
	} else {
		v.acceptMeta(zn.Meta)
	}
	plainTitle, hasTitle := zn.InhMeta.Get(meta.KeyTitle)
	if hasTitle {
		v.b.WriteStrings("<title>", v.evalValue(plainTitle, evalMeta), "</title>")
	}
	v.acceptMeta(zn.InhMeta, evalMeta)
	v.b.WriteString("\n</head>\n<body>\n")
	if hasTitle {
		if ilnTitle := evalMeta(plainTitle); ilnTitle != nil {
			v.b.WriteString("<h1>")
			ast.Walk(v, ilnTitle)
			v.b.WriteString("</h1>\n")
		}
	v.b.WriteString("\n</head>\n<body>\n")
	ast.WalkBlockSlice(v, zn.Ast)
	}
	ast.Walk(v, zn.Ast)
	v.writeEndnotes()
	v.b.WriteString("</body>\n</html>")
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as HTML5.
func (he *htmlEncoder) WriteMeta(w io.Writer, m *meta.Meta) (int, error) {
func (he *htmlEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(he, w)

	// Write title
	if title, ok := m.Get(meta.KeyTitle); ok {
		textEnc := encoder.Create(api.EncoderText, nil)
		var sb strings.Builder
		textEnc.WriteInlines(&sb, parser.ParseMetadata(title))
		v.b.WriteStrings("<meta name=\"zs-", meta.KeyTitle, "\" content=\"")
		v.writeQuotedEscaped(sb.String())
		v.writeQuotedEscaped(v.evalValue(title, evalMeta))
		v.b.WriteString("\">")
	}

	// Write other metadata
	v.acceptMeta(m)
	v.acceptMeta(m, evalMeta)
	length, err := v.b.Flush()
	return length, err
}

func (he *htmlEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return he.WriteBlocks(w, zn.Ast)
}

// WriteBlocks encodes a block slice.
func (he *htmlEncoder) WriteBlocks(w io.Writer, bs ast.BlockSlice) (int, error) {
func (he *htmlEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(he, w)
	ast.WalkBlockSlice(v, bs)
	ast.Walk(v, bln)
	v.writeEndnotes()
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (he *htmlEncoder) WriteInlines(w io.Writer, is ast.InlineSlice) (int, error) {
func (he *htmlEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(he, w)
	if env := he.env; env != nil {
		v.inInteractive = env.Interactive
	}
	ast.WalkInlineSlice(v, is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

Changes to encoder/htmlenc/inline.go.

29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
29
30
31
32
33
34
35





36
37
38
39
40
41
42







-
-
-
-
-







		}
	} else {
		v.b.WriteByte('\n')
	}
}

func (v *visitor) visitLink(ln *ast.LinkNode) {
	ln, n := v.env.AdaptLink(ln)
	if n != nil {
		ast.Walk(v, n)
		return
	}
	v.lang.push(ln.Attrs)
	defer v.lang.pop()

	switch ln.Ref.State {
	case ast.RefStateSelf, ast.RefStateFound, ast.RefStateHosted, ast.RefStateBased:
		v.writeAHref(ln.Ref, ln.Attrs, ln.Inlines)
	case ast.RefStateBroken:
66
67
68
69
70
71
72
73

74
75
76
77
78
79

80
81

82
83
84
85
86
87
88
89
90

91
92
93
94
95

96
97
98
99
100
101

102
103



104


105
106

107
108
109

110
111
112


113
114

115
116

117
118

119


120
121

122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141

142
143

144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164


165
166
167
168
169
170
171
172
173

174
175
176
177
178
179
180
61
62
63
64
65
66
67

68
69
70
71
72
73

74
75

76
77
78
79
80
81
82
83
84

85
86
87
88
89

90






91
92
93
94
95
96

97
98
99

100
101
102

103
104


105
106
107

108


109
110
111
112

113
114
115

116
117
118
119
120
121
122
123
124








125
126
127

128
129

130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149


150
151
152
153
154
155
156
157
158
159

160
161
162
163
164
165
166
167







-
+





-
+

-
+








-
+




-
+
-
-
-
-
-
-
+


+
+
+
-
+
+

-
+


-
+

-
-
+
+

-
+
-
-
+


+
-
+
+

-
+








-
-
-
-
-
-
-
-



-
+

-
+



















-
-
+
+








-
+







		}
		v.b.WriteString("<a href=\"")
		v.writeQuotedEscaped(ln.Ref.Value)
		v.b.WriteByte('"')
		v.visitAttributes(ln.Attrs)
		v.b.WriteByte('>')
		v.inInteractive = true
		ast.WalkInlineSlice(v, ln.Inlines)
		ast.Walk(v, ln.Inlines)
		v.inInteractive = false
		v.b.WriteString("</a>")
	}
}

func (v *visitor) writeAHref(ref *ast.Reference, attrs *ast.Attributes, ins ast.InlineSlice) {
func (v *visitor) writeAHref(ref *ast.Reference, attrs *ast.Attributes, iln *ast.InlineListNode) {
	if v.env.IsInteractive(v.inInteractive) {
		v.writeSpan(ins, attrs)
		v.writeSpan(iln, attrs)
		return
	}
	v.b.WriteString("<a href=\"")
	v.writeReference(ref)
	v.b.WriteByte('"')
	v.visitAttributes(attrs)
	v.b.WriteByte('>')
	v.inInteractive = true
	ast.WalkInlineSlice(v, ins)
	ast.Walk(v, iln)
	v.inInteractive = false
	v.b.WriteString("</a>")
}

func (v *visitor) visitImage(in *ast.ImageNode) {
func (v *visitor) visitEmbed(en *ast.EmbedNode) {
	in, n := v.env.AdaptImage(in)
	if n != nil {
		ast.Walk(v, n)
		return
	}
	v.lang.push(in.Attrs)
	v.lang.push(en.Attrs)
	defer v.lang.pop()

	switch m := en.Material.(type) {
	case *ast.ReferenceMaterialNode:
		v.b.WriteString("<img src=\"")
	if in.Ref == nil {
		v.writeReference(m.Ref)
	case *ast.BLOBMaterialNode:
		v.b.WriteString("<img src=\"data:image/")
		switch in.Syntax {
		switch m.Syntax {
		case "svg":
			v.b.WriteString("svg+xml;utf8,")
			v.writeQuotedEscaped(string(in.Blob))
			v.writeQuotedEscaped(string(m.Blob))
		default:
			v.b.WriteStrings(in.Syntax, ";base64,")
			v.b.WriteBase64(in.Blob)
			v.b.WriteStrings(m.Syntax, ";base64,")
			v.b.WriteBase64(m.Blob)
		}
	} else {
	default:
		v.b.WriteString("<img src=\"")
		v.writeReference(in.Ref)
		panic(fmt.Sprintf("Unknown material type %t for %v", en.Material, en.Material))
	}
	v.b.WriteString("\" alt=\"")
	if en.Inlines != nil {
	ast.WalkInlineSlice(v, in.Inlines)
		ast.Walk(v, en.Inlines)
	}
	v.b.WriteByte('"')
	v.visitAttributes(in.Attrs)
	v.visitAttributes(en.Attrs)
	if v.env.IsXHTML() {
		v.b.WriteString(" />")
	} else {
		v.b.WriteByte('>')
	}
}

func (v *visitor) visitCite(cn *ast.CiteNode) {
	cn, n := v.env.AdaptCite(cn)
	if n != nil {
		ast.Walk(v, n)
		return
	}
	if cn == nil {
		return
	}
	v.lang.push(cn.Attrs)
	defer v.lang.pop()
	v.b.WriteString(cn.Key)
	if len(cn.Inlines) > 0 {
	if cn.Inlines != nil {
		v.b.WriteString(", ")
		ast.WalkInlineSlice(v, cn.Inlines)
		ast.Walk(v, cn.Inlines)
	}
}

func (v *visitor) visitFootnote(fn *ast.FootnoteNode) {
	v.lang.push(fn.Attrs)
	defer v.lang.pop()
	if v.env.IsInteractive(v.inInteractive) {
		return
	}

	n := strconv.Itoa(v.env.AddFootnote(fn))
	v.b.WriteStrings("<sup id=\"fnref:", n, "\"><a href=\"#fn:", n, "\" class=\"zs-footnote-ref\" role=\"doc-noteref\">", n, "</a></sup>")
	// TODO: what to do with Attrs?
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	if v.env.IsInteractive(v.inInteractive) {
		return
	}
	if len(mn.Text) > 0 {
		v.b.WriteStrings("<a id=\"", mn.Text, "\"></a>")
	if fragment := mn.Fragment; fragment != "" {
		v.b.WriteStrings("<a id=\"", fragment, "\"></a>")
	}
}

func (v *visitor) visitFormat(fn *ast.FormatNode) {
	v.lang.push(fn.Attrs)
	defer v.lang.pop()

	var code string
	attrs := fn.Attrs
	attrs := fn.Attrs.Clone()
	switch fn.Kind {
	case ast.FormatItalic:
		code = "i"
	case ast.FormatEmph:
		code = "em"
	case ast.FormatBold:
		code = "b"
207
208
209
210
211
212
213
214

215
216
217
218

219
220
221
222

223
224
225
226
227
228
229
194
195
196
197
198
199
200

201
202
203
204

205
206
207
208

209
210
211
212
213
214
215
216







-
+



-
+



-
+







		return
	default:
		panic(fmt.Sprintf("Unknown format kind %v", fn.Kind))
	}
	v.b.WriteStrings("<", code)
	v.visitAttributes(attrs)
	v.b.WriteByte('>')
	ast.WalkInlineSlice(v, fn.Inlines)
	ast.Walk(v, fn.Inlines)
	v.b.WriteStrings("</", code, ">")
}

func (v *visitor) writeSpan(ins ast.InlineSlice, attrs *ast.Attributes) {
func (v *visitor) writeSpan(iln *ast.InlineListNode, attrs *ast.Attributes) {
	v.b.WriteString("<span")
	v.visitAttributes(attrs)
	v.b.WriteByte('>')
	ast.WalkInlineSlice(v, ins)
	ast.Walk(v, iln)
	v.b.WriteString("</span>")

}

var langQuotes = map[string][2]string{
	meta.ValueLangEN: {"&ldquo;", "&rdquo;"},
	"de":             {"&bdquo;", "&ldquo;"},
248
249
250
251
252
253
254
255

256
257
258
259
260
261
262
235
236
237
238
239
240
241

242
243
244
245
246
247
248
249







-
+







	if withSpan {
		v.b.WriteString("<span")
		v.visitAttributes(fn.Attrs)
		v.b.WriteByte('>')
	}
	openingQ, closingQ := getQuotes(v.lang.top())
	v.b.WriteString(openingQ)
	ast.WalkInlineSlice(v, fn.Inlines)
	ast.Walk(v, fn.Inlines)
	v.b.WriteString(closingQ)
	if withSpan {
		v.b.WriteString("</span>")
	}
}

func (v *visitor) visitLiteral(ln *ast.LiteralNode) {

Changes to encoder/htmlenc/visitor.go.

13
14
15
16
17
18
19

20
21
22
23
24
25
26
27
28
29
30

31
32
33

34
35
36
37
38
39
40
41
42
43
44




45
46
47
48
49
50
51
52

53
54
55
56
57
58
59
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43



44
45
46
47
48
49
50
51
52
53
54

55
56
57
58
59
60
61
62







+










-
+



+








-
-
-
+
+
+
+







-
+








import (
	"io"
	"sort"
	"strconv"
	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	env           *encoder.Environment
	b             encoder.BufWriter
	visibleSpace  bool // Show space character in raw text
	visibleSpace  bool // Show space character in plain text
	inVerse       bool // In verse block
	inInteractive bool // Rendered interactive HTML code
	lang          langStack
	textEnc       encoder.Encoder
}

func newVisitor(he *htmlEncoder, w io.Writer) *visitor {
	var lang string
	if he.env != nil {
		lang = he.env.Lang
	}
	return &visitor{
		env:  he.env,
		b:    encoder.NewBufWriter(w),
		lang: newLangStack(lang),
		env:     he.env,
		b:       encoder.NewBufWriter(w),
		lang:    newLangStack(lang),
		textEnc: encoder.Create(api.EncoderText, nil),
	}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.ParaNode:
		v.b.WriteString("<p>")
		ast.WalkInlineSlice(v, n.Inlines)
		ast.Walk(v, n.Inlines)
		v.writeEndPara()
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.visitHeading(n)
86
87
88
89
90
91
92
93
94


95
96
97
98
99
100
101
89
90
91
92
93
94
95


96
97
98
99
100
101
102
103
104







-
-
+
+







		} else {
			v.b.WriteByte(' ')
		}
	case *ast.BreakNode:
		v.visitBreak(n)
	case *ast.LinkNode:
		v.visitLink(n)
	case *ast.ImageNode:
		v.visitImage(n)
	case *ast.EmbedNode:
		v.visitEmbed(n)
	case *ast.CiteNode:
		v.visitCite(n)
	case *ast.FootnoteNode:
		v.visitFootnote(n)
	case *ast.MarkNode:
		v.visitMark(n)
	case *ast.FormatNode:
109
110
111
112
113
114
115
116
117
118















119
120

121
122
123




124
125
126
127



128
129

130
131






















132
133
134
135
136
137
138
112
113
114
115
116
117
118



119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136



137
138
139
140




141
142
143
144

145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176







-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+


+
-
-
-
+
+
+
+
-
-
-
-
+
+
+

-
+


+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







}

var mapMetaKey = map[string]string{
	meta.KeyCopyright: "copyright",
	meta.KeyLicense:   "license",
}

func (v *visitor) acceptMeta(m *meta.Meta) {
	for _, pair := range m.Pairs(true) {
		if env := v.env; env != nil && env.IgnoreMeta[pair.Key] {
func (v *visitor) acceptMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) {
	ignore := v.setupIgnoreSet()
	ignore[meta.KeyTitle] = true
	if tags, ok := m.Get(meta.KeyAllTags); ok {
		v.writeTags(tags)
		ignore[meta.KeyAllTags] = true
		ignore[meta.KeyTags] = true
	} else if tags, ok := m.Get(meta.KeyTags); ok {
		v.writeTags(tags)
		ignore[meta.KeyTags] = true
	}

	for _, p := range m.Pairs(true) {
		key := p.Key
		if ignore[key] {
			continue
		}
		value := p.Value
		if pair.Key == meta.KeyTitle {
			continue
		}
		if m.Type(key) == meta.TypeZettelmarkup {
			if v := v.evalValue(value, evalMeta); v != "" {
				value = v
			}
		if pair.Key == meta.KeyTags {
			v.writeTags(pair.Value)
		} else if key, ok := mapMetaKey[pair.Key]; ok {
			v.writeMeta("", key, pair.Value)
		}
		if mKey, ok := mapMetaKey[key]; ok {
			v.writeMeta("", mKey, value)
		} else {
			v.writeMeta("zs-", pair.Key, pair.Value)
			v.writeMeta("zs-", key, value)
		}
	}
}

func (v *visitor) evalValue(value string, evalMeta encoder.EvalMetaFunc) string {
	var sb strings.Builder
	_, err := v.textEnc.WriteInlines(&sb, evalMeta(value))
	if err == nil {
		return sb.String()
	}
	return ""
}

func (v *visitor) setupIgnoreSet() map[string]bool {
	if v.env == nil || v.env.IgnoreMeta == nil {
		return make(map[string]bool)
	}
	result := make(map[string]bool, len(v.env.IgnoreMeta))
	for k, v := range v.env.IgnoreMeta {
		if v {
			result[k] = true
		}
	}
	return result
}

func (v *visitor) writeTags(tags string) {
	v.b.WriteString("\n<meta name=\"keywords\" content=\"")
	for i, val := range meta.ListFromValue(tags) {
		if i > 0 {
			v.b.WriteString(", ")
154
155
156
157
158
159
160
161

162
163
164
165
166
167
168
169
170
171
172
173

174
175
176
177
178
179
180
192
193
194
195
196
197
198

199
200
201
202
203
204
205
206
207
208
209
210

211
212
213
214
215
216
217
218







-
+











-
+







		v.b.WriteString("<ol class=\"zs-endnotes\">\n")
		for i := 0; i < len(footnotes); i++ {
			// Do not use a range loop above, because a footnote may contain
			// a footnote. Therefore v.enc.footnote may grow during the loop.
			fn := footnotes[i]
			n := strconv.Itoa(i + 1)
			v.b.WriteStrings("<li id=\"fn:", n, "\" role=\"doc-endnote\">")
			ast.WalkInlineSlice(v, fn.Inlines)
			ast.Walk(v, fn.Inlines)
			v.b.WriteStrings(
				" <a href=\"#fnref:",
				n,
				"\" class=\"zs-footnote-backref\" role=\"doc-backlink\">&#x21a9;&#xfe0e;</a></li>\n")
		}
		v.b.WriteString("</ol>\n")
	}
}

// visitAttributes write HTML attributes
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a == nil || len(a.Attrs) == 0 {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		if k != "-" {
			keys = append(keys, k)
		}

Deleted encoder/jsonenc/djsonenc.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
































































































































































































































































































































































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package jsonenc encodes the abstract syntax tree into JSON.
package jsonenc

import (
	"fmt"
	"io"
	"sort"
	"strconv"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/encfun"
)

func init() {
	encoder.Register(api.EncoderDJSON, encoder.Info{
		Create: func(env *encoder.Environment) encoder.Encoder { return &jsonDetailEncoder{env: env} },
	})
}

type jsonDetailEncoder struct {
	env *encoder.Environment
}

// WriteZettel writes the encoded zettel to the writer.
func (je *jsonDetailEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, inhMeta bool) (int, error) {
	v := newDetailVisitor(w, je)
	v.b.WriteString("{\"meta\":{\"title\":")
	v.walkInlineSlice(encfun.MetaAsInlineSlice(zn.InhMeta, meta.KeyTitle))
	if inhMeta {
		v.writeMeta(zn.InhMeta)
	} else {
		v.writeMeta(zn.Meta)
	}
	v.b.WriteByte('}')
	v.b.WriteString(",\"content\":")
	v.walkBlockSlice(zn.Ast)
	v.b.WriteByte('}')
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as JSON.
func (je *jsonDetailEncoder) WriteMeta(w io.Writer, m *meta.Meta) (int, error) {
	v := newDetailVisitor(w, je)
	v.b.WriteString("{\"title\":")
	v.walkInlineSlice(encfun.MetaAsInlineSlice(m, meta.KeyTitle))
	v.writeMeta(m)
	v.b.WriteByte('}')
	length, err := v.b.Flush()
	return length, err
}

func (je *jsonDetailEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return je.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes a block slice to the writer
func (je *jsonDetailEncoder) WriteBlocks(w io.Writer, bs ast.BlockSlice) (int, error) {
	v := newDetailVisitor(w, je)
	v.walkBlockSlice(bs)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (je *jsonDetailEncoder) WriteInlines(w io.Writer, is ast.InlineSlice) (int, error) {
	v := newDetailVisitor(w, je)
	v.walkInlineSlice(is)
	length, err := v.b.Flush()
	return length, err
}

// detailVisitor writes the abstract syntax tree to an io.Writer.
type detailVisitor struct {
	b   encoder.BufWriter
	env *encoder.Environment
}

func newDetailVisitor(w io.Writer, je *jsonDetailEncoder) *detailVisitor {
	return &detailVisitor{b: encoder.NewBufWriter(w), env: je.env}
}

func (v *detailVisitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.ParaNode:
		v.writeNodeStart("Para")
		v.writeContentStart('i')
		v.walkInlineSlice(n.Inlines)
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.visitHeading(n)
	case *ast.HRuleNode:
		v.writeNodeStart("Hrule")
		v.visitAttributes(n.Attrs)
	case *ast.NestedListNode:
		v.visitNestedList(n)
	case *ast.DescriptionListNode:
		v.visitDescriptionList(n)
	case *ast.TableNode:
		v.visitTable(n)
	case *ast.BLOBNode:
		v.writeNodeStart("Blob")
		v.writeContentStart('q')
		writeEscaped(&v.b, n.Title)
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Syntax)
		v.writeContentStart('o')
		v.b.WriteBase64(n.Blob)
		v.b.WriteByte('"')
	case *ast.TextNode:
		v.writeNodeStart("Text")
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Text)
	case *ast.TagNode:
		v.writeNodeStart("Tag")
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Tag)
	case *ast.SpaceNode:
		v.writeNodeStart("Space")
		if l := len(n.Lexeme); l > 1 {
			v.writeContentStart('n')
			v.b.WriteString(strconv.Itoa(l))
		}
	case *ast.BreakNode:
		if n.Hard {
			v.writeNodeStart("Hard")
		} else {
			v.writeNodeStart("Soft")
		}
	case *ast.LinkNode:
		n, n2 := v.env.AdaptLink(n)
		if n2 != nil {
			ast.Walk(v, n2)
			return nil
		}
		v.writeNodeStart("Link")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('q')
		writeEscaped(&v.b, mapRefState[n.Ref.State])
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Ref.String())
		v.writeContentStart('i')
		v.walkInlineSlice(n.Inlines)
	case *ast.ImageNode:
		v.visitImage(n)
	case *ast.CiteNode:
		v.writeNodeStart("Cite")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Key)
		if len(n.Inlines) > 0 {
			v.writeContentStart('i')
			v.walkInlineSlice(n.Inlines)
		}
	case *ast.FootnoteNode:
		v.writeNodeStart("Footnote")
		v.visitAttributes(n.Attrs)
		v.writeContentStart('i')
		v.walkInlineSlice(n.Inlines)
	case *ast.MarkNode:
		v.writeNodeStart("Mark")
		if len(n.Text) > 0 {
			v.writeContentStart('s')
			writeEscaped(&v.b, n.Text)
		}
	case *ast.FormatNode:
		v.writeNodeStart(mapFormatKind[n.Kind])
		v.visitAttributes(n.Attrs)
		v.writeContentStart('i')
		v.walkInlineSlice(n.Inlines)
	case *ast.LiteralNode:
		kind, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown literal kind %v", n.Kind))
		}
		v.writeNodeStart(kind)
		v.visitAttributes(n.Attrs)
		v.writeContentStart('s')
		writeEscaped(&v.b, n.Text)
	default:
		return v
	}
	v.b.WriteByte('}')
	return nil
}

var mapVerbatimKind = map[ast.VerbatimKind]string{
	ast.VerbatimProg:    "CodeBlock",
	ast.VerbatimComment: "CommentBlock",
	ast.VerbatimHTML:    "HTMLBlock",
}

func (v *detailVisitor) visitVerbatim(vn *ast.VerbatimNode) {
	kind, ok := mapVerbatimKind[vn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown verbatim kind %v", vn.Kind))
	}
	v.writeNodeStart(kind)
	v.visitAttributes(vn.Attrs)
	v.writeContentStart('l')
	for i, line := range vn.Lines {
		v.writeComma(i)
		writeEscaped(&v.b, line)
	}
	v.b.WriteByte(']')
}

var mapRegionKind = map[ast.RegionKind]string{
	ast.RegionSpan:  "SpanBlock",
	ast.RegionQuote: "QuoteBlock",
	ast.RegionVerse: "VerseBlock",
}

func (v *detailVisitor) visitRegion(rn *ast.RegionNode) {
	kind, ok := mapRegionKind[rn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown region kind %v", rn.Kind))
	}
	v.writeNodeStart(kind)
	v.visitAttributes(rn.Attrs)
	v.writeContentStart('b')
	v.walkBlockSlice(rn.Blocks)
	if len(rn.Inlines) > 0 {
		v.writeContentStart('i')
		v.walkInlineSlice(rn.Inlines)
	}
}

func (v *detailVisitor) visitHeading(hn *ast.HeadingNode) {
	v.writeNodeStart("Heading")
	v.visitAttributes(hn.Attrs)
	v.writeContentStart('n')
	v.b.WriteString(strconv.Itoa(hn.Level))
	if slug := hn.Slug; len(slug) > 0 {
		v.writeContentStart('s')
		v.b.WriteStrings("\"", slug, "\"")
	}
	v.writeContentStart('i')
	v.walkInlineSlice(hn.Inlines)
}

var mapNestedListKind = map[ast.NestedListKind]string{
	ast.NestedListOrdered:   "OrderedList",
	ast.NestedListUnordered: "BulletList",
	ast.NestedListQuote:     "QuoteList",
}

func (v *detailVisitor) visitNestedList(ln *ast.NestedListNode) {
	v.writeNodeStart(mapNestedListKind[ln.Kind])
	v.writeContentStart('c')
	for i, item := range ln.Items {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, in := range item {
			v.writeComma(j)
			ast.Walk(v, in)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *detailVisitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.writeNodeStart("DescriptionList")
	v.writeContentStart('g')
	for i, def := range dn.Descriptions {
		v.writeComma(i)
		v.b.WriteByte('[')
		v.walkInlineSlice(def.Term)

		if len(def.Descriptions) > 0 {
			for _, b := range def.Descriptions {
				v.b.WriteString(",[")
				for j, dn := range b {
					v.writeComma(j)
					ast.Walk(v, dn)
				}
				v.b.WriteByte(']')
			}
		}
		v.b.WriteByte(']')
	}
	v.b.WriteByte(']')
}

func (v *detailVisitor) visitTable(tn *ast.TableNode) {
	v.writeNodeStart("Table")
	v.writeContentStart('p')

	// Table header
	v.b.WriteByte('[')
	for i, cell := range tn.Header {
		v.writeComma(i)
		v.writeCell(cell)
	}
	v.b.WriteString("],")

	// Table rows
	v.b.WriteByte('[')
	for i, row := range tn.Rows {
		v.writeComma(i)
		v.b.WriteByte('[')
		for j, cell := range row {
			v.writeComma(j)
			v.writeCell(cell)
		}
		v.b.WriteByte(']')
	}
	v.b.WriteString("]]")
}

var alignmentCode = map[ast.Alignment]string{
	ast.AlignDefault: "[\"\",",
	ast.AlignLeft:    "[\"<\",",
	ast.AlignCenter:  "[\":\",",
	ast.AlignRight:   "[\">\",",
}

func (v *detailVisitor) writeCell(cell *ast.TableCell) {
	v.b.WriteString(alignmentCode[cell.Align])
	v.walkInlineSlice(cell.Inlines)
	v.b.WriteByte(']')
}

var mapRefState = map[ast.RefState]string{
	ast.RefStateInvalid:  "invalid",
	ast.RefStateZettel:   "zettel",
	ast.RefStateSelf:     "self",
	ast.RefStateFound:    "zettel",
	ast.RefStateBroken:   "broken",
	ast.RefStateHosted:   "local",
	ast.RefStateBased:    "based",
	ast.RefStateExternal: "external",
}

func (v *detailVisitor) visitImage(in *ast.ImageNode) {
	in, n := v.env.AdaptImage(in)
	if n != nil {
		ast.Walk(v, n)
		return
	}
	v.writeNodeStart("Image")
	v.visitAttributes(in.Attrs)
	if in.Ref == nil {
		v.writeContentStart('j')
		v.b.WriteString("\"s\":")
		writeEscaped(&v.b, in.Syntax)
		switch in.Syntax {
		case "svg":
			v.writeContentStart('q')
			writeEscaped(&v.b, string(in.Blob))
		default:
			v.writeContentStart('o')
			v.b.WriteBase64(in.Blob)
			v.b.WriteByte('"')
		}
		v.b.WriteByte('}')
	} else {
		v.writeContentStart('s')
		writeEscaped(&v.b, in.Ref.String())
	}
	if len(in.Inlines) > 0 {
		v.writeContentStart('i')
		v.walkInlineSlice(in.Inlines)
	}
}

var mapFormatKind = map[ast.FormatKind]string{
	ast.FormatItalic:    "Italic",
	ast.FormatEmph:      "Emph",
	ast.FormatBold:      "Bold",
	ast.FormatStrong:    "Strong",
	ast.FormatMonospace: "Mono",
	ast.FormatStrike:    "Strikethrough",
	ast.FormatDelete:    "Delete",
	ast.FormatUnder:     "Underline",
	ast.FormatInsert:    "Insert",
	ast.FormatSuper:     "Super",
	ast.FormatSub:       "Sub",
	ast.FormatQuote:     "Quote",
	ast.FormatQuotation: "Quotation",
	ast.FormatSmall:     "Small",
	ast.FormatSpan:      "Span",
}

var mapLiteralKind = map[ast.LiteralKind]string{
	ast.LiteralProg:    "Code",
	ast.LiteralKeyb:    "Input",
	ast.LiteralOutput:  "Output",
	ast.LiteralComment: "Comment",
	ast.LiteralHTML:    "HTML",
}

func (v *detailVisitor) walkBlockSlice(bns ast.BlockSlice) {
	v.b.WriteByte('[')
	for i, bn := range bns {
		v.writeComma(i)
		ast.Walk(v, bn)
	}
	v.b.WriteByte(']')
}

func (v *detailVisitor) walkInlineSlice(ins ast.InlineSlice) {
	v.b.WriteByte('[')
	for i, in := range ins {
		v.writeComma(i)
		ast.Walk(v, in)
	}
	v.b.WriteByte(']')
}

// visitAttributes write JSON attributes
func (v *detailVisitor) visitAttributes(a *ast.Attributes) {
	if a == nil || len(a.Attrs) == 0 {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)
	}
	sort.Strings(keys)

	v.b.WriteString(",\"a\":{\"")
	for i, k := range keys {
		if i > 0 {
			v.b.WriteString("\",\"")
		}
		v.b.Write(Escape(k))
		v.b.WriteString("\":\"")
		v.b.Write(Escape(a.Attrs[k]))
	}
	v.b.WriteString("\"}")
}

func (v *detailVisitor) writeNodeStart(t string) {
	v.b.WriteStrings("{\"t\":\"", t, "\"")
}

var contentCode = map[rune][]byte{
	'b': []byte(",\"b\":"),   // List of blocks
	'c': []byte(",\"c\":["),  // List of list of blocks
	'g': []byte(",\"g\":["),  // General list
	'i': []byte(",\"i\":"),   // List of inlines
	'j': []byte(",\"j\":{"),  // Embedded JSON object
	'l': []byte(",\"l\":["),  // List of lines
	'n': []byte(",\"n\":"),   // Number
	'o': []byte(",\"o\":\""), // Byte object
	'p': []byte(",\"p\":["),  // Generic tuple
	'q': []byte(",\"q\":"),   // String, if 's' is also needed
	's': []byte(",\"s\":"),   // String
	't': []byte("Content code 't' is not allowed"),
	'y': []byte("Content code 'y' is not allowed"), // field after 'j'
}

func (v *detailVisitor) writeContentStart(code rune) {
	if b, ok := contentCode[code]; ok {
		v.b.Write(b)
		return
	}
	panic("Unknown content code " + strconv.Itoa(int(code)))
}

func (v *detailVisitor) writeMeta(m *meta.Meta) {
	for _, p := range m.Pairs(true) {
		if p.Key == meta.KeyTitle {
			continue
		}
		v.b.WriteString(",\"")
		v.b.Write(Escape(p.Key))
		v.b.WriteString("\":")
		if m.Type(p.Key).IsSet {
			v.writeSetValue(p.Value)
		} else {
			v.b.WriteByte('"')
			v.b.Write(Escape(p.Value))
			v.b.WriteByte('"')
		}
	}
}

func (v *detailVisitor) writeSetValue(value string) {
	v.b.WriteByte('[')
	for i, val := range meta.ListFromValue(value) {
		v.writeComma(i)
		v.b.WriteByte('"')
		v.b.Write(Escape(val))
		v.b.WriteByte('"')
	}
	v.b.WriteByte(']')
}

func (v *detailVisitor) writeComma(pos int) {
	if pos > 0 {
		v.b.WriteByte(',')
	}
}

Deleted encoder/jsonenc/jsonenc.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
















































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package jsonenc encodes the abstract syntax tree into some JSON formats.
package jsonenc

import (
	"bytes"
	"io"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
)

func init() {
	encoder.Register(api.EncoderJSON, encoder.Info{
		Create:  func(*encoder.Environment) encoder.Encoder { return &jsonEncoder{} },
		Default: true,
	})
}

// jsonEncoder is just a stub. It is not implemented. The real implementation
// is in file web/adapter/json.go
type jsonEncoder struct{}

// WriteZettel writes the encoded zettel to the writer.
func (je *jsonEncoder) WriteZettel(
	w io.Writer, zn *ast.ZettelNode, inhMeta bool) (int, error) {
	return 0, encoder.ErrNoWriteZettel
}

// WriteMeta encodes meta data as HTML5.
func (je *jsonEncoder) WriteMeta(w io.Writer, meta *meta.Meta) (int, error) {
	return 0, encoder.ErrNoWriteMeta
}

func (je *jsonEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return 0, encoder.ErrNoWriteContent
}

// WriteBlocks writes a block slice to the writer
func (je *jsonEncoder) WriteBlocks(w io.Writer, bs ast.BlockSlice) (int, error) {
	return 0, encoder.ErrNoWriteBlocks
}

// WriteInlines writes an inline slice to the writer
func (je *jsonEncoder) WriteInlines(w io.Writer, is ast.InlineSlice) (int, error) {
	return 0, encoder.ErrNoWriteInlines
}

var (
	jsBackslash   = []byte{'\\', '\\'}
	jsDoubleQuote = []byte{'\\', '"'}
	jsNewline     = []byte{'\\', 'n'}
	jsTab         = []byte{'\\', 't'}
	jsCr          = []byte{'\\', 'r'}
	jsUnicode     = []byte{'\\', 'u', '0', '0', '0', '0'}
	jsHex         = []byte("0123456789ABCDEF")
)

// Escape returns the given string as a byte slice, where every non-printable
// rune is made printable.
func Escape(s string) []byte {
	var buf bytes.Buffer

	last := 0
	for i, ch := range s {
		var b []byte
		switch ch {
		case '\t':
			b = jsTab
		case '\r':
			b = jsCr
		case '\n':
			b = jsNewline
		case '"':
			b = jsDoubleQuote
		case '\\':
			b = jsBackslash
		default:
			if ch < ' ' {
				b = jsUnicode
				b[2] = '0'
				b[3] = '0'
				b[4] = jsHex[ch>>4]
				b[5] = jsHex[ch&0xF]
			} else {
				continue
			}
		}
		buf.WriteString(s[last:i])
		buf.Write(b)
		last = i + 1
	}
	buf.WriteString(s[last:])
	return buf.Bytes()
}

func writeEscaped(b *encoder.BufWriter, s string) {
	b.WriteByte('"')
	b.Write(Escape(s))
	b.WriteByte('"')
}

Changes to encoder/nativeenc/nativeenc.go.

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46

47
48
49
50
51

52
53
54
55
56
57

58
59

60
61
62
63
64
65
66
67
68
69

70
71

72
73
74
75
76
77

78
79

80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96




97
98
99

100
101
102
103
104
105
106
107

108
109
110
111
112
113
114
115
116
117
17
18
19
20
21
22
23


24
25
26
27
28
29
30
31
32
33
34
35
36

37

38





39



40

41
42
43
44
45
46

47
48

49
50
51
52
53
54
55
56
57
58

59
60

61
62
63
64
65
66

67
68

69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

93
94
95
96
97
98
99


100



101
102
103
104
105
106
107







-
-













-
+
-

-
-
-
-
-
+
-
-
-

-
+





-
+

-
+









-
+

-
+





-
+

-
+

















+
+
+
+


-
+






-
-
+
-
-
-







	"sort"
	"strconv"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/encfun"
	"zettelstore.de/z/parser"
)

func init() {
	encoder.Register(api.EncoderNative, encoder.Info{
		Create: func(env *encoder.Environment) encoder.Encoder { return &nativeEncoder{env: env} },
	})
}

type nativeEncoder struct {
	env *encoder.Environment
}

// WriteZettel encodes the zettel to the writer.
func (ne *nativeEncoder) WriteZettel(
func (ne *nativeEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	w io.Writer, zn *ast.ZettelNode, inhMeta bool) (int, error) {
	v := newVisitor(w, ne)
	v.b.WriteString("[Title ")
	v.walkInlineSlice(encfun.MetaAsInlineSlice(zn.InhMeta, meta.KeyTitle))
	v.b.WriteByte(']')
	if inhMeta {
		v.acceptMeta(zn.InhMeta, false)
	v.acceptMeta(zn.InhMeta, evalMeta)
	} else {
		v.acceptMeta(zn.Meta, false)
	}
	v.b.WriteByte('\n')
	v.walkBlockSlice(zn.Ast)
	ast.Walk(v, zn.Ast)
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data in native format.
func (ne *nativeEncoder) WriteMeta(w io.Writer, m *meta.Meta) (int, error) {
func (ne *nativeEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(w, ne)
	v.acceptMeta(m, true)
	v.acceptMeta(m, evalMeta)
	length, err := v.b.Flush()
	return length, err
}

func (ne *nativeEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return ne.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes a block slice to the writer
func (ne *nativeEncoder) WriteBlocks(w io.Writer, bs ast.BlockSlice) (int, error) {
func (ne *nativeEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(w, ne)
	v.walkBlockSlice(bs)
	ast.Walk(v, bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (ne *nativeEncoder) WriteInlines(w io.Writer, is ast.InlineSlice) (int, error) {
func (ne *nativeEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(w, ne)
	v.walkInlineSlice(is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b     encoder.BufWriter
	level int
	env   *encoder.Environment
}

func newVisitor(w io.Writer, enc *nativeEncoder) *visitor {
	return &visitor{b: encoder.NewBufWriter(w), env: enc.env}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockListNode:
		v.visitBlockList(n)
	case *ast.InlineListNode:
		v.walkInlineList(n)
	case *ast.ParaNode:
		v.b.WriteString("[Para ")
		v.walkInlineSlice(n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
		v.visitRegion(n)
	case *ast.HeadingNode:
		v.b.WriteStrings("[Heading ", strconv.Itoa(n.Level), " \"", n.Slug, "\"")
		v.visitAttributes(n.Attrs)
		v.visitHeading(n)
		v.b.WriteByte(' ')
		v.walkInlineSlice(n.Inlines)
		v.b.WriteByte(']')
	case *ast.HRuleNode:
		v.b.WriteString("[Hrule")
		v.visitAttributes(n.Attrs)
		v.b.WriteByte(']')
	case *ast.NestedListNode:
		v.visitNestedList(n)
	case *ast.DescriptionListNode:
144
145
146
147
148
149
150
151
152


153
154
155
156
157
158
159

160
161

162
163
164
165
166
167
168

169
170
171

172
173
174
175
176
177
178
179
180
181

182
183
184
185
186
187
188
134
135
136
137
138
139
140


141
142
143
144
145
146
147
148

149
150

151
152
153
154
155
156
157

158
159
160

161





162
163
164
165

166
167
168
169
170
171
172
173







-
-
+
+






-
+

-
+






-
+


-
+
-
-
-
-
-




-
+







		if n.Hard {
			v.b.WriteString("Break")
		} else {
			v.b.WriteString("Space")
		}
	case *ast.LinkNode:
		v.visitLink(n)
	case *ast.ImageNode:
		v.visitImage(n)
	case *ast.EmbedNode:
		v.visitEmbed(n)
	case *ast.CiteNode:
		v.b.WriteString("Cite")
		v.visitAttributes(n.Attrs)
		v.b.WriteString(" \"")
		v.writeEscaped(n.Key)
		v.b.WriteByte('"')
		if len(n.Inlines) > 0 {
		if n.Inlines != nil {
			v.b.WriteString(" [")
			v.walkInlineSlice(n.Inlines)
			ast.Walk(v, n.Inlines)
			v.b.WriteByte(']')
		}
	case *ast.FootnoteNode:
		v.b.WriteString("Footnote")
		v.visitAttributes(n.Attrs)
		v.b.WriteString(" [")
		v.walkInlineSlice(n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
	case *ast.MarkNode:
		v.b.WriteString("Mark")
		v.visitMark(n)
		if len(n.Text) > 0 {
			v.b.WriteString(" \"")
			v.writeEscaped(n.Text)
			v.b.WriteByte('"')
		}
	case *ast.FormatNode:
		v.b.Write(mapFormatKind[n.Kind])
		v.visitAttributes(n.Attrs)
		v.b.WriteString(" [")
		v.walkInlineSlice(n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
	case *ast.LiteralNode:
		kind, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown literal kind %v", n.Kind))
		}
		v.b.Write(kind)
198
199
200
201
202
203
204
205

206
207
208

209
210
211
212
213
214
215
216
217
218
219
220
221
222




223
224
225
226





227
228
229
230








231
232
233
234
235
236
237
183
184
185
186
187
188
189

190



191


192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207




208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231







-
+
-
-
-
+
-
-












+
+
+
+
-
-
-
-
+
+
+
+
+




+
+
+
+
+
+
+
+








var (
	rawBackslash   = []byte{'\\', '\\'}
	rawDoubleQuote = []byte{'\\', '"'}
	rawNewline     = []byte{'\\', 'n'}
)

func (v *visitor) acceptMeta(m *meta.Meta, withTitle bool) {
func (v *visitor) acceptMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) {
	if withTitle {
		v.b.WriteString("[Title ")
		v.walkInlineSlice(parser.ParseMetadata(m.GetDefault(meta.KeyTitle, "")))
	v.writeZettelmarkup("Title", m.GetDefault(meta.KeyTitle, ""), evalMeta)
		v.b.WriteByte(']')
	}
	v.writeMetaString(m, meta.KeyRole, "Role")
	v.writeMetaList(m, meta.KeyTags, "Tags")
	v.writeMetaString(m, meta.KeySyntax, "Syntax")
	pairs := m.PairsRest(true)
	if len(pairs) == 0 {
		return
	}
	v.b.WriteString("\n[Header")
	v.level++
	for i, p := range pairs {
		v.writeComma(i)
		v.writeNewLine()
		key, value := p.Key, p.Value
		if meta.Type(key) == meta.TypeZettelmarkup {
			v.writeZettelmarkup(key, value, evalMeta)
		} else {
		v.b.WriteByte('[')
		v.b.WriteStrings(p.Key, " \"")
		v.writeEscaped(p.Value)
		v.b.WriteString("\"]")
			v.b.WriteByte('[')
			v.b.WriteStrings(key, " \"")
			v.writeEscaped(value)
			v.b.WriteString("\"]")
		}
	}
	v.level--
	v.b.WriteByte(']')
}

func (v *visitor) writeZettelmarkup(key, value string, evalMeta encoder.EvalMetaFunc) {
	v.b.WriteByte('[')
	v.b.WriteString(key)
	v.b.WriteByte(' ')
	ast.Walk(v, evalMeta(value))
	v.b.WriteByte(']')
}

func (v *visitor) writeMetaString(m *meta.Meta, key, native string) {
	if val, ok := m.Get(key); ok && len(val) > 0 {
		v.b.WriteStrings("\n[", native, " \"", val, "\"]")
	}
}

282
283
284
285
286
287
288
289

290
291
292

293
294
295
296

297
298
299
300
301











302
303
304
305
306
307
308
276
277
278
279
280
281
282

283
284
285

286
287
288
289

290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313







-
+


-
+



-
+





+
+
+
+
+
+
+
+
+
+
+







	}
	v.b.Write(kind)
	v.visitAttributes(rn.Attrs)
	v.level++
	v.writeNewLine()
	v.b.WriteByte('[')
	v.level++
	v.walkBlockSlice(rn.Blocks)
	ast.Walk(v, rn.Blocks)
	v.level--
	v.b.WriteByte(']')
	if len(rn.Inlines) > 0 {
	if rn.Inlines != nil {
		v.b.WriteByte(',')
		v.writeNewLine()
		v.b.WriteString("[Cite ")
		v.walkInlineSlice(rn.Inlines)
		ast.Walk(v, rn.Inlines)
		v.b.WriteByte(']')
	}
	v.level--
	v.b.WriteByte(']')
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	v.b.WriteStrings("[Heading ", strconv.Itoa(hn.Level))
	if fragment := hn.Fragment; fragment != "" {
		v.b.WriteStrings(" #", fragment)
	}
	v.visitAttributes(hn.Attrs)
	v.b.WriteByte(' ')
	ast.Walk(v, hn.Inlines)
	v.b.WriteByte(']')
}

var mapNestedListKind = map[ast.NestedListKind][]byte{
	ast.NestedListOrdered:   []byte("[OrderedList"),
	ast.NestedListUnordered: []byte("[BulletList"),
	ast.NestedListQuote:     []byte("[QuoteList"),
}

331
332
333
334
335
336
337
338

339
340
341
342
343
344
345
336
337
338
339
340
341
342

343
344
345
346
347
348
349
350







-
+







func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	v.b.WriteString("[DescriptionList")
	v.level++
	for i, descr := range dn.Descriptions {
		v.writeComma(i)
		v.writeNewLine()
		v.b.WriteString("[Term [")
		v.walkInlineSlice(descr.Term)
		ast.Walk(v, descr.Term)
		v.b.WriteByte(']')

		if len(descr.Descriptions) > 0 {
			v.level++
			for _, b := range descr.Descriptions {
				v.b.WriteByte(',')
				v.writeNewLine()
395
396
397
398
399
400
401
402

403
404

405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434

435
436
437
438
439
440
441
442
443
444
445




















446
447
448
449
450
451
452
453
454
455
456













457
458




459
460
461


462
463

464
465
466
467
468
469
470
400
401
402
403
404
405
406

407
408

409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425





426
427
428
429
430
431
432
433

434
435
436
437
438







439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458











459
460
461
462
463
464
465
466
467
468
469
470
471


472
473
474
475
476


477
478


479
480
481
482
483
484
485
486







-
+

-
+
















-
-
-
-
-








-
+




-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+
+
+

-
-
+
+
-
-
+







	ast.AlignLeft:    " Left",
	ast.AlignCenter:  " Center",
	ast.AlignRight:   " Right",
}

func (v *visitor) writeCell(cell *ast.TableCell) {
	v.b.WriteStrings("[Cell", alignString[cell.Align])
	if len(cell.Inlines) > 0 {
	if !cell.Inlines.IsEmpty() {
		v.b.WriteByte(' ')
		v.walkInlineSlice(cell.Inlines)
		ast.Walk(v, cell.Inlines)
	}
	v.b.WriteByte(']')
}

var mapRefState = map[ast.RefState]string{
	ast.RefStateInvalid:  "INVALID",
	ast.RefStateZettel:   "ZETTEL",
	ast.RefStateSelf:     "SELF",
	ast.RefStateFound:    "ZETTEL",
	ast.RefStateBroken:   "BROKEN",
	ast.RefStateHosted:   "LOCAL",
	ast.RefStateBased:    "BASED",
	ast.RefStateExternal: "EXTERNAL",
}

func (v *visitor) visitLink(ln *ast.LinkNode) {
	ln, n := v.env.AdaptLink(ln)
	if n != nil {
		ast.Walk(v, n)
		return
	}
	v.b.WriteString("Link")
	v.visitAttributes(ln.Attrs)
	v.b.WriteByte(' ')
	v.b.WriteString(mapRefState[ln.Ref.State])
	v.b.WriteString(" \"")
	v.writeEscaped(ln.Ref.String())
	v.b.WriteString("\" [")
	if !ln.OnlyRef {
		v.walkInlineSlice(ln.Inlines)
		ast.Walk(v, ln.Inlines)
	}
	v.b.WriteByte(']')
}

func (v *visitor) visitImage(in *ast.ImageNode) {
	in, n := v.env.AdaptImage(in)
	if n != nil {
		ast.Walk(v, n)
		return
	}
	v.b.WriteString("Image")
func (v *visitor) visitEmbed(en *ast.EmbedNode) {
	v.b.WriteString("Embed")
	v.visitAttributes(en.Attrs)
	switch m := en.Material.(type) {
	case *ast.ReferenceMaterialNode:
		v.b.WriteByte(' ')
		v.b.WriteString(mapRefState[m.Ref.State])
		v.b.WriteString(" \"")
		v.writeEscaped(m.Ref.String())
		v.b.WriteByte('"')
	case *ast.BLOBMaterialNode:
		v.b.WriteStrings(" {\"", m.Syntax, "\" \"")
		switch m.Syntax {
		case "svg":
			v.writeEscaped(string(m.Blob))
		default:
			v.b.WriteString("\" \"")
			v.b.WriteBase64(m.Blob)
		}
		v.b.WriteString("\"}")
	v.visitAttributes(in.Attrs)
	if in.Ref == nil {
		v.b.WriteStrings(" {\"", in.Syntax, "\" \"")
		switch in.Syntax {
		case "svg":
			v.writeEscaped(string(in.Blob))
		default:
			v.b.WriteString("\" \"")
			v.b.WriteBase64(in.Blob)
		}
		v.b.WriteString("\"}")
	default:
		panic(fmt.Sprintf("Unknown material type %t for %v", en.Material, en.Material))
	}

	if en.Inlines != nil {
		v.b.WriteString(" [")
		ast.Walk(v, en.Inlines)
		v.b.WriteByte(']')
	}
}

func (v *visitor) visitMark(mn *ast.MarkNode) {
	v.b.WriteString("Mark")
	} else {
		v.b.WriteStrings(" \"", in.Ref.String(), "\"")
	if text := mn.Text; text != "" {
		v.b.WriteString(" \"")
		v.writeEscaped(text)
		v.b.WriteByte('"')
	}
	if len(in.Inlines) > 0 {
		v.b.WriteString(" [")
	if fragment := mn.Fragment; fragment != "" {
		v.b.WriteString(" #")
		v.walkInlineSlice(in.Inlines)
		v.b.WriteByte(']')
		v.writeEscaped(fragment)
	}
}

var mapFormatKind = map[ast.FormatKind][]byte{
	ast.FormatItalic:    []byte("Italic"),
	ast.FormatEmph:      []byte("Emph"),
	ast.FormatBold:      []byte("Bold"),
486
487
488
489
490
491
492
493
494


495
496
497
498
499
500
501
502
503


504
505
506
507
508
509
510
511

512
513
514
515
516
517
518
502
503
504
505
506
507
508


509
510
511
512
513
514
515
516
517


518
519
520
521
522
523
524
525
526

527
528
529
530
531
532
533
534







-
-
+
+







-
-
+
+







-
+







	ast.LiteralProg:    []byte("Code"),
	ast.LiteralKeyb:    []byte("Input"),
	ast.LiteralOutput:  []byte("Output"),
	ast.LiteralComment: []byte("Comment"),
	ast.LiteralHTML:    []byte("HTML"),
}

func (v *visitor) walkBlockSlice(bns ast.BlockSlice) {
	for i, bn := range bns {
func (v *visitor) visitBlockList(bln *ast.BlockListNode) {
	for i, bn := range bln.List {
		if i > 0 {
			v.b.WriteByte(',')
			v.writeNewLine()
		}
		ast.Walk(v, bn)
	}
}
func (v *visitor) walkInlineSlice(ins ast.InlineSlice) {
	for i, in := range ins {
func (v *visitor) walkInlineList(iln *ast.InlineListNode) {
	for i, in := range iln.List {
		v.writeComma(i)
		ast.Walk(v, in)
	}
}

// visitAttributes write native attributes
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a == nil || len(a.Attrs) == 0 {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)
	}
	sort.Strings(keys)

Deleted encoder/rawenc/rawenc.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69





































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package rawenc encodes the abstract syntax tree as raw content.
package rawenc

import (
	"io"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
)

func init() {
	encoder.Register(api.EncoderRaw, encoder.Info{
		Create: func(*encoder.Environment) encoder.Encoder { return &rawEncoder{} },
	})
}

type rawEncoder struct{}

// WriteZettel writes the encoded zettel to the writer.
func (re *rawEncoder) WriteZettel(
	w io.Writer, zn *ast.ZettelNode, inhMeta bool) (int, error) {
	b := encoder.NewBufWriter(w)
	if inhMeta {
		zn.InhMeta.Write(&b, true)
	} else {
		zn.Meta.Write(&b, true)
	}
	b.WriteByte('\n')
	b.WriteString(zn.Content.AsString())
	length, err := b.Flush()
	return length, err
}

// WriteMeta encodes meta data as HTML5.
func (re *rawEncoder) WriteMeta(w io.Writer, m *meta.Meta) (int, error) {
	b := encoder.NewBufWriter(w)
	m.Write(&b, true)
	length, err := b.Flush()
	return length, err
}

func (re *rawEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	b := encoder.NewBufWriter(w)
	b.WriteString(zn.Content.AsString())
	length, err := b.Flush()
	return length, err
}

// WriteBlocks writes a block slice to the writer
func (re *rawEncoder) WriteBlocks(w io.Writer, bs ast.BlockSlice) (int, error) {
	return 0, encoder.ErrNoWriteBlocks
}

// WriteInlines writes an inline slice to the writer
func (re *rawEncoder) WriteInlines(w io.Writer, is ast.InlineSlice) (int, error) {
	return 0, encoder.ErrNoWriteInlines
}

Changes to encoder/textenc/textenc.go.

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

34
35
36

37
38
39
40

41
42
43
44
45
46
47


48
49
50















51
52
53
54
55
56
57
58
59
60
61
62














63
64
65
66
67

68
69
70
71
72
73
74
75
76
77
78
79

80
81

82
83
84
85
86
87

88
89

90
91
92
93
94
95
96
97
98
99
100
101
102
103
104


105
106
107


108
109
110
111
112
113
114
115
116


117
118

119
120
121
122
123
124
125
126

127
128
129
130
131

132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150

151
152
153
154
155
156
157

158
159
160
161
162
163
164
165
166
167
168
169
170
171

172
173
174
175
176
177
178
179
180
181
182
183
184













































185
186
187
188

189
190
191
192
193


194
195
196
197
198
199
200
201
202
203
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28
29
30
31

32
33


34




35
36
37
38
39
40


41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60












61
62
63
64
65
66
67
68
69
70
71
72
73
74





75




76
77
78
79
80
81
82

83
84

85
86
87
88
89
90

91
92

93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111


112
113






114


115
116
117

118
119
120
121





122


123
124

125










126
127







128

129
130
131
132
133

134
135
136
137
138
139
140
141
142
143
144
145
146
147

148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209

210
211
212
213


214
215
216
217
218
219
220
221
222
223
224
225







-











-
+

-
-
+
-
-
-
-
+





-
-
+
+



+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
-
+
-
-
-
-







-
+

-
+





-
+

-
+















+
+

-
-
+
+
-
-
-
-
-
-

-
-
+
+

-
+



-
-
-
-
-
+
-
-


-
+
-
-
-
-
-
-
-
-
-
-


-
-
-
-
-
-
-
+
-





-
+













-
+













+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+



-
+



-
-
+
+










import (
	"io"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
)

func init() {
	encoder.Register(api.EncoderText, encoder.Info{
		Create: func(*encoder.Environment) encoder.Encoder { return &textEncoder{} },
	})
}

type textEncoder struct{}

// WriteZettel writes metadata and content.
func (te *textEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, inhMeta bool) (int, error) {
func (te *textEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(w)
	if inhMeta {
		te.WriteMeta(&v.b, zn.InhMeta)
	te.WriteMeta(&v.b, zn.InhMeta, evalMeta)
	} else {
		te.WriteMeta(&v.b, zn.Meta)
	}
	v.acceptBlockSlice(zn.Ast)
	v.visitBlockList(zn.Ast)
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes metadata as text.
func (te *textEncoder) WriteMeta(w io.Writer, m *meta.Meta) (int, error) {
	b := encoder.NewBufWriter(w)
func (te *textEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	buf := encoder.NewBufWriter(w)
	for _, pair := range m.Pairs(true) {
		switch meta.Type(pair.Key) {
		case meta.TypeBool:
			writeBool(&buf, pair.Value)
		case meta.TypeTagSet:
			writeTagSet(&buf, meta.ListFromValue(pair.Value))
		case meta.TypeZettelmarkup:
			te.WriteInlines(&buf, evalMeta(pair.Value))
		default:
			buf.WriteString(pair.Value)
		}
		buf.WriteByte('\n')
	}
	length, err := buf.Flush()
	return length, err
}

func writeBool(buf *encoder.BufWriter, val string) {
			if meta.BoolValue(pair.Value) {
				b.WriteString("true")
			} else {
				b.WriteString("false")
			}
		case meta.TypeTagSet:
			for i, tag := range meta.ListFromValue(pair.Value) {
				if i > 0 {
					b.WriteByte(' ')
				}
				b.WriteString(meta.CleanTag(tag))
			}
	if meta.BoolValue(val) {
		buf.WriteString("true")
	} else {
		buf.WriteString("false")
	}
}

func writeTagSet(buf *encoder.BufWriter, tags []string) {
	for i, tag := range tags {
		if i > 0 {
			buf.WriteByte(' ')
		}
		buf.WriteString(meta.CleanTag(tag))
	}
		case meta.TypeZettelmarkup:
			te.WriteInlines(w, parser.ParseMetadata(pair.Value))
		default:
			b.WriteString(pair.Value)
		}

		b.WriteByte('\n')
	}
	length, err := b.Flush()
	return length, err
}

func (te *textEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return te.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes the content of a block slice to the writer.
func (te *textEncoder) WriteBlocks(w io.Writer, bs ast.BlockSlice) (int, error) {
func (*textEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(w)
	v.acceptBlockSlice(bs)
	v.visitBlockList(bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (te *textEncoder) WriteInlines(w io.Writer, is ast.InlineSlice) (int, error) {
func (*textEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(w)
	ast.WalkInlineSlice(v, is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b encoder.BufWriter
}

func newVisitor(w io.Writer) *visitor {
	return &visitor{b: encoder.NewBufWriter(w)}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockListNode:
		v.visitBlockList(n)
	case *ast.VerbatimNode:
		if n.Kind == ast.VerbatimComment {
			return nil
		v.visitVerbatim(n)
		return nil
		}
		for i, line := range n.Lines {
			v.writePosChar(i, '\n')
			v.b.WriteString(line)
		}
		return nil
	case *ast.RegionNode:
		v.acceptBlockSlice(n.Blocks)
		if len(n.Inlines) > 0 {
		v.visitBlockList(n.Blocks)
		if n.Inlines != nil {
			v.b.WriteByte('\n')
			ast.WalkInlineSlice(v, n.Inlines)
			ast.Walk(v, n.Inlines)
		}
		return nil
	case *ast.NestedListNode:
		for i, item := range n.Items {
			v.writePosChar(i, '\n')
			for j, it := range item {
				v.writePosChar(j, '\n')
				ast.Walk(v, it)
		v.visitNestedList(n)
			}
		}
		return nil
	case *ast.DescriptionListNode:
		for i, descr := range n.Descriptions {
		v.visitDescriptionList(n)
			v.writePosChar(i, '\n')
			ast.WalkInlineSlice(v, descr.Term)
			for _, b := range descr.Descriptions {
				v.b.WriteByte('\n')
				for k, d := range b {
					v.writePosChar(k, '\n')
					ast.Walk(v, d)
				}
			}
		}
		return nil
	case *ast.TableNode:
		if len(n.Header) > 0 {
			v.writeRow(n.Header)
			v.b.WriteByte('\n')
		}
		for i, row := range n.Rows {
			v.writePosChar(i, '\n')
			v.writeRow(row)
		v.visitTable(n)
		}
		return nil
	case *ast.TextNode:
		v.b.WriteString(n.Text)
		return nil
	case *ast.TagNode:
		v.b.WriteStrings("#", n.Tag)
		v.b.WriteString(n.Tag)
		return nil
	case *ast.SpaceNode:
		v.b.WriteByte(' ')
		return nil
	case *ast.BreakNode:
		if n.Hard {
			v.b.WriteByte('\n')
		} else {
			v.b.WriteByte(' ')
		}
		return nil
	case *ast.LinkNode:
		if !n.OnlyRef {
			ast.WalkInlineSlice(v, n.Inlines)
			ast.Walk(v, n.Inlines)
		}
		return nil
	case *ast.FootnoteNode:
		v.b.WriteByte(' ')
		return v // No 'return nil' to write text
	case *ast.LiteralNode:
		if n.Kind != ast.LiteralComment {
			v.b.WriteString(n.Text)
		}
	}
	return v
}

func (v *visitor) visitVerbatim(vn *ast.VerbatimNode) {
	if vn.Kind == ast.VerbatimComment {
		return
	}
	for i, line := range vn.Lines {
		v.writePosChar(i, '\n')
		v.b.WriteString(line)
	}
}

func (v *visitor) visitNestedList(ln *ast.NestedListNode) {
	for i, item := range ln.Items {
		v.writePosChar(i, '\n')
		for j, it := range item {
			v.writePosChar(j, '\n')
			ast.Walk(v, it)
		}
	}
}

func (v *visitor) visitDescriptionList(dl *ast.DescriptionListNode) {
	for i, descr := range dl.Descriptions {
		v.writePosChar(i, '\n')
		ast.Walk(v, descr.Term)
		for _, b := range descr.Descriptions {
			v.b.WriteByte('\n')
			for k, d := range b {
				v.writePosChar(k, '\n')
				ast.Walk(v, d)
			}
		}
	}
}

func (v *visitor) visitTable(tn *ast.TableNode) {
	if len(tn.Header) > 0 {
		v.writeRow(tn.Header)
		v.b.WriteByte('\n')
	}
	for i, row := range tn.Rows {
		v.writePosChar(i, '\n')
		v.writeRow(row)
	}
}

func (v *visitor) writeRow(row ast.TableRow) {
	for i, cell := range row {
		v.writePosChar(i, ' ')
		ast.WalkInlineSlice(v, cell.Inlines)
		ast.Walk(v, cell.Inlines)
	}
}

func (v *visitor) acceptBlockSlice(bns ast.BlockSlice) {
	for i, bn := range bns {
func (v *visitor) visitBlockList(bns *ast.BlockListNode) {
	for i, bn := range bns.List {
		v.writePosChar(i, '\n')
		ast.Walk(v, bn)
	}
}

func (v *visitor) writePosChar(pos int, ch byte) {
	if pos > 0 {
		v.b.WriteByte(ch)
	}
}

Changes to encoder/zmkenc/zmkenc.go.

27
28
29
30
31
32
33
34

35
36
37
38



39
40

41
42

43
44
45
46
47
48
49


















50
51
52
53
54
55
56
57

58
59

60
61
62
63
64
65

66
67

68
69
70
71
72
73
74
27
28
29
30
31
32
33

34

35


36
37
38
39

40
41

42
43
44
45
46
47


48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

73
74

75
76
77
78
79
80

81
82

83
84
85
86
87
88
89
90







-
+
-

-
-
+
+
+

-
+

-
+





-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







-
+

-
+





-
+

-
+







		Create: func(*encoder.Environment) encoder.Encoder { return &zmkEncoder{} },
	})
}

type zmkEncoder struct{}

// WriteZettel writes the encoded zettel to the writer.
func (ze *zmkEncoder) WriteZettel(
func (ze *zmkEncoder) WriteZettel(w io.Writer, zn *ast.ZettelNode, evalMeta encoder.EvalMetaFunc) (int, error) {
	w io.Writer, zn *ast.ZettelNode, inhMeta bool) (int, error) {
	v := newVisitor(w, ze)
	if inhMeta {
		zn.InhMeta.WriteAsHeader(&v.b, true)
	v.acceptMeta(zn.InhMeta, evalMeta)
	if zn.InhMeta.YamlSep {
		v.b.WriteString("---\n")
	} else {
		zn.Meta.WriteAsHeader(&v.b, true)
		v.b.WriteByte('\n')
	}
	ast.WalkBlockSlice(v, zn.Ast)
	ast.Walk(v, zn.Ast)
	length, err := v.b.Flush()
	return length, err
}

// WriteMeta encodes meta data as zmk.
func (ze *zmkEncoder) WriteMeta(w io.Writer, m *meta.Meta) (int, error) {
	return m.Write(w, true)
func (ze *zmkEncoder) WriteMeta(w io.Writer, m *meta.Meta, evalMeta encoder.EvalMetaFunc) (int, error) {
	v := newVisitor(w, ze)
	v.acceptMeta(m, evalMeta)
	length, err := v.b.Flush()
	return length, err
}

func (v *visitor) acceptMeta(m *meta.Meta, evalMeta encoder.EvalMetaFunc) {
	for _, p := range m.Pairs(true) {
		key := p.Key
		v.b.WriteStrings(key, ": ")
		if meta.Type(key) == meta.TypeZettelmarkup {
			ast.Walk(v, evalMeta(p.Value))
		} else {
			v.b.WriteString(p.Value)
		}
		v.b.WriteByte('\n')
	}
}

func (ze *zmkEncoder) WriteContent(w io.Writer, zn *ast.ZettelNode) (int, error) {
	return ze.WriteBlocks(w, zn.Ast)
}

// WriteBlocks writes the content of a block slice to the writer.
func (ze *zmkEncoder) WriteBlocks(w io.Writer, bs ast.BlockSlice) (int, error) {
func (ze *zmkEncoder) WriteBlocks(w io.Writer, bln *ast.BlockListNode) (int, error) {
	v := newVisitor(w, ze)
	ast.WalkBlockSlice(v, bs)
	ast.Walk(v, bln)
	length, err := v.b.Flush()
	return length, err
}

// WriteInlines writes an inline slice to the writer
func (ze *zmkEncoder) WriteInlines(w io.Writer, is ast.InlineSlice) (int, error) {
func (ze *zmkEncoder) WriteInlines(w io.Writer, iln *ast.InlineListNode) (int, error) {
	v := newVisitor(w, ze)
	ast.WalkInlineSlice(v, is)
	ast.Walk(v, iln)
	length, err := v.b.Flush()
	return length, err
}

// visitor writes the abstract syntax tree to an io.Writer.
type visitor struct {
	b      encoder.BufWriter
82
83
84
85
86
87
88
89

90
91
92
93
94
95
96
98
99
100
101
102
103
104

105
106
107
108
109
110
111
112







-
+







		enc: enc,
	}
}

func (v *visitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.ParaNode:
		ast.WalkInlineSlice(v, n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte('\n')
		if len(v.prefix) == 0 {
			v.b.WriteByte('\n')
		}
	case *ast.VerbatimNode:
		v.visitVerbatim(n)
	case *ast.RegionNode:
117
118
119
120
121
122
123
124
125


126
127
128
129
130

131
132
133
134
135
136
137
133
134
135
136
137
138
139


140
141
142
143
144
145

146
147
148
149
150
151
152
153







-
-
+
+




-
+







		v.b.WriteStrings("#", n.Tag)
	case *ast.SpaceNode:
		v.b.WriteString(n.Lexeme)
	case *ast.BreakNode:
		v.visitBreak(n)
	case *ast.LinkNode:
		v.visitLink(n)
	case *ast.ImageNode:
		v.visitImage(n)
	case *ast.EmbedNode:
		v.visitEmbed(n)
	case *ast.CiteNode:
		v.visitCite(n)
	case *ast.FootnoteNode:
		v.b.WriteString("[^")
		ast.WalkInlineSlice(v, n.Inlines)
		ast.Walk(v, n.Inlines)
		v.b.WriteByte(']')
		v.visitAttributes(n.Attrs)
	case *ast.MarkNode:
		v.b.WriteStrings("[!", n.Text, "]")
	case *ast.FormatNode:
		v.visitFormat(n)
	case *ast.LiteralNode:
164
165
166
167
168
169
170
171

172
173

174
175

176
177
178
179
180
181
182
183
184
185

186
187
188
189
190
191
192
180
181
182
183
184
185
186

187
188

189
190

191
192
193
194
195
196
197
198
199
200

201
202
203
204
205
206
207
208







-
+

-
+

-
+









-
+







	kind, ok := mapRegionKind[rn.Kind]
	if !ok {
		panic(fmt.Sprintf("Unknown region kind %d", rn.Kind))
	}
	v.b.WriteString(kind)
	v.visitAttributes(rn.Attrs)
	v.b.WriteByte('\n')
	ast.WalkBlockSlice(v, rn.Blocks)
	ast.Walk(v, rn.Blocks)
	v.b.WriteString(kind)
	if len(rn.Inlines) > 0 {
	if rn.Inlines != nil {
		v.b.WriteByte(' ')
		ast.WalkInlineSlice(v, rn.Inlines)
		ast.Walk(v, rn.Inlines)
	}
	v.b.WriteByte('\n')
}

func (v *visitor) visitHeading(hn *ast.HeadingNode) {
	for i := 0; i <= hn.Level; i++ {
		v.b.WriteByte('=')
	}
	v.b.WriteByte(' ')
	ast.WalkInlineSlice(v, hn.Inlines)
	ast.Walk(v, hn.Inlines)
	v.visitAttributes(hn.Attrs)
	v.b.WriteByte('\n')
}

var mapNestedListKind = map[ast.NestedListKind]byte{
	ast.NestedListOrdered:   '#',
	ast.NestedListUnordered: '*',
213
214
215
216
217
218
219
220

221
222
223
224
225
226
227
229
230
231
232
233
234
235

236
237
238
239
240
241
242
243







-
+







	v.prefix = v.prefix[:len(v.prefix)-1]
	v.b.WriteByte('\n')
}

func (v *visitor) visitDescriptionList(dn *ast.DescriptionListNode) {
	for _, descr := range dn.Descriptions {
		v.b.WriteString("; ")
		ast.WalkInlineSlice(v, descr.Term)
		ast.Walk(v, descr.Term)
		v.b.WriteByte('\n')

		for _, b := range descr.Descriptions {
			v.b.WriteString(": ")
			ast.WalkDescriptionSlice(v, b)
			v.b.WriteByte('\n')
		}
239
240
241
242
243
244
245
246

247
248
249
250
251
252
253
254
255
256
257
258
259

260
261
262
263
264
265
266
255
256
257
258
259
260
261

262
263
264
265
266
267
268
269
270
271
272
273
274

275
276
277
278
279
280
281
282







-
+












-
+







	if len(tn.Header) > 0 {
		for pos, cell := range tn.Header {
			v.b.WriteString("|=")
			colAlign := tn.Align[pos]
			if cell.Align != colAlign {
				v.b.WriteString(alignCode[cell.Align])
			}
			ast.WalkInlineSlice(v, cell.Inlines)
			ast.Walk(v, cell.Inlines)
			if colAlign != ast.AlignDefault {
				v.b.WriteString(alignCode[colAlign])
			}
		}
		v.b.WriteByte('\n')
	}
	for _, row := range tn.Rows {
		for pos, cell := range row {
			v.b.WriteByte('|')
			if cell.Align != tn.Align[pos] {
				v.b.WriteString(alignCode[cell.Align])
			}
			ast.WalkInlineSlice(v, cell.Inlines)
			ast.Walk(v, cell.Inlines)
		}
		v.b.WriteByte('\n')
	}
	v.b.WriteByte('\n')
}

var escapeSeqs = map[string]bool{
318
319
320
321
322
323
324
325

326
327
328
329
330
331
332



333
334
335


336
337
338





339
340
341
342
343
344

345
346

347
348
349
350
351
352
353
334
335
336
337
338
339
340

341
342
343
344
345
346


347
348
349
350


351
352
353
354

355
356
357
358
359
360
361
362
363
364

365
366

367
368
369
370
371
372
373
374







-
+





-
-
+
+
+

-
-
+
+


-
+
+
+
+
+





-
+

-
+







		}
	}
}

func (v *visitor) visitLink(ln *ast.LinkNode) {
	v.b.WriteString("[[")
	if !ln.OnlyRef {
		ast.WalkInlineSlice(v, ln.Inlines)
		ast.Walk(v, ln.Inlines)
		v.b.WriteByte('|')
	}
	v.b.WriteStrings(ln.Ref.String(), "]]")
}

func (v *visitor) visitImage(in *ast.ImageNode) {
	if in.Ref != nil {
func (v *visitor) visitEmbed(en *ast.EmbedNode) {
	switch m := en.Material.(type) {
	case *ast.ReferenceMaterialNode:
		v.b.WriteString("{{")
		if len(in.Inlines) > 0 {
			ast.WalkInlineSlice(v, in.Inlines)
		if en.Inlines != nil {
			ast.Walk(v, en.Inlines)
			v.b.WriteByte('|')
		}
		v.b.WriteStrings(in.Ref.String(), "}}")
		v.b.WriteStrings(m.Ref.String(), "}}")
	case *ast.BLOBMaterialNode:
		panic("TODO")
	default:
		panic(fmt.Sprintf("Unknown material type %t for %v", en.Material, en.Material))
	}
}

func (v *visitor) visitCite(cn *ast.CiteNode) {
	v.b.WriteStrings("[@", cn.Key)
	if len(cn.Inlines) > 0 {
	if cn.Inlines != nil {
		v.b.WriteString(", ")
		ast.WalkInlineSlice(v, cn.Inlines)
		ast.Walk(v, cn.Inlines)
	}
	v.b.WriteByte(']')
	v.visitAttributes(cn.Attrs)
}

var mapFormatKind = map[ast.FormatKind][]byte{
	ast.FormatItalic:    []byte("//"),
376
377
378
379
380
381
382
383

384
385
386
387
388
389
390
397
398
399
400
401
402
403

404
405
406
407
408
409
410
411







-
+







	switch fn.Kind {
	case ast.FormatEmph, ast.FormatStrong, ast.FormatInsert, ast.FormatDelete:
		attrs = attrs.Clone()
		attrs.Set("-", "")
	}

	v.b.Write(kind)
	ast.WalkInlineSlice(v, fn.Inlines)
	ast.Walk(v, fn.Inlines)
	v.b.Write(kind)
	v.visitAttributes(attrs)
}

func (v *visitor) visitLiteral(ln *ast.LiteralNode) {
	switch ln.Kind {
	case ast.LiteralProg:
409
410
411
412
413
414
415
416

417
418
419
420
421
422
423
430
431
432
433
434
435
436

437
438
439
440
441
442
443
444







-
+







	v.writeEscaped(text, code)
	v.b.WriteBytes(code, code)
	v.visitAttributes(attrs)
}

// visitAttributes write HTML attributes
func (v *visitor) visitAttributes(a *ast.Attributes) {
	if a == nil || len(a.Attrs) == 0 {
	if a.IsEmpty() {
		return
	}
	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)
	}
	sort.Strings(keys)

Added evaluator/evaluator.go.
































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package evaluator interprets and evaluates the AST.
package evaluator

import (
	"context"
	"errors"
	"fmt"
	"strconv"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/parser/cleaner"
)

// Environment contains values to control the evaluation.
type Environment struct {
	EmbedImage   bool
	GetTagRef    func(string) *ast.Reference
	GetHostedRef func(string) *ast.Reference
	GetFoundRef  func(zid id.Zid, fragment string) *ast.Reference
}

// Port contains all methods to retrieve zettel (or part of it) to evaluate a zettel.
type Port interface {
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)
	GetZettel(context.Context, id.Zid) (domain.Zettel, error)
}

var emptyEnv Environment

// EvaluateZettel evaluates the given zettel in the given context, with the
// given ports, and the given environment.
func EvaluateZettel(ctx context.Context, port Port, env *Environment, rtConfig config.Config, zn *ast.ZettelNode) {
	evaluateNode(ctx, port, env, rtConfig, zn.Ast)
	cleaner.CleanBlockList(zn.Ast)
}

// EvaluateInline evaluates the given inline list in the given context, with
// the given ports, and the given environment.
func EvaluateInline(ctx context.Context, port Port, env *Environment, rtConfig config.Config, iln *ast.InlineListNode) {
	evaluateNode(ctx, port, env, rtConfig, iln)
	cleaner.CleanInlineList(iln)
}

func evaluateNode(ctx context.Context, port Port, env *Environment, rtConfig config.Config, n ast.Node) {
	if env == nil {
		env = &emptyEnv
	}
	e := evaluator{
		ctx:        ctx,
		port:       port,
		env:        env,
		rtConfig:   rtConfig,
		astMap:     map[id.Zid]*ast.ZettelNode{},
		embedMap:   map[string]*ast.InlineListNode{},
		embedCount: 0,
		marker:     &ast.ZettelNode{},
	}
	ast.Walk(&e, n)
}

type evaluator struct {
	ctx        context.Context
	port       Port
	env        *Environment
	rtConfig   config.Config
	astMap     map[id.Zid]*ast.ZettelNode
	marker     *ast.ZettelNode
	embedMap   map[string]*ast.InlineListNode
	embedCount int
}

func (e *evaluator) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.InlineListNode:
		e.visitInlineList(n)
	default:
		return e
	}
	return nil
}

func (e *evaluator) visitInlineList(iln *ast.InlineListNode) {
	for i := 0; i < len(iln.List); i++ {
		in := iln.List[i]
		ast.Walk(e, in)
		switch n := in.(type) {
		case *ast.TagNode:
			iln.List[i] = e.visitTag(n)
		case *ast.LinkNode:
			iln.List[i] = e.evalLinkNode(n)
		case *ast.EmbedNode:
			in := e.evalEmbedNode(n)
			if ln, ok := in.(*ast.InlineListNode); ok {
				iln.List = replaceWithInlineNodes(iln.List, i, ln.List)
				i += len(ln.List) - 1
			} else {
				iln.List[i] = in
			}
		}
	}
}

func replaceWithInlineNodes(ins []ast.InlineNode, i int, replaceIns []ast.InlineNode) []ast.InlineNode {
	if len(replaceIns) == 1 {
		ins[i] = replaceIns[0]
		return ins
	}
	newIns := make([]ast.InlineNode, 0, len(ins)+len(replaceIns)-1)
	if i > 0 {
		newIns = append(newIns, ins[:i]...)
	}
	if len(replaceIns) > 0 {
		newIns = append(newIns, replaceIns...)
	}
	if i+1 < len(ins) {
		newIns = append(newIns, ins[i+1:]...)
	}
	return newIns
}

func (e *evaluator) visitTag(tn *ast.TagNode) ast.InlineNode {
	if gtr := e.env.GetTagRef; gtr != nil {
		fullTag := "#" + tn.Tag
		return &ast.LinkNode{
			Ref:     e.env.GetTagRef(fullTag),
			Inlines: ast.CreateInlineListNodeFromWords(fullTag),
		}
	}
	return tn
}

func (e *evaluator) evalLinkNode(ln *ast.LinkNode) ast.InlineNode {
	ref := ln.Ref
	if ref == nil {
		return ln
	}
	if ref.State == ast.RefStateBased {
		if ghr := e.env.GetHostedRef; ghr != nil {
			ln.Ref = ghr(ref.Value[1:])
		}
		return ln
	}
	if ref.State != ast.RefStateZettel {
		return ln
	}
	zid, err := id.Parse(ref.URL.Path)
	if err != nil {
		panic(err)
	}
	_, err = e.port.GetMeta(box.NoEnrichContext(e.ctx), zid)
	if errors.Is(err, &box.ErrNotAllowed{}) {
		return &ast.FormatNode{
			Kind:    ast.FormatSpan,
			Attrs:   ln.Attrs,
			Inlines: ln.Inlines,
		}
	} else if err != nil {
		ln.Ref.State = ast.RefStateBroken
		return ln
	}

	if gfr := e.env.GetFoundRef; gfr != nil {
		ln.Ref = gfr(zid, ref.URL.EscapedFragment())
	}
	return ln
}

func (e *evaluator) evalEmbedNode(en *ast.EmbedNode) ast.InlineNode {
	switch en.Material.(type) {
	case *ast.ReferenceMaterialNode:
	case *ast.BLOBMaterialNode:
		return en
	default:
		panic(fmt.Sprintf("Unknown material type %t for %v", en.Material, en.Material))
	}

	ref := en.Material.(*ast.ReferenceMaterialNode)
	switch ref.Ref.State {
	case ast.RefStateInvalid:
		return e.createErrorImage(en)
	case ast.RefStateZettel, ast.RefStateFound:
	case ast.RefStateSelf:
		return createErrorText(en, "Self", "embed", "reference:")
	case ast.RefStateBroken:
		return e.createErrorImage(en)
	case ast.RefStateHosted, ast.RefStateBased, ast.RefStateExternal:
		return en
	default:
		panic(fmt.Sprintf("Unknown state %v for reference %v", ref.Ref.State, ref.Ref))
	}

	zid, err := id.Parse(ref.Ref.URL.Path)
	if err != nil {
		panic(err)
	}
	zettel, err := e.port.GetZettel(box.NoEnrichContext(e.ctx), zid)
	if err != nil {
		return e.createErrorImage(en)
	}

	syntax := e.getSyntax(zettel.Meta)
	if parser.IsImageFormat(syntax) {
		return e.embedImage(en, zettel, syntax)
	}
	if !parser.IsTextParser(syntax) {
		// Not embeddable.
		return createErrorText(en, "Not", "embeddable (syntax="+syntax+"):")
	}

	zn, ok := e.astMap[zid]
	if zn == e.marker {
		return createErrorText(en, "Recursive", "transclusion:")
	}
	if !ok {
		e.astMap[zid] = e.marker
		zn = e.evaluateEmbeddedZettel(zettel, syntax)
		e.astMap[zid] = zn
	}

	result, ok := e.embedMap[ref.Ref.Value]
	if !ok {
		// Search for text to be embedded.
		result = findInlineList(zn.Ast, ref.Ref.URL.Fragment)
		e.embedMap[ref.Ref.Value] = result
		if result.IsEmpty() {
			return createErrorText(en, "Nothing", "to", "transclude:")
		}
	}

	e.embedCount++
	if maxTrans := e.rtConfig.GetMaxTransclusions(); e.embedCount > maxTrans {
		return createErrorText(en, "Too", "many", "transclusions ("+strconv.Itoa(maxTrans)+"):")
	}
	return result
}

func (e *evaluator) getSyntax(m *meta.Meta) string {
	if cfg := e.rtConfig; cfg != nil {
		return config.GetSyntax(m, cfg)
	}
	return m.GetDefault(meta.KeySyntax, "")
}

func (e *evaluator) createErrorImage(en *ast.EmbedNode) *ast.EmbedNode {
	zid := id.EmojiZid
	if !e.env.EmbedImage {
		en.Material = &ast.ReferenceMaterialNode{Ref: ast.ParseReference(zid.String())}
		return en
	}
	zettel, err := e.port.GetZettel(box.NoEnrichContext(e.ctx), zid)
	if err == nil {
		return doEmbedImage(en, zettel, e.getSyntax(zettel.Meta))
	}
	panic(err)
}

func (e *evaluator) embedImage(en *ast.EmbedNode, zettel domain.Zettel, syntax string) *ast.EmbedNode {
	if e.env.EmbedImage {
		return doEmbedImage(en, zettel, syntax)
	}
	return en
}

func doEmbedImage(en *ast.EmbedNode, zettel domain.Zettel, syntax string) *ast.EmbedNode {
	en.Material = &ast.BLOBMaterialNode{
		Blob:   zettel.Content.AsBytes(),
		Syntax: syntax,
	}
	return en
}

func createErrorText(en *ast.EmbedNode, msgWords ...string) ast.InlineNode {
	ref := en.Material.(*ast.ReferenceMaterialNode)
	ln := &ast.LinkNode{
		Ref:     ref.Ref,
		Inlines: ast.CreateInlineListNodeFromWords(ref.Ref.String()),
		OnlyRef: true,
	}
	text := ast.CreateInlineListNodeFromWords(msgWords...)
	text.Append(&ast.SpaceNode{Lexeme: " "}, ln)
	fn := &ast.FormatNode{
		Kind:    ast.FormatMonospace,
		Inlines: text,
	}
	fn = &ast.FormatNode{
		Kind:    ast.FormatBold,
		Inlines: ast.CreateInlineListNode(fn),
	}
	fn.Attrs = fn.Attrs.AddClass("error")
	return fn
}

func (e *evaluator) evaluateEmbeddedZettel(zettel domain.Zettel, syntax string) *ast.ZettelNode {
	zn := parser.ParseZettel(zettel, syntax, e.rtConfig)
	ast.Walk(e, zn.Ast)
	return zn
}

func findInlineList(bnl *ast.BlockListNode, fragment string) *ast.InlineListNode {
	if fragment == "" {
		return firstFirstTopLevelParagraph(bnl.List)
	}
	fs := fragmentSearcher{
		fragment: fragment,
		result:   nil,
	}
	ast.Walk(&fs, bnl)
	return fs.result
}

func firstFirstTopLevelParagraph(bns []ast.BlockNode) *ast.InlineListNode {
	for _, bn := range bns {
		pn, ok := bn.(*ast.ParaNode)
		if !ok {
			continue
		}
		inl := pn.Inlines
		if inl != nil && len(inl.List) > 0 {
			return inl
		}
	}
	return nil
}

type fragmentSearcher struct {
	fragment string
	result   *ast.InlineListNode
}

func (fs *fragmentSearcher) Visit(node ast.Node) ast.Visitor {
	if fs.result != nil {
		return nil
	}
	switch n := node.(type) {
	case *ast.BlockListNode:
		for i, bn := range n.List {
			if hn, ok := bn.(*ast.HeadingNode); ok && hn.Fragment == fs.fragment {
				fs.result = firstFirstTopLevelParagraph(n.List[i+1:])
				return nil
			}
			ast.Walk(fs, bn)
		}
	case *ast.InlineListNode:
		for i, in := range n.List {
			if mn, ok := in.(*ast.MarkNode); ok && mn.Fragment == fs.fragment {
				fs.result = ast.CreateInlineListNode(skipSpaceNodes(n.List[i+1:])...)
				return nil
			}
			ast.Walk(fs, in)
		}
	default:
		return fs
	}
	return nil
}

func skipSpaceNodes(ins []ast.InlineNode) []ast.InlineNode {
	for i, in := range ins {
		switch in.(type) {
		case *ast.SpaceNode:
		case *ast.BreakNode:
		default:
			return ins[i:]
		}
	}
	return nil
}

Changes to go.mod.

1
2
3

4
5
6

7
8

9
10
11

12


1
2

3
4
5

6
7

8
9
10

11
12
13
14


-
+


-
+

-
+


-
+

+
+
module zettelstore.de/z

go 1.16
go 1.17

require (
	github.com/fsnotify/fsnotify v1.4.9
	github.com/fsnotify/fsnotify v1.5.1
	github.com/pascaldekloe/jwt v1.10.0
	github.com/yuin/goldmark v1.4.0
	github.com/yuin/goldmark v1.4.1
	golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e
	golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b
	golang.org/x/text v0.3.6
	golang.org/x/text v0.3.7
)

require golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c // indirect

Changes to go.sum.

1
2


3
4
5
6


7
8
9
10
11
12
13



14
15
16
17
18
19


20


1
2
3
4


5
6
7
8
9

10


11
12
13
14
15
16
17


18
19
20
-
-
+
+


-
-
+
+



-

-
-
+
+
+




-
-
+
+

github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI=
github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=
github.com/pascaldekloe/jwt v1.10.0 h1:ktcIUV4TPvh404R5dIBEnPCsSwj0sqi3/0+XafE5gJs=
github.com/pascaldekloe/jwt v1.10.0/go.mod h1:TKhllgThT7TOP5rGr2zMLKEDZRAgJfBbtKyVeRsNB9A=
github.com/yuin/goldmark v1.4.0 h1:OtISOGfH6sOWa1/qXqqAiOIAO6Z5J3AEAE18WAq6BiQ=
github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark v1.4.1 h1:/vn0k+RBvwlxEmP5E7SZMqNxPhfMVFEJiykr15/0XKM=
github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e h1:gsTQYXdTw2Gq7RBsWvlQ91b+aEQ6bXFUngBGuR8sPpI=
golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1 h1:SrN+KX8Art/Sf4HNj6Zcz06G7VEz+7w9tdXTPOZ7+l4=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c h1:F1jZWGFhYfh0Ci55sIpILtKKK8p3i2/krTr0H1rg74I=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=

Changes to kernel/impl/cfg.go.

43
44
45
46
47
48
49
50
51
52
53
54
55







56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

73
74
75
76
77
78
79
43
44
45
46
47
48
49






50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81







-
-
-
-
-
-
+
+
+
+
+
+
+

















+







				if vis == meta.VisibilityUnknown {
					return nil
				}
				return vis
			},
			true,
		},
		meta.KeyExpertMode:     {"Expert mode", parseBool, true},
		meta.KeyFooterHTML:     {"Footer HTML", parseString, true},
		meta.KeyHomeZettel:     {"Home zettel", parseZid, true},
		meta.KeyMarkerExternal: {"Marker external URL", parseString, true},
		meta.KeySiteName:       {"Site name", parseString, true},
		meta.KeyYAMLHeader:     {"YAML header", parseBool, true},
		meta.KeyExpertMode:       {"Expert mode", parseBool, true},
		meta.KeyFooterHTML:       {"Footer HTML", parseString, true},
		meta.KeyHomeZettel:       {"Home zettel", parseZid, true},
		meta.KeyMarkerExternal:   {"Marker external URL", parseString, true},
		meta.KeyMaxTransclusions: {"Maximum transclusions", parseInt, true},
		meta.KeySiteName:         {"Site name", parseString, true},
		meta.KeyYAMLHeader:       {"YAML header", parseBool, true},
		meta.KeyZettelFileSyntax: {
			"Zettel file syntax",
			func(val string) interface{} { return strings.Fields(val) },
			true,
		},
	}
	cs.next = interfaceMap{
		meta.KeyDefaultCopyright:  "",
		meta.KeyDefaultLang:       meta.ValueLangEN,
		meta.KeyDefaultRole:       meta.ValueRoleZettel,
		meta.KeyDefaultSyntax:     meta.ValueSyntaxZmk,
		meta.KeyDefaultTitle:      "Untitled",
		meta.KeyDefaultVisibility: meta.VisibilityLogin,
		meta.KeyExpertMode:        false,
		meta.KeyFooterHTML:        "",
		meta.KeyHomeZettel:        id.DefaultHomeZid,
		meta.KeyMarkerExternal:    "&#10138;",
		meta.KeyMaxTransclusions:  1024,
		meta.KeySiteName:          "Zettelstore",
		meta.KeyYAMLHeader:        false,
		meta.KeyZettelFileSyntax:  nil,
	}
}

func (cs *configService) Start(kern *myKernel) error {
230
231
232
233
234
235
236











237
238
239
240
241
242
243
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256







+
+
+
+
+
+
+
+
+
+
+







	}
	cfg.mx.RLock()
	val, _ = cfg.orig.Get(meta.KeyDefaultVisibility)
	vis := meta.GetVisibility(val)
	cfg.mx.RUnlock()
	return vis
}

// GetMaxTransclusions return the maximum number of indirect transclusions.
func (cfg *myConfig) GetMaxTransclusions() int {
	cfg.mx.RLock()
	val, ok := cfg.data.GetNumber(meta.KeyMaxTransclusions)
	cfg.mx.RUnlock()
	if ok && val > 0 {
		return val
	}
	return 1024
}

// GetYAMLHeader returns the current value of the "yaml-header" key.
func (cfg *myConfig) GetYAMLHeader() bool { return cfg.getBool(meta.KeyYAMLHeader) }

// GetMarkerExternal returns the current value of the "marker-external" key.
func (cfg *myConfig) GetMarkerExternal() string {
	return cfg.getString(meta.KeyMarkerExternal)

Changes to kernel/impl/cmd.go.

188
189
190
191
192
193
194
195

196
197
198
199
200
201
202
188
189
190
191
192
193
194

195
196
197
198
199
200
201
202







-
+







		func(sess *cmdSession, cmd string, args []string) bool { sess.kern.Shutdown(false); return false },
	},
	"start": {"start service", cmdStart},
	"stat":  {"show service statistics", cmdStat},
	"stop":  {"stop service", cmdStop},
}

func cmdHelp(sess *cmdSession, cmd string, args []string) bool {
func cmdHelp(sess *cmdSession, _ string, _ []string) bool {
	cmds := make([]string, 0, len(commands))
	for key := range commands {
		if key == "" {
			continue
		}
		cmds = append(cmds, key)
	}
218
219
220
221
222
223
224
225

226
227
228
229
230

231
232
233
234
235
236
237
218
219
220
221
222
223
224

225
226
227
228
229

230
231
232
233
234
235
236
237







-
+




-
+







	table := [][]string{{"Key", "Description"}}
	for _, kd := range srv.ConfigDescriptions() {
		table = append(table, []string{kd.Key, kd.Descr})
	}
	sess.printTable(table)
	return true
}
func cmdGetConfig(sess *cmdSession, cmd string, args []string) bool {
func cmdGetConfig(sess *cmdSession, _ string, args []string) bool {
	showConfig(sess, args,
		listCurConfig, func(srv service, key string) interface{} { return srv.GetConfig(key) })
	return true
}
func cmdNextConfig(sess *cmdSession, cmd string, args []string) bool {
func cmdNextConfig(sess *cmdSession, _ string, args []string) bool {
	showConfig(sess, args,
		listNextConfig, func(srv service, key string) interface{} { return srv.GetNextConfig(key) })
	return true
}
func showConfig(sess *cmdSession, args []string,
	listConfig func(*cmdSession, service), getConfig func(service, string) interface{}) {

296
297
298
299
300
301
302
303

304
305
306
307
308
309
310
296
297
298
299
300
301
302

303
304
305
306
307
308
309
310







-
+







	newValue := strings.Join(args[2:], " ")
	if !srvD.srv.SetConfig(args[1], newValue) {
		sess.println("Unable to set key", args[1], "to value", newValue)
	}
	return true
}

func cmdServices(sess *cmdSession, cmd string, args []string) bool {
func cmdServices(sess *cmdSession, _ string, _ []string) bool {
	names := make([]string, 0, len(sess.kern.srvNames))
	for name := range sess.kern.srvNames {
		names = append(names, name)
	}
	sort.Strings(names)

	table := [][]string{{"Service", "Status"}}
386
387
388
389
390
391
392
393

394
395
396
397
398
399
400
386
387
388
389
390
391
392

393
394
395
396
397
398
399
400







-
+







	if !ok {
		sess.println("Unknown service", args[0])
		return 0, false
	}
	return srvD.srvnum, true
}

func cmdMetrics(sess *cmdSession, cmd string, args []string) bool {
func cmdMetrics(sess *cmdSession, _ string, _ []string) bool {
	var samples []metrics.Sample
	all := metrics.All()
	for _, d := range all {
		if d.Kind == metrics.KindFloat64Histogram {
			continue
		}
		samples = append(samples, metrics.Sample{Name: d.Name})
428
429
430
431
432
433
434
435

436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455

456
457
458
459
460
461
462
428
429
430
431
432
433
434

435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454

455
456
457
458
459
460
461
462







-
+



















-
+







		}
		table = append(table, []string{sVal, descr})
	}
	sess.printTable(table)
	return true
}

func cmdDumpIndex(sess *cmdSession, cmd string, args []string) bool {
func cmdDumpIndex(sess *cmdSession, _ string, _ []string) bool {
	sess.kern.DumpIndex(sess.w)
	return true
}
func cmdDumpRecover(sess *cmdSession, cmd string, args []string) bool {
	if len(args) == 0 {
		sess.usage(cmd, "RECOVER")
		sess.println("-- A valid value for RECOVER can be obtained via 'stat core'.")
		return true
	}
	lines := sess.kern.core.RecoverLines(args[0])
	if len(lines) == 0 {
		return true
	}
	for _, line := range lines {
		sess.println(line)
	}
	return true
}

func cmdEnvironment(sess *cmdSession, cmd string, args []string) bool {
func cmdEnvironment(sess *cmdSession, _ string, _ []string) bool {
	workDir, err := os.Getwd()
	if err != nil {
		workDir = err.Error()
	}
	execName, err := os.Executable()
	if err != nil {
		execName = err.Error()

Changes to kernel/impl/config.go.

221
222
223
224
225
226
227








228
229
230
231
232
233
234
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242







+
+
+
+
+
+
+
+







	}
	switch val[0] {
	case '0', 'f', 'F', 'n', 'N':
		return false
	}
	return true
}

func parseInt(val string) interface{} {
	i, err := strconv.Atoi(val)
	if err == nil {
		return i
	}
	return 0
}

func parseZid(val string) interface{} {
	if zid, err := id.Parse(val); err == nil {
		return zid
	}
	return id.Invalid
}

Changes to kernel/impl/impl.go.

93
94
95
96
97
98
99
100

101
102
103
104
105
106
107
93
94
95
96
97
98
99

100
101
102
103
104
105
106
107







-
+







		for _, dep := range deps {
			kern.depStop[dep] = append(kern.depStop[dep], srv)
		}
	}
	return kern
}

func (kern *myKernel) Start(headline bool, lineServer bool) {
func (kern *myKernel) Start(headline, lineServer bool) {
	for _, srvD := range kern.srvs {
		srvD.srv.Freeze()
	}
	kern.wg.Add(1)
	signal.Notify(kern.interrupt, os.Interrupt, syscall.SIGTERM)
	go func() {
		// Wait for interrupt.

Changes to parser/blob/blob.go.

16
17
18
19
20
21
22
23
24
25
26






27
28
29
30
31
32






33
34
35
36
37
38






39
40
41
42

43
44
45
46
47

48
49
50
51
52
53

54
55
56
57


58
16
17
18
19
20
21
22




23
24
25
26
27
28
29
30




31
32
33
34
35
36
37
38




39
40
41
42
43
44
45
46
47

48
49
50
51
52

53
54
55
56
57
58

59
60
61


62
63
64







-
-
-
-
+
+
+
+
+
+


-
-
-
-
+
+
+
+
+
+


-
-
-
-
+
+
+
+
+
+



-
+




-
+





-
+


-
-
+
+

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
	parser.Register(&parser.Info{
		Name:         "gif",
		AltNames:     nil,
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          "gif",
		AltNames:      nil,
		IsTextParser:  false,
		IsImageFormat: true,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
	parser.Register(&parser.Info{
		Name:         "jpeg",
		AltNames:     []string{"jpg"},
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          "jpeg",
		AltNames:      []string{"jpg"},
		IsTextParser:  false,
		IsImageFormat: true,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
	parser.Register(&parser.Info{
		Name:         "png",
		AltNames:     nil,
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          "png",
		AltNames:      nil,
		IsTextParser:  false,
		IsImageFormat: true,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) *ast.BlockListNode {
	if p := parser.Get(syntax); p != nil {
		syntax = p.Name
	}
	title, _ := m.Get(meta.KeyTitle)
	return ast.BlockSlice{
	return &ast.BlockListNode{List: []ast.BlockNode{
		&ast.BLOBNode{
			Title:  title,
			Syntax: syntax,
			Blob:   []byte(inp.Src),
		},
	}
	}}
}

func parseInlines(inp *input.Input, syntax string) ast.InlineSlice {
	return ast.InlineSlice{}
func parseInlines(*input.Input, string) *ast.InlineListNode {
	return nil
}

Changes to parser/cleaner/cleaner.go.

17
18
19
20
21
22
23



24
25
26





27
28
29
30
31

32
33
34

35
36
37
38

39
40
41
42
43
44
45

46
47
48
49
50
51
52
53
54
55
56

57
58
59
60
61
62

63
64
65
66
67
68
69
70
71
72
73
74
75

































76

77
78
79
80
81
82
83
17
18
19
20
21
22
23
24
25
26



27
28
29
30
31
32
33
34
35

36
37
38

39
40
41
42

43
44
45
46
47
48
49

50
51
52









53



54
55

56








57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94

95
96
97
98
99
100
101
102







+
+
+
-
-
-
+
+
+
+
+




-
+


-
+



-
+






-
+


-
-
-
-
-
-
-
-
-
+
-
-
-


-
+
-
-
-
-
-
-
-
-





+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+








	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/strfun"
)

// CleanBlockList cleans the given block list.
func CleanBlockList(bln *ast.BlockListNode) { cleanNode(bln) }

// CleanupBlockSlice cleans the given block slice.
func CleanupBlockSlice(bs ast.BlockSlice) {
	cv := cleanupVisitor{
// CleanInlineList cleans the given inline list.
func CleanInlineList(iln *ast.InlineListNode) { cleanNode(iln) }

func cleanNode(n ast.Node) {
	cv := cleanVisitor{
		textEnc: encoder.Create(api.EncoderText, nil),
		hasMark: false,
		doMark:  false,
	}
	ast.WalkBlockSlice(&cv, bs)
	ast.Walk(&cv, n)
	if cv.hasMark {
		cv.doMark = true
		ast.WalkBlockSlice(&cv, bs)
		ast.Walk(&cv, n)
	}
}

type cleanupVisitor struct {
type cleanVisitor struct {
	textEnc encoder.Encoder
	ids     map[string]ast.Node
	hasMark bool
	doMark  bool
}

func (cv *cleanupVisitor) Visit(node ast.Node) ast.Visitor {
func (cv *cleanVisitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.HeadingNode:
		if cv.doMark || n == nil || n.Inlines == nil {
			return nil
		}
		var sb strings.Builder
		_, err := cv.textEnc.WriteInlines(&sb, n.Inlines)
		if err != nil {
			return nil
		}
		s := strfun.Slugify(sb.String())
		cv.visitHeading(n)
		if len(s) > 0 {
			n.Slug = cv.addIdentifier(s, n)
		}
		return nil
	case *ast.MarkNode:
		if !cv.doMark {
		cv.visitMark(n)
			cv.hasMark = true
			return nil
		}
		if n.Text == "" {
			n.Text = cv.addIdentifier("*", n)
			return nil
		}
		n.Text = cv.addIdentifier(n.Text, n)
		return nil
	}
	return cv
}

func (cv *cleanVisitor) visitHeading(hn *ast.HeadingNode) {
	if cv.doMark || hn == nil || hn.Inlines.IsEmpty() {
		return
	}
	if hn.Slug == "" {
		var sb strings.Builder
		_, err := cv.textEnc.WriteInlines(&sb, hn.Inlines)
		if err != nil {
			return
		}
		hn.Slug = strfun.Slugify(sb.String())
	}
	if hn.Slug != "" {
		hn.Fragment = cv.addIdentifier(hn.Slug, hn)
	}
}

func (cv *cleanVisitor) visitMark(mn *ast.MarkNode) {
	if !cv.doMark {
		cv.hasMark = true
		return
	}
	if mn.Text == "" {
		mn.Slug = ""
		mn.Fragment = cv.addIdentifier("*", mn)
		return
	}
	if mn.Slug == "" {
		mn.Slug = strfun.Slugify(mn.Text)
	}
	mn.Fragment = cv.addIdentifier(mn.Slug, mn)
}

func (cv *cleanupVisitor) addIdentifier(id string, node ast.Node) string {
func (cv *cleanVisitor) addIdentifier(id string, node ast.Node) string {
	if cv.ids == nil {
		cv.ids = map[string]ast.Node{id: node}
		return id
	}
	if n, ok := cv.ids[id]; ok && n != node {
		prefix := id + "-"
		for count := 1; ; count++ {

Changes to parser/markdown/markdown.go.

26
27
28
29
30
31
32
33
34
35
36






37
38
39
40

41
42

43
44
45

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

64
65
66
67

68
69
70
71
72
73

74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105


106
107
108
109
110
111
112
113
114
115

116
117
118
119
120

121
122
123
124
125
126
127
26
27
28
29
30
31
32




33
34
35
36
37
38
39
40
41

42
43

44
45
46

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

65
66
67
68

69
70
71
72
73
74

75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105


106
107


108
109
110
111
112
113
114

115
116
117
118
119

120
121
122
123
124
125
126
127







-
-
-
-
+
+
+
+
+
+



-
+

-
+


-
+

















-
+



-
+





-
+














-
+















-
-
+
+
-
-







-
+




-
+







	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
	parser.Register(&parser.Info{
		Name:         "markdown",
		AltNames:     []string{"md"},
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          "markdown",
		AltNames:      []string{"md"},
		IsTextParser:  true,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
func parseBlocks(inp *input.Input, _ *meta.Meta, _ string) *ast.BlockListNode {
	p := parseMarkdown(inp)
	return p.acceptBlockSlice(p.docNode)
	return p.acceptBlockChildren(p.docNode)
}

func parseInlines(inp *input.Input, syntax string) ast.InlineSlice {
func parseInlines(*input.Input, string) *ast.InlineListNode {
	panic("markdown.parseInline not yet implemented")
}

func parseMarkdown(inp *input.Input) *mdP {
	source := []byte(inp.Src[inp.Pos:])
	parser := gm.DefaultParser()
	node := parser.Parse(gmText.NewReader(source))
	textEnc := encoder.Create(api.EncoderText, nil)
	return &mdP{source: source, docNode: node, textEnc: textEnc}
}

type mdP struct {
	source  []byte
	docNode gmAst.Node
	textEnc encoder.Encoder
}

func (p *mdP) acceptBlockSlice(docNode gmAst.Node) ast.BlockSlice {
func (p *mdP) acceptBlockChildren(docNode gmAst.Node) *ast.BlockListNode {
	if docNode.Type() != gmAst.TypeDocument {
		panic(fmt.Sprintf("Expected document, but got node type %v", docNode.Type()))
	}
	result := make(ast.BlockSlice, 0, docNode.ChildCount())
	result := make([]ast.BlockNode, 0, docNode.ChildCount())
	for child := docNode.FirstChild(); child != nil; child = child.NextSibling() {
		if block := p.acceptBlock(child); block != nil {
			result = append(result, block)
		}
	}
	return result
	return &ast.BlockListNode{List: result}
}

func (p *mdP) acceptBlock(node gmAst.Node) ast.ItemNode {
	if node.Type() != gmAst.TypeBlock {
		panic(fmt.Sprintf("Expected block node, but got node type %v", node.Type()))
	}
	switch n := node.(type) {
	case *gmAst.Paragraph:
		return p.acceptParagraph(n)
	case *gmAst.TextBlock:
		return p.acceptTextBlock(n)
	case *gmAst.Heading:
		return p.acceptHeading(n)
	case *gmAst.ThematicBreak:
		return p.acceptThematicBreak(n)
		return p.acceptThematicBreak()
	case *gmAst.CodeBlock:
		return p.acceptCodeBlock(n)
	case *gmAst.FencedCodeBlock:
		return p.acceptFencedCodeBlock(n)
	case *gmAst.Blockquote:
		return p.acceptBlockquote(n)
	case *gmAst.List:
		return p.acceptList(n)
	case *gmAst.HTMLBlock:
		return p.acceptHTMLBlock(n)
	}
	panic(fmt.Sprintf("Unhandled block node of kind %v", node.Kind()))
}

func (p *mdP) acceptParagraph(node *gmAst.Paragraph) ast.ItemNode {
	if ins := p.acceptInlineSlice(node); len(ins) > 0 {
		return &ast.ParaNode{
	if iln := p.acceptInlineChildren(node); iln != nil && len(iln.List) > 0 {
		return &ast.ParaNode{Inlines: iln}
			Inlines: ins,
		}
	}
	return nil
}

func (p *mdP) acceptHeading(node *gmAst.Heading) *ast.HeadingNode {
	return &ast.HeadingNode{
		Level:   node.Level,
		Inlines: p.acceptInlineSlice(node),
		Inlines: p.acceptInlineChildren(node),
		Attrs:   nil,
	}
}

func (p *mdP) acceptThematicBreak(node *gmAst.ThematicBreak) *ast.HRuleNode {
func (*mdP) acceptThematicBreak() *ast.HRuleNode {
	return &ast.HRuleNode{
		Attrs: nil, //TODO
	}
}

func (p *mdP) acceptCodeBlock(node *gmAst.CodeBlock) *ast.VerbatimNode {
	return &ast.VerbatimNode{
201
202
203
204
205
206
207
208
209


210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232


233
234
235
236
237
238

239
240
241

242
243
244
245
246
247
248
201
202
203
204
205
206
207


208
209


210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228


229
230
231
232
233
234
235

236
237
238

239
240
241
242
243
244
245
246







-
-
+
+
-
-



















-
-
+
+





-
+


-
+







			result = append(result, item)
		}
	}
	return result
}

func (p *mdP) acceptTextBlock(node *gmAst.TextBlock) ast.ItemNode {
	if ins := p.acceptInlineSlice(node); len(ins) > 0 {
		return &ast.ParaNode{
	if iln := p.acceptInlineChildren(node); iln != nil && len(iln.List) > 0 {
		return &ast.ParaNode{Inlines: iln}
			Inlines: ins,
		}
	}
	return nil
}

func (p *mdP) acceptHTMLBlock(node *gmAst.HTMLBlock) *ast.VerbatimNode {
	lines := p.acceptRawText(node)
	if node.HasClosure() {
		closure := string(node.ClosureLine.Value(p.source))
		if l := len(closure); l > 1 && closure[l-1] == '\n' {
			closure = closure[:l-1]
		}
		lines = append(lines, closure)
	}
	return &ast.VerbatimNode{
		Kind:  ast.VerbatimHTML,
		Lines: lines,
	}
}

func (p *mdP) acceptInlineSlice(node gmAst.Node) ast.InlineSlice {
	result := make(ast.InlineSlice, 0, node.ChildCount())
func (p *mdP) acceptInlineChildren(node gmAst.Node) *ast.InlineListNode {
	result := make([]ast.InlineNode, 0, node.ChildCount())
	for child := node.FirstChild(); child != nil; child = child.NextSibling() {
		if inlines := p.acceptInline(child); inlines != nil {
			result = append(result, inlines...)
		}
	}
	return result
	return ast.CreateInlineListNode(result...)
}

func (p *mdP) acceptInline(node gmAst.Node) ast.InlineSlice {
func (p *mdP) acceptInline(node gmAst.Node) []ast.InlineNode {
	if node.Type() != gmAst.TypeInline {
		panic(fmt.Sprintf("Expected inline node, but got %v", node.Type()))
	}
	switch n := node.(type) {
	case *gmAst.Text:
		return p.acceptText(n)
	case *gmAst.CodeSpan:
257
258
259
260
261
262
263
264

265
266
267
268
269
270

271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286

287
288

289
290

291
292
293
294
295
296
297
255
256
257
258
259
260
261

262
263
264
265
266
267

268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283

284
285

286
287

288
289
290
291
292
293
294
295







-
+





-
+















-
+

-
+

-
+







		return p.acceptAutoLink(n)
	case *gmAst.RawHTML:
		return p.acceptRawHTML(n)
	}
	panic(fmt.Sprintf("Unhandled inline node %v", node.Kind()))
}

func (p *mdP) acceptText(node *gmAst.Text) ast.InlineSlice {
func (p *mdP) acceptText(node *gmAst.Text) []ast.InlineNode {
	segment := node.Segment
	if node.IsRaw() {
		return splitText(string(segment.Value(p.source)))
	}
	ins := splitText(string(segment.Value(p.source)))
	result := make(ast.InlineSlice, 0, len(ins)+1)
	result := make([]ast.InlineNode, 0, len(ins)+1)
	for _, in := range ins {
		if tn, ok := in.(*ast.TextNode); ok {
			tn.Text = cleanText(tn.Text, true)
		}
		result = append(result, in)
	}
	if node.HardLineBreak() {
		result = append(result, &ast.BreakNode{Hard: true})
	} else if node.SoftLineBreak() {
		result = append(result, &ast.BreakNode{Hard: false})
	}
	return result
}

// splitText transform the text into a sequence of TextNode and SpaceNode
func splitText(text string) ast.InlineSlice {
func splitText(text string) []ast.InlineNode {
	if text == "" {
		return ast.InlineSlice{}
		return nil
	}
	result := make(ast.InlineSlice, 0, 1)
	result := make([]ast.InlineNode, 0, 1)

	state := 0 // 0=unknown,1=non-spaces,2=spaces
	lastPos := 0
	for pos, ch := range text {
		if input.IsSpace(ch) {
			if state == 1 {
				result = append(result, &ast.TextNode{Text: text[lastPos:pos]})
354
355
356
357
358
359
360
361
362


363
364
365
366
367
368
369
352
353
354
355
356
357
358


359
360
361
362
363
364
365
366
367







-
-
+
+







	}
	if lastPos < len(text) {
		sb.WriteString(text[lastPos:])
	}
	return sb.String()
}

func (p *mdP) acceptCodeSpan(node *gmAst.CodeSpan) ast.InlineSlice {
	return ast.InlineSlice{
func (p *mdP) acceptCodeSpan(node *gmAst.CodeSpan) []ast.InlineNode {
	return []ast.InlineNode{
		&ast.LiteralNode{
			Kind:  ast.LiteralProg,
			Attrs: nil, //TODO
			Text:  cleanCodeSpan(string(node.Text(p.source))),
		},
	}
}
386
387
388
389
390
391
392
393

394
395
396
397
398

399
400
401
402

403
404
405
406
407

408
409
410
411
412
413

414
415
416

417
418
419
420
421
422
423

424
425
426
427
428
429
430
431
432
433





434
435
436
437
438
439


440
441

442
443
444
445
446
447
448
449
450

451
452
453
454


455
456

457
458
459
460
461
462
463
464
465
466
467

468
469
470

471
472
473
474
475
476
477

478
479
480
481
482
483

484
485
486
487
488
489
490
384
385
386
387
388
389
390

391
392
393
394
395

396
397
398
399

400
401
402
403
404

405
406
407
408
409
410

411
412
413

414
415
416
417
418
419
420

421
422
423
424
425
426





427
428
429
430
431
432
433
434
435


436
437
438

439
440
441
442
443
444
445
446


447




448
449


450
451
452
453
454
455
456
457
458
459
460

461
462
463

464
465
466
467
468
469
470

471
472
473
474
475
476

477
478
479
480
481
482
483
484







-
+




-
+



-
+




-
+





-
+


-
+






-
+





-
-
-
-
-
+
+
+
+
+




-
-
+
+

-
+







-
-
+
-
-
-
-
+
+
-
-
+










-
+


-
+






-
+





-
+







	if lastPos == 0 {
		return text
	}
	sb.WriteString(text[lastPos:])
	return sb.String()
}

func (p *mdP) acceptEmphasis(node *gmAst.Emphasis) ast.InlineSlice {
func (p *mdP) acceptEmphasis(node *gmAst.Emphasis) []ast.InlineNode {
	kind := ast.FormatEmph
	if node.Level == 2 {
		kind = ast.FormatStrong
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.FormatNode{
			Kind:    kind,
			Attrs:   nil, //TODO
			Inlines: p.acceptInlineSlice(node),
			Inlines: p.acceptInlineChildren(node),
		},
	}
}

func (p *mdP) acceptLink(node *gmAst.Link) ast.InlineSlice {
func (p *mdP) acceptLink(node *gmAst.Link) []ast.InlineNode {
	ref := ast.ParseReference(cleanText(string(node.Destination), true))
	var attrs *ast.Attributes
	if title := string(node.Title); len(title) > 0 {
		attrs = attrs.Set("title", cleanText(title, true))
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.LinkNode{
			Ref:     ref,
			Inlines: p.acceptInlineSlice(node),
			Inlines: p.acceptInlineChildren(node),
			OnlyRef: false,
			Attrs:   attrs,
		},
	}
}

func (p *mdP) acceptImage(node *gmAst.Image) ast.InlineSlice {
func (p *mdP) acceptImage(node *gmAst.Image) []ast.InlineNode {
	ref := ast.ParseReference(cleanText(string(node.Destination), true))
	var attrs *ast.Attributes
	if title := string(node.Title); len(title) > 0 {
		attrs = attrs.Set("title", cleanText(title, true))
	}
	return ast.InlineSlice{
		&ast.ImageNode{
			Ref:     ref,
			Inlines: p.flattenInlineSlice(node),
			Attrs:   attrs,
	return []ast.InlineNode{
		&ast.EmbedNode{
			Material: &ast.ReferenceMaterialNode{Ref: ref},
			Inlines:  p.flattenInlineList(node),
			Attrs:    attrs,
		},
	}
}

func (p *mdP) flattenInlineSlice(node gmAst.Node) ast.InlineSlice {
	ins := p.acceptInlineSlice(node)
func (p *mdP) flattenInlineList(node gmAst.Node) *ast.InlineListNode {
	iln := p.acceptInlineChildren(node)
	var sb strings.Builder
	_, err := p.textEnc.WriteInlines(&sb, ins)
	_, err := p.textEnc.WriteInlines(&sb, iln)
	if err != nil {
		panic(err)
	}
	text := sb.String()
	if text == "" {
		return nil
	}
	return ast.InlineSlice{
		&ast.TextNode{
	return ast.CreateInlineListNode(&ast.TextNode{Text: text})
			Text: text,
		},
	}
}
}


func (p *mdP) acceptAutoLink(node *gmAst.AutoLink) ast.InlineSlice {
func (p *mdP) acceptAutoLink(node *gmAst.AutoLink) []ast.InlineNode {
	url := node.URL(p.source)
	if node.AutoLinkType == gmAst.AutoLinkEmail &&
		!bytes.HasPrefix(bytes.ToLower(url), []byte("mailto:")) {
		url = append([]byte("mailto:"), url...)
	}
	ref := ast.ParseReference(cleanText(string(url), false))
	label := node.Label(p.source)
	if len(label) == 0 {
		label = url
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.LinkNode{
			Ref:     ref,
			Inlines: ast.InlineSlice{&ast.TextNode{Text: string(label)}},
			Inlines: ast.CreateInlineListNode(&ast.TextNode{Text: string(label)}),
			OnlyRef: true,
			Attrs:   nil, //TODO
		},
	}
}

func (p *mdP) acceptRawHTML(node *gmAst.RawHTML) ast.InlineSlice {
func (p *mdP) acceptRawHTML(node *gmAst.RawHTML) []ast.InlineNode {
	segs := make([]string, 0, node.Segments.Len())
	for i := 0; i < node.Segments.Len(); i++ {
		segment := node.Segments.At(i)
		segs = append(segs, string(segment.Value(p.source)))
	}
	return ast.InlineSlice{
	return []ast.InlineNode{
		&ast.LiteralNode{
			Kind:  ast.LiteralHTML,
			Attrs: nil, // TODO: add HTML as language
			Text:  strings.Join(segs, ""),
		},
	}
}

Changes to parser/none/none.go.

16
17
18
19
20
21
22
23
24
25
26






27
28
29
30

31
32
33
34
35
36

37
38
39
40
41

42
43
44

45
46
47
48
49
50

51
52
53
54
55

56
57
58
59
60
61
62
63
64
65
66

67
68
69
70
71
72
73
74
75

76
77
78
79
80
81

82
83
84

85
86

87
88
89
90
91


92
93
94
95
96
97
98
99
100


101
102


103
16
17
18
19
20
21
22




23
24
25
26
27
28
29
30
31

32
33
34
35
36
37

38
39
40
41
42

43
44
45

46
47
48
49
50
51

52
53
54
55
56

57
58
59
60
61
62
63
64
65
66
67

68
69
70
71
72
73
74
75
76

77
78
79
80
81
82

83
84
85

86
87

88
89
90
91


92
93









94
95


96
97








-
-
-
-
+
+
+
+
+
+



-
+





-
+




-
+


-
+





-
+




-
+










-
+








-
+





-
+


-
+

-
+



-
-
+
+
-
-
-
-
-
-
-
-
-
+
+
-
-
+
+
-
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
	parser.Register(&parser.Info{
		Name:         meta.ValueSyntaxNone,
		AltNames:     []string{},
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          meta.ValueSyntaxNone,
		AltNames:      []string{},
		IsTextParser:  false,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
func parseBlocks(_ *input.Input, m *meta.Meta, _ string) *ast.BlockListNode {
	descrlist := &ast.DescriptionListNode{}
	for _, p := range m.Pairs(true) {
		descrlist.Descriptions = append(
			descrlist.Descriptions, getDescription(p.Key, p.Value))
	}
	return ast.BlockSlice{descrlist}
	return &ast.BlockListNode{List: []ast.BlockNode{descrlist}}
}

func getDescription(key, value string) ast.Description {
	return ast.Description{
		Term: ast.InlineSlice{&ast.TextNode{Text: key}},
		Term: ast.CreateInlineListNode(&ast.TextNode{Text: key}),
		Descriptions: []ast.DescriptionSlice{{
			&ast.ParaNode{
				Inlines: convertToInlineSlice(value, meta.Type(key)),
				Inlines: convertToInlineList(value, meta.Type(key)),
			}},
		},
	}
}

func convertToInlineSlice(value string, dt *meta.DescriptionType) ast.InlineSlice {
func convertToInlineList(value string, dt *meta.DescriptionType) *ast.InlineListNode {
	var sliceData []string
	if dt.IsSet {
		sliceData = meta.ListFromValue(value)
		if len(sliceData) == 0 {
			return ast.InlineSlice{}
			return &ast.InlineListNode{}
		}
	} else {
		sliceData = []string{value}
	}
	var makeLink bool
	switch dt {
	case meta.TypeID, meta.TypeIDSet:
		makeLink = true
	}

	result := make(ast.InlineSlice, 0, 2*len(sliceData)-1)
	result := make([]ast.InlineNode, 0, 2*len(sliceData)-1)
	for i, val := range sliceData {
		if i > 0 {
			result = append(result, &ast.SpaceNode{Lexeme: " "})
		}
		tn := &ast.TextNode{Text: val}
		if makeLink {
			result = append(result, &ast.LinkNode{
				Ref:     ast.ParseReference(val),
				Inlines: ast.InlineSlice{tn},
				Inlines: ast.CreateInlineListNode(tn),
			})
		} else {
			result = append(result, tn)
		}
	}
	return result
	return ast.CreateInlineListNode(result...)
}

func parseInlines(inp *input.Input, syntax string) ast.InlineSlice {
func parseInlines(inp *input.Input, _ string) *ast.InlineListNode {
	inp.SkipToEOL()
	return ast.InlineSlice{
	return ast.CreateInlineListNode(
		&ast.FormatNode{
			Kind:  ast.FormatSpan,
			Attrs: &ast.Attributes{Attrs: map[string]string{"class": "warning"}},
			Inlines: ast.InlineSlice{
				&ast.TextNode{Text: "parser.meta.ParseInlines:"},
			Inlines: ast.CreateInlineListNodeFromWords(
				"parser.meta.ParseInlines:", "not", "possible", "("+inp.Src[0:inp.Pos]+")",
				&ast.SpaceNode{Lexeme: " "},
				&ast.TextNode{Text: "not"},
				&ast.SpaceNode{Lexeme: " "},
				&ast.TextNode{Text: "possible"},
				&ast.SpaceNode{Lexeme: " "},
				&ast.TextNode{Text: "("},
				&ast.TextNode{Text: inp.Src[0:inp.Pos]},
				&ast.TextNode{Text: ")"},
			},
			),
		},
		},
	}
	)
}
}

Changes to parser/parser.go.

8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34






35
36
37
38
39
40
41
42

43
44
45
46
47

48
49
50
51
52









53
54
55
56
57
58
59
60
61
62
63
64
65


















66
67
68
69
70




71
72
73
74

75
76
77
78
79
80
81


82
83
84
85
86


87


88
89
90
91
92
93
94
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31




32
33
34
35
36
37
38
39
40
41
42
43
44

45
46
47
48
49

50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96




97
98
99
100
101
102
103

104
105
106
107
108
109


110
111
112
113
114
115
116
117
118

119
120
121
122
123
124
125
126
127







+
















-
-
-
-
+
+
+
+
+
+







-
+




-
+





+
+
+
+
+
+
+
+
+













+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
-
-
-
+
+
+
+



-
+





-
-
+
+





+
+
-
+
+







// under this license.
//-----------------------------------------------------------------------------

// Package parser provides a generic interface to a range of different parsers.
package parser

import (
	"fmt"
	"log"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser/cleaner"
)

// Info describes a single parser.
//
// Before ParseBlocks() or ParseInlines() is called, ensure the input stream to
// be valid. This can ce achieved on calling inp.Next() after the input stream
// was created.
type Info struct {
	Name         string
	AltNames     []string
	ParseBlocks  func(*input.Input, *meta.Meta, string) ast.BlockSlice
	ParseInlines func(*input.Input, string) ast.InlineSlice
	Name          string
	AltNames      []string
	IsTextParser  bool
	IsImageFormat bool
	ParseBlocks   func(*input.Input, *meta.Meta, string) *ast.BlockListNode
	ParseInlines  func(*input.Input, string) *ast.InlineListNode
}

var registry = map[string]*Info{}

// Register the parser (info) for later retrieval.
func Register(pi *Info) *Info {
	if _, ok := registry[pi.Name]; ok {
		log.Fatalf("Parser %q already registered", pi.Name)
		panic(fmt.Sprintf("Parser %q already registered", pi.Name))
	}
	registry[pi.Name] = pi
	for _, alt := range pi.AltNames {
		if _, ok := registry[alt]; ok {
			log.Fatalf("Parser %q already registered", alt)
			panic(fmt.Sprintf("Parser %q already registered", alt))
		}
		registry[alt] = pi
	}
	return pi
}

// GetSyntaxes returns a list of syntaxes implemented by all registered parsers.
func GetSyntaxes() []string {
	result := make([]string, 0, len(registry))
	for syntax := range registry {
		result = append(result, syntax)
	}
	return result
}

// Get the parser (info) by name. If name not found, use a default parser.
func Get(name string) *Info {
	if pi := registry[name]; pi != nil {
		return pi
	}
	if pi := registry["plain"]; pi != nil {
		return pi
	}
	log.Printf("No parser for %q found", name)
	panic("No default parser registered")
}

// IsTextParser returns whether the given syntax parses text into an AST or not.
func IsTextParser(syntax string) bool {
	pi, ok := registry[syntax]
	if !ok {
		return false
	}
	return pi.IsTextParser
}

// IsImageFormat returns whether the given syntax is known to be an image format.
func IsImageFormat(syntax string) bool {
	pi, ok := registry[syntax]
	if !ok {
		return false
	}
	return pi.IsImageFormat
}

// ParseBlocks parses some input and returns a slice of block nodes.
func ParseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
	bs := Get(syntax).ParseBlocks(inp, m, syntax)
	cleaner.CleanupBlockSlice(bs)
	return bs
func ParseBlocks(inp *input.Input, m *meta.Meta, syntax string) *ast.BlockListNode {
	bln := Get(syntax).ParseBlocks(inp, m, syntax)
	cleaner.CleanBlockList(bln)
	return bln
}

// ParseInlines parses some input and returns a slice of inline nodes.
func ParseInlines(inp *input.Input, syntax string) ast.InlineSlice {
func ParseInlines(inp *input.Input, syntax string) *ast.InlineListNode {
	return Get(syntax).ParseInlines(inp, syntax)
}

// ParseMetadata parses a string as Zettelmarkup, resulting in an inline slice.
// Typically used to parse the title or other metadata of type Zettelmarkup.
func ParseMetadata(title string) ast.InlineSlice {
	return ParseInlines(input.NewInput(title), meta.ValueSyntaxZmk)
func ParseMetadata(value string) *ast.InlineListNode {
	return ParseInlines(input.NewInput(value), meta.ValueSyntaxZmk)
}

// ParseZettel parses the zettel based on the syntax.
func ParseZettel(zettel domain.Zettel, syntax string, rtConfig config.Config) *ast.ZettelNode {
	m := zettel.Meta
	inhMeta := m
	if rtConfig != nil {
	inhMeta := rtConfig.AddDefaultValues(m)
		inhMeta = rtConfig.AddDefaultValues(inhMeta)
	}
	if syntax == "" {
		syntax, _ = inhMeta.Get(meta.KeySyntax)
	}
	parseMeta := inhMeta
	if syntax == meta.ValueSyntaxNone {
		parseMeta = m
	}

Added parser/parser_test.go.



































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package parser provides a generic interface to a range of different parsers.
package parser_test

import (
	"testing"

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/parser"

	_ "zettelstore.de/z/parser/blob"       // Allow to use BLOB parser.
	_ "zettelstore.de/z/parser/markdown"   // Allow to use markdown parser.
	_ "zettelstore.de/z/parser/none"       // Allow to use none parser.
	_ "zettelstore.de/z/parser/plain"      // Allow to use plain parser.
	_ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser.
)

func TestParserType(t *testing.T) {
	syntaxes := parser.GetSyntaxes()
	syntaxSet := make(map[string]bool, len(syntaxes))
	for _, syntax := range syntaxes {
		syntaxSet[syntax] = true
	}
	testCases := []struct {
		syntax string
		text   bool
		image  bool
	}{
		{"html", false, false},
		{"css", false, false},
		{"gif", false, true},
		{"jpeg", false, true},
		{"jpg", false, true},
		{"markdown", true, false},
		{"md", true, false},
		{"mustache", false, false},
		{meta.ValueSyntaxNone, false, false},
		{"plain", false, false},
		{"png", false, true},
		{"svg", false, true},
		{"text", false, false},
		{"txt", false, false},
		{meta.ValueSyntaxZmk, true, false},
	}
	for _, tc := range testCases {
		delete(syntaxSet, tc.syntax)
		if got := parser.IsTextParser(tc.syntax); got != tc.text {
			t.Errorf("Syntax %q is text: %v, but got %v", tc.syntax, tc.text, got)
		}
		if got := parser.IsImageFormat(tc.syntax); got != tc.image {
			t.Errorf("Syntax %q is image: %v, but got %v", tc.syntax, tc.image, got)
		}
	}
	for syntax := range syntaxSet {
		t.Errorf("Forgot to test syntax %q", syntax)
	}
}

Changes to parser/plain/plain.go.

18
19
20
21
22
23
24
25
26
27
28






29
30
31
32
33






34
35
36
37
38






39
40
41
42
43






44
45
46
47
48






49
50
51
52
53


54
55
56


57
58
59


60
61
62
63
64
65

66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

81
82
83

84
85
86

87
88
89
90
91
92




93
94

95
96
97
98
99



100
101
102
103

104
105
106
107


108
109

110
111
112
113
114
115


116
117
118
119

120
121
122
123
124
125
126
127
128
129
130
131
132
18
19
20
21
22
23
24




25
26
27
28
29
30
31
32



33
34
35
36
37
38
39
40



41
42
43
44
45
46
47
48



49
50
51
52
53
54
55
56



57
58
59
60
61
62
63
64
65


66
67
68


69
70
71


72
73
74
75
76
77
78

79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

94
95
96

97
98
99

100
101





102
103
104
105


106
107
108



109
110
111
112
113


114




115
116


117
118
119
120
121


122
123
124
125
126

127
128
129
130
131
132
133
134
135
136
137
138
139
140







-
-
-
-
+
+
+
+
+
+


-
-
-
+
+
+
+
+
+


-
-
-
+
+
+
+
+
+


-
-
-
+
+
+
+
+
+


-
-
-
+
+
+
+
+
+



-
-
+
+

-
-
+
+

-
-
+
+





-
+














-
+


-
+


-
+

-
-
-
-
-
+
+
+
+
-
-
+


-
-
-
+
+
+


-
-
+
-
-
-
-
+
+
-
-
+




-
-
+
+



-
+













	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
	parser.Register(&parser.Info{
		Name:         "txt",
		AltNames:     []string{"plain", "text"},
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          "txt",
		AltNames:      []string{"plain", "text"},
		IsTextParser:  false,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
	parser.Register(&parser.Info{
		Name:         "html",
		ParseBlocks:  parseBlocksHTML,
		ParseInlines: parseInlinesHTML,
		Name:          "html",
		AltNames:      []string{},
		IsTextParser:  false,
		IsImageFormat: false,
		ParseBlocks:   parseBlocksHTML,
		ParseInlines:  parseInlinesHTML,
	})
	parser.Register(&parser.Info{
		Name:         "css",
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          "css",
		AltNames:      []string{},
		IsTextParser:  false,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
	parser.Register(&parser.Info{
		Name:         "svg",
		ParseBlocks:  parseSVGBlocks,
		ParseInlines: parseSVGInlines,
		Name:          "svg",
		AltNames:      []string{},
		IsTextParser:  false,
		IsImageFormat: true,
		ParseBlocks:   parseSVGBlocks,
		ParseInlines:  parseSVGInlines,
	})
	parser.Register(&parser.Info{
		Name:         "mustache",
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          "mustache",
		AltNames:      []string{},
		IsTextParser:  false,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
	return doParseBlocks(inp, m, syntax, ast.VerbatimProg)
func parseBlocks(inp *input.Input, _ *meta.Meta, syntax string) *ast.BlockListNode {
	return doParseBlocks(inp, syntax, ast.VerbatimProg)
}
func parseBlocksHTML(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
	return doParseBlocks(inp, m, syntax, ast.VerbatimHTML)
func parseBlocksHTML(inp *input.Input, _ *meta.Meta, syntax string) *ast.BlockListNode {
	return doParseBlocks(inp, syntax, ast.VerbatimHTML)
}
func doParseBlocks(inp *input.Input, m *meta.Meta, syntax string, kind ast.VerbatimKind) ast.BlockSlice {
	return ast.BlockSlice{
func doParseBlocks(inp *input.Input, syntax string, kind ast.VerbatimKind) *ast.BlockListNode {
	return &ast.BlockListNode{List: []ast.BlockNode{
		&ast.VerbatimNode{
			Kind:  kind,
			Attrs: &ast.Attributes{Attrs: map[string]string{"": syntax}},
			Lines: readLines(inp),
		},
	}
	}}
}

func readLines(inp *input.Input) (lines []string) {
	for {
		inp.EatEOL()
		posL := inp.Pos
		if inp.Ch == input.EOS {
			return lines
		}
		inp.SkipToEOL()
		lines = append(lines, inp.Src[posL:inp.Pos])
	}
}

func parseInlines(inp *input.Input, syntax string) ast.InlineSlice {
func parseInlines(inp *input.Input, syntax string) *ast.InlineListNode {
	return doParseInlines(inp, syntax, ast.LiteralProg)
}
func parseInlinesHTML(inp *input.Input, syntax string) ast.InlineSlice {
func parseInlinesHTML(inp *input.Input, syntax string) *ast.InlineListNode {
	return doParseInlines(inp, syntax, ast.LiteralHTML)
}
func doParseInlines(inp *input.Input, syntax string, kind ast.LiteralKind) ast.InlineSlice {
func doParseInlines(inp *input.Input, syntax string, kind ast.LiteralKind) *ast.InlineListNode {
	inp.SkipToEOL()
	return ast.InlineSlice{
		&ast.LiteralNode{
			Kind:  kind,
			Attrs: &ast.Attributes{Attrs: map[string]string{"": syntax}},
			Text:  inp.Src[0:inp.Pos],
	return ast.CreateInlineListNode(&ast.LiteralNode{
		Kind:  kind,
		Attrs: &ast.Attributes{Attrs: map[string]string{"": syntax}},
		Text:  inp.Src[0:inp.Pos],
		},
	}
	})
}

func parseSVGBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
	ins := parseSVGInlines(inp, syntax)
	if ins == nil {
func parseSVGBlocks(inp *input.Input, _ *meta.Meta, syntax string) *ast.BlockListNode {
	iln := parseSVGInlines(inp, syntax)
	if iln == nil {
		return nil
	}
	return ast.BlockSlice{
		&ast.ParaNode{
	return &ast.BlockListNode{List: []ast.BlockNode{&ast.ParaNode{Inlines: iln}}}
			Inlines: ins,
		},
	}
}
}


func parseSVGInlines(inp *input.Input, syntax string) ast.InlineSlice {
func parseSVGInlines(inp *input.Input, syntax string) *ast.InlineListNode {
	svgSrc := scanSVG(inp)
	if svgSrc == "" {
		return nil
	}
	return ast.InlineSlice{
		&ast.ImageNode{
	return ast.CreateInlineListNode(&ast.EmbedNode{
		Material: &ast.BLOBMaterialNode{
			Blob:   []byte(svgSrc),
			Syntax: syntax,
		},
	}
	})
}

func scanSVG(inp *input.Input) string {
	for input.IsSpace(inp.Ch) {
		inp.Next()
	}
	svgSrc := inp.Src[inp.Pos:]
	if !strings.HasPrefix(svgSrc, "<svg ") {
		return ""
	}
	// TODO: check proper end </svg>
	return svgSrc
}

Changes to parser/zettelmark/block.go.

14
15
16
17
18
19
20
21
22


23
24
25

26
27
28
29

30
31
32
33
34
35
36
37
38

39
40
41
42
43
44
45
14
15
16
17
18
19
20


21
22
23
24

25
26
27
28

29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45







-
-
+
+


-
+



-
+








-
+







import (
	"fmt"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/input"
)

// parseBlockSlice parses a sequence of blocks.
func (cp *zmkP) parseBlockSlice() ast.BlockSlice {
// parseBlockList parses a sequence of blocks.
func (cp *zmkP) parseBlockList() *ast.BlockListNode {
	inp := cp.inp
	var lastPara *ast.ParaNode
	result := make(ast.BlockSlice, 0, 2)
	bs := make([]ast.BlockNode, 0, 2)
	for inp.Ch != input.EOS {
		bn, cont := cp.parseBlock(lastPara)
		if bn != nil {
			result = append(result, bn)
			bs = append(bs, bn)
		}
		if !cont {
			lastPara, _ = bn.(*ast.ParaNode)
		}
	}
	if cp.nestingLevel != 0 {
		panic("Nesting level was not decremented")
	}
	return result
	return &ast.BlockListNode{List: bs}
}

// parseBlock parses one block.
func (cp *zmkP) parseBlock(lastPara *ast.ParaNode) (res ast.BlockNode, cont bool) {
	inp := cp.inp
	pos := inp.Pos
	if cp.nestingLevel <= maxNestingLevel {
76
77
78
79
80
81
82
83

84
85
86
87

88
89
90
91
92
93
94
95
96
97
98

99
100
101
102
103
104
105
76
77
78
79
80
81
82

83
84
85
86

87
88
89
90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
105







-
+



-
+










-
+







			bn, success = cp.parseNestedList()
		case ';':
			cp.lists = nil
			cp.table = nil
			bn, success = cp.parseDefTerm()
		case ' ':
			cp.table = nil
			bn, success = cp.parseIndent()
			bn, success = nil, cp.parseIndent()
		case '|':
			cp.lists = nil
			cp.descrl = nil
			bn, success = cp.parseRow()
			bn, success = cp.parseRow(), true
		}

		if success {
			return bn, false
		}
	}
	inp.SetPos(pos)
	cp.clearStacked()
	pn := cp.parsePara()
	if lastPara != nil {
		lastPara.Inlines = append(lastPara.Inlines, pn.Inlines...)
		lastPara.Inlines.Append(pn.Inlines.List...)
		return nil, true
	}
	return pn, false
}

func (cp *zmkP) cleanupListsAfterEOL() {
	for _, l := range cp.lists {
124
125
126
127
128
129
130
131

132
133
134
135
136
137

138
139
140
141
142
143
144
124
125
126
127
128
129
130

131
132
133
134
135
136

137
138
139
140
141
142
143
144







-
+





-
+







		return cp.parseRegion()
	}
	return cp.parseDefDescr()
}

// parsePara parses paragraphed inline material.
func (cp *zmkP) parsePara() *ast.ParaNode {
	pn := &ast.ParaNode{}
	pn := ast.NewParaNode()
	for {
		in := cp.parseInline()
		if in == nil {
			return pn
		}
		pn.Inlines = append(pn.Inlines, in)
		pn.Inlines.Append(in)
		if _, ok := in.(*ast.BreakNode); ok {
			ch := cp.inp.Ch
			switch ch {
			// Must contain all cases from above switch in parseBlock.
			case input.EOS, '\n', '\r', '`', runeModGrave, '%', '"', '<', '=', '-', '*', '#', '>', ';', ':', ' ', '|':
				return pn
			}
216
217
218
219
220
221
222
223






224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240

241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260





261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280

281
282
283
284
285
286
287
288

289

290
291
292
293
294
295
296
297
298
299
300
301
302
216
217
218
219
220
221
222

223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244

245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264

265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288

289
290
291
292
293
294
295
296
297
298

299
300
301
302
303
304

305
306
307
308
309
310
311







-
+
+
+
+
+
+
















-
+



















-
+
+
+
+
+



















-
+








+
-
+





-







		return nil, false
	}
	attrs := cp.parseAttributes(true)
	inp.SkipToEOL()
	if inp.Ch == input.EOS {
		return nil, false
	}
	rn = &ast.RegionNode{Kind: kind, Attrs: attrs}
	rn = &ast.RegionNode{
		Kind:    kind,
		Attrs:   attrs,
		Blocks:  &ast.BlockListNode{},
		Inlines: nil,
	}
	var lastPara *ast.ParaNode
	inp.EatEOL()
	for {
		posL := inp.Pos
		switch inp.Ch {
		case fch:
			if cp.countDelim(fch) >= cnt {
				cp.parseRegionLastLine(rn)
				return rn, true
			}
			inp.SetPos(posL)
		case input.EOS:
			return nil, false
		}
		bn, cont := cp.parseBlock(lastPara)
		if bn != nil {
			rn.Blocks = append(rn.Blocks, bn)
			rn.Blocks.List = append(rn.Blocks.List, bn)
		}
		if !cont {
			lastPara, _ = bn.(*ast.ParaNode)
		}
	}
}

func (cp *zmkP) parseRegionLastLine(rn *ast.RegionNode) {
	cp.clearStacked() // remove any lists defined in the region
	cp.skipSpace()
	for {
		switch cp.inp.Ch {
		case input.EOS, '\n', '\r':
			return
		}
		in := cp.parseInline()
		if in == nil {
			return
		}
		rn.Inlines = append(rn.Inlines, in)
		if rn.Inlines == nil {
			rn.Inlines = ast.CreateInlineListNode(in)
		} else {
			rn.Inlines.Append(in)
		}
	}

}

// parseHeading parses a head line.
func (cp *zmkP) parseHeading() (hn *ast.HeadingNode, success bool) {
	inp := cp.inp
	lvl := cp.countDelim(inp.Ch)
	if lvl < 3 {
		return nil, false
	}
	if lvl > 7 {
		lvl = 7
	}
	if inp.Ch != ' ' {
		return nil, false
	}
	inp.Next()
	cp.skipSpace()
	hn = &ast.HeadingNode{Level: lvl - 1}
	hn = &ast.HeadingNode{Level: lvl - 1, Inlines: &ast.InlineListNode{}}
	for {
		if input.IsEOLEOS(inp.Ch) {
			return hn, true
		}
		in := cp.parseInline()
		if in == nil {
			return hn, true
		}
		hn.Inlines.Append(in)
		if inp.Ch == '{' {
		if inp.Ch == '{' && inp.Peek() != '{' {
			attrs := cp.parseAttributes(true)
			hn.Attrs = attrs
			inp.SkipToEOL()
			return hn, true
		}
		hn.Inlines = append(hn.Inlines, in)
	}
}

// parseHRule parses a horizontal rule.
func (cp *zmkP) parseHRule() (hn *ast.HRuleNode, success bool) {
	inp := cp.inp
	if cp.countDelim(inp.Ch) < 3 {
327
328
329
330
331
332
333
334

335
336
337
338
339
340
341
336
337
338
339
340
341
342

343
344
345
346
347
348
349
350







-
+








	if len(kinds) < len(cp.lists) {
		cp.lists = cp.lists[:len(kinds)]
	}
	ln, newLnCount := cp.buildNestedList(kinds)
	pn := cp.parseLinePara()
	if pn == nil {
		pn = &ast.ParaNode{}
		pn = ast.NewParaNode()
	}
	ln.Items = append(ln.Items, ast.ItemSlice{pn})
	return cp.cleanupParsedNestedList(newLnCount)
}

func (cp *zmkP) parseNestedListKinds() []ast.NestedListKind {
	inp := cp.inp
420
421
422
423
424
425
426



427

428
429
430
431
432
433
434
429
430
431
432
433
434
435
436
437
438

439
440
441
442
443
444
445
446







+
+
+
-
+







		in := cp.parseInline()
		if in == nil {
			if descrl.Descriptions[defPos].Term == nil {
				return nil, false
			}
			return res, true
		}
		if descrl.Descriptions[defPos].Term == nil {
			descrl.Descriptions[defPos].Term = &ast.InlineListNode{}
		}
		descrl.Descriptions[defPos].Term = append(descrl.Descriptions[defPos].Term, in)
		descrl.Descriptions[defPos].Term.Append(in)
		if _, ok := in.(*ast.BreakNode); ok {
			return res, true
		}
	}
}

// parseDefDescr parses a description of a definition list.
455
456
457
458
459
460
461
462

463
464
465
466
467
468
469
470
471
472
473

474
475
476

477
478

479
480
481
482
483
484
485
486
487
488
489
490
491
492

493
494
495
496

497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515

516
517
518
519
520
521
522
523
524
525
526
527
528
529
530

531
532
533
534
535
536
537
538
539
540

541
542
543
544
545
546
547
548
549

550
551
552
553
554
555
556
557

558
559
560
561

562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578

579
580
581

582
583
584
585
586
587
588
589
590

591
592
593

594
595
596

597
598
599

600
601

602
603
467
468
469
470
471
472
473

474
475
476
477
478
479
480
481
482
483
484

485
486
487

488
489

490
491
492
493
494
495
496
497
498
499
500
501
502
503

504
505
506
507

508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526

527
528
529
530
531
532
533
534
535
536
537
538
539
540
541

542
543
544
545
546
547
548
549
550
551

552
553
554
555
556
557
558
559
560

561
562
563
564
565
566
567
568

569
570
571
572

573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589

590
591
592

593
594
595
596
597
598
599
600
601

602
603
604

605
606
607

608
609
610

611
612

613
614
615







-
+










-
+


-
+

-
+













-
+



-
+


















-
+














-
+









-
+








-
+







-
+



-
+
















-
+


-
+








-
+


-
+


-
+


-
+

-
+


	cp.lists = nil
	cp.table = nil
	descrl.Descriptions[defPos].Descriptions = append(descrl.Descriptions[defPos].Descriptions, ast.DescriptionSlice{pn})
	return nil, true
}

// parseIndent parses initial spaces to continue a list.
func (cp *zmkP) parseIndent() (res ast.BlockNode, success bool) {
func (cp *zmkP) parseIndent() bool {
	inp := cp.inp
	cnt := 0
	for {
		inp.Next()
		if inp.Ch != ' ' {
			break
		}
		cnt++
	}
	if cp.lists != nil {
		return nil, cp.parseIndentForList(cnt)
		return cp.parseIndentForList(cnt)
	}
	if cp.descrl != nil {
		return nil, cp.parseIndentForDescription(cnt)
		return cp.parseIndentForDescription(cnt)
	}
	return nil, false
	return false
}

func (cp *zmkP) parseIndentForList(cnt int) bool {
	if len(cp.lists) < cnt {
		cnt = len(cp.lists)
	}
	cp.lists = cp.lists[:cnt]
	if cnt == 0 {
		return false
	}
	ln := cp.lists[cnt-1]
	pn := cp.parseLinePara()
	if pn == nil {
		pn = &ast.ParaNode{}
		pn = ast.NewParaNode()
	}
	lbn := ln.Items[len(ln.Items)-1]
	if lpn, ok := lbn[len(lbn)-1].(*ast.ParaNode); ok {
		lpn.Inlines = append(lpn.Inlines, pn.Inlines...)
		lpn.Inlines.Append(pn.Inlines.List...)
	} else {
		ln.Items[len(ln.Items)-1] = append(ln.Items[len(ln.Items)-1], pn)
	}
	return true
}

func (cp *zmkP) parseIndentForDescription(cnt int) bool {
	defPos := len(cp.descrl.Descriptions) - 1
	if cnt < 1 || defPos < 0 {
		return false
	}
	if len(cp.descrl.Descriptions[defPos].Descriptions) == 0 {
		// Continuation of a definition term
		for {
			in := cp.parseInline()
			if in == nil {
				return true
			}
			cp.descrl.Descriptions[defPos].Term = append(cp.descrl.Descriptions[defPos].Term, in)
			cp.descrl.Descriptions[defPos].Term.Append(in)
			if _, ok := in.(*ast.BreakNode); ok {
				return true
			}
		}
	}

	// Continuation of a definition description
	pn := cp.parseLinePara()
	if pn == nil {
		return false
	}
	descrPos := len(cp.descrl.Descriptions[defPos].Descriptions) - 1
	lbn := cp.descrl.Descriptions[defPos].Descriptions[descrPos]
	if lpn, ok := lbn[len(lbn)-1].(*ast.ParaNode); ok {
		lpn.Inlines = append(lpn.Inlines, pn.Inlines...)
		lpn.Inlines.Append(pn.Inlines.List...)
	} else {
		descrPos := len(cp.descrl.Descriptions[defPos].Descriptions) - 1
		cp.descrl.Descriptions[defPos].Descriptions[descrPos] = append(cp.descrl.Descriptions[defPos].Descriptions[descrPos], pn)
	}
	return true
}

// parseLinePara parses one line of inline material.
func (cp *zmkP) parseLinePara() *ast.ParaNode {
	pn := &ast.ParaNode{}
	pn := ast.NewParaNode()
	for {
		in := cp.parseInline()
		if in == nil {
			if pn.Inlines == nil {
				return nil
			}
			return pn
		}
		pn.Inlines = append(pn.Inlines, in)
		pn.Inlines.Append(in)
		if _, ok := in.(*ast.BreakNode); ok {
			return pn
		}
	}
}

// parseRow parse one table row.
func (cp *zmkP) parseRow() (res ast.BlockNode, success bool) {
func (cp *zmkP) parseRow() ast.BlockNode {
	inp := cp.inp
	if inp.Peek() == '%' {
		inp.SkipToEOL()
		return nil, true
		return nil
	}
	row := ast.TableRow{}
	for {
		inp.Next()
		cell := cp.parseCell()
		if cell != nil {
			row = append(row, cell)
		}
		switch inp.Ch {
		case '\n', '\r':
			inp.EatEOL()
			fallthrough
		case input.EOS:
			// add to table
			if cp.table == nil {
				cp.table = &ast.TableNode{Rows: []ast.TableRow{row}}
				return cp.table, true
				return cp.table
			}
			cp.table.Rows = append(cp.table.Rows, row)
			return nil, true
			return nil
		}
		// inp.Ch must be '|'
	}
}

// parseCell parses one single cell of a table row.
func (cp *zmkP) parseCell() *ast.TableCell {
	inp := cp.inp
	var slice ast.InlineSlice
	var l []ast.InlineNode
	for {
		if input.IsEOLEOS(inp.Ch) {
			if len(slice) == 0 {
			if len(l) == 0 {
				return nil
			}
			return &ast.TableCell{Inlines: slice}
			return &ast.TableCell{Inlines: ast.CreateInlineListNode(l...)}
		}
		if inp.Ch == '|' {
			return &ast.TableCell{Inlines: slice}
			return &ast.TableCell{Inlines: ast.CreateInlineListNode(l...)}
		}
		slice = append(slice, cp.parseInline())
		l = append(l, cp.parseInline())
	}
}

Changes to parser/zettelmark/inline.go.

15
16
17
18
19
20
21
22
23


24
25

26
27
28
29

30
31
32
33

34
35
36
37
38
39
40
15
16
17
18
19
20
21


22
23
24

25
26
27
28

29
30
31
32

33
34
35
36
37
38
39
40







-
-
+
+

-
+



-
+



-
+







	"fmt"
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/input"
)

// parseInlineSlice parses a sequence of Inlines until EOS.
func (cp *zmkP) parseInlineSlice() ast.InlineSlice {
// parseInlineList parses a sequence of Inlines until EOS.
func (cp *zmkP) parseInlineList() *ast.InlineListNode {
	inp := cp.inp
	var ins ast.InlineSlice
	var ins []ast.InlineNode
	for inp.Ch != input.EOS {
		in := cp.parseInline()
		if in == nil {
			return ins
			break
		}
		ins = append(ins, in)
	}
	return ins
	return ast.CreateInlineListNode(ins...)
}

func (cp *zmkP) parseInline() ast.InlineNode {
	inp := cp.inp
	pos := inp.Pos
	if cp.nestingLevel <= maxNestingLevel {
		cp.nestingLevel++
60
61
62
63
64
65
66
67

68
69
70
71
72
73
74
60
61
62
63
64
65
66

67
68
69
70
71
72
73
74







-
+







				in, success = cp.parseFootnote()
			case '!':
				in, success = cp.parseMark()
			}
		case '{':
			inp.Next()
			if inp.Ch == '{' {
				in, success = cp.parseImage()
				in, success = cp.parseEmbed()
			}
		case '#':
			return cp.parseTag()
		case '%':
			in, success = cp.parseComment()
		case '/', '*', '_', '~', '\'', '^', ',', '<', '"', ';', ':':
			in, success = cp.parseFormat()
152
153
154
155
156
157
158
159

160
161
162
163
164
165


166
167
168
169
170

171
172
173
174
175
176
177
178
179

180
181
182
183
184
185
186
187

188
189
190
191
192
193
194
152
153
154
155
156
157
158

159
160
161
162
163


164
165
166
167
168
169

170
171
172
173
174
175
176
177
178

179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195







-
+




-
-
+
+




-
+








-
+








+








func (cp *zmkP) parseSoftBreak() *ast.BreakNode {
	cp.inp.EatEOL()
	return &ast.BreakNode{}
}

func (cp *zmkP) parseLink() (*ast.LinkNode, bool) {
	if ref, ins, ok := cp.parseReference(']'); ok {
	if ref, iln, ok := cp.parseReference(']'); ok {
		attrs := cp.parseAttributes(false)
		if len(ref) > 0 {
			onlyRef := false
			r := ast.ParseReference(ref)
			if ins == nil {
				ins = ast.InlineSlice{&ast.TextNode{Text: ref}}
			if iln == nil {
				iln = ast.CreateInlineListNode(&ast.TextNode{Text: ref})
				onlyRef = true
			}
			return &ast.LinkNode{
				Ref:     r,
				Inlines: ins,
				Inlines: iln,
				OnlyRef: onlyRef,
				Attrs:   attrs,
			}, true
		}
	}
	return nil, false
}

func (cp *zmkP) parseReference(closeCh rune) (ref string, ins ast.InlineSlice, ok bool) {
func (cp *zmkP) parseReference(closeCh rune) (ref string, iln *ast.InlineListNode, ok bool) {
	inp := cp.inp
	inp.Next()
	cp.skipSpace()
	pos := inp.Pos
	hasSpace, ok := cp.readReferenceToSep(closeCh)
	if !ok {
		return "", nil, false
	}
	var ins []ast.InlineNode
	if inp.Ch == '|' { // First part must be inline text
		if pos == inp.Pos { // [[| or {{|
			return "", nil, false
		}
		cp.inp = input.NewInput(inp.Src[pos:inp.Pos])
		for {
			in := cp.parseInline()
212
213
214
215
216
217
218

219



220
221
222
223
224
225
226
213
214
215
216
217
218
219
220

221
222
223
224
225
226
227
228
229
230







+
-
+
+
+







	}
	ref = inp.Src[pos:inp.Pos]
	inp.Next()
	if inp.Ch != closeCh {
		return "", nil, false
	}
	inp.Next()
	if len(ins) == 0 {
	return ref, ins, true
		return ref, nil, true
	}
	return ref, ast.CreateInlineListNode(ins...), true
}

func (cp *zmkP) readReferenceToSep(closeCh rune) (bool, bool) {
	hasSpace := false
	inp := cp.inp
	for {
		switch inp.Ch {
303
304
305
306
307
308
309
310

311
312
313
314



315

316
317
318

319
320

321
322
323
324
325
326
327
328
329
330
331
332

333
334
335
336
337







338
339
340
341





342
343
344
345
346
347
348
307
308
309
310
311
312
313

314
315
316
317
318
319
320
321

322
323
324

325
326

327
328
329
330
331
332
333
334
335
336
337
338
339
340





341
342
343
344
345
346
347
348
349
350

351
352
353
354
355
356
357
358
359
360
361
362







-
+




+
+
+
-
+


-
+

-
+












+
-
-
-
-
-
+
+
+
+
+
+
+



-
+
+
+
+
+







	}
	attrs := cp.parseAttributes(false)
	return &ast.CiteNode{Key: inp.Src[pos:posL], Inlines: ins, Attrs: attrs}, true
}

func (cp *zmkP) parseFootnote() (*ast.FootnoteNode, bool) {
	cp.inp.Next()
	ins, ok := cp.parseLinkLikeRest()
	iln, ok := cp.parseLinkLikeRest()
	if !ok {
		return nil, false
	}
	attrs := cp.parseAttributes(false)
	if iln == nil {
		iln = &ast.InlineListNode{}
	}
	return &ast.FootnoteNode{Inlines: ins, Attrs: attrs}, true
	return &ast.FootnoteNode{Inlines: iln, Attrs: attrs}, true
}

func (cp *zmkP) parseLinkLikeRest() (ast.InlineSlice, bool) {
func (cp *zmkP) parseLinkLikeRest() (*ast.InlineListNode, bool) {
	cp.skipSpace()
	var ins ast.InlineSlice
	var ins []ast.InlineNode
	inp := cp.inp
	for inp.Ch != ']' {
		in := cp.parseInline()
		if in == nil {
			return nil, false
		}
		ins = append(ins, in)
		if _, ok := in.(*ast.BreakNode); ok && input.IsEOLEOS(inp.Ch) {
			return nil, false
		}
	}
	inp.Next()
	if len(ins) == 0 {
	return ins, true
}

func (cp *zmkP) parseImage() (ast.InlineNode, bool) {
	if ref, ins, ok := cp.parseReference('}'); ok {
		return nil, true
	}
	return ast.CreateInlineListNode(ins...), true
}

func (cp *zmkP) parseEmbed() (ast.InlineNode, bool) {
	if ref, iln, ok := cp.parseReference('}'); ok {
		attrs := cp.parseAttributes(false)
		if len(ref) > 0 {
			r := ast.ParseReference(ref)
			return &ast.ImageNode{Ref: r, Inlines: ins, Attrs: attrs}, true
			return &ast.EmbedNode{
				Material: &ast.ReferenceMaterialNode{Ref: r},
				Inlines:  iln,
				Attrs:    attrs,
			}, true
		}
	}
	return nil, false
}

func (cp *zmkP) parseMark() (*ast.MarkNode, bool) {
	inp := cp.inp
414
415
416
417
418
419
420
421

422
423
424
425
426
427
428
429
430
431
432
433

434
435
436
437
438

439
440
441
442
443
444
445
428
429
430
431
432
433
434

435
436
437
438
439
440
441
442
443
444
445
446

447
448
449
450
451

452
453
454
455
456
457
458
459







-
+











-
+




-
+







		panic(fmt.Sprintf("%q is not a formatting char", fch))
	}
	inp.Next() // read 2nd formatting character
	if inp.Ch != fch {
		return nil, false
	}
	inp.Next()
	fn := &ast.FormatNode{Kind: kind}
	fn := &ast.FormatNode{Kind: kind, Inlines: &ast.InlineListNode{}}
	for {
		if inp.Ch == input.EOS {
			return nil, false
		}
		if inp.Ch == fch {
			inp.Next()
			if inp.Ch == fch {
				inp.Next()
				fn.Attrs = cp.parseAttributes(false)
				return fn, true
			}
			fn.Inlines = append(fn.Inlines, &ast.TextNode{Text: string(fch)})
			fn.Inlines.Append(&ast.TextNode{Text: string(fch)})
		} else if in := cp.parseInline(); in != nil {
			if _, ok := in.(*ast.BreakNode); ok && input.IsEOLEOS(inp.Ch) {
				return nil, false
			}
			fn.Inlines = append(fn.Inlines, in)
			fn.Inlines.Append(in)
		}
	}
}

var mapRuneLiteral = map[rune]ast.LiteralKind{
	'`':          ast.LiteralProg,
	runeModGrave: ast.LiteralProg,

Changes to parser/zettelmark/post-processor.go.

14
15
16
17
18
19
20
21

22
23

24
25
26
27

28
29

30
31
32
33
34
35
36
37
38




39
40

41
42
43
44


45
46
47
48
49
50

51
52
53

54
55
56

57
58
59
60
61

62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

80
81
82
83



84
85

86
87

88





89
90
91
92
93
94
95
96
97
98
99












































100
101
102
103


104
105
106

107
108
109

110
111
112
113
114
115
116
14
15
16
17
18
19
20

21
22

23
24
25
26

27
28

29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45



46
47




48

49
50


51

52

53





54
55

















56
57



58
59
60
61

62
63

64
65
66
67
68
69
70











71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116


117
118
119
120

121
122
123

124
125
126
127
128
129
130
131







-
+

-
+



-
+

-
+









+
+
+
+

-
+

-
-
-
+
+
-
-
-
-

-
+

-
-
+
-

-
+
-
-
-
-
-
+

-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+

-
-
-
+
+
+

-
+

-
+

+
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+


-
-
+
+


-
+


-
+







import (
	"strings"

	"zettelstore.de/z/ast"
)

// postProcessBlocks is the entry point for post-processing a list of block nodes.
func postProcessBlocks(bs ast.BlockSlice) ast.BlockSlice {
func postProcessBlocks(bs *ast.BlockListNode) {
	pp := postProcessor{}
	return pp.processBlockSlice(bs)
	ast.Walk(&pp, bs)
}

// postProcessInlines is the entry point for post-processing a list of inline nodes.
func postProcessInlines(is ast.InlineSlice) ast.InlineSlice {
func postProcessInlines(iln *ast.InlineListNode) {
	pp := postProcessor{}
	return pp.processInlineSlice(is)
	ast.Walk(&pp, iln)
}

// postProcessor is a visitor that cleans the abstract syntax tree.
type postProcessor struct {
	inVerse bool
}

func (pp *postProcessor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.BlockListNode:
		pp.visitBlockList(n)
	case *ast.InlineListNode:
		pp.visitInlineList(n)
	case *ast.ParaNode:
		n.Inlines = pp.processInlineSlice(n.Inlines)
		return pp
	case *ast.RegionNode:
		oldVerse := pp.inVerse
		if n.Kind == ast.RegionVerse {
			pp.inVerse = true
		pp.visitRegion(n)
		return pp
		}
		n.Blocks = pp.processBlockSlice(n.Blocks)
		pp.inVerse = oldVerse
		n.Inlines = pp.processInlineSlice(n.Inlines)
	case *ast.HeadingNode:
		n.Inlines = pp.processInlineSlice(n.Inlines)
		return pp
	case *ast.NestedListNode:
		for i, item := range n.Items {
			n.Items[i] = pp.processItemSlice(item)
		pp.visitNestedList(n)
		}
	case *ast.DescriptionListNode:
		for i, def := range n.Descriptions {
		pp.visitDescriptionList(n)
			n.Descriptions[i].Term = pp.processInlineSlice(def.Term)
			for j, b := range def.Descriptions {
				n.Descriptions[i].Descriptions[j] = pp.processDescriptionSlice(b)
			}
		}
		return pp
	case *ast.TableNode:
		width := tableWidth(n)
		n.Align = make([]ast.Alignment, width)
		for i := 0; i < width; i++ {
			n.Align[i] = ast.AlignDefault
		}
		if len(n.Rows) > 0 && isHeaderRow(n.Rows[0]) {
			n.Header = n.Rows[0]
			n.Rows = n.Rows[1:]
			pp.visitTableHeader(n)
		}
		if len(n.Header) > 0 {
			n.Header = appendCells(n.Header, width, n.Align)
			for i, cell := range n.Header {
				pp.processCell(cell, n.Align[i])
			}
		}
		pp.visitTableRows(n, width)
		pp.visitTable(n)
	case *ast.LinkNode:
		n.Inlines = pp.processInlineSlice(n.Inlines)
	case *ast.ImageNode:
		n.Inlines = pp.processInlineSlice(n.Inlines)
		return pp
	case *ast.EmbedNode:
		return pp
	case *ast.CiteNode:
		n.Inlines = pp.processInlineSlice(n.Inlines)
		return pp
	case *ast.FootnoteNode:
		n.Inlines = pp.processInlineSlice(n.Inlines)
		return pp
	case *ast.FormatNode:
		pp.visitFormat(n)
		return pp
	}
	return nil
}
		if n.Attrs != nil && n.Attrs.HasDefault() {
			if newKind, ok := mapSemantic[n.Kind]; ok {
				n.Attrs.RemoveDefault()
				n.Kind = newKind
			}
		}
		n.Inlines = pp.processInlineSlice(n.Inlines)
	}
	return nil
}


func (pp *postProcessor) visitRegion(rn *ast.RegionNode) {
	oldVerse := pp.inVerse
	if rn.Kind == ast.RegionVerse {
		pp.inVerse = true
	}
	pp.visitBlockList(rn.Blocks)
	pp.inVerse = oldVerse
}

func (pp *postProcessor) visitNestedList(ln *ast.NestedListNode) {
	for i, item := range ln.Items {
		ln.Items[i] = pp.processItemSlice(item)
	}
}

func (pp *postProcessor) visitDescriptionList(dn *ast.DescriptionListNode) {
	for i, def := range dn.Descriptions {
		for j, b := range def.Descriptions {
			dn.Descriptions[i].Descriptions[j] = pp.processDescriptionSlice(b)
		}
	}
}

func (pp *postProcessor) visitTable(tn *ast.TableNode) {
	width := tableWidth(tn)
	tn.Align = make([]ast.Alignment, width)
	for i := 0; i < width; i++ {
		tn.Align[i] = ast.AlignDefault
	}
	if len(tn.Rows) > 0 && isHeaderRow(tn.Rows[0]) {
		tn.Header = tn.Rows[0]
		tn.Rows = tn.Rows[1:]
		pp.visitTableHeader(tn)
	}
	if len(tn.Header) > 0 {
		tn.Header = appendCells(tn.Header, width, tn.Align)
		for i, cell := range tn.Header {
			pp.processCell(cell, tn.Align[i])
		}
	}
	pp.visitTableRows(tn, width)
}

func (pp *postProcessor) visitTableHeader(tn *ast.TableNode) {
	for pos, cell := range tn.Header {
		inlines := cell.Inlines
		if len(inlines) == 0 {
		ins := cell.Inlines.List
		if len(ins) == 0 {
			continue
		}
		if textNode, ok := inlines[0].(*ast.TextNode); ok {
		if textNode, ok := ins[0].(*ast.TextNode); ok {
			textNode.Text = strings.TrimPrefix(textNode.Text, "=")
		}
		if textNode, ok := inlines[len(inlines)-1].(*ast.TextNode); ok {
		if textNode, ok := ins[len(ins)-1].(*ast.TextNode); ok {
			if tnl := len(textNode.Text); tnl > 0 {
				if align := getAlignment(textNode.Text[tnl-1]); align != ast.AlignDefault {
					tn.Align[pos] = align
					textNode.Text = textNode.Text[0 : tnl-1]
				}
			}
		}
136
137
138
139
140
141
142
143




144
145
146
147
148

149
150


151
152
153
154
155
156
157
151
152
153
154
155
156
157

158
159
160
161
162
163
164
165
166
167


168
169
170
171
172
173
174
175
176







-
+
+
+
+





+
-
-
+
+







		}
	}
	return width
}

func appendCells(row ast.TableRow, width int, colAlign []ast.Alignment) ast.TableRow {
	for len(row) < width {
		row = append(row, &ast.TableCell{Align: colAlign[len(row)]})
		row = append(row, &ast.TableCell{
			Align:   colAlign[len(row)],
			Inlines: &ast.InlineListNode{},
		})
	}
	return row
}

func isHeaderRow(row ast.TableRow) bool {
	for _, cell := range row {
	for i := 0; i < len(row); i++ {
		if inlines := row[i].Inlines; len(inlines) > 0 {
		iln := cell.Inlines
		if inlines := iln.List; len(inlines) > 0 {
			if textNode, ok := inlines[0].(*ast.TextNode); ok {
				if strings.HasPrefix(textNode.Text, "=") {
					return true
				}
			}
		}
	}
169
170
171
172
173
174
175
176
177


178
179
180


181
182
183
184

185
186
187
188
189


190









191
192
193
194
195
196
197
198
199
200
201
202
203
204
















205

206


207
208
209


210
211

212
213

214
215
216
217
218
219
220
221
222
223


224
225


226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243

244
245
246
247
248
249
250
188
189
190
191
192
193
194


195
196



197
198
199
200
201

202
203
204
205
206
207
208
209

210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227





228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245

246
247
248


249
250
251

252
253

254
255
256
257
258
259
260
261
262


263
264
265

266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284

285
286
287
288
289
290
291
292







-
-
+
+
-
-
-
+
+



-
+





+
+
-
+
+
+
+
+
+
+
+
+









-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

+
-
+
+

-
-
+
+

-
+

-
+








-
-
+
+

-
+
+

















-
+







	default:
		return ast.AlignDefault
	}
}

// processCell tries to recognize cell formatting.
func (pp *postProcessor) processCell(cell *ast.TableCell, colAlign ast.Alignment) {
	if len(cell.Inlines) == 0 {
		return
	iln := cell.Inlines
	ins := iln.List
	}
	if textNode, ok := cell.Inlines[0].(*ast.TextNode); ok && len(textNode.Text) > 0 {
		align := getAlignment(textNode.Text[0])
	if tn := initialText(ins); tn != nil {
		align := getAlignment(tn.Text[0])
		if align == ast.AlignDefault {
			cell.Align = colAlign
		} else {
			textNode.Text = textNode.Text[1:]
			tn.Text = tn.Text[1:]
			cell.Align = align
		}
	} else {
		cell.Align = colAlign
	}
	ast.Walk(pp, iln)
}
	cell.Inlines = pp.processInlineSlice(cell.Inlines)

func initialText(ins []ast.InlineNode) *ast.TextNode {
	if len(ins) == 0 {
		return nil
	}
	if tn, ok := ins[0].(*ast.TextNode); ok && len(tn.Text) > 0 {
		return tn
	}
	return nil
}

var mapSemantic = map[ast.FormatKind]ast.FormatKind{
	ast.FormatItalic: ast.FormatEmph,
	ast.FormatBold:   ast.FormatStrong,
	ast.FormatUnder:  ast.FormatInsert,
	ast.FormatStrike: ast.FormatDelete,
}

// processBlockSlice post-processes a slice of blocks.
// It is one of the working horses for post-processing.
func (pp *postProcessor) processBlockSlice(bns ast.BlockSlice) ast.BlockSlice {
	if len(bns) == 0 {
		return nil
func (pp *postProcessor) visitFormat(fn *ast.FormatNode) {
	if fn.Attrs.HasDefault() {
		if newKind, ok := mapSemantic[fn.Kind]; ok {
			fn.Attrs.RemoveDefault()
			fn.Kind = newKind
		}
	}
}

func (pp *postProcessor) visitBlockList(bln *ast.BlockListNode) {
	if bln == nil {
		return
	}
	if len(bln.List) == 0 {
		bln.List = nil
		return
	}
	for _, bn := range bln.List {
	ast.WalkBlockSlice(pp, bns)
		ast.Walk(pp, bn)
	}
	fromPos, toPos := 0, 0
	for fromPos < len(bns) {
		bns[toPos] = bns[fromPos]
	for fromPos < len(bln.List) {
		bln.List[toPos] = bln.List[fromPos]
		fromPos++
		switch bn := bns[toPos].(type) {
		switch bn := bln.List[toPos].(type) {
		case *ast.ParaNode:
			if len(bn.Inlines) > 0 {
			if len(bn.Inlines.List) > 0 {
				toPos++
			}
		case *nullItemNode:
		case *nullDescriptionNode:
		default:
			toPos++
		}
	}
	for pos := toPos; pos < len(bns); pos++ {
		bns[pos] = nil // Allow excess nodes to be garbage collected.
	for pos := toPos; pos < len(bln.List); pos++ {
		bln.List[pos] = nil // Allow excess nodes to be garbage collected.
	}
	return bns[:toPos:toPos]
	bln.List = bln.List[:toPos:toPos]

}

// processItemSlice post-processes a slice of items.
// It is one of the working horses for post-processing.
func (pp *postProcessor) processItemSlice(ins ast.ItemSlice) ast.ItemSlice {
	if len(ins) == 0 {
		return nil
	}
	for _, in := range ins {
		ast.Walk(pp, in)
	}
	fromPos, toPos := 0, 0
	for fromPos < len(ins) {
		ins[toPos] = ins[fromPos]
		fromPos++
		switch in := ins[toPos].(type) {
		case *ast.ParaNode:
			if in != nil && len(in.Inlines) > 0 {
			if in != nil && len(in.Inlines.List) > 0 {
				toPos++
			}
		case *nullItemNode:
		case *nullDescriptionNode:
		default:
			toPos++
		}
266
267
268
269
270
271
272
273

274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291







292

293
294



295
296

297
298
299
300
301




302
303
304
305
306
307
308




309
310
311

312

313
314

315

316
317
318

319
320
321
322
323
324
325
326
327


328
329
330

331
332
333
334
335
336
337
338
339
340
341

342
343

344
345
346
347
348
349
350
308
309
310
311
312
313
314

315
316
317
318
319
320
321
322
323
324
325
326
327
328





329
330
331
332
333
334
335
336
337


338
339
340
341

342
343




344
345
346
347

348
349
350



351
352
353
354
355
356
357
358

359
360
361
362

363
364
365

366
367
368
369
370
371
372
373
374

375
376
377
378

379
380
381
382
383
384
385
386
387
388
389

390


391
392
393
394
395
396
397
398







-
+













-
-
-
-
-
+
+
+
+
+
+
+

+
-
-
+
+
+

-
+

-
-
-
-
+
+
+
+
-



-
-
-
+
+
+
+



+
-
+


+
-
+


-
+








-
+
+


-
+










-
+
-
-
+







	}
	fromPos, toPos := 0, 0
	for fromPos < len(dns) {
		dns[toPos] = dns[fromPos]
		fromPos++
		switch dn := dns[toPos].(type) {
		case *ast.ParaNode:
			if len(dn.Inlines) > 0 {
			if len(dn.Inlines.List) > 0 {
				toPos++
			}
		case *nullDescriptionNode:
		default:
			toPos++
		}
	}
	for pos := toPos; pos < len(dns); pos++ {
		dns[pos] = nil // Allow excess nodes to be garbage collected.
	}
	return dns[:toPos:toPos]
}

// processInlineSlice post-processes a slice of inline nodes.
// It is one of the working horses for post-processing.
func (pp *postProcessor) processInlineSlice(ins ast.InlineSlice) ast.InlineSlice {
	if len(ins) == 0 {
		return nil
func (pp *postProcessor) visitInlineList(iln *ast.InlineListNode) {
	if iln == nil {
		return
	}
	if len(iln.List) == 0 {
		iln.List = nil
		return
	}
	for _, in := range iln.List {
	ast.WalkInlineSlice(pp, ins)

		ast.Walk(pp, in)
	}

	if !pp.inVerse {
		ins = processInlineSliceHead(ins)
		processInlineSliceHead(iln)
	}
	toPos := pp.processInlineSliceCopy(ins)
	toPos = pp.processInlineSliceTail(ins, toPos)
	ins = ins[:toPos:toPos]
	pp.processInlineSliceInplace(ins)
	toPos := pp.processInlineSliceCopy(iln)
	toPos = pp.processInlineSliceTail(iln, toPos)
	iln.List = iln.List[:toPos:toPos]
	pp.processInlineListInplace(iln)
	return ins
}

// processInlineSliceHead removes leading spaces and empty text.
func processInlineSliceHead(ins ast.InlineSlice) ast.InlineSlice {
	for i := 0; i < len(ins); i++ {
		switch in := ins[i].(type) {
func processInlineSliceHead(iln *ast.InlineListNode) {
	ins := iln.List
	for i, in := range ins {
		switch in := in.(type) {
		case *ast.SpaceNode:
		case *ast.TextNode:
			if len(in.Text) > 0 {
				iln.List = ins[i:]
				return ins[i:]
				return
			}
		default:
			iln.List = ins[i:]
			return ins[i:]
			return
		}
	}
	return ins[0:0]
	iln.List = ins[0:0]
}

// processInlineSliceCopy goes forward through the slice and tries to eliminate
// elements that follow the current element.
//
// Two text nodes are merged into one.
//
// Two spaces following a break are merged into a hard break.
func (pp *postProcessor) processInlineSliceCopy(ins ast.InlineSlice) int {
func (pp *postProcessor) processInlineSliceCopy(iln *ast.InlineListNode) int {
	ins := iln.List
	maxPos := len(ins)
	for {
		again, toPos := pp.processInlineSliceCopyLoop(ins, maxPos)
		again, toPos := pp.processInlineSliceCopyLoop(iln, maxPos)
		for pos := toPos; pos < maxPos; pos++ {
			ins[pos] = nil // Allow excess nodes to be garbage collected.
		}
		if !again {
			return toPos
		}
		maxPos = toPos
	}
}

func (pp *postProcessor) processInlineSliceCopyLoop(
func (pp *postProcessor) processInlineSliceCopyLoop(iln *ast.InlineListNode, maxPos int) (bool, int) {
	ins ast.InlineSlice, maxPos int) (bool, int) {

	ins := iln.List
	again := false
	fromPos, toPos := 0, 0
	for fromPos < maxPos {
		ins[toPos] = ins[fromPos]
		fromPos++
		switch in := ins[toPos].(type) {
		case *ast.TextNode:
380
381
382
383
384
385
386
387


388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406


407
408
409
410
411
412
413
414
415
428
429
430
431
432
433
434

435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453


454
455
456
457
458
459
460
461
462
463
464







-
+
+

















-
-
+
+









		}
		toPos++
	}
	return again, toPos
}

// processInlineSliceTail removes empty text nodes, breaks and spaces at the end.
func (pp *postProcessor) processInlineSliceTail(ins ast.InlineSlice, toPos int) int {
func (pp *postProcessor) processInlineSliceTail(iln *ast.InlineListNode, toPos int) int {
	ins := iln.List
	for toPos > 0 {
		switch n := ins[toPos-1].(type) {
		case *ast.TextNode:
			if len(n.Text) > 0 {
				return toPos
			}
		case *ast.BreakNode:
		case *ast.SpaceNode:
		default:
			return toPos
		}
		toPos--
		ins[toPos] = nil // Kill node to enable garbage collection
	}
	return toPos
}

func (pp *postProcessor) processInlineSliceInplace(ins ast.InlineSlice) {
	for _, in := range ins {
func (pp *postProcessor) processInlineListInplace(iln *ast.InlineListNode) {
	for _, in := range iln.List {
		if n, ok := in.(*ast.TextNode); ok {
			if n.Text == "..." {
				n.Text = "\u2026"
			} else if len(n.Text) == 4 && strings.IndexByte(",;:!?", n.Text[3]) >= 0 && n.Text[:3] == "..." {
				n.Text = "\u2026" + n.Text[3:]
			}
		}
	}
}

Changes to parser/zettelmark/zettelmark.go.

18
19
20
21
22
23
24
25
26
27
28






29
30
31
32

33
34
35



36
37
38

39
40
41



42
43
44
45
46
47
48
18
19
20
21
22
23
24




25
26
27
28
29
30
31
32
33

34
35


36
37
38
39
40

41
42


43
44
45
46
47
48
49
50
51
52







-
-
-
-
+
+
+
+
+
+



-
+

-
-
+
+
+


-
+

-
-
+
+
+







	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

func init() {
	parser.Register(&parser.Info{
		Name:         meta.ValueSyntaxZmk,
		AltNames:     nil,
		ParseBlocks:  parseBlocks,
		ParseInlines: parseInlines,
		Name:          meta.ValueSyntaxZmk,
		AltNames:      nil,
		IsTextParser:  true,
		IsImageFormat: false,
		ParseBlocks:   parseBlocks,
		ParseInlines:  parseInlines,
	})
}

func parseBlocks(inp *input.Input, m *meta.Meta, syntax string) ast.BlockSlice {
func parseBlocks(inp *input.Input, _ *meta.Meta, _ string) *ast.BlockListNode {
	parser := &zmkP{inp: inp}
	bs := parser.parseBlockSlice()
	return postProcessBlocks(bs)
	bns := parser.parseBlockList()
	postProcessBlocks(bns)
	return bns
}

func parseInlines(inp *input.Input, syntax string) ast.InlineSlice {
func parseInlines(inp *input.Input, _ string) *ast.InlineListNode {
	parser := &zmkP{inp: inp}
	is := parser.parseInlineSlice()
	return postProcessInlines(is)
	iln := parser.parseInlineList()
	postProcessInlines(iln)
	return iln
}

type zmkP struct {
	inp          *input.Input             // Input stream
	lists        []*ast.NestedListNode    // Stack of lists
	table        *ast.TableNode           // Current table
	descrl       *ast.DescriptionListNode // Current description list

Changes to parser/zettelmark/zettelmark_test.go.

46
47
48
49
50
51
52
53

54
55
56
57
58
59
60
46
47
48
49
50
51
52

53
54
55
56
57
58
59
60







-
+








	for tcn, tc := range tcs {
		t.Run(fmt.Sprintf("TC=%02d,src=%q", tcn, tc.source), func(st *testing.T) {
			st.Helper()
			inp := input.NewInput(tc.source)
			bns := parser.ParseBlocks(inp, nil, meta.ValueSyntaxZmk)
			var tv TestVisitor
			ast.WalkBlockSlice(&tv, bns)
			ast.Walk(&tv, bns)
			got := tv.String()
			if tc.want != got {
				st.Errorf("\nwant=%q\n got=%q", tc.want, got)
			}
		})
	}
}
205
206
207
208
209
210
211
212

213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228



229
230
231
232
233


234
235
236
237
238
239
240
241
242
243
244
245
246
247













248
249
250
251
252
253
254
205
206
207
208
209
210
211

212
213
214
215
216
217
218
219
220
221
222
223
224
225



226
227
228
229
230
231


232
233
234













235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254







-
+













-
-
-
+
+
+



-
-
+
+

-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+







		{"[^abc\ndef]", "(PARA (FN abc SB def))"},
		{"[^abc\n\ndef]", "(PARA [^abc)(PARA def])"},
		{"[^abc[^def]]", "(PARA (FN abc (FN def)))"},
		{"[^abc]{-}", "(PARA (FN abc)[ATTR -])"},
	})
}

func TestImage(t *testing.T) {
func TestEmbed(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"{", "(PARA {)"},
		{"{{", "(PARA {{)"},
		{"{{|", "(PARA {{|)"},
		{"{{}", "(PARA {{})"},
		{"{{|}", "(PARA {{|})"},
		{"{{}}", "(PARA {{}})"},
		{"{{|}}", "(PARA {{|}})"},
		{"{{ }}", "(PARA {{ SP }})"},
		{"{{\n}}", "(PARA {{ SB }})"},
		{"{{a }}", "(PARA {{a SP }})"},
		{"{{a\n}}", "(PARA {{a SB }})"},
		{"{{a}}", "(PARA (IMAGE a))"},
		{"{{12345678901234}}", "(PARA (IMAGE 12345678901234))"},
		{"{{ a}}", "(PARA (IMAGE a))"},
		{"{{a}}", "(PARA (EMBED a))"},
		{"{{12345678901234}}", "(PARA (EMBED 12345678901234))"},
		{"{{ a}}", "(PARA (EMBED a))"},
		{"{{a}", "(PARA {{a})"},
		{"{{|a}}", "(PARA {{|a}})"},
		{"{{b|}}", "(PARA {{b|}})"},
		{"{{b|a}}", "(PARA (IMAGE a b))"},
		{"{{b| a}}", "(PARA (IMAGE a b))"},
		{"{{b|a}}", "(PARA (EMBED a b))"},
		{"{{b| a}}", "(PARA (EMBED a b))"},
		{"{{b|a}", "(PARA {{b|a})"},
		{"{{b\nc|a}}", "(PARA (IMAGE a b SB c))"},
		{"{{b c|a#n}}", "(PARA (IMAGE a#n b SP c))"},
		{"{{a}}{go}", "(PARA (IMAGE a)[ATTR go])"},
		{"{{{{a}}|b}}", "(PARA (IMAGE %7B%7Ba) |b}})"},
		{"{{\\|}}", "(PARA (IMAGE %5C%7C))"},
		{"{{\\||a}}", "(PARA (IMAGE a |))"},
		{"{{b\\||a}}", "(PARA (IMAGE a b|))"},
		{"{{b\\|c|a}}", "(PARA (IMAGE a b|c))"},
		{"{{\\}}}", "(PARA (IMAGE %5C%7D))"},
		{"{{\\}|a}}", "(PARA (IMAGE a }))"},
		{"{{b\\}|a}}", "(PARA (IMAGE a b}))"},
		{"{{\\}\\||a}}", "(PARA (IMAGE a }|))"},
		{"{{http://a|http://a}}", "(PARA (IMAGE http://a http://a))"},
		{"{{b\nc|a}}", "(PARA (EMBED a b SB c))"},
		{"{{b c|a#n}}", "(PARA (EMBED a#n b SP c))"},
		{"{{a}}{go}", "(PARA (EMBED a)[ATTR go])"},
		{"{{{{a}}|b}}", "(PARA (EMBED %7B%7Ba) |b}})"},
		{"{{\\|}}", "(PARA (EMBED %5C%7C))"},
		{"{{\\||a}}", "(PARA (EMBED a |))"},
		{"{{b\\||a}}", "(PARA (EMBED a b|))"},
		{"{{b\\|c|a}}", "(PARA (EMBED a b|c))"},
		{"{{\\}}}", "(PARA (EMBED %5C%7D))"},
		{"{{\\}|a}}", "(PARA (EMBED a }))"},
		{"{{b\\}|a}}", "(PARA (EMBED a b}))"},
		{"{{\\}\\||a}}", "(PARA (EMBED a }|))"},
		{"{{http://a|http://a}}", "(PARA (EMBED http://a http://a))"},
	})
}

func TestTag(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"#", "(PARA #)"},
262
263
264
265
266
267
268
269


270
271


272
273
274



275
276
277
278
279
280
281
282
283
262
263
264
265
266
267
268

269
270
271

272
273
274


275
276
277


278
279
280
281
282
283
284







-
+
+

-
+
+

-
-
+
+
+
-
-







}

func TestMark(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[!", "(PARA [!)"},
		{"[!\n", "(PARA [!)"},
		{"[!]", "(PARA (MARK *))"},
		{"[!]", "(PARA (MARK #*))"},
		{"[!][!]", "(PARA (MARK #*) (MARK #*-1))"},
		{"[! ]", "(PARA [! SP ])"},
		{"[!a]", "(PARA (MARK a))"},
		{"[!a]", "(PARA (MARK \"a\" #a))"},
		{"[!a][!a]", "(PARA (MARK \"a\" #a) (MARK \"a\" #a-1))"},
		{"[!a ]", "(PARA [!a SP ])"},
		{"[!a_]", "(PARA (MARK a_))"},
		{"[!a-b]", "(PARA (MARK a-b))"},
		{"[!a_]", "(PARA (MARK \"a_\" #a))"},
		{"[!a_][!a]", "(PARA (MARK \"a_\" #a) (MARK \"a\" #a-1))"},
		{"[!a-b]", "(PARA (MARK \"a-b\" #a-b))"},
		{"[!a][!a]", "(PARA (MARK a) (MARK a-1))"},
		{"[!][!]", "(PARA (MARK *) (MARK *-1))"},
	})
}

func TestComment(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"%", "(PARA %)"},
452
453
454
455
456
457
458
459
460
461
462
463
464
465







466
467

468
469
470
471
472










473
474
475
476
477
478
479
453
454
455
456
457
458
459







460
461
462
463
464
465
466
467

468
469
470
471


472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488







-
-
-
-
-
-
-
+
+
+
+
+
+
+

-
+



-
-
+
+
+
+
+
+
+
+
+
+







	t.Parallel()
	checkTcs(t, TestCases{
		{"=h", "(PARA =h)"},
		{"= h", "(PARA = SP h)"},
		{"==h", "(PARA ==h)"},
		{"== h", "(PARA == SP h)"},
		{"===h", "(PARA ===h)"},
		{"=== h", "(H2 h)"},
		{"===  h", "(H2 h)"},
		{"==== h", "(H3 h)"},
		{"===== h", "(H4 h)"},
		{"====== h", "(H5 h)"},
		{"======= h", "(H6 h)"},
		{"======== h", "(H6 h)"},
		{"=== h", "(H2 h #h)"},
		{"===  h", "(H2 h #h)"},
		{"==== h", "(H3 h #h)"},
		{"===== h", "(H4 h #h)"},
		{"====== h", "(H5 h #h)"},
		{"======= h", "(H6 h #h)"},
		{"======== h", "(H6 h #h)"},
		{"=", "(PARA =)"},
		{"=== h=//=a//", "(H2 h= {/ =a})"},
		{"=== h=//=a//", "(H2 h= {/ =a} #h-a)"},
		{"=\n", "(PARA =)"},
		{"a=", "(PARA a=)"},
		{" =", "(PARA =)"},
		{"=== h\na", "(H2 h)(PARA a)"},
		{"=== h i {-}", "(H2 h SP i)[ATTR -]"},
		{"=== h\na", "(H2 h #h)(PARA a)"},
		{"=== h i {-}", "(H2 h SP i #h-i)[ATTR -]"},
		{"=== h {{a}}", "(H2 h SP (EMBED a) #h)"},
		{"=== h{{a}}", "(H2 h (EMBED a) #h)"},
		{"=== {{a}}", "(H2 (EMBED a))"},
		{"=== h {{a}}{-}", "(H2 h SP (EMBED a)[ATTR -] #h)"},
		{"=== h {{a}} {-}", "(H2 h SP (EMBED a) #h)[ATTR -]"},
		{"=== h {-}{{a}}", "(H2 h #h)[ATTR -]"},
		{"=== h{id=abc}", "(H2 h #h)[ATTR id=abc]"},
		{"=== h\n=== h", "(H2 h #h)(H2 h #h-1)"},
	})
}

func TestHRule(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"-", "(PARA -)"},
566
567
568
569
570
571
572


573
574
575
576
577
578
579

580
581
582
583
584
585
586
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598







+
+







+







	})
}

func TestTable(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"|", "(TAB (TR))"},
		{"||", "(TAB (TR (TD)))"},
		{"| |", "(TAB (TR (TD)))"},
		{"|a", "(TAB (TR (TD a)))"},
		{"|a|", "(TAB (TR (TD a)))"},
		{"|a| ", "(TAB (TR (TD a)(TD)))"},
		{"|a|b", "(TAB (TR (TD a)(TD b)))"},
		{"|a|b\n|c|d", "(TAB (TR (TD a)(TD b))(TR (TD c)(TD d)))"},
		{"|%", ""},
		{"|a|b\n|%---\n|c|d", "(TAB (TR (TD a)(TD b))(TR (TD c)(TD d)))"},
		{"|a|b\n|c", "(TAB (TR (TD a)(TD b))(TR (TD c)(TD)))"},
	})
}

func TestBlockAttr(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{":::go\n:::", "(SPAN)[ATTR =go]"},
663
664
665
666
667
668
669


670
671
672

673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692

693
694

695
696

697
698

699
700
701
702
703
704
705





706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723

724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739

740
741
742
743
744
745
746
747
748
749
750
751
752
753
754

755
756
757
758
759
760
761
675
676
677
678
679
680
681
682
683
684
685

686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705

706
707

708
709

710
711

712
713
714
715
716
717
718

719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740

741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756

757
758
759
760
761
762
763
764
765
766
767
768
769
770
771

772
773
774
775
776
777
778
779







+
+


-
+



















-
+

-
+

-
+

-
+






-
+
+
+
+
+

















-
+















-
+














-
+







	b strings.Builder
}

func (tv *TestVisitor) String() string { return tv.b.String() }

func (tv *TestVisitor) Visit(node ast.Node) ast.Visitor {
	switch n := node.(type) {
	case *ast.InlineListNode:
		tv.visitInlineList(n)
	case *ast.ParaNode:
		tv.b.WriteString("(PARA")
		tv.visitInlineSlice(n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.b.WriteByte(')')
	case *ast.VerbatimNode:
		code, ok := mapVerbatimKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown verbatim code %v", n.Kind))
		}
		tv.b.WriteString(code)
		for _, line := range n.Lines {
			tv.b.WriteByte('\n')
			tv.b.WriteString(line)
		}
		tv.b.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.RegionNode:
		code, ok := mapRegionKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("Unknown region code %v", n.Kind))
		}
		tv.b.WriteString(code)
		if n.Blocks != nil {
		if n.Blocks != nil && len(n.Blocks.List) > 0 {
			tv.b.WriteByte(' ')
			ast.WalkBlockSlice(tv, n.Blocks)
			ast.Walk(tv, n.Blocks)
		}
		if len(n.Inlines) > 0 {
		if n.Inlines != nil {
			tv.b.WriteString(" (LINE")
			tv.visitInlineSlice(n.Inlines)
			ast.Walk(tv, n.Inlines)
			tv.b.WriteByte(')')
		}
		tv.b.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.HeadingNode:
		fmt.Fprintf(&tv.b, "(H%d", n.Level)
		tv.visitInlineSlice(n.Inlines)
		ast.Walk(tv, n.Inlines)
		if n.Fragment != "" {
			tv.b.WriteString(" #")
			tv.b.WriteString(n.Fragment)
		}
		tv.b.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.HRuleNode:
		tv.b.WriteString("(HR)")
		tv.visitAttributes(n.Attrs)
	case *ast.NestedListNode:
		tv.b.WriteString(mapNestedListKind[n.Kind])
		for _, item := range n.Items {
			tv.b.WriteString(" {")
			ast.WalkItemSlice(tv, item)
			tv.b.WriteByte('}')
		}
		tv.b.WriteByte(')')
	case *ast.DescriptionListNode:
		tv.b.WriteString("(DL")
		for _, def := range n.Descriptions {
			tv.b.WriteString(" (DT")
			tv.visitInlineSlice(def.Term)
			ast.Walk(tv, def.Term)
			tv.b.WriteByte(')')
			for _, b := range def.Descriptions {
				tv.b.WriteString(" (DD ")
				ast.WalkDescriptionSlice(tv, b)
				tv.b.WriteByte(')')
			}
		}
		tv.b.WriteByte(')')
	case *ast.TableNode:
		tv.b.WriteString("(TAB")
		if len(n.Header) > 0 {
			tv.b.WriteString(" (TR")
			for _, cell := range n.Header {
				tv.b.WriteString(" (TH")
				tv.b.WriteString(alignString[cell.Align])
				tv.visitInlineSlice(cell.Inlines)
				ast.Walk(tv, cell.Inlines)
				tv.b.WriteString(")")
			}
			tv.b.WriteString(")")
		}
		if len(n.Rows) > 0 {
			tv.b.WriteString(" ")
			for _, row := range n.Rows {
				tv.b.WriteString("(TR")
				for i, cell := range row {
					if i == 0 {
						tv.b.WriteString(" ")
					}
					tv.b.WriteString("(TD")
					tv.b.WriteString(alignString[cell.Align])
					tv.visitInlineSlice(cell.Inlines)
					ast.Walk(tv, cell.Inlines)
					tv.b.WriteString(")")
				}
				tv.b.WriteString(")")
			}
		}
		tv.b.WriteString(")")
	case *ast.BLOBNode:
778
779
780
781
782
783
784
785

786
787
788
789
790
791
792














793
794

795


796
797
798
799
800

801
802
803
804


805
806
807






808
809
810
811
812

813
814
815
816
817
818
819
820
821
822

823
824
825
826
827


828
829
830
831
832
833
834
796
797
798
799
800
801
802

803
804
805





806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822

823
824
825
826
827
828

829
830
831
832
833
834
835



836
837
838
839
840
841
842
843
844
845

846
847
848
849
850
851
852
853
854
855

856
857
858
859
860
861
862
863
864
865
866
867
868
869
870







-
+


-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+


+
-
+
+




-
+




+
+
-
-
-
+
+
+
+
+
+




-
+









-
+





+
+







		if n.Hard {
			tv.b.WriteString("HB")
		} else {
			tv.b.WriteString("SB")
		}
	case *ast.LinkNode:
		fmt.Fprintf(&tv.b, "(LINK %v", n.Ref)
		tv.visitInlineSlice(n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.b.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.ImageNode:
		fmt.Fprintf(&tv.b, "(IMAGE %v", n.Ref)
		tv.visitInlineSlice(n.Inlines)
		tv.b.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.EmbedNode:
		switch m := n.Material.(type) {
		case *ast.ReferenceMaterialNode:
			fmt.Fprintf(&tv.b, "(EMBED %v", m.Ref)
			if n.Inlines != nil {
				ast.Walk(tv, n.Inlines)
			}
			tv.b.WriteByte(')')
			tv.visitAttributes(n.Attrs)
		case *ast.BLOBMaterialNode:
			panic("TODO: zmktest blob")
		default:
			panic(fmt.Sprintf("Unknown material type %t for %v", n.Material, n.Material))
		}
	case *ast.CiteNode:
		fmt.Fprintf(&tv.b, "(CITE %s", n.Key)
		if n.Inlines != nil {
		tv.visitInlineSlice(n.Inlines)
			ast.Walk(tv, n.Inlines)
		}
		tv.b.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.FootnoteNode:
		tv.b.WriteString("(FN")
		tv.visitInlineSlice(n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.b.WriteByte(')')
		tv.visitAttributes(n.Attrs)
	case *ast.MarkNode:
		tv.b.WriteString("(MARK")
		if n.Text != "" {
			tv.b.WriteString(" \"")
		if len(n.Text) > 0 {
			tv.b.WriteByte(' ')
			tv.b.WriteString(n.Text)
			tv.b.WriteString(n.Text)
			tv.b.WriteByte('"')
		}
		if n.Fragment != "" {
			tv.b.WriteString(" #")
			tv.b.WriteString(n.Fragment)
		}
		tv.b.WriteByte(')')
	case *ast.FormatNode:
		fmt.Fprintf(&tv.b, "{%c", mapFormatKind[n.Kind])
		tv.visitInlineSlice(n.Inlines)
		ast.Walk(tv, n.Inlines)
		tv.b.WriteByte('}')
		tv.visitAttributes(n.Attrs)
	case *ast.LiteralNode:
		code, ok := mapLiteralKind[n.Kind]
		if !ok {
			panic(fmt.Sprintf("No element for code %v", n.Kind))
		}
		tv.b.WriteByte('{')
		tv.b.WriteRune(code)
		if len(n.Text) > 0 {
		if n.Text != "" {
			tv.b.WriteByte(' ')
			tv.b.WriteString(n.Text)
		}
		tv.b.WriteByte('}')
		tv.visitAttributes(n.Attrs)
	default:
		return tv
	}
	return nil
}

var mapVerbatimKind = map[ast.VerbatimKind]string{
	ast.VerbatimProg: "(PROG",
}
869
870
871
872
873
874
875
876
877


878
879
880
881
882
883
884

885
886
887
888
889
890
891
905
906
907
908
909
910
911


912
913
914
915
916
917
918
919

920
921
922
923
924
925
926
927







-
-
+
+






-
+







var mapLiteralKind = map[ast.LiteralKind]rune{
	ast.LiteralProg:    '`',
	ast.LiteralKeyb:    '+',
	ast.LiteralOutput:  '=',
	ast.LiteralComment: '%',
}

func (tv *TestVisitor) visitInlineSlice(ins ast.InlineSlice) {
	for _, in := range ins {
func (tv *TestVisitor) visitInlineList(iln *ast.InlineListNode) {
	for _, in := range iln.List {
		tv.b.WriteByte(' ')
		ast.Walk(tv, in)
	}
}

func (tv *TestVisitor) visitAttributes(a *ast.Attributes) {
	if a == nil || len(a.Attrs) == 0 {
	if a.IsEmpty() {
		return
	}
	tv.b.WriteString("[ATTR")

	keys := make([]string, 0, len(a.Attrs))
	for k := range a.Attrs {
		keys = append(keys, k)

Changes to search/search.go.

218
219
220
221
222
223
224
225
226
227




228
229
230
231
232
233
234

235
236
237
238

239
240
241
242
243
244
245
218
219
220
221
222
223
224



225
226
227
228
229
230
231
232
233
234

235
236
237
238

239
240
241
242
243
244
245
246







-
-
-
+
+
+
+






-
+



-
+







		return 0
	}
	s.mx.RLock()
	defer s.mx.RUnlock()
	return s.limit
}

// HasComputedMetaKey returns true, if the search references a metadata key which
// a computed value.
func (s *Search) HasComputedMetaKey() bool {
// EnrichNeeded returns true, if the search references a metadata key that
// is calculated via metadata enrichments. In most cases this is a computed
// value. Metadata "tags" is an exception to this rule.
func (s *Search) EnrichNeeded() bool {
	if s == nil {
		return false
	}
	s.mx.RLock()
	defer s.mx.RUnlock()
	for key := range s.tags {
		if meta.IsComputed(key) {
		if meta.IsComputed(key) || key == meta.KeyTags {
			return true
		}
	}
	if order := s.order; order != "" && meta.IsComputed(order) {
	if order := s.order; order != "" && (meta.IsComputed(order) || order == meta.KeyTags) {
		return true
	}
	return false
}

// CompileMatch returns a function to match meta data based on select specification.
func (s *Search) CompileMatch(searcher Searcher) MetaMatchFunc {

Changes to search/select.go.

171
172
173
174
175
176
177
178

179
180
181
182
183
184
185
171
172
173
174
175
176
177

178
179
180
181
182
183
184
185







-
+







			return true
		}
	}
	return false
}

func createMatchTagSetFunc(values []opValue) matchFunc {
	tagValues := processTagSet(preprocessSet(values))
	tagValues := processTagSet(preprocessSet(sliceToLower(values)))
	return func(value string) bool {
		tags := meta.ListFromValue(value)
		// Remove leading '#' from each tag
		for i, tag := range tags {
			tags[i] = meta.CleanTag(tag)
		}
		for _, neededTags := range tagValues {
272
273
274
275
276
277
278
279

280
281
282
283
284

285
286
287
288
289
290
291
292
293

294
295
296
297
298
299







300
301
302
303
304
305
306
272
273
274
275
276
277
278

279
280
281
282
283

284
285
286
287
288
289
290
291
292

293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313







-
+




-
+








-
+






+
+
+
+
+
+
+







		return true
	}
}

func makeSearchMetaMatchFunc(posSpecs, negSpecs []matchSpec, nomatch []string) MetaMatchFunc {
	return func(m *meta.Meta) bool {
		for _, key := range nomatch {
			if _, ok := m.Get(key); ok {
			if _, ok := getMeta(m, key); ok {
				return false
			}
		}
		for _, s := range posSpecs {
			if value, ok := m.Get(s.key); !ok || !s.match(value) {
			if value, ok := getMeta(m, s.key); !ok || !s.match(value) {
				return false
			}
		}
		for _, s := range negSpecs {
			if s.match == nil {
				if _, ok := m.Get(s.key); ok {
					return false
				}
			} else if value, ok := m.Get(s.key); !ok || s.match(value) {
			} else if value, ok := getMeta(m, s.key); !ok || s.match(value) {
				return false
			}
		}
		return true
	}
}

func getMeta(m *meta.Meta, key string) (string, bool) {
	if key == meta.KeyTags {
		return m.Get(meta.KeyAllTags)
	}
	return m.Get(key)
}

func sliceToLower(sl []opValue) []opValue {
	result := make([]opValue, 0, len(sl))
	for _, s := range sl {
		result = append(result, opValue{
			value: strings.ToLower(s.value),
			op:    s.op,

Changes to strfun/escape.go.

72
73
74
75
76
77
78
79
80
81













































72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126










+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
			html = htmlAmp
		default:
			continue
		}
		io.WriteString(w, s[last:i])
		w.Write(html)
		last = i + 1
	}
	io.WriteString(w, s[last:])
}

var (
	jsBackslash   = []byte{'\\', '\\'}
	jsDoubleQuote = []byte{'\\', '"'}
	jsNewline     = []byte{'\\', 'n'}
	jsTab         = []byte{'\\', 't'}
	jsCr          = []byte{'\\', 'r'}
	jsUnicode     = []byte{'\\', 'u', '0', '0', '0', '0'}
	jsHex         = []byte("0123456789ABCDEF")
)

// JSONEscape returns the given string as a byte slice, where every non-printable
// rune is made printable.
func JSONEscape(w io.Writer, s string) {
	last := 0
	for i, ch := range s {
		var b []byte
		switch ch {
		case '\t':
			b = jsTab
		case '\r':
			b = jsCr
		case '\n':
			b = jsNewline
		case '"':
			b = jsDoubleQuote
		case '\\':
			b = jsBackslash
		default:
			if ch < ' ' {
				b = jsUnicode
				b[2] = '0'
				b[3] = '0'
				b[4] = jsHex[ch>>4]
				b[5] = jsHex[ch&0xF]
			} else {
				continue
			}
		}
		io.WriteString(w, s[last:i])
		w.Write(b)
		last = i + 1
	}
	io.WriteString(w, s[last:])
}

Changes to template/mustache.go.

69
70
71
72
73
74
75
76
77
78
79




80
81
82
83
84
85
86
87
88

89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107




108
109
110
111
112
113

114
115
116
117
118
119
120
69
70
71
72
73
74
75




76
77
78
79
80
81
82
83
84
85
86
87

88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103




104
105
106
107
108
109
110
111
112

113
114
115
116
117
118
119
120







-
-
-
-
+
+
+
+








-
+















-
-
-
-
+
+
+
+





-
+







)

type varNode struct {
	name string
	raw  bool
}

func (e *varNode) node()         {}
func (e *varNode) Type() TagType { return Variable }
func (e *varNode) Name() string  { return e.name }
func (e *varNode) Tags() []Tag   { panic("mustache: Tags on Variable type") }
func (*varNode) node()          {}
func (*varNode) Type() TagType  { return Variable }
func (e *varNode) Name() string { return e.name }
func (*varNode) Tags() []Tag    { panic("mustache: Tags on Variable type") }

type sectionNode struct {
	name      string
	inverted  bool
	startline int
	nodes     []node
}

func (e *sectionNode) node() {}
func (*sectionNode) node() {}
func (e *sectionNode) Type() TagType {
	if e.inverted {
		return InvertedSection
	}
	return Section
}
func (e *sectionNode) Name() string { return e.name }
func (e *sectionNode) Tags() []Tag  { return extractTags(e.nodes) }

type partialNode struct {
	name   string
	indent string
	prov   PartialProvider
}

func (e *partialNode) node()         {}
func (e *partialNode) Type() TagType { return Partial }
func (e *partialNode) Name() string  { return e.name }
func (e *partialNode) Tags() []Tag   { return nil }
func (*partialNode) node()          {}
func (*partialNode) Type() TagType  { return Partial }
func (e *partialNode) Name() string { return e.name }
func (*partialNode) Tags() []Tag    { return nil }

type textNode struct {
	text []byte
}

func (e *textNode) node() {}
func (*textNode) node() {}

// Template represents a compiled mustache template
type Template struct {
	data    string
	otag    string
	ctag    string
	p       int
294
295
296
297
298
299
300
301

302
303
304
305
306

307
308
309
310
311
312
313
294
295
296
297
298
299
300

301
302
303
304
305

306
307
308
309
310
311
312
313







-
+




-
+







		tmpl.p = eow + 2
		tmpl.curline++
		return true
	}
	return false
}

func (tmpl *Template) parsePartial(name, indent string) (*partialNode, error) {
func (tmpl *Template) parsePartial(name, indent string) *partialNode {
	return &partialNode{
		name:   name,
		indent: indent,
		prov:   tmpl.partial,
	}, nil
	}
}

func (tmpl *Template) parseSection(section *sectionNode) error {
	for {
		textResult, err := tmpl.readText()
		text := textResult.text
		padding := textResult.padding
346
347
348
349
350
351
352
353

354
355
356
357
358
359
360
361
362
363
346
347
348
349
350
351
352

353



354
355
356
357
358
359
360







-
+
-
-
-







			name := strings.TrimSpace(tag[1:])
			if name != section.name {
				return parseError{tmpl.curline, "interleaved closing tag: " + name}
			}
			return nil
		case '>':
			name := strings.TrimSpace(tag[1:])
			partial, err := tmpl.parsePartial(name, textResult.padding)
			partial := tmpl.parsePartial(name, textResult.padding)
			if err != nil {
				return err
			}
			section.nodes = append(section.nodes, partial)
		case '=':
			if tag[len(tag)-1] != '=' {
				return parseError{tmpl.curline, "Invalid meta tag"}
			}
			tag = strings.TrimSpace(tag[1 : len(tag)-1])
			newtags := strings.SplitN(tag, " ", 2)
417
418
419
420
421
422
423
424

425
426
427
428
429
430
431
432
433
434
414
415
416
417
418
419
420

421



422
423
424
425
426
427
428







-
+
-
-
-







				return err
			}
			tmpl.nodes = append(tmpl.nodes, sn)
		case '/':
			return parseError{tmpl.curline, "unmatched close tag"}
		case '>':
			name := strings.TrimSpace(tag[1:])
			partial, err := tmpl.parsePartial(name, textResult.padding)
			partial := tmpl.parsePartial(name, textResult.padding)
			if err != nil {
				return err
			}
			tmpl.nodes = append(tmpl.nodes, partial)
		case '=':
			if tag[len(tag)-1] != '=' {
				return parseError{tmpl.curline, "Invalid meta tag"}
			}
			tag = strings.TrimSpace(tag[1 : len(tag)-1])
			newtags := strings.SplitN(tag, " ", 2)
697
698
699
700
701
702
703
704

705
706
707
708
709
710
711
691
692
693
694
695
696
697

698
699
700
701
702
703
704
705







-
+







	return "", &ErrPartialNotFound{name}
}

// emptyProvider will always returns an empty string.
type emptyProvider struct{}

// Get accepts the name of a partial and returns the parsed partial.
func (ep *emptyProvider) Get(name string) (string, error) { return "", nil }
func (*emptyProvider) Get(string) (string, error) { return "", nil }

// EmptyProvider is a partial provider that will always return an empty string.
var EmptyProvider emptyProvider

var nonEmptyLine = regexp.MustCompile(`(?m:^(.+)$)`)

func getPartials(partials PartialProvider, name, indent string) (*Template, error) {

Added testdata/content/embed/20200215204700.zettel.




1
2
3
+
+
+
title: Simple Test

{{abc}}

Deleted testdata/content/image/20200215204700.zettel.

1
2
3



-
-
-
title: Simple Test

{{abc}}

Changes to testdata/content/table/20200215204700.zettel.

1
2
3

1
2
3
4



+
title: Simple Test

|c1|c2|c3|
|d1||d3

Changes to tests/markdown_test.go.

18
19
20
21
22
23
24
25
26


27
28
29
30
31
32
33
18
19
20
21
22
23
24


25
26
27
28
29
30
31
32
33







-
-
+
+







	"regexp"
	"strings"
	"testing"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/encoder"
	_ "zettelstore.de/z/encoder/htmlenc"
	_ "zettelstore.de/z/encoder/jsonenc"
	_ "zettelstore.de/z/encoder/djsonenc"
	_ "zettelstore.de/z/encoder/htmlenc"
	_ "zettelstore.de/z/encoder/nativeenc"
	_ "zettelstore.de/z/encoder/textenc"
	_ "zettelstore.de/z/encoder/zmkenc"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	_ "zettelstore.de/z/parser/markdown"
	_ "zettelstore.de/z/parser/zettelmark"
69
70
71
72
73
74
75
76
77


78
79

80
81
82
83
84
85
86
69
70
71
72
73
74
75


76
77
78

79
80
81
82
83
84
85
86







-
-
+
+

-
+







}

var reHeadingID = regexp.MustCompile(` id="[^"]*"`)

func TestEncoderAvailability(t *testing.T) {
	t.Parallel()
	encoderMissing := false
	for _, format := range formats {
		enc := encoder.Create(format, nil)
	for _, enc := range encodings {
		enc := encoder.Create(enc, nil)
		if enc == nil {
			t.Errorf("No encoder for %q found", format)
			t.Errorf("No encoder for %q found", enc)
			encoderMissing = true
		}
	}
	if encoderMissing {
		panic("At least one encoder is missing. See test log")
	}
}
105
106
107
108
109
110
111
112

113
114
115
116
117



118
119
120
121
122
123

124
125
126
127
128
129
130
105
106
107
108
109
110
111

112
113
114



115
116
117
118
119
120
121
122

123
124
125
126
127
128
129
130







-
+


-
-
-
+
+
+





-
+







		if _, found := excMap[tc.Markdown]; !found {
			testHTMLEncoding(t, tc, ast)
		}
		testZmkEncoding(t, tc, ast)
	}
}

func testAllEncodings(t *testing.T, tc markdownTestCase, ast ast.BlockSlice) {
func testAllEncodings(t *testing.T, tc markdownTestCase, ast *ast.BlockListNode) {
	var sb strings.Builder
	testID := tc.Example*100 + 1
	for _, format := range formats {
		t.Run(fmt.Sprintf("Encode %v %v", format, testID), func(st *testing.T) {
			encoder.Create(format, nil).WriteBlocks(&sb, ast)
	for _, enc := range encodings {
		t.Run(fmt.Sprintf("Encode %v %v", enc, testID), func(st *testing.T) {
			encoder.Create(enc, nil).WriteBlocks(&sb, ast)
			sb.Reset()
		})
	}
}

func testHTMLEncoding(t *testing.T, tc markdownTestCase, ast ast.BlockSlice) {
func testHTMLEncoding(t *testing.T, tc markdownTestCase, ast *ast.BlockListNode) {
	htmlEncoder := encoder.Create(api.EncoderHTML, &encoder.Environment{Xhtml: true})
	var sb strings.Builder
	testID := tc.Example*100 + 1
	t.Run(fmt.Sprintf("Encode md html %v", testID), func(st *testing.T) {
		htmlEncoder.WriteBlocks(&sb, ast)
		gotHTML := sb.String()
		sb.Reset()
141
142
143
144
145
146
147
148

149
150
151
152
153
154
155
141
142
143
144
145
146
147

148
149
150
151
152
153
154
155







-
+







			if gotHTML != mdHTML {
				st.Errorf("\nCMD: %q\nExp: %q\nGot: %q", tc.Markdown, mdHTML, gotHTML)
			}
		}
	})
}

func testZmkEncoding(t *testing.T, tc markdownTestCase, ast ast.BlockSlice) {
func testZmkEncoding(t *testing.T, tc markdownTestCase, ast *ast.BlockListNode) {
	zmkEncoder := encoder.Create(api.EncoderZmk, nil)
	var sb strings.Builder
	testID := tc.Example*100 + 1
	t.Run(fmt.Sprintf("Encode zmk %14d", testID), func(st *testing.T) {
		zmkEncoder.WriteBlocks(&sb, ast)
		gotFirst := sb.String()
		sb.Reset()

Changes to tests/regression_test.go.

29
30
31
32
33
34
35
36
37


38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
29
30
31
32
33
34
35


36
37
38
39
40
41
42
43
44

45
46
47
48
49
50
51
52







-
-
+
+







-
+







	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"

	_ "zettelstore.de/z/box/dirbox"
	_ "zettelstore.de/z/encoder/htmlenc"
	_ "zettelstore.de/z/encoder/jsonenc"
	_ "zettelstore.de/z/encoder/djsonenc"
	_ "zettelstore.de/z/encoder/htmlenc"
	_ "zettelstore.de/z/encoder/nativeenc"
	_ "zettelstore.de/z/encoder/textenc"
	_ "zettelstore.de/z/encoder/zmkenc"
	_ "zettelstore.de/z/parser/blob"
	_ "zettelstore.de/z/parser/zettelmark"
)

var formats = []api.EncodingEnum{
var encodings = []api.EncodingEnum{
	api.EncoderHTML,
	api.EncoderDJSON,
	api.EncoderNative,
	api.EncoderText,
}

func getFileBoxes(wd, kind string) (root string, boxes []box.ManagedBox) {
71
72
73
74
75
76
77
78
79


80
81
82
83

84
85
86
87
88
89
90
71
72
73
74
75
76
77


78
79
80
81
82

83
84
85
86
87
88
89
90







-
-
+
+



-
+







		}
	}
	return root, boxes
}

type noEnrich struct{}

func (nf *noEnrich) Enrich(context.Context, *meta.Meta, int) {}
func (nf *noEnrich) Remove(context.Context, *meta.Meta)      {}
func (*noEnrich) Enrich(context.Context, *meta.Meta, int) {}
func (*noEnrich) Remove(context.Context, *meta.Meta)      {}

type noAuth struct{}

func (na *noAuth) IsReadonly() bool { return false }
func (*noAuth) IsReadonly() bool { return false }

func trimLastEOL(s string) string {
	if lastPos := len(s) - 1; lastPos >= 0 && s[lastPos] == '\n' {
		return s[:lastPos]
	}
	return s
}
109
110
111
112
113
114
115
116

117
118
119

120
121
122
123
124
125

126
127
128
129
130
131
132
109
110
111
112
113
114
115

116
117
118

119
120
121
122
123
124

125
126
127
128
129
130
131
132







-
+


-
+





-
+







	gotContent = trimLastEOL(gotContent)
	wantContent = trimLastEOL(wantContent)
	if gotContent != wantContent {
		t.Errorf("\nWant: %q\nGot:  %q", wantContent, gotContent)
	}
}

func checkBlocksFile(t *testing.T, resultName string, zn *ast.ZettelNode, format api.EncodingEnum) {
func checkBlocksFile(t *testing.T, resultName string, zn *ast.ZettelNode, enc api.EncodingEnum) {
	t.Helper()
	var env encoder.Environment
	if enc := encoder.Create(format, &env); enc != nil {
	if enc := encoder.Create(enc, &env); enc != nil {
		var sb strings.Builder
		enc.WriteBlocks(&sb, zn.Ast)
		checkFileContent(t, resultName, sb.String())
		return
	}
	panic(fmt.Sprintf("Unknown writer format %q", format))
	panic(fmt.Sprintf("Unknown writer encoding %q", enc))
}

func checkZmkEncoder(t *testing.T, zn *ast.ZettelNode) {
	zmkEncoder := encoder.Create(api.EncoderZmk, nil)
	var sb strings.Builder
	zmkEncoder.WriteBlocks(&sb, zn.Ast)
	gotFirst := sb.String()
147
148
149
150
151
152
153
154
155
156
157
158
159
160

161

162
163
164
165
166
167
168
169
170
171
172
173
174




175
176
177
178
179
180
181
147
148
149
150
151
152
153


154
155
156
157
158
159

160
161
162
163
164
165
166
167
168
169




170
171
172
173
174
175
176
177
178
179
180







-
-





+
-
+









-
-
-
-
+
+
+
+







	u, err := url.Parse(p.Location())
	if err != nil {
		panic("Unable to parse URL '" + p.Location() + "': " + err.Error())
	}
	return u.Path[len(root):]
}

func match(*meta.Meta) bool { return true }

func checkContentBox(t *testing.T, p box.ManagedBox, wd, boxName string) {
	ss := p.(box.StartStopper)
	if err := ss.Start(context.Background()); err != nil {
		panic(err)
	}
	metaList := []*meta.Meta{}
	metaList, err := p.SelectMeta(context.Background(), match)
	err := p.ApplyMeta(context.Background(), func(m *meta.Meta) { metaList = append(metaList, m) })
	if err != nil {
		panic(err)
	}
	for _, meta := range metaList {
		zettel, err := p.GetZettel(context.Background(), meta.Zid)
		if err != nil {
			panic(err)
		}
		z := parser.ParseZettel(zettel, "", testConfig)
		for _, format := range formats {
			t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, format), func(st *testing.T) {
				resultName := filepath.Join(wd, "result", "content", boxName, z.Zid.String()+"."+format.String())
				checkBlocksFile(st, resultName, z, format)
		for _, enc := range encodings {
			t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, enc), func(st *testing.T) {
				resultName := filepath.Join(wd, "result", "content", boxName, z.Zid.String()+"."+enc.String())
				checkBlocksFile(st, resultName, z, enc)
			})
		}
		t.Run(fmt.Sprintf("%s::%d", p.Location(), meta.Zid), func(st *testing.T) {
			checkZmkEncoder(st, z)
		})
	}
	if err := ss.Stop(context.Background()); err != nil {
191
192
193
194
195
196
197
198

199
200
201

202
203

204
205
206
207

208
209
210
211
212
213
214

215

216
217
218
219
220
221
222
223
224
225
226
227
228




229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251













252
253

254

255
256
257
258
259
260
261
190
191
192
193
194
195
196

197
198
199

200
201

202
203
204
205

206
207
208
209
210
211
212
213
214

215
216
217
218
219
220
221
222
223
224




225
226
227
228
229
230
231
232
233
234
235
236
237
238













239
240
241
242
243
244
245
246
247
248
249
250
251
252

253
254
255
256
257
258
259
260
261
262







-
+


-
+

-
+



-
+







+
-
+









-
-
-
-
+
+
+
+










-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+

-
+

+







	}
	root, boxes := getFileBoxes(wd, "content")
	for _, p := range boxes {
		checkContentBox(t, p, wd, getBoxName(p, root))
	}
}

func checkMetaFile(t *testing.T, resultName string, zn *ast.ZettelNode, format api.EncodingEnum) {
func checkMetaFile(t *testing.T, resultName string, zn *ast.ZettelNode, enc api.EncodingEnum) {
	t.Helper()

	if enc := encoder.Create(format, nil); enc != nil {
	if enc := encoder.Create(enc, nil); enc != nil {
		var sb strings.Builder
		enc.WriteMeta(&sb, zn.Meta)
		enc.WriteMeta(&sb, zn.Meta, parser.ParseMetadata)
		checkFileContent(t, resultName, sb.String())
		return
	}
	panic(fmt.Sprintf("Unknown writer format %q", format))
	panic(fmt.Sprintf("Unknown writer encoding %q", enc))
}

func checkMetaBox(t *testing.T, p box.ManagedBox, wd, boxName string) {
	ss := p.(box.StartStopper)
	if err := ss.Start(context.Background()); err != nil {
		panic(err)
	}
	metaList := []*meta.Meta{}
	metaList, err := p.SelectMeta(context.Background(), match)
	err := p.ApplyMeta(context.Background(), func(m *meta.Meta) { metaList = append(metaList, m) })
	if err != nil {
		panic(err)
	}
	for _, meta := range metaList {
		zettel, err := p.GetZettel(context.Background(), meta.Zid)
		if err != nil {
			panic(err)
		}
		z := parser.ParseZettel(zettel, "", testConfig)
		for _, format := range formats {
			t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, format), func(st *testing.T) {
				resultName := filepath.Join(wd, "result", "meta", boxName, z.Zid.String()+"."+format.String())
				checkMetaFile(st, resultName, z, format)
		for _, enc := range encodings {
			t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, enc), func(st *testing.T) {
				resultName := filepath.Join(wd, "result", "meta", boxName, z.Zid.String()+"."+enc.String())
				checkMetaFile(st, resultName, z, enc)
			})
		}
	}
	if err := ss.Stop(context.Background()); err != nil {
		panic(err)
	}
}

type myConfig struct{}

func (cfg *myConfig) AddDefaultValues(m *meta.Meta) *meta.Meta { return m }
func (cfg *myConfig) GetDefaultTitle() string                  { return "" }
func (cfg *myConfig) GetDefaultRole() string                   { return meta.ValueRoleZettel }
func (cfg *myConfig) GetDefaultSyntax() string                 { return meta.ValueSyntaxZmk }
func (cfg *myConfig) GetDefaultLang() string                   { return "" }
func (cfg *myConfig) GetDefaultVisibility() meta.Visibility    { return meta.VisibilityPublic }
func (cfg *myConfig) GetFooterHTML() string                    { return "" }
func (cfg *myConfig) GetHomeZettel() id.Zid                    { return id.Invalid }
func (cfg *myConfig) GetListPageSize() int                     { return 0 }
func (cfg *myConfig) GetMarkerExternal() string                { return "" }
func (cfg *myConfig) GetSiteName() string                      { return "" }
func (cfg *myConfig) GetYAMLHeader() bool                      { return false }
func (cfg *myConfig) GetZettelFileSyntax() []string            { return nil }
func (*myConfig) AddDefaultValues(m *meta.Meta) *meta.Meta { return m }
func (*myConfig) GetDefaultTitle() string                  { return "" }
func (*myConfig) GetDefaultRole() string                   { return meta.ValueRoleZettel }
func (*myConfig) GetDefaultSyntax() string                 { return meta.ValueSyntaxZmk }
func (*myConfig) GetDefaultLang() string                   { return "" }
func (*myConfig) GetDefaultVisibility() meta.Visibility    { return meta.VisibilityPublic }
func (*myConfig) GetFooterHTML() string                    { return "" }
func (*myConfig) GetHomeZettel() id.Zid                    { return id.Invalid }
func (*myConfig) GetListPageSize() int                     { return 0 }
func (*myConfig) GetMarkerExternal() string                { return "" }
func (*myConfig) GetSiteName() string                      { return "" }
func (*myConfig) GetYAMLHeader() bool                      { return false }
func (*myConfig) GetZettelFileSyntax() []string            { return nil }

func (cfg *myConfig) GetExpertMode() bool                      { return false }
func (*myConfig) GetExpertMode() bool                          { return false }
func (cfg *myConfig) GetVisibility(*meta.Meta) meta.Visibility { return cfg.GetDefaultVisibility() }
func (*myConfig) GetMaxTransclusions() int                     { return 1024 }

var testConfig = &myConfig{}

func TestMetaRegression(t *testing.T) {
	t.Parallel()
	wd, err := os.Getwd()
	if err != nil {

Added tests/result/content/embed/20200215204700.djson.


1
+
[{"t":"Para","i":[{"t":"Embed","s":"abc"}]}]

Added tests/result/content/embed/20200215204700.html.


1
+
<p><img src="abc" alt=""></p>

Added tests/result/content/embed/20200215204700.native.


1
+
[Para Embed EXTERNAL "abc"]

Added tests/result/content/embed/20200215204700.text.

Changes to tests/result/content/heading/20200215204700.native.

1


1
-
+
[Heading 2 "first" Text "First"]
[Heading 2 #first Text "First"]

Deleted tests/result/content/image/20200215204700.djson.

1

-
[{"t":"Para","i":[{"t":"Image","s":"abc"}]}]

Deleted tests/result/content/image/20200215204700.html.

1

-
<p><img src="abc" alt=""></p>

Deleted tests/result/content/image/20200215204700.native.

1

-
[Para Image "abc"]

Deleted tests/result/content/image/20200215204700.text.

Changes to tests/result/content/mark/20200215204700.djson.

1


1
-
+
[{"t":"Para","i":[{"t":"Mark","s":"mark"}]}]
[{"t":"Para","i":[{"t":"Mark","s":"mark","q":"mark"}]}]

Changes to tests/result/content/mark/20200215204700.native.

1


1
-
+
[Para Mark "mark"]
[Para Mark "mark" #mark]

Changes to tests/result/content/table/20200215204700.djson.

1


1
-
+
[{"t":"Table","p":[[],[[["",[{"t":"Text","s":"c1"}]],["",[{"t":"Text","s":"c2"}]],["",[{"t":"Text","s":"c3"}]]]]]}]
[{"t":"Table","p":[[],[[["",[{"t":"Text","s":"c1"}]],["",[{"t":"Text","s":"c2"}]],["",[{"t":"Text","s":"c3"}]]],[["",[{"t":"Text","s":"d1"}]],["",[]],["",[{"t":"Text","s":"d3"}]]]]]}]

Changes to tests/result/content/table/20200215204700.html.

1
2
3

4
5
1
2
3
4
5
6



+


<table>
<tbody>
<tr><td>c1</td><td>c2</td><td>c3</td></tr>
<tr><td>d1</td><td></td><td>d3</td></tr>
</tbody>
</table>

Changes to tests/result/content/table/20200215204700.native.

1
2


1

2
3

-
+
+
[Table
 [Row [Cell Default Text "c1"],[Cell Default Text "c2"],[Cell Default Text "c3"]]]
 [Row [Cell Default Text "c1"],[Cell Default Text "c2"],[Cell Default Text "c3"]],
 [Row [Cell Default Text "d1"],[Cell Default],[Cell Default Text "d3"]]]

Changes to tests/result/content/table/20200215204700.text.

1

1
2

+
c1 c2 c3
d1  d3

Changes to tools/build.go.

25
26
27
28
29
30
31


32
33
34
35
36
37
38
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40







+
+







	"path/filepath"
	"regexp"
	"strings"
	"time"

	"zettelstore.de/z/strfun"
)

var directProxy = []string{"GOPROXY=direct"}

func executeCommand(env []string, name string, arg ...string) (string, error) {
	logCommand("EXEC", env, name, arg)
	var out bytes.Buffer
	cmd := prepareCommand(env, name, arg, &out)
	err := cmd.Run()
	return out.String(), err
124
125
126
127
128
129
130
131

132
133
134
135
136
137
138
126
127
128
129
130
131
132

133
134
135
136
137
138
139
140







-
+








func getVersion() string {
	base, vcs := getVersionData()
	return calcVersion(base, vcs)
}

func findExec(cmd string) string {
	if path, err := executeCommand(nil, "which", "shadow"); err == nil && path != "" {
	if path, err := executeCommand(nil, "which", cmd); err == nil && path != "" {
		return path
	}
	return ""
}

func cmdCheck() error {
	if err := checkGoTest("./..."); err != nil {
152
153
154
155
156
157
158
159

160
161
162
163
164
165
166
154
155
156
157
158
159
160

161
162
163
164
165
166
167
168







-
+







	}
	return checkFossilExtra()
}

func checkGoTest(pkg string, testParams ...string) error {
	args := []string{"test", pkg}
	args = append(args, testParams...)
	out, err := executeCommand(nil, "go", args...)
	out, err := executeCommand(directProxy, "go", args...)
	if err != nil {
		for _, line := range strfun.SplitLines(out) {
			if strings.HasPrefix(line, "ok") || strings.HasPrefix(line, "?") {
				continue
			}
			fmt.Fprintln(os.Stderr, line)
		}
297
298
299
300
301
302
303
304

305
306
307
308
309
310
311
299
300
301
302
303
304
305

306
307
308
309
310
311
312
313







-
+







		return false
	}
	conn.Close()
	return true
}

func cmdBuild() error {
	return doBuild(nil, getVersion(), "bin/zettelstore")
	return doBuild(directProxy, getVersion(), "bin/zettelstore")
}

func doBuild(env []string, version, target string) error {
	out, err := executeCommand(
		env,
		"go", "build",
		"-tags", "osusergo,netgo",
402
403
404
405
406
407
408

409
410
411
412
413
414
415
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418







+







		{"arm", "linux", []string{"GOARM=6"}, "zettelstore"},
		{"amd64", "darwin", nil, "iZettelstore"},
		{"arm64", "darwin", nil, "iZettelstore"},
		{"amd64", "windows", nil, "zettelstore.exe"},
	}
	for _, rel := range releases {
		env := append(rel.env, "GOARCH="+rel.arch, "GOOS="+rel.os)
		env = append(env, directProxy...)
		zsName := filepath.Join("releases", rel.name)
		if err := doBuild(env, calcVersion(base, fossil), zsName); err != nil {
			return err
		}
		zipName := fmt.Sprintf("zettelstore-%v-%v-%v.zip", base, rel.os, rel.arch)
		if err := createReleaseZip(zsName, zipName, rel.name); err != nil {
			return err
467
468
469
470
471
472
473














474
475
476
477
478
479
480
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497







+
+
+
+
+
+
+
+
+
+
+
+
+
+








func cmdClean() error {
	for _, dir := range []string{"bin", "releases"} {
		err := os.RemoveAll(dir)
		if err != nil {
			return err
		}
	}
	out, err := executeCommand(nil, "go", "clean", "./...")
	if err != nil {
		return err
	}
	if len(out) > 0 {
		fmt.Println(out)
	}
	out, err = executeCommand(nil, "go", "clean", "-cache", "-modcache", "-testcache")
	if err != nil {
		return err
	}
	if len(out) > 0 {
		fmt.Println(out)
	}
	return nil
}

func cmdHelp() {
	fmt.Println(`Usage: go run tools/build.go [-v] COMMAND

Changes to usecase/context.go.

48
49
50
51
52
53
54
55

56
57
58
59
60
61
62
48
49
50
51
52
53
54

55
56
57
58
59
60
61
62







-
+







// Run executes the use case.
func (uc ZettelContext) Run(ctx context.Context, zid id.Zid, dir ZettelContextDirection, depth, limit int) (result []*meta.Meta, err error) {
	start, err := uc.port.GetMeta(ctx, zid)
	if err != nil {
		return nil, err
	}
	tasks := ztlCtx{depth: depth}
	uc.addInitialTasks(ctx, &tasks, start)
	tasks.add(start, 0)
	visited := id.NewSet()
	isBackward := dir == ZettelContextBoth || dir == ZettelContextBackward
	isForward := dir == ZettelContextBoth || dir == ZettelContextForward
	for !tasks.empty() {
		m, curDepth := tasks.pop()
		if _, ok := visited[m.Zid]; ok {
			continue
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
92
93
94
95
96
97
98




99
100
101
102
103
104
105







-
-
-
-







				}
			}
		}
	}
	return result, nil
}

func (uc ZettelContext) addInitialTasks(ctx context.Context, tasks *ztlCtx, start *meta.Meta) {
	tasks.add(start, 0)
}

func (uc ZettelContext) addID(ctx context.Context, tasks *ztlCtx, depth int, value string) {
	if zid, err := id.Parse(value); err == nil {
		if m, err := uc.port.GetMeta(ctx, zid); err == nil {
			tasks.add(m, depth)
		}
	}
}

Added usecase/evaluate.go.









































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"context"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/parser"
)

// Evaluate is the data for this use case.
type Evaluate struct {
	rtConfig  config.Config
	getZettel GetZettel
	getMeta   GetMeta
}

// NewEvaluate creates a new use case.
func NewEvaluate(rtConfig config.Config, getZettel GetZettel, getMeta GetMeta) Evaluate {
	return Evaluate{
		rtConfig:  rtConfig,
		getZettel: getZettel,
		getMeta:   getMeta,
	}
}

// Run executes the use case.
func (uc *Evaluate) Run(ctx context.Context, zid id.Zid, syntax string, env *evaluator.Environment) (*ast.ZettelNode, error) {
	zettel, err := uc.getZettel.Run(ctx, zid)
	if err != nil {
		return nil, err
	}
	zn, err := parser.ParseZettel(zettel, syntax, uc.rtConfig), nil
	if err != nil {
		return nil, err
	}

	evaluator.EvaluateZettel(ctx, uc, env, uc.rtConfig, zn)
	return zn, nil
}

// RunMetadata executes the use case for a metadata value.
func (uc *Evaluate) RunMetadata(ctx context.Context, value string, env *evaluator.Environment) *ast.InlineListNode {
	iln := parser.ParseMetadata(value)
	evaluator.EvaluateInline(ctx, uc, env, uc.rtConfig, iln)
	return iln
}

// GetMeta retrieves the metadata of a given zettel identifier.
func (uc *Evaluate) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	return uc.getMeta.Run(ctx, zid)
}

// GetZettel retrieves the full zettel of a given zettel identifier.
func (uc *Evaluate) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	return uc.getZettel.Run(ctx, zid)
}

Changes to usecase/list_tags.go.

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
10
11
12
13
14
15
16

17
18
19
20
21
22
23







-








// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"context"

	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

// ListTagsPort is the interface used by this use case.
type ListTagsPort interface {
	// SelectMeta returns all zettel meta data that match the selection criteria.
36
37
38
39
40
41
42
43

44
45
46
47
48
49

50
51
52
53
54
55
56
35
36
37
38
39
40
41

42
43
44
45
46
47

48
49
50
51
52
53
54
55







-
+





-
+







}

// TagData associates tags with a list of all zettel meta that use this tag
type TagData map[string][]*meta.Meta

// Run executes the use case.
func (uc ListTags) Run(ctx context.Context, minCount int) (TagData, error) {
	metas, err := uc.port.SelectMeta(box.NoEnrichContext(ctx), nil)
	metas, err := uc.port.SelectMeta(ctx, nil)
	if err != nil {
		return nil, err
	}
	result := make(TagData)
	for _, m := range metas {
		if tl, ok := m.GetList(meta.KeyTags); ok && len(tl) > 0 {
		if tl, ok := m.GetList(meta.KeyAllTags); ok && len(tl) > 0 {
			for _, t := range tl {
				result[t] = append(result[t], m)
			}
		}
	}
	if minCount > 1 {
		for t, ms := range result {

Changes to usecase/order.go.

23
24
25
26
27
28
29
30
31


32
33
34
35
36


37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52
53
54
55
23
24
25
26
27
28
29


30
31
32
33
34


35
36
37
38
39
40
41
42

43
44
45
46
47
48
49
50
51
52
53
54
55







-
-
+
+



-
-
+
+






-
+












type ZettelOrderPort interface {
	// GetMeta retrieves just the meta data of a specific zettel.
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
}

// ZettelOrder is the data for this use case.
type ZettelOrder struct {
	port        ZettelOrderPort
	parseZettel ParseZettel
	port     ZettelOrderPort
	evaluate Evaluate
}

// NewZettelOrder creates a new use case.
func NewZettelOrder(port ZettelOrderPort, parseZettel ParseZettel) ZettelOrder {
	return ZettelOrder{port: port, parseZettel: parseZettel}
func NewZettelOrder(port ZettelOrderPort, evaluate Evaluate) ZettelOrder {
	return ZettelOrder{port: port, evaluate: evaluate}
}

// Run executes the use case.
func (uc ZettelOrder) Run(ctx context.Context, zid id.Zid, syntax string) (
	start *meta.Meta, result []*meta.Meta, err error,
) {
	zn, err := uc.parseZettel.Run(ctx, zid, syntax)
	zn, err := uc.evaluate.Run(ctx, zid, syntax, nil)
	if err != nil {
		return nil, nil, err
	}
	for _, ref := range collect.Order(zn) {
		if zid, err := id.Parse(ref.URL.Path); err == nil {
			if m, err := uc.port.GetMeta(ctx, zid); err == nil {
				result = append(result, m)
			}
		}
	}
	return zn.Meta, result, nil
}

Changes to usecase/parse_zettel.go.

1
2

3
4
5
6
7
8
9
1

2
3
4
5
6
7
8
9

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------
28
29
30
31
32
33
34
35
36

37
38
39
40
41
42
43
28
29
30
31
32
33
34


35
36
37
38
39
40
41
42







-
-
+








// NewParseZettel creates a new use case.
func NewParseZettel(rtConfig config.Config, getZettel GetZettel) ParseZettel {
	return ParseZettel{rtConfig: rtConfig, getZettel: getZettel}
}

// Run executes the use case.
func (uc ParseZettel) Run(
	ctx context.Context, zid id.Zid, syntax string) (*ast.ZettelNode, error) {
func (uc ParseZettel) Run(ctx context.Context, zid id.Zid, syntax string) (*ast.ZettelNode, error) {
	zettel, err := uc.getZettel.Run(ctx, zid)
	if err != nil {
		return nil, err
	}

	return parser.ParseZettel(zettel, syntax, uc.rtConfig), nil
}

Changes to usecase/search.go.

33
34
35
36
37
38
39
40

41
42
43
44
33
34
35
36
37
38
39

40
41
42
43
44







-
+




// NewSearch creates a new use case.
func NewSearch(port SearchPort) Search {
	return Search{port: port}
}

// Run executes the use case.
func (uc Search) Run(ctx context.Context, s *search.Search) ([]*meta.Meta, error) {
	if !s.HasComputedMetaKey() {
	if !s.EnrichNeeded() {
		ctx = box.NoEnrichContext(ctx)
	}
	return uc.port.SelectMeta(ctx, s)
}

Changes to web/adapter/api/api.go.

32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
47
48

49
50
51
52

53
54
55

56
57
58


59
60
61
62



63
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46
47

48
49
50
51

52
53
54

55
56


57
58
59



60
61
62
63







-
+








-
+



-
+


-
+

-
-
+
+

-
-
-
+
+
+

	auth     server.Auth

	tokenLifetime time.Duration
}

// New creates a new API object.
func New(b server.Builder, authz auth.AuthzManager, token auth.TokenManager, auth server.Auth, rtConfig config.Config) *API {
	api := &API{
	a := &API{
		b:        b,
		authz:    authz,
		token:    token,
		auth:     auth,
		rtConfig: rtConfig,

		tokenLifetime: kernel.Main.GetConfig(kernel.WebService, kernel.WebTokenLifetimeAPI).(time.Duration),
	}
	return api
	return a
}

// GetURLPrefix returns the configured URL prefix of the web server.
func (api *API) GetURLPrefix() string { return api.b.GetURLPrefix() }
func (a *API) GetURLPrefix() string { return a.b.GetURLPrefix() }

// NewURLBuilder creates a new URL builder object with the given key.
func (api *API) NewURLBuilder(key byte) *api.URLBuilder { return api.b.NewURLBuilder(key) }
func (a *API) NewURLBuilder(key byte) *api.URLBuilder { return a.b.NewURLBuilder(key) }

func (api *API) getAuthData(ctx context.Context) *server.AuthData {
	return api.auth.GetAuthData(ctx)
func (a *API) getAuthData(ctx context.Context) *server.AuthData {
	return a.auth.GetAuthData(ctx)
}
func (api *API) withAuth() bool { return api.authz.WithAuth() }
func (api *API) getToken(ident *meta.Meta) ([]byte, error) {
	return api.token.GetToken(ident, api.tokenLifetime, auth.KindJSON)
func (a *API) withAuth() bool { return a.authz.WithAuth() }
func (a *API) getToken(ident *meta.Meta) ([]byte, error) {
	return a.token.GetToken(ident, a.tokenLifetime, auth.KindJSON)
}

Changes to web/adapter/api/content_type.go.

9
10
11
12
13
14
15


16

17
18
19
20



21
22
23
24



25
26
27
28
29


30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49


50
51
52

53
54
55
56
57
58
59
9
10
11
12
13
14
15
16
17

18
19



20
21
22




23
24
25

26
27


28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47


48
49
50
51

52
53
54
55
56
57
58
59







+
+
-
+

-
-
-
+
+
+
-
-
-
-
+
+
+
-


-
-
+
+


















-
-
+
+


-
+







//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import "zettelstore.de/z/api"

const ctHTML = "text/html; charset=utf-8"
const ctJSON = "application/json"
const plainText = "text/plain; charset=utf-8"
const ctPlainText = "text/plain; charset=utf-8"

var mapFormat2CT = map[api.EncodingEnum]string{
	api.EncoderHTML:   "text/html; charset=utf-8",
	api.EncoderNative: plainText,
var mapEncoding2CT = map[api.EncodingEnum]string{
	api.EncoderHTML:   ctHTML,
	api.EncoderNative: ctPlainText,
	api.EncoderJSON:   "application/json",
	api.EncoderDJSON:  "application/json",
	api.EncoderText:   plainText,
	api.EncoderZmk:    plainText,
	api.EncoderDJSON:  ctJSON,
	api.EncoderText:   ctPlainText,
	api.EncoderZmk:    ctPlainText,
	api.EncoderRaw:    plainText, // In some cases...
}

func format2ContentType(format api.EncodingEnum) string {
	ct, ok := mapFormat2CT[format]
func encoding2ContentType(enc api.EncodingEnum) string {
	ct, ok := mapEncoding2CT[enc]
	if !ok {
		return "application/octet-stream"
	}
	return ct
}

var mapSyntax2CT = map[string]string{
	"css":      "text/css; charset=utf-8",
	"gif":      "image/gif",
	"html":     "text/html; charset=utf-8",
	"jpeg":     "image/jpeg",
	"jpg":      "image/jpeg",
	"js":       "text/javascript; charset=utf-8",
	"pdf":      "application/pdf",
	"png":      "image/png",
	"svg":      "image/svg+xml",
	"xml":      "text/xml; charset=utf-8",
	"zmk":      "text/x-zmk; charset=utf-8",
	"plain":    plainText,
	"text":     plainText,
	"plain":    ctPlainText,
	"text":     ctPlainText,
	"markdown": "text/markdown; charset=utf-8",
	"md":       "text/markdown; charset=utf-8",
	"mustache": plainText,
	"mustache": ctPlainText,
	//"graphviz":      "text/vnd.graphviz; charset=utf-8",
}

func syntax2contentType(syntax string) (string, bool) {
	contentType, ok := mapSyntax2CT[syntax]
	return contentType, ok
}

Changes to web/adapter/api/create_zettel.go.

15
16
17
18
19
20
21
22



























23
24
25

26
27
28

29
30
31
32
33
34
35
36
37
38
39

40
41

42
43
44

45
46
47
48
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

52
53
54

55
56
57
58
59
60
61
62
63
64
65

66
67

68
69
70

71
72
73
74
75








+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+


-
+


-
+










-
+

-
+


-
+




	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakePostCreatePlainZettelHandler creates a new HTTP handler to store content of
// an existing zettel.
func (a *API) MakePostCreatePlainZettelHandler(createZettel usecase.CreateZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zettel, err := buildZettelFromPlainData(r, id.Invalid)
		if err != nil {
			adapter.ReportUsecaseError(w, adapter.NewErrBadRequest(err.Error()))
			return
		}

		newZid, err := createZettel.Run(ctx, zettel)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		u := a.NewURLBuilder('z').SetZid(newZid).String()
		h := w.Header()
		h.Set(zsapi.HeaderContentType, ctPlainText)
		h.Set(zsapi.HeaderLocation, u)
		w.WriteHeader(http.StatusCreated)
		if _, err = w.Write(newZid.Bytes()); err != nil {
			adapter.InternalServerError(w, "Write Plain", err)
		}
	}
}

// MakePostCreateZettelHandler creates a new HTTP handler to store content of
// an existing zettel.
func (api *API) MakePostCreateZettelHandler(createZettel usecase.CreateZettel) http.HandlerFunc {
func (a *API) MakePostCreateZettelHandler(createZettel usecase.CreateZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zettel, err := buildZettelFromData(r, id.Invalid)
		zettel, err := buildZettelFromJSONData(r, id.Invalid)
		if err != nil {
			adapter.ReportUsecaseError(w, adapter.NewErrBadRequest(err.Error()))
			return
		}

		newZid, err := createZettel.Run(ctx, zettel)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		u := api.NewURLBuilder('z').SetZid(newZid).String()
		u := a.NewURLBuilder('j').SetZid(newZid).String()
		h := w.Header()
		h.Set(zsapi.HeaderContentType, format2ContentType(zsapi.EncoderJSON))
		h.Set(zsapi.HeaderContentType, ctJSON)
		h.Set(zsapi.HeaderLocation, u)
		w.WriteHeader(http.StatusCreated)
		if err = encodeJSONData(w, zsapi.ZidJSON{ID: newZid.String(), URL: u}); err != nil {
		if err = encodeJSONData(w, zsapi.ZidJSON{ID: newZid.String()}); err != nil {
			adapter.InternalServerError(w, "Write JSON", err)
		}
	}
}

Changes to web/adapter/api/delete_zettel.go.

16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
16
17
18
19
20
21
22

23
24
25
26
27
28
29
30







-
+








	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeDeleteZettelHandler creates a new HTTP handler to delete a zettel.
func (api *API) MakeDeleteZettelHandler(deleteZettel usecase.DeleteZettel) http.HandlerFunc {
func MakeDeleteZettelHandler(deleteZettel usecase.DeleteZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}

Added web/adapter/api/get_eval_zettel.go.


























































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetEvalZettelHandler creates a new HTTP handler to return a evaluated zettel.
func (a *API) MakeGetEvalZettelHandler(evaluate usecase.Evaluate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}

		ctx := r.Context()
		q := r.URL.Query()
		enc, encStr := adapter.GetEncoding(r, q, encoder.GetDefaultEncoding())
		part := getPart(q, partContent)
		var embedImage bool
		if enc == zsapi.EncoderHTML {
			embedImage = true
		}
		env := evaluator.Environment{
			EmbedImage: embedImage,
		}
		zn, err := evaluate.Run(ctx, zid, q.Get(meta.KeySyntax), &env)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		evalMeta := func(value string) *ast.InlineListNode {
			return evaluate.RunMetadata(ctx, value, &env)
		}
		a.writeEncodedZettelPart(w, zn, evalMeta, enc, encStr, part)
	}
}

Changes to web/adapter/api/get_links.go.

9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45
46
47
48
49
50
51
52



53
54
55
56



57
58
59
60
61
62
63







64
65
66
67





68
69
70


71
72

73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109



110
111
112
113

114
115

116
117
118
119
120
121
122
9
10
11
12
13
14
15

16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35

36
37
38
39
40
41
42









43
44
45




46
47
48







49
50
51
52
53
54
55




56
57
58
59
60



61
62


63
64
65
66
67

































68
69
70
71

72

73
74

75
76
77
78
79
80
81
82







-











-
+








-
+






-
-
-
-
-
-
-
-
-
+
+
+
-
-
-
-
+
+
+
-
-
-
-
-
-
-
+
+
+
+
+
+
+
-
-
-
-
+
+
+
+
+
-
-
-
+
+
-
-
+




-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+

-

-
+

-
+







//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"net/http"
	"strconv"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/collect"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetLinksHandler creates a new API handler to return links to other material.
func (api *API) MakeGetLinksHandler(parseZettel usecase.ParseZettel) http.HandlerFunc {
func MakeGetLinksHandler(evaluate usecase.Evaluate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		ctx := r.Context()
		q := r.URL.Query()
		zn, err := parseZettel.Run(ctx, zid, q.Get(meta.KeySyntax))
		zn, err := evaluate.Run(ctx, zid, q.Get(meta.KeySyntax), nil)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		summary := collect.References(zn)

		kind := getKindFromValue(q.Get("kind"))
		matter := getMatterFromValue(q.Get("matter"))
		if !validKindMatter(kind, matter) {
			adapter.BadRequest(w, "Invalid kind/matter")
			return
		}

		outData := zsapi.ZettelLinksJSON{
			ID:  zid.String(),
		outData := zsapi.ZettelLinksJSON{ID: zid.String()}
		// TODO: calculate incoming links from other zettel (via "backward" metadata?)
		outData.Linked.Incoming = nil
			URL: api.NewURLBuilder('z').SetZid(zid).String(),
		}
		if kind&kindLink != 0 {
			api.setupLinkJSONRefs(summary, matter, &outData)
		zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Links)
		outData.Linked.Outgoing = idRefs(zetRefs)
		outData.Linked.Local = stringRefs(locRefs)
			if matter&matterMeta != 0 {
				for _, p := range zn.Meta.PairsRest(false) {
					if meta.Type(p.Key) == meta.TypeURL {
						outData.Links.Meta = append(outData.Links.Meta, p.Value)
					}
				}
			}
		outData.Linked.External = stringRefs(extRefs)
		for _, p := range zn.Meta.PairsRest(false) {
			if meta.Type(p.Key) == meta.TypeURL {
				outData.Linked.Meta = append(outData.Linked.Meta, p.Value)
			}
		}

		}
		if kind&kindImage != 0 {
			api.setupImageJSONRefs(summary, matter, &outData)
		}
		zetRefs, locRefs, extRefs = collect.DivideReferences(summary.Embeds)
		outData.Embedded.Outgoing = idRefs(zetRefs)
		outData.Embedded.Local = stringRefs(locRefs)
		outData.Embedded.External = stringRefs(extRefs)

		if kind&kindCite != 0 {
			outData.Cites = stringCites(summary.Cites)
		}
		outData.Cites = stringCites(summary.Cites)


		w.Header().Set(zsapi.HeaderContentType, format2ContentType(zsapi.EncoderJSON))
		w.Header().Set(zsapi.HeaderContentType, ctJSON)
		encodeJSONData(w, outData)
	}
}

func (api *API) setupLinkJSONRefs(summary collect.Summary, matter matterType, outData *zsapi.ZettelLinksJSON) {
	if matter&matterIncoming != 0 {
		// TODO: calculate incoming links from other zettel (via "backward" metadata?)
		outData.Links.Incoming = []zsapi.ZidJSON{}
	}
	zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Links)
	if matter&matterOutgoing != 0 {
		outData.Links.Outgoing = api.idURLRefs(zetRefs)
	}
	if matter&matterLocal != 0 {
		outData.Links.Local = stringRefs(locRefs)
	}
	if matter&matterExternal != 0 {
		outData.Links.External = stringRefs(extRefs)
	}
}

func (api *API) setupImageJSONRefs(summary collect.Summary, matter matterType, outData *zsapi.ZettelLinksJSON) {
	zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Images)
	if matter&matterOutgoing != 0 {
		outData.Images.Outgoing = api.idURLRefs(zetRefs)
	}
	if matter&matterLocal != 0 {
		outData.Images.Local = stringRefs(locRefs)
	}
	if matter&matterExternal != 0 {
		outData.Images.External = stringRefs(extRefs)
	}
}

func (api *API) idURLRefs(refs []*ast.Reference) []zsapi.ZidJSON {
	result := make([]zsapi.ZidJSON, 0, len(refs))
	for _, ref := range refs {
func idRefs(refs []*ast.Reference) []string {
	result := make([]string, len(refs))
	for i, ref := range refs {
		path := ref.URL.Path
		ub := api.NewURLBuilder('z').AppendPath(path)
		if fragment := ref.URL.Fragment; len(fragment) > 0 {
			ub.SetFragment(fragment)
			path = path + "#" + fragment
		}
		result = append(result, zsapi.ZidJSON{ID: path, URL: ub.String()})
		result[i] = path
	}
	return result
}

func stringRefs(refs []*ast.Reference) []string {
	result := make([]string, 0, len(refs))
	for _, ref := range refs {
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
92
93
94
95
96
97
98























































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
		if _, ok := mapKey[cn.Key]; !ok {
			mapKey[cn.Key] = true
			result = append(result, cn.Key)
		}
	}
	return result
}

type kindType int

const (
	_ kindType = 1 << iota
	kindLink
	kindImage
	kindCite
)

var mapKind = map[string]kindType{
	"":      kindLink | kindImage | kindCite,
	"link":  kindLink,
	"image": kindImage,
	"cite":  kindCite,
	"both":  kindLink | kindImage,
	"all":   kindLink | kindImage | kindCite,
}

func getKindFromValue(value string) kindType {
	if k, ok := mapKind[value]; ok {
		return k
	}
	if n, err := strconv.Atoi(value); err == nil && n > 0 {
		return kindType(n)
	}
	return 0
}

type matterType int

const (
	_ matterType = 1 << iota
	matterIncoming
	matterOutgoing
	matterLocal
	matterExternal
	matterMeta
)

var mapMatter = map[string]matterType{
	"":         matterIncoming | matterOutgoing | matterLocal | matterExternal | matterMeta,
	"incoming": matterIncoming,
	"outgoing": matterOutgoing,
	"local":    matterLocal,
	"external": matterExternal,
	"meta":     matterMeta,
	"zettel":   matterIncoming | matterOutgoing,
	"material": matterLocal | matterExternal | matterMeta,
	"all":      matterIncoming | matterOutgoing | matterLocal | matterExternal | matterMeta,
}

func getMatterFromValue(value string) matterType {
	if m, ok := mapMatter[value]; ok {
		return m
	}
	if n, err := strconv.Atoi(value); err == nil && n > 0 {
		return matterType(n)
	}
	return 0
}

func validKindMatter(kind kindType, matter matterType) bool {
	if kind == 0 {
		return false
	}
	if kind&kindLink != 0 {
		return matter != 0
	}
	if kind&kindImage != 0 {
		if matter == 0 || matter == matterIncoming {
			return false
		}
		return true
	}
	if kind&kindCite != 0 {
		return matter == matterOutgoing
	}
	return false
}

Changes to web/adapter/api/get_order.go.

18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32
33
34
35
36
37
38

39
40
41







-
+













-
+


	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetOrderHandler creates a new API handler to return zettel references
// of a given zettel.
func (api *API) MakeGetOrderHandler(zettelOrder usecase.ZettelOrder) http.HandlerFunc {
func MakeGetOrderHandler(zettelOrder usecase.ZettelOrder) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		ctx := r.Context()
		q := r.URL.Query()
		start, metas, err := zettelOrder.Run(ctx, zid, q.Get(meta.KeySyntax))
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		api.writeMetaList(w, start, metas)
		writeMetaList(w, start, metas)
	}
}

Added web/adapter/api/get_parsed_zettel.go.

















































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"fmt"
	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetParsedZettelHandler creates a new HTTP handler to return a parsed zettel.
func (a *API) MakeGetParsedZettelHandler(parseZettel usecase.ParseZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}

		q := r.URL.Query()
		enc, encStr := adapter.GetEncoding(r, q, encoder.GetDefaultEncoding())
		part := getPart(q, partContent)
		zn, err := parseZettel.Run(r.Context(), zid, q.Get(meta.KeySyntax))
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		a.writeEncodedZettelPart(w, zn, parser.ParseMetadata, enc, encStr, part)
	}
}

func (a *API) writeEncodedZettelPart(
	w http.ResponseWriter, zn *ast.ZettelNode,
	evalMeta encoder.EvalMetaFunc,
	enc zsapi.EncodingEnum, encStr string, part partType,
) {
	env := encoder.Environment{
		Lang:           config.GetLang(zn.InhMeta, a.rtConfig),
		Xhtml:          false,
		MarkerExternal: "",
		NewWindow:      false,
		IgnoreMeta:     map[string]bool{meta.KeyLang: true},
	}
	encdr := encoder.Create(enc, &env)
	if encdr == nil {
		adapter.BadRequest(w, fmt.Sprintf("Zettel %q not available in encoding %q", zn.Meta.Zid.String(), encStr))
		return
	}
	w.Header().Set(zsapi.HeaderContentType, encoding2ContentType(enc))
	var err error
	switch part {
	case partZettel:
		_, err = encdr.WriteZettel(w, zn, evalMeta)
	case partMeta:
		_, err = encdr.WriteMeta(w, zn.InhMeta, evalMeta)
	case partContent:
		_, err = encdr.WriteContent(w, zn)
	}
	if err != nil {
		adapter.InternalServerError(w, "Write encoded zettel", err)
	}
}

Changes to web/adapter/api/get_role_list.go.

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
33
34
35
36
37


38
39
40
41


42
43
8
9
10
11
12
13
14

15
16
17

18
19
20
21
22

23
24
25
26
27
28
29
30





31
32




33
34









-



-





-
+







-
-
-
-
-
+
+
-
-
-
-
+
+
-
-
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"fmt"
	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeListRoleHandler creates a new HTTP handler for the use case "list some zettel".
func (api *API) MakeListRoleHandler(listRole usecase.ListRole) http.HandlerFunc {
func MakeListRoleHandler(listRole usecase.ListRole) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		roleList, err := listRole.Run(r.Context())
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}

		format, formatText := adapter.GetFormat(r, r.URL.Query(), encoder.GetDefaultFormat())
		switch format {
		case zsapi.EncoderJSON:
			w.Header().Set(zsapi.HeaderContentType, format2ContentType(format))
			encodeJSONData(w, zsapi.RoleListJSON{Roles: roleList})
		w.Header().Set(zsapi.HeaderContentType, ctJSON)
		encodeJSONData(w, zsapi.RoleListJSON{Roles: roleList})
		default:
			adapter.BadRequest(w, fmt.Sprintf("Role list not available in format %q", formatText))
		}

	}
}
	}
}

Changes to web/adapter/api/get_tags_list.go.

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47










48
49
50
51


52
8
9
10
11
12
13
14

15
16
17
18

19
20
21
22
23

24
25
26
27
28
29
30
31
32













33
34
35
36
37
38
39
40
41
42




43
44








-




-





-
+








-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
-
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"fmt"
	"net/http"
	"strconv"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeListTagsHandler creates a new HTTP handler for the use case "list some zettel".
func (api *API) MakeListTagsHandler(listTags usecase.ListTags) http.HandlerFunc {
func MakeListTagsHandler(listTags usecase.ListTags) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		iMinCount, _ := strconv.Atoi(r.URL.Query().Get("min"))
		tagData, err := listTags.Run(r.Context(), iMinCount)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}

		format, formatText := adapter.GetFormat(r, r.URL.Query(), encoder.GetDefaultFormat())
		switch format {
		case zsapi.EncoderJSON:
			w.Header().Set(zsapi.HeaderContentType, format2ContentType(format))
			tagMap := make(map[string][]string, len(tagData))
			for tag, metaList := range tagData {
				zidList := make([]string, 0, len(metaList))
				for _, m := range metaList {
					zidList = append(zidList, m.Zid.String())
				}
				tagMap[tag] = zidList
			}
			encodeJSONData(w, zsapi.TagListJSON{Tags: tagMap})
		w.Header().Set(zsapi.HeaderContentType, ctJSON)
		tagMap := make(map[string][]string, len(tagData))
		for tag, metaList := range tagData {
			zidList := make([]string, 0, len(metaList))
			for _, m := range metaList {
				zidList = append(zidList, m.Zid.String())
			}
			tagMap[tag] = zidList
		}
		encodeJSONData(w, zsapi.TagListJSON{Tags: tagMap})
		default:
			adapter.BadRequest(w, fmt.Sprintf("Tags list not available in format %q", formatText))
		}
	}
	}
}
}

Changes to web/adapter/api/get_zettel.go.

8
9
10
11
12
13
14
15
16

17
18
19
20
21
22
23
24


25
26
27
28
29
30
31


32
33

34
35
36
37
38
39
40
41
42
43
44









45
46
47

48
49
50

51
52
53
54
55
56


57
58
59
60





61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76


77
78
79

80
81
82
83
84

85
86
87
88
89
90


91
92
93
94
95
96

97
98
99
100
101








102
103

104
105
106
107





108
109
110
111
112
113
114

115
116
117

118
119
120







121
122
123
124
125
126



127


128
129

130
8
9
10
11
12
13
14


15
16
17
18

19
20


21
22

23
24
25
26


27
28
29

30
31

32
33







34
35
36
37
38
39
40
41
42

43

44

45

46






47
48




49
50
51
52
53


54
55












56
57
58


59





60






61
62






63





64
65
66
67
68
69
70
71


72




73
74
75
76
77







78



79



80
81
82
83
84
85
86






87
88
89

90
91
92

93
94







-
-
+



-


-
-
+
+
-




-
-
+
+

-
+

-


-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
-

-
+
-

-
+
-
-
-
-
-
-
+
+
-
-
-
-
+
+
+
+
+
-
-


-
-
-
-
-
-
-
-
-
-
-
-
+
+

-
-
+
-
-
-
-
-
+
-
-
-
-
-
-
+
+
-
-
-
-
-
-
+
-
-
-
-
-
+
+
+
+
+
+
+
+
-
-
+
-
-
-
-
+
+
+
+
+
-
-
-
-
-
-
-
+
-
-
-
+
-
-
-
+
+
+
+
+
+
+
-
-
-
-
-
-
+
+
+
-
+
+

-
+

// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"errors"
	"fmt"
	"context"
	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetZettelHandler creates a new HTTP handler to return a rendered zettel.
func (api *API) MakeGetZettelHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc {
// MakeGetZettelHandler creates a new HTTP handler to return a zettel.
func MakeGetZettelHandler(getZettel usecase.GetZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		z, err := getZettelFromPath(r.Context(), w, r, getZettel)
		if err != nil {
			http.NotFound(w, r)
			return
		}

		ctx := r.Context()
		q := r.URL.Query()
		format, _ := adapter.GetFormat(r, q, encoder.GetDefaultFormat())
		if format == zsapi.EncoderRaw {
			ctx = box.NoEnrichContext(ctx)
		}

		w.Header().Set(zsapi.HeaderContentType, ctJSON)
		content, encoding := z.Content.Encode()
		err = encodeJSONData(w, zsapi.ZettelJSON{
			ID:       z.Meta.Zid.String(),
			Meta:     z.Meta.Map(),
			Encoding: encoding,
			Content:  content,
		})
		zn, err := parseZettel.Run(ctx, zid, q.Get(meta.KeySyntax))
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			adapter.InternalServerError(w, "Write Zettel JSON", err)
			return
		}

	}
		part := getPart(q, partZettel)
		if part == partUnknown {
			adapter.BadRequest(w, "Unknown _part parameter")
			return
		}
		switch format {
}

		case zsapi.EncoderJSON, zsapi.EncoderDJSON:
			w.Header().Set(zsapi.HeaderContentType, format2ContentType(format))
			err = api.getWriteMetaZettelFunc(ctx, format, part, partZettel, getMeta)(w, zn)
			if err != nil {
// MakeGetPlainZettelHandler creates a new HTTP handler to return a zettel in plain formar
func (a *API) MakeGetPlainZettelHandler(getZettel usecase.GetZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		z, err := getZettelFromPath(box.NoEnrichContext(r.Context()), w, r, getZettel)
		if err != nil {
				adapter.InternalServerError(w, "Write D/JSON", err)
			}
			return
		}

		env := encoder.Environment{
			LinkAdapter:    adapter.MakeLinkAdapter(ctx, api, 'z', getMeta, part.DefString(partZettel), format),
			ImageAdapter:   adapter.MakeImageAdapter(ctx, api, getMeta),
			CiteAdapter:    nil,
			Lang:           config.GetLang(zn.InhMeta, api.rtConfig),
			Xhtml:          false,
			MarkerExternal: "",
			NewWindow:      false,
			IgnoreMeta:     map[string]bool{meta.KeyLang: true},
		}
		switch part {

		switch getPart(r.URL.Query(), partContent) {
		case partZettel:
			err = writeZettelPartZettel(w, zn, format, env)
		case partMeta:
			_, err = z.Meta.Write(w, false)
			err = writeZettelPartMeta(w, zn, format)
		case partContent:
			err = api.writeZettelPartContent(w, zn, format, env)
		}
		if err != nil {
			if err == nil {
			if errors.Is(err, adapter.ErrNoSuchFormat) {
				adapter.BadRequest(w, fmt.Sprintf("Zettel %q not available in format %q", zid.String(), format))
				return
			}
			adapter.InternalServerError(w, "Get zettel", err)
		}
				_, err = w.Write([]byte{'\n'})
			}
	}
}

func writeZettelPartZettel(w http.ResponseWriter, zn *ast.ZettelNode, format zsapi.EncodingEnum, env encoder.Environment) error {
	enc := encoder.Create(format, &env)
	if enc == nil {
			if err == nil {
		return adapter.ErrNoSuchFormat
	}
	inhMeta := false
	if format != zsapi.EncoderRaw {
		w.Header().Set(zsapi.HeaderContentType, format2ContentType(format))
				_, err = z.Content.Write(w)
			}
		case partMeta:
			w.Header().Set(zsapi.HeaderContentType, ctPlainText)
			_, err = z.Meta.Write(w, false)
		case partContent:
			if ct, ok := syntax2contentType(config.GetSyntax(z.Meta, a.rtConfig)); ok {
				w.Header().Set(zsapi.HeaderContentType, ct)
		inhMeta = true
	}
			}
	_, err := enc.WriteZettel(w, zn, inhMeta)
	return err
}

			_, err = z.Content.Write(w)
		}
		if err != nil {
			adapter.InternalServerError(w, "Write plain zettel", err)
		}
func writeZettelPartMeta(w http.ResponseWriter, zn *ast.ZettelNode, format zsapi.EncodingEnum) error {
	w.Header().Set(zsapi.HeaderContentType, format2ContentType(format))
	if enc := encoder.Create(format, nil); enc != nil {
		if format == zsapi.EncoderRaw {
			_, err := enc.WriteMeta(w, zn.Meta)
			return err
		}
	}
		_, err := enc.WriteMeta(w, zn.InhMeta)
		return err
	}
}
	return adapter.ErrNoSuchFormat
}


func getZettelFromPath(ctx context.Context, w http.ResponseWriter, r *http.Request, getZettel usecase.GetZettel) (domain.Zettel, error) {
	zid, err := id.Parse(r.URL.Path[1:])
	if err != nil {
		http.NotFound(w, r)
		return domain.Zettel{}, err
	}
func (api *API) writeZettelPartContent(w http.ResponseWriter, zn *ast.ZettelNode, format zsapi.EncodingEnum, env encoder.Environment) error {
	if format == zsapi.EncoderRaw {
		if ct, ok := syntax2contentType(config.GetSyntax(zn.Meta, api.rtConfig)); ok {
			w.Header().Add(zsapi.HeaderContentType, ct)
		}
	} else {

	z, err := getZettel.Run(ctx, zid)
	if err != nil {
		w.Header().Set(zsapi.HeaderContentType, format2ContentType(format))
		adapter.ReportUsecaseError(w, err)
		return domain.Zettel{}, err
	}
	return writeContent(w, zn, format, &env)
	return z, nil
}

Changes to web/adapter/api/get_zettel_context.go.

17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31







-
+







	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeZettelContextHandler creates a new HTTP handler for the use case "zettel context".
func (api *API) MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc {
func MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		q := r.URL.Query()
40
41
42
43
44
45
46
47

48
49
40
41
42
43
44
45
46

47
48
49







-
+


		}
		ctx := r.Context()
		metaList, err := getContext.Run(ctx, zid, dir, depth, limit)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		api.writeMetaList(w, metaList[0], metaList[1:])
		writeMetaList(w, metaList[0], metaList[1:])
	}
}

Changes to web/adapter/api/get_zettel_list.go.

12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28
29

30
31
32
33
34
35
36

37
38
39
40



41
42
43
44
45















46









47

48
49
50
51
52
53
54
55
56




57
58
59
60
61

62
63
64


65
66
67
68
69
70
71
72
73
74
75

76
77
78
79
80
81
82
83
84
85
86
87
12
13
14
15
16
17
18


19


20
21
22
23
24


25



26
27
28

29




30
31
32
33
34



35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

60
61
62
63
64
65




66
67
68
69





70



71
72









73

74

75






76


77







-
-
+
-
-





-
-
+
-
-
-



-
+
-
-
-
-
+
+
+


-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

+
+
+
+
+
+
+
+
+
-
+





-
-
-
-
+
+
+
+
-
-
-
-
-
+
-
-
-
+
+
-
-
-
-
-
-
-
-
-

-
+
-

-
-
-
-
-
-

-
-

package api

import (
	"fmt"
	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/config"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeListMetaHandler creates a new HTTP handler for the use case "list some zettel".
func (api *API) MakeListMetaHandler(
	listMeta usecase.ListMeta,
func MakeListMetaHandler(listMeta usecase.ListMeta) http.HandlerFunc {
	getMeta usecase.GetMeta,
	parseZettel usecase.ParseZettel,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		q := r.URL.Query()
		s := adapter.GetSearch(q, false)
		s := adapter.GetSearch(q)
		format, formatText := adapter.GetFormat(r, q, encoder.GetDefaultFormat())
		part := getPart(q, partMeta)
		if part == partUnknown {
			adapter.BadRequest(w, "Unknown _part parameter")
		metaList, err := listMeta.Run(ctx, s)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		ctx1 := ctx
		if format == zsapi.EncoderHTML || (!s.HasComputedMetaKey() && (part == partID || part == partContent)) {
			ctx1 = box.NoEnrichContext(ctx1)

		result := make([]zsapi.ZidMetaJSON, 0, len(metaList))
		for _, m := range metaList {
			result = append(result, zsapi.ZidMetaJSON{
				ID:   m.Zid.String(),
				Meta: m.Map(),
			})
		}

		w.Header().Set(zsapi.HeaderContentType, ctJSON)
		err = encodeJSONData(w, zsapi.ZettelListJSON{
			List: result,
		})
		if err != nil {
			adapter.InternalServerError(w, "Write Zettel list JSON", err)
		}
	}
}

// MakeListPlainHandler creates a new HTTP handler for the use case "list some zettel".
func (a *API) MakeListPlainHandler(listMeta usecase.ListMeta) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		q := r.URL.Query()
		s := adapter.GetSearch(q)
		metaList, err := listMeta.Run(ctx1, s)
		metaList, err := listMeta.Run(ctx, s)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}

		w.Header().Set(zsapi.HeaderContentType, format2ContentType(format))
		switch format {
		case zsapi.EncoderHTML:
			api.renderListMetaHTML(w, metaList)
		w.Header().Set(zsapi.HeaderContentType, ctPlainText)
		for _, m := range metaList {
			_, err = fmt.Fprintln(w, m.Zid.String(), config.GetTitle(m, a.rtConfig))
			if err != nil {
		case zsapi.EncoderJSON, zsapi.EncoderDJSON:
			api.renderListMetaXJSON(ctx, w, metaList, format, part, partMeta, getMeta, parseZettel)
		case zsapi.EncoderNative, zsapi.EncoderRaw, zsapi.EncoderText, zsapi.EncoderZmk:
			adapter.NotImplemented(w, fmt.Sprintf("Zettel list in format %q not yet implemented", formatText))
		default:
				break
			adapter.BadRequest(w, fmt.Sprintf("Zettel list not available in format %q", formatText))
		}
	}
			}
		}
}

func (api *API) renderListMetaHTML(w http.ResponseWriter, metaList []*meta.Meta) {
	env := encoder.Environment{Interactive: true}
	buf := encoder.NewBufWriter(w)
	buf.WriteStrings("<html lang=\"", api.rtConfig.GetDefaultLang(), "\">\n<body>\n<ul>\n")
	for _, m := range metaList {
		title := m.GetDefault(meta.KeyTitle, "")
		htmlTitle, err := adapter.FormatInlines(parser.ParseMetadata(title), zsapi.EncoderHTML, &env)
		if err != nil {
			adapter.InternalServerError(w, "Format HTML inlines", err)
			adapter.InternalServerError(w, "Write Zettel list plain", err)
			return
		}
		buf.WriteStrings(
			"<li><a href=\"",
			api.NewURLBuilder('z').SetZid(m.Zid).AppendQuery(zsapi.QueryKeyFormat, zsapi.FormatHTML).String(),
			"\">",
			htmlTitle,
			"</a></li>\n")
	}
	buf.WriteString("</ul>\n</body>\n</html>")
	buf.Flush()
}

Changes to web/adapter/api/json.go.

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302

303
304
305
306
307
308
309

310
311
312
313
314
315
316
317
318

319
320
321
322
323
324
325
8
9
10
11
12
13
14

15
16
17
18
19

20
21
22



23










































































































































































































































































24
25
26
27
28
29
30

31
32
33
34

35
36

37
38
39

40
41
42
43
44

45
46
47
48
49
50
51
52







-





-



-
-
-

-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







-
+



-


-
+


-





-
+







// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"context"
	"encoding/json"
	"io"
	"net/http"

	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

type jsonContent struct {
	ID       string `json:"id"`
	URL      string `json:"url"`
	Encoding string `json:"encoding"`
	Content  string `json:"content"`
}

var (
	djsonMetaHeader    = []byte(",\"meta\":")
	djsonContentHeader = []byte(",\"content\":")
	djsonHeader1       = []byte("{\"id\":\"")
	djsonHeader2       = []byte("\",\"url\":\"")
	djsonHeader3       = []byte("?_format=")
	djsonHeader4       = []byte("\"")
	djsonFooter        = []byte("}")
)

func (api *API) writeDJSONHeader(w io.Writer, zid id.Zid) error {
	_, err := w.Write(djsonHeader1)
	if err == nil {
		_, err = w.Write(zid.Bytes())
	}
	if err == nil {
		_, err = w.Write(djsonHeader2)
	}
	if err == nil {
		_, err = io.WriteString(w, api.NewURLBuilder('z').SetZid(zid).String())
	}
	if err == nil {
		_, err = w.Write(djsonHeader3)
		if err == nil {
			_, err = io.WriteString(w, zsapi.FormatDJSON)
		}
	}
	if err == nil {
		_, err = w.Write(djsonHeader4)
	}
	return err
}

func (api *API) renderListMetaXJSON(
	ctx context.Context,
	w http.ResponseWriter,
	metaList []*meta.Meta,
	format zsapi.EncodingEnum,
	part, defPart partType,
	getMeta usecase.GetMeta,
	parseZettel usecase.ParseZettel,
) {
	prepareZettel := api.getPrepareZettelFunc(ctx, parseZettel, part)
	writeZettel := api.getWriteMetaZettelFunc(ctx, format, part, defPart, getMeta)
	err := writeListXJSON(w, metaList, prepareZettel, writeZettel)
	if err != nil {
		adapter.InternalServerError(w, "Get list", err)
	}
}

type prepareZettelFunc func(m *meta.Meta) (*ast.ZettelNode, error)

func (api *API) getPrepareZettelFunc(ctx context.Context, parseZettel usecase.ParseZettel, part partType) prepareZettelFunc {
	switch part {
	case partZettel, partContent:
		return func(m *meta.Meta) (*ast.ZettelNode, error) {
			return parseZettel.Run(ctx, m.Zid, "")
		}
	case partMeta, partID:
		return func(m *meta.Meta) (*ast.ZettelNode, error) {
			return &ast.ZettelNode{
				Meta:    m,
				Content: domain.NewContent(""),
				Zid:     m.Zid,
				InhMeta: api.rtConfig.AddDefaultValues(m),
				Ast:     nil,
			}, nil
		}
	}
	return nil
}

type writeZettelFunc func(io.Writer, *ast.ZettelNode) error

func (api *API) getWriteMetaZettelFunc(ctx context.Context, format zsapi.EncodingEnum,
	part, defPart partType, getMeta usecase.GetMeta) writeZettelFunc {
	switch part {
	case partZettel:
		return api.getWriteZettelFunc(ctx, format, defPart, getMeta)
	case partMeta:
		return api.getWriteMetaFunc(ctx, format)
	case partContent:
		return api.getWriteContentFunc(ctx, format, defPart, getMeta)
	case partID:
		return api.getWriteIDFunc(ctx, format)
	default:
		panic(part)
	}
}

func (api *API) getWriteZettelFunc(ctx context.Context, format zsapi.EncodingEnum,
	defPart partType, getMeta usecase.GetMeta) writeZettelFunc {
	if format == zsapi.EncoderJSON {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			content, encoding := zn.Content.Encode()
			return encodeJSONData(w, zsapi.ZettelJSON{
				ID:       zn.Zid.String(),
				URL:      api.NewURLBuilder('z').SetZid(zn.Zid).String(),
				Meta:     zn.InhMeta.Map(),
				Encoding: encoding,
				Content:  content,
			})
		}
	}
	enc := encoder.Create(zsapi.EncoderDJSON, nil)
	if enc == nil {
		panic("no DJSON encoder found")
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonMetaHeader)
		if err != nil {
			return err
		}
		_, err = enc.WriteMeta(w, zn.InhMeta)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonContentHeader)
		if err != nil {
			return err
		}
		err = writeContent(w, zn, zsapi.EncoderDJSON, &encoder.Environment{
			LinkAdapter: adapter.MakeLinkAdapter(
				ctx, api, 'z', getMeta, partZettel.DefString(defPart), zsapi.EncoderDJSON),
			ImageAdapter: adapter.MakeImageAdapter(ctx, api, getMeta)})
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}
func (api *API) getWriteMetaFunc(ctx context.Context, format zsapi.EncodingEnum) writeZettelFunc {
	if format == zsapi.EncoderJSON {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			return encodeJSONData(w, zsapi.ZidMetaJSON{
				ID:   zn.Zid.String(),
				URL:  api.NewURLBuilder('z').SetZid(zn.Zid).String(),
				Meta: zn.InhMeta.Map(),
			})
		}
	}
	enc := encoder.Create(zsapi.EncoderDJSON, nil)
	if enc == nil {
		panic("no DJSON encoder found")
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonMetaHeader)
		if err != nil {
			return err
		}
		_, err = enc.WriteMeta(w, zn.InhMeta)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}
func (api *API) getWriteContentFunc(ctx context.Context, format zsapi.EncodingEnum,
	defPart partType, getMeta usecase.GetMeta) writeZettelFunc {
	if format == zsapi.EncoderJSON {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			content, encoding := zn.Content.Encode()
			return encodeJSONData(w, jsonContent{
				ID:       zn.Zid.String(),
				URL:      api.NewURLBuilder('z').SetZid(zn.Zid).String(),
				Encoding: encoding,
				Content:  content,
			})
		}
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonContentHeader)
		if err != nil {
			return err
		}
		err = writeContent(w, zn, zsapi.EncoderDJSON, &encoder.Environment{
			LinkAdapter: adapter.MakeLinkAdapter(
				ctx, api, 'z', getMeta, partContent.DefString(defPart), zsapi.EncoderDJSON),
			ImageAdapter: adapter.MakeImageAdapter(ctx, api, getMeta)})
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}
func (api *API) getWriteIDFunc(ctx context.Context, format zsapi.EncodingEnum) writeZettelFunc {
	if format == zsapi.EncoderJSON {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			return encodeJSONData(w, zsapi.ZidJSON{
				ID:  zn.Zid.String(),
				URL: api.NewURLBuilder('z').SetZid(zn.Zid).String(),
			})
		}
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}

var (
	jsonListHeader = []byte("{\"list\":[")
	jsonListSep    = []byte{','}
	jsonListFooter = []byte("]}")
)

func writeListXJSON(w http.ResponseWriter, metaList []*meta.Meta, prepareZettel prepareZettelFunc, writeZettel writeZettelFunc) error {
	_, err := w.Write(jsonListHeader)
	for i, m := range metaList {
		if err != nil {
			return err
		}
		if i > 0 {
			_, err = w.Write(jsonListSep)
			if err != nil {
				return err
			}
		}
		zn, err1 := prepareZettel(m)
		if err1 != nil {
			return err1
		}
		err = writeZettel(w, zn)
	}
	if err == nil {
		_, err = w.Write(jsonListFooter)
	}
	return err
}

func writeContent(w io.Writer, zn *ast.ZettelNode, format zsapi.EncodingEnum, env *encoder.Environment) error {
	enc := encoder.Create(format, env)
	if enc == nil {
		return adapter.ErrNoSuchFormat
	}

	_, err := enc.WriteContent(w, zn)
	return err
}

func encodeJSONData(w io.Writer, data interface{}) error {
	enc := json.NewEncoder(w)
	enc.SetEscapeHTML(false)
	return enc.Encode(data)
}

func (api *API) writeMetaList(w http.ResponseWriter, m *meta.Meta, metaList []*meta.Meta) error {
func writeMetaList(w http.ResponseWriter, m *meta.Meta, metaList []*meta.Meta) error {
	outList := make([]zsapi.ZidMetaJSON, len(metaList))
	for i, m := range metaList {
		outList[i].ID = m.Zid.String()
		outList[i].URL = api.NewURLBuilder('z').SetZid(m.Zid).String()
		outList[i].Meta = m.Map()
	}
	w.Header().Set(zsapi.HeaderContentType, format2ContentType(zsapi.EncoderJSON))
	w.Header().Set(zsapi.HeaderContentType, ctJSON)
	return encodeJSONData(w, zsapi.ZidMetaRelatedList{
		ID:   m.Zid.String(),
		URL:  api.NewURLBuilder('z').SetZid(m.Zid).String(),
		Meta: m.Map(),
		List: outList,
	})
}

func buildZettelFromData(r *http.Request, zid id.Zid) (domain.Zettel, error) {
func buildZettelFromJSONData(r *http.Request, zid id.Zid) (domain.Zettel, error) {
	var zettel domain.Zettel
	dec := json.NewDecoder(r.Body)
	var zettelData zsapi.ZettelDataJSON
	if err := dec.Decode(&zettelData); err != nil {
		return zettel, err
	}
	m := meta.New(zid)

Changes to web/adapter/api/login.go.

19
20
21
22
23
24
25
26

27
28
29


30
31
32
33
34
35
36

37
38
39
40
41
42
43
44
45
46
47
48
49


50
51
52
53
54
55
56
19
20
21
22
23
24
25

26
27


28
29
30
31
32
33
34
35

36
37
38
39
40
41
42
43
44
45
46
47


48
49
50
51
52
53
54
55
56







-
+

-
-
+
+






-
+











-
-
+
+







	zsapi "zettelstore.de/z/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakePostLoginHandler creates a new HTTP handler to authenticate the given user via API.
func (api *API) MakePostLoginHandler(ucAuth usecase.Authenticate) http.HandlerFunc {
func (a *API) MakePostLoginHandler(ucAuth usecase.Authenticate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		if !api.withAuth() {
			w.Header().Set(zsapi.HeaderContentType, format2ContentType(zsapi.EncoderJSON))
		if !a.withAuth() {
			w.Header().Set(zsapi.HeaderContentType, ctJSON)
			writeJSONToken(w, "freeaccess", 24*366*10*time.Hour)
			return
		}
		var token []byte
		if ident, cred := retrieveIdentCred(r); ident != "" {
			var err error
			token, err = ucAuth.Run(r.Context(), ident, cred, api.tokenLifetime, auth.KindJSON)
			token, err = ucAuth.Run(r.Context(), ident, cred, a.tokenLifetime, auth.KindJSON)
			if err != nil {
				adapter.ReportUsecaseError(w, err)
				return
			}
		}
		if len(token) == 0 {
			w.Header().Set("WWW-Authenticate", `Bearer realm="Default"`)
			http.Error(w, "Authentication failed", http.StatusUnauthorized)
			return
		}

		w.Header().Set(zsapi.HeaderContentType, format2ContentType(zsapi.EncoderJSON))
		writeJSONToken(w, string(token), api.tokenLifetime)
		w.Header().Set(zsapi.HeaderContentType, ctJSON)
		writeJSONToken(w, string(token), a.tokenLifetime)
	}
}

func retrieveIdentCred(r *http.Request) (string, string) {
	if ident, cred, ok := adapter.GetCredentialsViaForm(r); ok {
		return ident, cred
	}
66
67
68
69
70
71
72
73

74
75
76

77
78
79
80
81
82
83
84
85

86
87
88
89
90
91

92
93
94
95
96
97


98
99
66
67
68
69
70
71
72

73
74
75

76
77
78
79
80
81
82
83
84

85
86
87
88
89
90

91
92
93
94
95


96
97
98
99







-
+


-
+








-
+





-
+




-
-
+
+


		Token:   token,
		Type:    "Bearer",
		Expires: int(lifetime / time.Second),
	})
}

// MakeRenewAuthHandler creates a new HTTP handler to renew the authenticate of a user.
func (api *API) MakeRenewAuthHandler() http.HandlerFunc {
func (a *API) MakeRenewAuthHandler() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		authData := api.getAuthData(ctx)
		authData := a.getAuthData(ctx)
		if authData == nil || len(authData.Token) == 0 || authData.User == nil {
			adapter.BadRequest(w, "Not authenticated")
			return
		}
		totalLifetime := authData.Expires.Sub(authData.Issued)
		currentLifetime := authData.Now.Sub(authData.Issued)
		// If we are in the first quarter of the tokens lifetime, return the token
		if currentLifetime*4 < totalLifetime {
			w.Header().Set(zsapi.HeaderContentType, format2ContentType(zsapi.EncoderJSON))
			w.Header().Set(zsapi.HeaderContentType, ctJSON)
			writeJSONToken(w, string(authData.Token), totalLifetime-currentLifetime)
			return
		}

		// Token is a little bit aged. Create a new one
		token, err := api.getToken(authData.User)
		token, err := a.getToken(authData.User)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		w.Header().Set(zsapi.HeaderContentType, format2ContentType(zsapi.EncoderJSON))
		writeJSONToken(w, string(token), api.tokenLifetime)
		w.Header().Set(zsapi.HeaderContentType, ctJSON)
		writeJSONToken(w, string(token), a.tokenLifetime)
	}
}

Changes to web/adapter/api/rename_zettel.go.

18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32







-
+







	"zettelstore.de/z/api"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeRenameZettelHandler creates a new HTTP handler to update a zettel.
func (api *API) MakeRenameZettelHandler(renameZettel usecase.RenameZettel) http.HandlerFunc {
func MakeRenameZettelHandler(renameZettel usecase.RenameZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		newZid, found := getDestinationZid(r)

Changes to web/adapter/api/request.go.

8
9
10
11
12
13
14


15
16
17




18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

43
44
45

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69














8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

29

30
31
32
33
34
35

36
37
38
39
40
41





42
43
44

45
46
47
48
49


50
51
52
53
54
55


56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79







+
+



+
+
+
+





-
+
-






-






-
-
-
-
-
+


-
+




-
-






-
-










+
+
+
+
+
+
+
+
+
+
+
+
+
+
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"io"
	"net/http"
	"net/url"

	"zettelstore.de/z/api"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
)

type partType int

const (
	partUnknown partType = iota
	_ partType = iota
	partID
	partMeta
	partContent
	partZettel
)

var partMap = map[string]partType{
	api.PartID:      partID,
	api.PartMeta:    partMeta,
	api.PartContent: partContent,
	api.PartZettel:  partZettel,
}

func getPart(q url.Values, defPart partType) partType {
	p := q.Get(api.QueryKeyPart)
	if p == "" {
		return defPart
	}
	if part, ok := partMap[p]; ok {
	if part, ok := partMap[q.Get(api.QueryKeyPart)]; ok {
		return part
	}
	return partUnknown
	return defPart
}

func (p partType) String() string {
	switch p {
	case partID:
		return "id"
	case partMeta:
		return "meta"
	case partContent:
		return "content"
	case partZettel:
		return "zettel"
	case partUnknown:
		return "unknown"
	}
	return ""
}

func (p partType) DefString(defPart partType) string {
	if p == defPart {
		return ""
	}
	return p.String()
}

func buildZettelFromPlainData(r *http.Request, zid id.Zid) (domain.Zettel, error) {
	b, err := io.ReadAll(r.Body)
	if err != nil {
		return domain.Zettel{}, err
	}
	inp := input.NewInput(string(b))
	m := meta.NewFromInput(zid, inp)
	return domain.Zettel{
		Meta:    m,
		Content: domain.NewContent(inp.Src[inp.Pos:]),
	}, nil

}

Changes to web/adapter/api/update_zettel.go.

14
15
16
17
18
19
20
21





















22
23

24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50

51
52
53
54
55
56
57
58
59
60
61
62








+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
+






-
+











import (
	"net/http"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeUpdatePlainZettelHandler creates a new HTTP handler to update a zettel.
func MakeUpdatePlainZettelHandler(updateZettel usecase.UpdateZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		zettel, err := buildZettelFromPlainData(r, zid)
		if err != nil {
			adapter.ReportUsecaseError(w, adapter.NewErrBadRequest(err.Error()))
			return
		}
		if err := updateZettel.Run(r.Context(), zettel, true); err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		w.WriteHeader(http.StatusNoContent)
	}
}

// MakeUpdateZettelHandler creates a new HTTP handler to update a zettel.
func (api *API) MakeUpdateZettelHandler(updateZettel usecase.UpdateZettel) http.HandlerFunc {
func MakeUpdateZettelHandler(updateZettel usecase.UpdateZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		zettel, err := buildZettelFromData(r, zid)
		zettel, err := buildZettelFromJSONData(r, zid)
		if err != nil {
			adapter.ReportUsecaseError(w, adapter.NewErrBadRequest(err.Error()))
			return
		}
		if err := updateZettel.Run(r.Context(), zettel, true); err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		w.WriteHeader(http.StatusNoContent)
	}
}

Deleted web/adapter/encoding.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138










































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package adapter provides handlers for web requests.
package adapter

import (
	"context"
	"errors"
	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/server"
)

// ErrNoSuchFormat signals an unsupported encoding format
var ErrNoSuchFormat = errors.New("no such format")

// FormatInlines returns a string representation of the inline slice.
func FormatInlines(is ast.InlineSlice, format api.EncodingEnum, env *encoder.Environment) (string, error) {
	enc := encoder.Create(format, env)
	if enc == nil {
		return "", ErrNoSuchFormat
	}

	var content strings.Builder
	_, err := enc.WriteInlines(&content, is)
	if err != nil {
		return "", err
	}
	return content.String(), nil
}

// MakeLinkAdapter creates an adapter to change a link node during encoding.
func MakeLinkAdapter(
	ctx context.Context,
	b server.Builder,
	key byte,
	getMeta usecase.GetMeta,
	part string,
	format api.EncodingEnum,
) func(*ast.LinkNode) ast.InlineNode {
	return func(origLink *ast.LinkNode) ast.InlineNode {
		origRef := origLink.Ref
		if origRef == nil {
			return origLink
		}
		if origRef.State == ast.RefStateBased {
			newLink := *origLink
			urlPrefix := b.GetURLPrefix()
			newRef := ast.ParseReference(urlPrefix + origRef.Value[1:])
			newRef.State = ast.RefStateHosted
			newLink.Ref = newRef
			return &newLink
		}
		if origRef.State != ast.RefStateZettel {
			return origLink
		}
		zid, err := id.Parse(origRef.URL.Path)
		if err != nil {
			panic(err)
		}
		_, err = getMeta.Run(box.NoEnrichContext(ctx), zid)
		if errors.Is(err, &box.ErrNotAllowed{}) {
			return &ast.FormatNode{
				Kind:    ast.FormatSpan,
				Attrs:   origLink.Attrs,
				Inlines: origLink.Inlines,
			}
		}
		var newRef *ast.Reference
		if err == nil {
			ub := b.NewURLBuilder(key).SetZid(zid)
			if part != "" {
				ub.AppendQuery(api.QueryKeyPart, part)
			}
			if format != api.EncoderUnknown {
				ub.AppendQuery(api.QueryKeyFormat, format.String())
			}
			if fragment := origRef.URL.EscapedFragment(); fragment != "" {
				ub.SetFragment(fragment)
			}

			newRef = ast.ParseReference(ub.String())
			newRef.State = ast.RefStateFound
		} else {
			newRef = ast.ParseReference(origRef.Value)
			newRef.State = ast.RefStateBroken
		}
		newLink := *origLink
		newLink.Ref = newRef
		return &newLink
	}
}

// MakeImageAdapter creates an adapter to change an image node during encoding.
func MakeImageAdapter(ctx context.Context, b server.Builder, getMeta usecase.GetMeta) func(*ast.ImageNode) ast.InlineNode {
	return func(origImage *ast.ImageNode) ast.InlineNode {
		if origImage.Ref == nil {
			return origImage
		}
		switch origImage.Ref.State {
		case ast.RefStateInvalid:
			return createZettelImage(b, origImage, id.EmojiZid, ast.RefStateInvalid)
		case ast.RefStateZettel:
			zid, err := id.Parse(origImage.Ref.Value)
			if err != nil {
				panic(err)
			}
			_, err = getMeta.Run(box.NoEnrichContext(ctx), zid)
			if err != nil {
				return createZettelImage(b, origImage, id.EmojiZid, ast.RefStateBroken)
			}
			return createZettelImage(b, origImage, zid, ast.RefStateFound)
		}
		return origImage
	}
}

func createZettelImage(b server.Builder, origImage *ast.ImageNode, zid id.Zid, state ast.RefState) *ast.ImageNode {
	newImage := *origImage
	newImage.Ref = ast.ParseReference(
		b.NewURLBuilder('z').SetZid(zid).AppendQuery("_part", "content").AppendQuery("_format", "raw").String())
	newImage.Ref.State = state
	return &newImage
}

Changes to web/adapter/request.go.

47
48
49
50
51
52
53
54
55
56
57
58





59
60
61


62
63
64


65
66

67
68
69

70
71
72
73


74
75
76
77
78
79
80
81
82



83
84
85

86
87
88


89
90
91
92

93
94
95
96

97
98

99
100

101
102

103
104

105
106
107

108
109
110
111
112
113
114
47
48
49
50
51
52
53





54
55
56
57
58
59


60
61
62


63
64
65

66
67
68

69
70
71


72
73
74
75
76
77
78
79



80
81
82
83
84

85
86


87
88
89
90
91

92

93
94

95
96

97
98

99
100

101
102

103
104
105

106
107
108
109
110
111
112
113







-
-
-
-
-
+
+
+
+
+

-
-
+
+

-
-
+
+

-
+


-
+


-
-
+
+






-
-
-
+
+
+


-
+

-
-
+
+



-
+
-


-
+

-
+

-
+

-
+

-
+


-
+







		if val, err := strconv.Atoi(s); err == nil {
			return val, true
		}
	}
	return 0, false
}

// GetFormat returns the data format selected by the caller.
func GetFormat(r *http.Request, q url.Values, defFormat api.EncodingEnum) (api.EncodingEnum, string) {
	format := q.Get(api.QueryKeyFormat)
	if len(format) > 0 {
		return api.Encoder(format), format
// GetEncoding returns the data encoding selected by the caller.
func GetEncoding(r *http.Request, q url.Values, defEncoding api.EncodingEnum) (api.EncodingEnum, string) {
	encoding := q.Get(api.QueryKeyEncoding)
	if len(encoding) > 0 {
		return api.Encoder(encoding), encoding
	}
	if format, ok := getOneFormat(r, api.HeaderAccept); ok {
		return api.Encoder(format), format
	if enc, ok := getOneEncoding(r, api.HeaderAccept); ok {
		return api.Encoder(enc), enc
	}
	if format, ok := getOneFormat(r, api.HeaderContentType); ok {
		return api.Encoder(format), format
	if enc, ok := getOneEncoding(r, api.HeaderContentType); ok {
		return api.Encoder(enc), enc
	}
	return defFormat, "*default*"
	return defEncoding, defEncoding.String()
}

func getOneFormat(r *http.Request, key string) (string, bool) {
func getOneEncoding(r *http.Request, key string) (string, bool) {
	if values, ok := r.Header[key]; ok {
		for _, value := range values {
			if format, ok := contentType2format(value); ok {
				return format, true
			if enc, ok := contentType2encoding(value); ok {
				return enc, true
			}
		}
	}
	return "", false
}

var mapCT2format = map[string]string{
	"application/json": api.FormatJSON,
	"text/html":        api.FormatHTML,
var mapCT2encoding = map[string]string{
	"application/json": "json",
	"text/html":        api.EncodingHTML,
}

func contentType2format(contentType string) (string, bool) {
func contentType2encoding(contentType string) (string, bool) {
	// TODO: only check before first ';'
	format, ok := mapCT2format[contentType]
	return format, ok
	enc, ok := mapCT2encoding[contentType]
	return enc, ok
}

// GetSearch retrieves the specified search and sorting options from a query.
func GetSearch(q url.Values, forSearch bool) (s *search.Search) {
func GetSearch(q url.Values) (s *search.Search) {
	sortQKey, orderQKey, offsetQKey, limitQKey, negateQKey, sQKey := getQueryKeys(forSearch)
	for key, values := range q {
		switch key {
		case sortQKey, orderQKey:
		case api.QueryKeySort, api.QueryKeyOrder:
			s = extractOrderFromQuery(values, s)
		case offsetQKey:
		case api.QueryKeyOffset:
			s = extractOffsetFromQuery(values, s)
		case limitQKey:
		case api.QueryKeyLimit:
			s = extractLimitFromQuery(values, s)
		case negateQKey:
		case api.QueryKeyNegate:
			s = s.SetNegate()
		case sQKey:
		case api.QueryKeySearch:
			s = setCleanedQueryValues(s, "", values)
		default:
			if !forSearch && meta.KeyIsValid(key) {
			if meta.KeyIsValid(key) {
				s = setCleanedQueryValues(s, key, values)
			}
		}
	}
	return s
}

141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
140
141
142
143
144
145
146







147
148
149
150
151
152
153







-
-
-
-
-
-
-







		if limit, err := strconv.Atoi(values[0]); err == nil {
			s = s.SetLimit(limit)
		}
	}
	return s
}

func getQueryKeys(forSearch bool) (string, string, string, string, string, string) {
	if forSearch {
		return "sort", "order", "offset", "limit", "negate", "s"
	}
	return "_sort", "_order", "_offset", "_limit", "_negate", "_s"
}

func setCleanedQueryValues(s *search.Search, key string, values []string) *search.Search {
	for _, val := range values {
		s = s.AddExpr(key, val)
	}
	return s
}

Changes to web/adapter/response.go.

13
14
15
16
17
18
19


20


21

22
23
24
25
26
27
28
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33







+
+

+
+

+








import (
	"errors"
	"fmt"
	"log"
	"net/http"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/server"
)

// ReportUsecaseError returns an appropriate HTTP status code for errors in use cases.
func ReportUsecaseError(w http.ResponseWriter, err error) {
	code, text := CodeMessageFromError(err)
	if code == http.StatusInternalServerError {
		log.Printf("%v: %v", text, err)
61
62
63
64
65
66
67


































66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
		return http.StatusInternalServerError, fmt.Sprintf("Zettelstore not operational: %v", err)
	}
	if errors.Is(err, box.ErrConflict) {
		return http.StatusConflict, "Zettelstore operations conflicted"
	}
	return http.StatusInternalServerError, err.Error()
}

// CreateTagReference builds a reference to list all tags.
func CreateTagReference(b server.Builder, key byte, enc, s string) *ast.Reference {
	u := b.NewURLBuilder(key).AppendQuery(api.QueryKeyEncoding, enc).AppendQuery(meta.KeyTags, s)
	ref := ast.ParseReference(u.String())
	ref.State = ast.RefStateHosted
	return ref
}

// CreateHostedReference builds a reference with state "hosted".
func CreateHostedReference(b server.Builder, s string) *ast.Reference {
	urlPrefix := b.GetURLPrefix()
	ref := ast.ParseReference(urlPrefix + s)
	ref.State = ast.RefStateHosted
	return ref
}

// CreateFoundReference builds a reference for a found zettel.
func CreateFoundReference(b server.Builder, key byte, part, enc string, zid id.Zid, fragment string) *ast.Reference {
	ub := b.NewURLBuilder(key).SetZid(zid)
	if part != "" {
		ub.AppendQuery(api.QueryKeyPart, part)
	}
	if enc != "" {
		ub.AppendQuery(api.QueryKeyEncoding, enc)
	}
	if fragment != "" {
		ub.SetFragment(fragment)
	}

	ref := ast.ParseReference(ub.String())
	ref.State = ast.RefStateFound
	return ref
}

Changes to web/adapter/webui/create_zettel.go.

30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45
46
47
48
49
50
51

52
53
54
55
56
57
58
59
60
61
62
63
64
65

66
67
68
69
70
71
72

73
74
75
76
77
78

79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94

95
96

97
98
99
100
101
102
103
30
31
32
33
34
35
36

37
38
39
40
41
42
43
44
45
46
47
48
49
50

51
52
53
54
55
56
57
58
59
60
61
62
63
64

65
66
67
68
69
70
71

72
73
74
75
76
77

78
79
80
81
82
83
84
85
86
87
88

89
90
91
92

93
94

95
96
97
98
99
100
101
102







-
+













-
+













-
+






-
+





-
+










-




-
+

-
+







)

// MakeGetCopyZettelHandler creates a new HTTP handler to display the
// HTML edit view of a copied zettel.
func (wui *WebUI) MakeGetCopyZettelHandler(getZettel usecase.GetZettel, copyZettel usecase.CopyZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getOrigZettel(ctx, w, r, getZettel, "Copy")
		origZettel, err := getOrigZettel(ctx, r, getZettel, "Copy")
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		wui.renderZettelForm(w, r, copyZettel.Run(origZettel), "Copy Zettel", "Copy Zettel")
	}
}

// MakeGetFolgeZettelHandler creates a new HTTP handler to display the
// HTML edit view of a follow-up zettel.
func (wui *WebUI) MakeGetFolgeZettelHandler(getZettel usecase.GetZettel, folgeZettel usecase.FolgeZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getOrigZettel(ctx, w, r, getZettel, "Folge")
		origZettel, err := getOrigZettel(ctx, r, getZettel, "Folge")
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		wui.renderZettelForm(w, r, folgeZettel.Run(origZettel), "Folge Zettel", "Folgezettel")
	}
}

// MakeGetNewZettelHandler creates a new HTTP handler to display the
// HTML edit view of a zettel.
func (wui *WebUI) MakeGetNewZettelHandler(getZettel usecase.GetZettel, newZettel usecase.NewZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getOrigZettel(ctx, w, r, getZettel, "New")
		origZettel, err := getOrigZettel(ctx, r, getZettel, "New")
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		m := origZettel.Meta
		title := parser.ParseInlines(input.NewInput(config.GetTitle(m, wui.rtConfig)), meta.ValueSyntaxZmk)
		textTitle, err := adapter.FormatInlines(title, api.EncoderText, nil)
		textTitle, err := encodeInlines(title, api.EncoderText, nil)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)}
		htmlTitle, err := adapter.FormatInlines(title, api.EncoderHTML, &env)
		htmlTitle, err := encodeInlines(title, api.EncoderHTML, &env)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		wui.renderZettelForm(w, r, newZettel.Run(origZettel), textTitle, htmlTitle)
	}
}

func getOrigZettel(
	ctx context.Context,
	w http.ResponseWriter,
	r *http.Request,
	getZettel usecase.GetZettel,
	op string,
) (domain.Zettel, error) {
	if format, formatText := adapter.GetFormat(r, r.URL.Query(), api.EncoderHTML); format != api.EncoderHTML {
	if enc, encText := adapter.GetEncoding(r, r.URL.Query(), api.EncoderHTML); enc != api.EncoderHTML {
		return domain.Zettel{}, adapter.NewErrBadRequest(
			fmt.Sprintf("%v zettel not possible in format %q", op, formatText))
			fmt.Sprintf("%v zettel not possible in encoding %q", op, encText))
	}
	zid, err := id.Parse(r.URL.Path[1:])
	if err != nil {
		return domain.Zettel{}, box.ErrNotFound
	}
	origZettel, err := getZettel.Run(box.NoEnrichContext(ctx), zid)
	if err != nil {

Changes to web/adapter/webui/delete_zettel.go.

25
26
27
28
29
30
31
32

33
34

35
36
37
38
39
40
41
25
26
27
28
29
30
31

32
33

34
35
36
37
38
39
40
41







-
+

-
+







)

// MakeGetDeleteZettelHandler creates a new HTTP handler to display the
// HTML delete view of a zettel.
func (wui *WebUI) MakeGetDeleteZettelHandler(getZettel usecase.GetZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		if format, formatText := adapter.GetFormat(r, r.URL.Query(), api.EncoderHTML); format != api.EncoderHTML {
		if enc, encText := adapter.GetEncoding(r, r.URL.Query(), api.EncoderHTML); enc != api.EncoderHTML {
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Delete zettel not possible in format %q", formatText)))
				fmt.Sprintf("Delete zettel not possible in encoding %q", encText)))
			return
		}

		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			wui.reportError(ctx, w, box.ErrNotFound)
			return

Changes to web/adapter/webui/edit_zettel.go.

37
38
39
40
41
42
43
44

45
46

47
48
49
50
51
52
53
37
38
39
40
41
42
43

44
45

46
47
48
49
50
51
52
53







-
+

-
+








		zettel, err := getZettel.Run(box.NoEnrichContext(ctx), zid)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}

		if format, formatText := adapter.GetFormat(r, r.URL.Query(), api.EncoderHTML); format != api.EncoderHTML {
		if enc, encText := adapter.GetEncoding(r, r.URL.Query(), api.EncoderHTML); enc != api.EncoderHTML {
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Edit zettel %q not possible in format %q", zid, formatText)))
				fmt.Sprintf("Edit zettel %q not possible in encoding %q", zid, encText)))
			return
		}

		user := wui.getUser(ctx)
		m := zettel.Meta
		var base baseData
		wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Edit Zettel", user, &base)

Changes to web/adapter/webui/get_info.go.

22
23
24
25
26
27
28
29

30
31
32
33
34
35
36
37
38
39
40


41
42
43
44
45

46
47
48
49
50

51
52
53
54
55
56
57

58
59

60
61
62
63
64
65
66
67
68
69

70
71
72
73
74
75
76

77












78
79

80
81
82

83
84
85

86
87
88
89

90
91
92
93
94

95
96
97
98
99
100
101
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36
37
38


39
40





41
42
43
44
45
46
47
48
49
50
51
52
53

54
55

56
57
58
59
60
61
62
63
64
65

66
67
68
69
70
71
72

73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

88
89
90

91
92
93

94
95
96
97

98
99
100
101
102

103
104
105
106
107
108
109
110







-
+









-
-
+
+
-
-
-
-
-
+





+






-
+

-
+









-
+






-
+

+
+
+
+
+
+
+
+
+
+
+
+

-
+


-
+


-
+



-
+




-
+







	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/collect"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/encfun"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

type metaDataInfo struct {
	Key   string
	Value string
}

type matrixElement struct {
	Text   string
type matrixLine struct {
	Header   string
	HasURL bool
	URL    string
}
type matrixLine struct {
	Elements []matrixElement
	Elements []simpleLink
}

// MakeGetInfoHandler creates a new HTTP handler for the use case "get zettel".
func (wui *WebUI) MakeGetInfoHandler(
	parseZettel usecase.ParseZettel,
	evaluate *usecase.Evaluate,
	getMeta usecase.GetMeta,
	getAllMeta usecase.GetAllMeta,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		q := r.URL.Query()
		if format, formatText := adapter.GetFormat(r, q, api.EncoderHTML); format != api.EncoderHTML {
		if enc, encText := adapter.GetEncoding(r, q, api.EncoderHTML); enc != api.EncoderHTML {
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Zettel info not available in format %q", formatText)))
				fmt.Sprintf("Zettel info not available in encoding %q", encText)))
			return
		}

		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			wui.reportError(ctx, w, box.ErrNotFound)
			return
		}

		zn, err := parseZettel.Run(ctx, zid, q.Get("syntax"))
		zn, err := parseZettel.Run(ctx, zid, q.Get(meta.KeySyntax))
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}

		summary := collect.References(zn)
		locLinks, extLinks := splitLocExtLinks(append(summary.Links, summary.Images...))
		locLinks, extLinks := splitLocExtLinks(append(summary.Links, summary.Embeds...))

		envEval := evaluator.Environment{
			EmbedImage: true,
			GetTagRef: func(s string) *ast.Reference {
				return adapter.CreateTagReference(wui, 'h', api.EncodingHTML, s)
			},
			GetHostedRef: func(s string) *ast.Reference {
				return adapter.CreateHostedReference(wui, s)
			},
			GetFoundRef: func(zid id.Zid, fragment string) *ast.Reference {
				return adapter.CreateFoundReference(wui, 'h', "", "", zid, fragment)
			},
		}
		lang := config.GetLang(zn.InhMeta, wui.rtConfig)
		env := encoder.Environment{Lang: lang}
		envHTML := encoder.Environment{Lang: lang}
		pairs := zn.Meta.Pairs(true)
		metaData := make([]metaDataInfo, len(pairs))
		getTitle := makeGetTitle(ctx, getMeta, &env)
		getTextTitle := wui.makeGetTextTitle(ctx, getMeta, evaluate)
		for i, p := range pairs {
			var html strings.Builder
			wui.writeHTMLMetaValue(&html, zn.Meta, p.Key, getTitle, &env)
			wui.writeHTMLMetaValue(ctx, &html, p.Key, p.Value, getTextTitle, evaluate, &envEval, &envHTML)
			metaData[i] = metaDataInfo{p.Key, html.String()}
		}
		shadowLinks := getShadowLinks(ctx, zid, getAllMeta)
		endnotes, err := formatBlocks(nil, api.EncoderHTML, &env)
		endnotes, err := encodeBlocks(&ast.BlockListNode{}, api.EncoderHTML, &envHTML)
		if err != nil {
			endnotes = ""
		}

		textTitle := encfun.MetaAsText(zn.InhMeta, meta.KeyTitle)
		textTitle := wui.encodeTitleAsText(ctx, zn.InhMeta, evaluate)
		user := wui.getUser(ctx)
		canCreate := wui.canCreate(ctx, user)
		var base baseData
		wui.makeBaseData(ctx, lang, textTitle, user, &base)
		wui.renderTemplate(ctx, w, id.InfoTemplateZid, &base, struct {
			Zid            string
			WebURL         string
113
114
115
116
117
118
119
120


121
122
123
124
125
126
127

128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145


146
147
148
149
150
151
152
122
123
124
125
126
127
128

129
130
131
132
133
134
135
136

137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154

155
156
157
158
159
160
161
162
163







-
+
+






-
+

















-
+
+







			MetaData       []metaDataInfo
			HasLinks       bool
			HasLocLinks    bool
			LocLinks       []localLink
			HasExtLinks    bool
			ExtLinks       []string
			ExtNewWindow   string
			Matrix         []matrixLine
			EvalMatrix     []matrixLine
			ParseMatrix    []matrixLine
			HasShadowLinks bool
			ShadowLinks    []string
			Endnotes       string
		}{
			Zid:            zid.String(),
			WebURL:         wui.NewURLBuilder('h').SetZid(zid).String(),
			ContextURL:     wui.NewURLBuilder('j').SetZid(zid).String(),
			ContextURL:     wui.NewURLBuilder('k').SetZid(zid).String(),
			CanWrite:       wui.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:        wui.NewURLBuilder('e').SetZid(zid).String(),
			CanFolge:       canCreate,
			FolgeURL:       wui.NewURLBuilder('f').SetZid(zid).String(),
			CanCopy:        canCreate && !zn.Content.IsBinary(),
			CopyURL:        wui.NewURLBuilder('c').SetZid(zid).String(),
			CanRename:      wui.canRename(ctx, user, zn.Meta),
			RenameURL:      wui.NewURLBuilder('b').SetZid(zid).String(),
			CanDelete:      wui.canDelete(ctx, user, zn.Meta),
			DeleteURL:      wui.NewURLBuilder('d').SetZid(zid).String(),
			MetaData:       metaData,
			HasLinks:       len(extLinks)+len(locLinks) > 0,
			HasLocLinks:    len(locLinks) > 0,
			LocLinks:       locLinks,
			HasExtLinks:    len(extLinks) > 0,
			ExtLinks:       extLinks,
			ExtNewWindow:   htmlAttrNewWindow(len(extLinks) > 0),
			Matrix:         wui.infoAPIMatrix(zid),
			EvalMatrix:     wui.infoAPIMatrix('v', zid),
			ParseMatrix:    wui.infoAPIMatrixPlain('p', zid),
			HasShadowLinks: len(shadowLinks) > 0,
			ShadowLinks:    shadowLinks,
			Endnotes:       endnotes,
		})
	}
}

171
172
173
174
175
176
177
178
179
180
181
182





183
184
185
186



187
188

189
190

191
192

193
194
195


196
197

198
199
200

201
















202
203
204
205
206
207
208
182
183
184
185
186
187
188





189
190
191
192
193
194



195
196
197
198

199
200

201


202
203


204
205
206

207
208
209

210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234







-
-
-
-
-
+
+
+
+
+

-
-
-
+
+
+

-
+

-
+
-
-
+

-
-
+
+

-
+


-
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







			continue
		}
		locLinks = append(locLinks, localLink{ref.IsValid(), ref.String()})
	}
	return locLinks, extLinks
}

func (wui *WebUI) infoAPIMatrix(zid id.Zid) []matrixLine {
	formats := encoder.GetFormats()
	formatTexts := make([]string, 0, len(formats))
	for _, f := range formats {
		formatTexts = append(formatTexts, f.String())
func (wui *WebUI) infoAPIMatrix(key byte, zid id.Zid) []matrixLine {
	encodings := encoder.GetEncodings()
	encTexts := make([]string, 0, len(encodings))
	for _, f := range encodings {
		encTexts = append(encTexts, f.String())
	}
	sort.Strings(formatTexts)
	defFormat := encoder.GetDefaultFormat().String()
	parts := []string{"zettel", "meta", "content"}
	sort.Strings(encTexts)
	defEncoding := encoder.GetDefaultEncoding().String()
	parts := []string{api.PartZettel, api.PartMeta, api.PartContent}
	matrix := make([]matrixLine, 0, len(parts))
	u := wui.NewURLBuilder('z').SetZid(zid)
	u := wui.NewURLBuilder(key).SetZid(zid)
	for _, part := range parts {
		row := make([]matrixElement, 0, len(formatTexts)+1)
		row := make([]simpleLink, len(encTexts))
		row = append(row, matrixElement{part, false, ""})
		for _, format := range formatTexts {
		for j, enc := range encTexts {
			u.AppendQuery(api.QueryKeyPart, part)
			if format != defFormat {
				u.AppendQuery(api.QueryKeyFormat, format)
			if enc != defEncoding {
				u.AppendQuery(api.QueryKeyEncoding, enc)
			}
			row = append(row, matrixElement{format, true, u.String()})
			row[j] = simpleLink{enc, u.String()}
			u.ClearQuery()
		}
		matrix = append(matrix, matrixLine{row})
		matrix = append(matrix, matrixLine{part, row})
	}
	return matrix
}

func (wui *WebUI) infoAPIMatrixPlain(key byte, zid id.Zid) []matrixLine {
	matrix := wui.infoAPIMatrix(key, zid)

	// Append plain and JSON format
	u := wui.NewURLBuilder('z').SetZid(zid)
	parts := []string{api.PartZettel, api.PartMeta, api.PartContent}
	for i, part := range parts {
		u.AppendQuery(api.QueryKeyPart, part)
		matrix[i].Elements = append(matrix[i].Elements, simpleLink{"plain", u.String()})
		u.ClearQuery()
	}
	u = wui.NewURLBuilder('j').SetZid(zid)
	matrix[0].Elements = append(matrix[0].Elements, simpleLink{"json", u.String()})
	return matrix
}

func getShadowLinks(ctx context.Context, zid id.Zid, getAllMeta usecase.GetAllMeta) []string {
	ml, err := getAllMeta.Run(ctx, zid)
	if err != nil || len(ml) < 2 {
		return nil

Changes to web/adapter/webui/get_zettel.go.

9
10
11
12
13
14
15

16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
32

33
34
35
36
37
38
39
40
41
42















43
44
45
46
47



48
49
50
51
52
53
54
55
56
57
58
59

60
61
62
63

64
65


66
67
68
69
70
71
72
73
74
75
76
77
78
79
80

81

82

83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105


106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123

124
125
126
127


128
129
130
131
132
133




134
135
136
137













138

139








140
141

142
143
144
145
146
147


148
149
150
151





152
153
154
155

156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176


177
178

179
180
181
182
183
184
185


186
187
188
189
190
191
192
193
194
195

196
197
198
199
200
201
202
203
204
205
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

27
28
29
30
31
32

33
34
35
36
37
38
39
40
41


42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66



67
68
69
70
71
72

73
74
75
76
77
78


79
80
81
82
83
84






85
86
87
88

89
90
91

92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109

110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132


133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149




150
151
152
153
154
155
156
157
158
159
160
161
162
163
164

165
166
167
168
169
170
171
172
173

174
175
176
177
178
179
180
181
182




183
184
185
186
187
188
189
190

191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210


211
212
213

214
215
216
217
218
219


220
221
222
223
224
225
226
227
228
229
230

231
232
233
234
235
236
237
238
239
240
241







+










-
+





-
+








-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+





+
+
+


-
-
-






-
+




+
-
-
+
+




-
-
-
-
-
-




-
+

+
-
+

















-





+
+
















-
-
+




+
+






+
+
+
+
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+

+
-
+
+
+
+
+
+
+
+

-
+






+
+
-
-
-
-
+
+
+
+
+



-
+



















-
-
+
+

-
+





-
-
+
+









-
+










//-----------------------------------------------------------------------------

// Package webui provides web-UI handlers for web requests.
package webui

import (
	"bytes"
	"errors"
	"net/http"
	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/encfun"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetHTMLZettelHandler creates a new HTTP handler for the use case "get zettel".
func (wui *WebUI) MakeGetHTMLZettelHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc {
func (wui *WebUI) MakeGetHTMLZettelHandler(evaluate *usecase.Evaluate, getMeta usecase.GetMeta) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			wui.reportError(ctx, w, box.ErrNotFound)
			return
		}

		syntax := r.URL.Query().Get("syntax")
		zn, err := parseZettel.Run(ctx, zid, syntax)
		q := r.URL.Query()
		env := evaluator.Environment{
			EmbedImage: true,
			GetTagRef: func(s string) *ast.Reference {
				return adapter.CreateTagReference(wui, 'h', api.EncodingHTML, s)
			},
			GetHostedRef: func(s string) *ast.Reference {
				return adapter.CreateHostedReference(wui, s)
			},
			GetFoundRef: func(zid id.Zid, fragment string) *ast.Reference {
				return adapter.CreateFoundReference(wui, 'h', "", "", zid, fragment)
			},
		}
		zn, err := evaluate.Run(ctx, zid, q.Get(meta.KeySyntax), &env)

		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}

		evalMeta := func(value string) *ast.InlineListNode {
			return evaluate.RunMetadata(ctx, value, &env)
		}
		lang := config.GetLang(zn.InhMeta, wui.rtConfig)
		envHTML := encoder.Environment{
			LinkAdapter:    adapter.MakeLinkAdapter(ctx, wui, 'h', getMeta, "", api.EncoderUnknown),
			ImageAdapter:   adapter.MakeImageAdapter(ctx, wui, getMeta),
			CiteAdapter:    nil,
			Lang:           lang,
			Xhtml:          false,
			MarkerExternal: wui.rtConfig.GetMarkerExternal(),
			NewWindow:      true,
			IgnoreMeta:     map[string]bool{meta.KeyTitle: true, meta.KeyLang: true},
		}
		metaHeader, err := formatMeta(zn.InhMeta, api.EncoderHTML, &envHTML)
		metaHeader, err := encodeMeta(zn.InhMeta, evalMeta, api.EncoderHTML, &envHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		textTitle := wui.encodeTitleAsText(ctx, zn.InhMeta, evaluate)
		htmlTitle, err := adapter.FormatInlines(
			encfun.MetaAsInlineSlice(zn.InhMeta, meta.KeyTitle), api.EncoderHTML, &envHTML)
		htmlTitle := wui.encodeTitleAsHTML(ctx, zn.InhMeta, evaluate, &env, &envHTML)
		htmlContent, err := encodeBlocks(zn.Ast, api.EncoderHTML, &envHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		htmlContent, err := formatBlocks(zn.Ast, api.EncoderHTML, &envHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		textTitle := encfun.MetaAsText(zn.InhMeta, meta.KeyTitle)
		user := wui.getUser(ctx)
		roleText := zn.Meta.GetDefault(meta.KeyRole, "*")
		tags := wui.buildTagInfos(zn.Meta)
		canCreate := wui.canCreate(ctx, user)
		getTitle := makeGetTitle(ctx, getMeta, &encoder.Environment{Lang: lang})
		getTextTitle := wui.makeGetTextTitle(ctx, getMeta, evaluate)
		extURL, hasExtURL := zn.Meta.Get(meta.KeyURL)
		folgeLinks := wui.encodeZettelLinks(zn.InhMeta, meta.KeyFolge, getTextTitle)
		backLinks := wui.formatBackLinks(zn.InhMeta, getTitle)
		backLinks := wui.encodeZettelLinks(zn.InhMeta, meta.KeyBack, getTextTitle)
		var base baseData
		wui.makeBaseData(ctx, lang, textTitle, user, &base)
		base.MetaHeader = metaHeader
		wui.renderTemplate(ctx, w, id.ZettelTemplateZid, &base, struct {
			HTMLTitle     string
			CanWrite      bool
			EditURL       string
			Zid           string
			InfoURL       string
			RoleText      string
			RoleURL       string
			HasTags       bool
			Tags          []simpleLink
			CanCopy       bool
			CopyURL       string
			CanFolge      bool
			FolgeURL      string
			FolgeRefs     string
			PrecursorRefs string
			HasExtURL     bool
			ExtURL        string
			ExtNewWindow  string
			Content       string
			HasFolgeLinks bool
			FolgeLinks    []simpleLink
			HasBackLinks  bool
			BackLinks     []simpleLink
		}{
			HTMLTitle:     htmlTitle,
			CanWrite:      wui.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:       wui.NewURLBuilder('e').SetZid(zid).String(),
			Zid:           zid.String(),
			InfoURL:       wui.NewURLBuilder('i').SetZid(zid).String(),
			RoleText:      roleText,
			RoleURL:       wui.NewURLBuilder('h').AppendQuery("role", roleText).String(),
			HasTags:       len(tags) > 0,
			Tags:          tags,
			CanCopy:       canCreate && !zn.Content.IsBinary(),
			CopyURL:       wui.NewURLBuilder('c').SetZid(zid).String(),
			CanFolge:      canCreate,
			FolgeURL:      wui.NewURLBuilder('f').SetZid(zid).String(),
			FolgeRefs:     wui.formatMetaKey(zn.InhMeta, meta.KeyFolge, getTitle),
			PrecursorRefs: wui.formatMetaKey(zn.InhMeta, meta.KeyPrecursor, getTitle),
			PrecursorRefs: wui.encodeIdentifierSet(zn.InhMeta, meta.KeyPrecursor, getTextTitle),
			ExtURL:        extURL,
			HasExtURL:     hasExtURL,
			ExtNewWindow:  htmlAttrNewWindow(envHTML.NewWindow && hasExtURL),
			Content:       htmlContent,
			HasFolgeLinks: len(folgeLinks) > 0,
			FolgeLinks:    folgeLinks,
			HasBackLinks:  len(backLinks) > 0,
			BackLinks:     backLinks,
		})
	}
}

// errNoSuchEncoding signals an unsupported encoding encoding
var errNoSuchEncoding = errors.New("no such encoding")

// encodeInlines returns a string representation of the inline slice.
func formatBlocks(bs ast.BlockSlice, format api.EncodingEnum, env *encoder.Environment) (string, error) {
	enc := encoder.Create(format, env)
	if enc == nil {
		return "", adapter.ErrNoSuchFormat
func encodeInlines(is *ast.InlineListNode, enc api.EncodingEnum, env *encoder.Environment) (string, error) {
	if is == nil {
		return "", nil
	}
	encdr := encoder.Create(enc, env)
	if encdr == nil {
		return "", errNoSuchEncoding
	}

	var content strings.Builder
	_, err := encdr.WriteInlines(&content, is)
	if err != nil {
		return "", err
	}
	return content.String(), nil

}

func encodeBlocks(bln *ast.BlockListNode, enc api.EncodingEnum, env *encoder.Environment) (string, error) {
	encdr := encoder.Create(enc, env)
	if encdr == nil {
		return "", errNoSuchEncoding
	}

	var content strings.Builder
	_, err := enc.WriteBlocks(&content, bs)
	_, err := encdr.WriteBlocks(&content, bln)
	if err != nil {
		return "", err
	}
	return content.String(), nil
}

func encodeMeta(
	m *meta.Meta, evalMeta encoder.EvalMetaFunc,
func formatMeta(m *meta.Meta, format api.EncodingEnum, env *encoder.Environment) (string, error) {
	enc := encoder.Create(format, env)
	if enc == nil {
		return "", adapter.ErrNoSuchFormat
	enc api.EncodingEnum, env *encoder.Environment,
) (string, error) {
	encdr := encoder.Create(enc, env)
	if encdr == nil {
		return "", errNoSuchEncoding
	}

	var content strings.Builder
	_, err := enc.WriteMeta(&content, m)
	_, err := encdr.WriteMeta(&content, m, evalMeta)
	if err != nil {
		return "", err
	}
	return content.String(), nil
}

func (wui *WebUI) buildTagInfos(m *meta.Meta) []simpleLink {
	var tagInfos []simpleLink
	if tags, ok := m.GetList(meta.KeyTags); ok {
		ub := wui.NewURLBuilder('h')
		tagInfos = make([]simpleLink, len(tags))
		for i, tag := range tags {
			tagInfos[i] = simpleLink{Text: tag, URL: ub.AppendQuery("tags", tag).String()}
			ub.ClearQuery()
		}
	}
	return tagInfos
}

func (wui *WebUI) formatMetaKey(m *meta.Meta, key string, getTitle getTitleFunc) string {
	if _, ok := m.Get(key); ok {
func (wui *WebUI) encodeIdentifierSet(m *meta.Meta, key string, getTextTitle getTextTitleFunc) string {
	if value, ok := m.Get(key); ok {
		var buf bytes.Buffer
		wui.writeHTMLMetaValue(&buf, m, key, getTitle, nil)
		wui.writeIdentifierSet(&buf, meta.ListFromValue(value), getTextTitle)
		return buf.String()
	}
	return ""
}

func (wui *WebUI) formatBackLinks(m *meta.Meta, getTitle getTitleFunc) []simpleLink {
	values, ok := m.GetList(meta.KeyBack)
func (wui *WebUI) encodeZettelLinks(m *meta.Meta, key string, getTextTitle getTextTitleFunc) []simpleLink {
	values, ok := m.GetList(key)
	if !ok || len(values) == 0 {
		return nil
	}
	result := make([]simpleLink, 0, len(values))
	for _, val := range values {
		zid, err := id.Parse(val)
		if err != nil {
			continue
		}
		if title, found := getTitle(zid, api.EncoderText); found > 0 {
		if title, found := getTextTitle(zid); found > 0 {
			url := wui.NewURLBuilder('h').SetZid(zid).String()
			if title == "" {
				result = append(result, simpleLink{Text: val, URL: url})
			} else {
				result = append(result, simpleLink{Text: title, URL: url})
			}
		}
	}
	return result
}

Changes to web/adapter/webui/home.go.

44
45
46
47
48
49
50
51

52
53
54
55
56
44
45
46
47
48
49
50

51
52
53
54
55
56







-
+





		}
		_, err := s.GetMeta(ctx, homeZid)
		if err == nil {
			redirectFound(w, r, wui.NewURLBuilder('h').SetZid(homeZid))
			return
		}
		if errors.Is(err, &box.ErrNotAllowed{}) && wui.authz.WithAuth() && wui.getUser(ctx) == nil {
			redirectFound(w, r, wui.NewURLBuilder('a'))
			redirectFound(w, r, wui.NewURLBuilder('i'))
			return
		}
		redirectFound(w, r, wui.NewURLBuilder('h'))
	}
}

Changes to web/adapter/webui/htmlmeta.go.

17
18
19
20
21
22
23

24
25
26
27

28
29
30
31
32
33
34
35
36








37
38

39
40

41
42

43
44

45
46
47

48
49
50

51
52

53
54
55

56
57
58

59
60
61
62

63
64

65
66
67

68
69
70

71
72

73
74
75
76
77
78


79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

94
95
96
97
98
99

100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

115
116
117
118
119

120
121
122
123
124
125
126
17
18
19
20
21
22
23
24
25
26
27

28
29
30

31
32
33
34


35
36
37
38
39
40
41
42
43

44
45

46
47

48
49

50
51


52

53

54
55

56
57


58

59

60
61
62
63

64
65

66
67


68

69

70
71

72
73
74
75
76


77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

93
94
95
96
97
98

99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

114
115
116
117
118

119
120
121
122
123
124
125
126







+



-
+


-




-
-
+
+
+
+
+
+
+
+

-
+

-
+

-
+

-
+

-
-
+
-

-
+

-
+

-
-
+
-

-
+



-
+

-
+

-
-
+
-

-
+

-
+




-
-
+
+














-
+





-
+














-
+




-
+







	"fmt"
	"io"
	"net/url"
	"time"

	"zettelstore.de/z/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

var space = []byte{' '}

func (wui *WebUI) writeHTMLMetaValue(w io.Writer, m *meta.Meta, key string, getTitle getTitleFunc, env *encoder.Environment) {
	switch kt := m.Type(key); kt {
func (wui *WebUI) writeHTMLMetaValue(
	ctx context.Context,
	w io.Writer, key, value string,
	getTextTitle getTextTitleFunc,
	evaluate *usecase.Evaluate, envEval *evaluator.Environment,
	envEnc *encoder.Environment,
) {
	switch kt := meta.Type(key); kt {
	case meta.TypeBool:
		wui.writeHTMLBool(w, key, m.GetBool(key))
		wui.writeHTMLBool(w, key, value)
	case meta.TypeCredential:
		writeCredential(w, m.GetDefault(key, "???c"))
		writeCredential(w, value)
	case meta.TypeEmpty:
		writeEmpty(w, m.GetDefault(key, "???e"))
		writeEmpty(w, value)
	case meta.TypeID:
		wui.writeIdentifier(w, m.GetDefault(key, "???i"), getTitle)
		wui.writeIdentifier(w, value, getTextTitle)
	case meta.TypeIDSet:
		if l, ok := m.GetList(key); ok {
			wui.writeIdentifierSet(w, l, getTitle)
		wui.writeIdentifierSet(w, meta.ListFromValue(value), getTextTitle)
		}
	case meta.TypeNumber:
		wui.writeNumber(w, key, m.GetDefault(key, "???n"))
		wui.writeNumber(w, key, value)
	case meta.TypeString:
		writeString(w, m.GetDefault(key, "???s"))
		writeString(w, value)
	case meta.TypeTagSet:
		if l, ok := m.GetList(key); ok {
			wui.writeTagSet(w, key, l)
		wui.writeTagSet(w, key, meta.ListFromValue(value))
		}
	case meta.TypeTimestamp:
		if ts, ok := m.GetTime(key); ok {
		if ts, ok := meta.TimeValue(value); ok {
			writeTimestamp(w, ts)
		}
	case meta.TypeURL:
		writeURL(w, m.GetDefault(key, "???u"))
		writeURL(w, value)
	case meta.TypeWord:
		wui.writeWord(w, key, m.GetDefault(key, "???w"))
		wui.writeWord(w, key, value)
	case meta.TypeWordSet:
		if l, ok := m.GetList(key); ok {
			wui.writeWordSet(w, key, l)
		wui.writeWordSet(w, key, meta.ListFromValue(value))
		}
	case meta.TypeZettelmarkup:
		writeZettelmarkup(w, m.GetDefault(key, "???z"), env)
		io.WriteString(w, encodeZmkMetadata(ctx, value, evaluate, envEval, api.EncoderHTML, envEnc))
	default:
		strfun.HTMLEscape(w, m.GetDefault(key, "???w"), false)
		strfun.HTMLEscape(w, value, false)
		fmt.Fprintf(w, " <b>(Unhandled type: %v, key: %v)</b>", kt, key)
	}
}

func (wui *WebUI) writeHTMLBool(w io.Writer, key string, val bool) {
	if val {
func (wui *WebUI) writeHTMLBool(w io.Writer, key, value string) {
	if meta.BoolValue(value) {
		wui.writeLink(w, key, "true", "True")
	} else {
		wui.writeLink(w, key, "false", "False")
	}
}

func writeCredential(w io.Writer, val string) {
	strfun.HTMLEscape(w, val, false)
}

func writeEmpty(w io.Writer, val string) {
	strfun.HTMLEscape(w, val, false)
}

func (wui *WebUI) writeIdentifier(w io.Writer, val string, getTitle getTitleFunc) {
func (wui *WebUI) writeIdentifier(w io.Writer, val string, getTextTitle getTextTitleFunc) {
	zid, err := id.Parse(val)
	if err != nil {
		strfun.HTMLEscape(w, val, false)
		return
	}
	title, found := getTitle(zid, api.EncoderText)
	title, found := getTextTitle(zid)
	switch {
	case found > 0:
		if title == "" {
			fmt.Fprintf(w, "<a href=\"%v\">%v</a>", wui.NewURLBuilder('h').SetZid(zid), zid)
		} else {
			fmt.Fprintf(w, "<a href=\"%v\" title=\"%v\">%v</a>", wui.NewURLBuilder('h').SetZid(zid), title, zid)
		}
	case found == 0:
		fmt.Fprintf(w, "<s>%v</s>", val)
	case found < 0:
		io.WriteString(w, val)
	}
}

func (wui *WebUI) writeIdentifierSet(w io.Writer, vals []string, getTitle getTitleFunc) {
func (wui *WebUI) writeIdentifierSet(w io.Writer, vals []string, getTextTitle getTextTitleFunc) {
	for i, val := range vals {
		if i > 0 {
			w.Write(space)
		}
		wui.writeIdentifier(w, val, getTitle)
		wui.writeIdentifier(w, val, getTextTitle)
	}
}

func (wui *WebUI) writeNumber(w io.Writer, key, val string) {
	wui.writeLink(w, key, val, val)
}

160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182

183


184
185



186
187
188
189
190
191
192



193
194
195
196
197
198





























199

200
160
161
162
163
164
165
166








167
168
169
170
171
172
173

174
175
176
177


178
179
180
181
182
183
184
185
186
187
188
189
190






191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222







-
-
-
-
-
-
-
-







-
+

+
+
-
-
+
+
+







+
+
+
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

+

	for i, word := range words {
		if i > 0 {
			w.Write(space)
		}
		wui.writeWord(w, key, word)
	}
}
func writeZettelmarkup(w io.Writer, val string, env *encoder.Environment) {
	title, err := adapter.FormatInlines(parser.ParseMetadata(val), api.EncoderHTML, env)
	if err != nil {
		strfun.HTMLEscape(w, val, false)
		return
	}
	io.WriteString(w, title)
}

func (wui *WebUI) writeLink(w io.Writer, key, value, text string) {
	fmt.Fprintf(w, "<a href=\"%v?%v=%v\">", wui.NewURLBuilder('h'), url.QueryEscape(key), url.QueryEscape(value))
	strfun.HTMLEscape(w, text, false)
	io.WriteString(w, "</a>")
}

type getTitleFunc func(id.Zid, api.EncodingEnum) (string, int)
type getTextTitleFunc func(id.Zid) (string, int)

func (wui *WebUI) makeGetTextTitle(
	ctx context.Context,
func makeGetTitle(ctx context.Context, getMeta usecase.GetMeta, env *encoder.Environment) getTitleFunc {
	return func(zid id.Zid, format api.EncodingEnum) (string, int) {
	getMeta usecase.GetMeta, evaluate *usecase.Evaluate,
) getTextTitleFunc {
	return func(zid id.Zid) (string, int) {
		m, err := getMeta.Run(box.NoEnrichContext(ctx), zid)
		if err != nil {
			if errors.Is(err, &box.ErrNotAllowed{}) {
				return "", -1
			}
			return "", 0
		}
		return wui.encodeTitleAsText(ctx, m, evaluate), 1
	}
}
		astTitle := parser.ParseMetadata(m.GetDefault(meta.KeyTitle, ""))
		title, err := adapter.FormatInlines(astTitle, format, env)
		if err == nil {
			return title, 1
		}
		return "", 1

func (wui *WebUI) encodeTitleAsHTML(
	ctx context.Context, m *meta.Meta,
	evaluate *usecase.Evaluate, envEval *evaluator.Environment,
	envHTML *encoder.Environment,
) string {
	plainTitle := config.GetTitle(m, wui.rtConfig)
	return encodeZmkMetadata(ctx, plainTitle, evaluate, envEval, api.EncoderHTML, envHTML)
}

func (wui *WebUI) encodeTitleAsText(
	ctx context.Context, m *meta.Meta, evaluate *usecase.Evaluate,
) string {
	plainTitle := config.GetTitle(m, wui.rtConfig)
	return encodeZmkMetadata(ctx, plainTitle, evaluate, nil, api.EncoderText, nil)
}

func encodeZmkMetadata(
	ctx context.Context, value string,
	evaluate *usecase.Evaluate, envEval *evaluator.Environment,
	enc api.EncodingEnum, envHTML *encoder.Environment,
) string {
	iln := evaluate.RunMetadata(ctx, value, envEval)
	if iln.IsEmpty() {
		return ""
	}
	result, err := encodeInlines(iln, enc, envHTML)
	if err == nil {
		return result
	}
	return err.Error()
}

Changes to web/adapter/webui/lists.go.

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46
47
48

49
50
51
52
53




54
55

56
57
58
59
60
61

62
63
64
65
66

67
68

69
70
71
72
73
74
75
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

48
49
50
51
52

53
54
55
56
57

58
59
60
61
62
63

64
65
66
67
68

69


70
71
72
73
74
75
76
77







-











+









-
+




-
+
+
+
+

-
+





-
+




-
+
-
-
+







	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/search"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeListHTMLMetaHandler creates a HTTP handler for rendering the list of
// zettel as HTML.
func (wui *WebUI) MakeListHTMLMetaHandler(
	listMeta usecase.ListMeta,
	listRole usecase.ListRole,
	listTags usecase.ListTags,
	evaluate *usecase.Evaluate,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		query := r.URL.Query()
		switch query.Get("_l") {
		case "r":
			wui.renderRolesList(w, r, listRole)
		case "t":
			wui.renderTagsList(w, r, listTags)
		default:
			wui.renderZettelList(w, r, listMeta)
			wui.renderZettelList(w, r, listMeta, evaluate)
		}
	}
}

func (wui *WebUI) renderZettelList(w http.ResponseWriter, r *http.Request, listMeta usecase.ListMeta) {
func (wui *WebUI) renderZettelList(
	w http.ResponseWriter, r *http.Request,
	listMeta usecase.ListMeta, evaluate *usecase.Evaluate,
) {
	query := r.URL.Query()
	s := adapter.GetSearch(query, false)
	s := adapter.GetSearch(query)
	ctx := r.Context()
	title := wui.listTitleSearch("Select", s)
	wui.renderMetaList(
		ctx, w, title, s,
		func(s *search.Search) ([]*meta.Meta, error) {
			if !s.HasComputedMetaKey() {
			if !s.EnrichNeeded() {
				ctx = box.NoEnrichContext(ctx)
			}
			return listMeta.Run(ctx, s)
		},
		func(offset int) string {
		evaluate,
			return wui.newPageURL('h', query, offset, "_offset", "_limit")
		})
	)
}

type roleInfo struct {
	Text string
	URL  string
}

100
101
102
103
104
105
106
107
108
109
110
111





112
113
114
115
116
117
118
102
103
104
105
106
107
108





109
110
111
112
113
114
115
116
117
118
119
120







-
-
-
-
-
+
+
+
+
+








type countInfo struct {
	Count string
	URL   string
}

type tagInfo struct {
	Name  string
	URL   string
	count int
	Count string
	Size  string
	Name   string
	URL    string
	iCount int
	Count  string
	Size   string
}

var fontSizes = [...]int{75, 83, 100, 117, 150, 200}

func (wui *WebUI) renderTagsList(w http.ResponseWriter, r *http.Request, listTags usecase.ListTags) {
	ctx := r.Context()
	iMinCount, _ := strconv.Atoi(r.URL.Query().Get("min"))
141
142
143
144
145
146
147
148

149
150
151
152
153
154
155
143
144
145
146
147
148
149

150
151
152
153
154
155
156
157







-
+







		countList = append(countList, count)
	}
	sort.Ints(countList)
	for pos, count := range countList {
		countMap[count] = fontSizes[(pos*len(fontSizes))/len(countList)]
	}
	for i := 0; i < len(tagsList); i++ {
		count := tagsList[i].count
		count := tagsList[i].iCount
		tagsList[i].Count = strconv.Itoa(count)
		tagsList[i].Size = strconv.Itoa(countMap[count])
	}

	var base baseData
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base)
	minCounts := make([]countInfo, 0, len(countList))
166
167
168
169
170
171
172
173

174
175
176
177
178
179
180
181

182
183
184
185
186
187
188
189
190

191
192
193
194
195

196
197

198
199
200
201
202

203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219

220
221
222
223
224
225
226
227

228
229
230
231
232
233
234
168
169
170
171
172
173
174

175




176
177
178

179
180
181
182
183
184
185
186
187

188
189
190
191
192

193


194
195
196
197
198

199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215

216





217
218

219
220
221
222
223
224
225
226







-
+
-
-
-
-



-
+








-
+




-
+
-
-
+




-
+
















-
+
-
-
-
-
-


-
+







		ListTagsURL: base.ListTagsURL,
		MinCounts:   minCounts,
		Tags:        tagsList,
	})
}

// MakeSearchHandler creates a new HTTP handler for the use case "search".
func (wui *WebUI) MakeSearchHandler(
func (wui *WebUI) MakeSearchHandler(ucSearch usecase.Search, evaluate *usecase.Evaluate) http.HandlerFunc {
	ucSearch usecase.Search,
	getMeta usecase.GetMeta,
	getZettel usecase.GetZettel,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		query := r.URL.Query()
		ctx := r.Context()
		s := adapter.GetSearch(query, true)
		s := adapter.GetSearch(query)
		if s == nil {
			redirectFound(w, r, wui.NewURLBuilder('h'))
			return
		}

		title := wui.listTitleSearch("Search", s)
		wui.renderMetaList(
			ctx, w, title, s, func(s *search.Search) ([]*meta.Meta, error) {
				if !s.HasComputedMetaKey() {
				if !s.EnrichNeeded() {
					ctx = box.NoEnrichContext(ctx)
				}
				return ucSearch.Run(ctx, s)
			},
			func(offset int) string {
			evaluate,
				return wui.newPageURL('f', query, offset, "offset", "limit")
			})
		)
	}
}

// MakeZettelContextHandler creates a new HTTP handler for the use case "zettel context".
func (wui *WebUI) MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc {
func (wui *WebUI) MakeZettelContextHandler(getContext usecase.ZettelContext, evaluate *usecase.Evaluate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			wui.reportError(ctx, w, box.ErrNotFound)
			return
		}
		q := r.URL.Query()
		dir := adapter.GetZCDirection(q.Get(api.QueryKeyDir))
		depth := getIntParameter(q, api.QueryKeyDepth, 5)
		limit := getIntParameter(q, api.QueryKeyLimit, 200)
		metaList, err := getContext.Run(ctx, zid, dir, depth, limit)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		metaLinks, err := wui.buildHTMLMetaList(metaList)
		metaLinks := wui.buildHTMLMetaList(ctx, metaList, evaluate)
		if err != nil {
			adapter.InternalServerError(w, "Build HTML meta list", err)
			return
		}

		depths := []string{"2", "3", "4", "5", "6", "7", "8", "9", "10"}
		depthLinks := make([]simpleLink, len(depths))
		depthURL := wui.NewURLBuilder('j').SetZid(zid)
		depthURL := wui.NewURLBuilder('k').SetZid(zid)
		for i, depth := range depths {
			depthURL.ClearQuery()
			switch dir {
			case usecase.ZettelContextBackward:
				depthURL.AppendQuery(api.QueryKeyDir, api.DirBackward)
			case usecase.ZettelContextForward:
				depthURL.AppendQuery(api.QueryKeyDir, api.DirForward)
266
267
268
269
270
271
272

273

274
275
276
277
278
279
280
281

282
283
284
285
286
287
288
289
290
291
292
258
259
260
261
262
263
264
265

266
267
268
269
270
271
272
273

274




275
276
277
278
279
280
281







+
-
+







-
+
-
-
-
-








func (wui *WebUI) renderMetaList(
	ctx context.Context,
	w http.ResponseWriter,
	title string,
	s *search.Search,
	ucMetaList func(sorter *search.Search) ([]*meta.Meta, error),
	evaluate *usecase.Evaluate,
	pageURL func(int) string) {
) {

	metaList, err := ucMetaList(s)
	if err != nil {
		wui.reportError(ctx, w, err)
		return
	}
	user := wui.getUser(ctx)
	metas, err := wui.buildHTMLMetaList(metaList)
	metas := wui.buildHTMLMetaList(ctx, metaList, evaluate)
	if err != nil {
		wui.reportError(ctx, w, err)
		return
	}
	var base baseData
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base)
	wui.renderTemplate(ctx, w, id.ListTemplateZid, &base, struct {
		Title string
		Metas []simpleLink
	}{
		Title: title,
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326



327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343

344
345
346
347

348
292
293
294
295
296
297
298















299

300
301
302
303
304
305
306
307
308
309
310
311

312




313

314
315
316
317

318
319







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-

-
+
+
+









-

-
-
-
-

-
+



-
+

	if s != nil {
		sb.WriteString(": ")
		s.Print(&sb)
	}
	return sb.String()
}

func (wui *WebUI) newPageURL(key byte, query url.Values, offset int, offsetKey, limitKey string) string {
	ub := wui.NewURLBuilder(key)
	for key, values := range query {
		if key != offsetKey && key != limitKey {
			for _, val := range values {
				ub.AppendQuery(key, val)
			}
		}
	}
	if offset > 0 {
		ub.AppendQuery(offsetKey, strconv.Itoa(offset))
	}
	return ub.String()
}

// buildHTMLMetaList builds a zettel list based on a meta list for HTML rendering.
func (wui *WebUI) buildHTMLMetaList(metaList []*meta.Meta) ([]simpleLink, error) {
func (wui *WebUI) buildHTMLMetaList(
	ctx context.Context, metaList []*meta.Meta, evaluate *usecase.Evaluate,
) []simpleLink {
	defaultLang := wui.rtConfig.GetDefaultLang()
	metas := make([]simpleLink, 0, len(metaList))
	for _, m := range metaList {
		var lang string
		if val, ok := m.Get(meta.KeyLang); ok {
			lang = val
		} else {
			lang = defaultLang
		}
		title, _ := m.Get(meta.KeyTitle)
		env := encoder.Environment{Lang: lang, Interactive: true}
		htmlTitle, err := adapter.FormatInlines(parser.ParseMetadata(title), api.EncoderHTML, &env)
		if err != nil {
			return nil, err
		}
		metas = append(metas, simpleLink{
			Text: htmlTitle,
			Text: wui.encodeTitleAsHTML(ctx, m, evaluate, nil, &env),
			URL:  wui.NewURLBuilder('h').SetZid(m.Zid).String(),
		})
	}
	return metas, nil
	return metas
}

Changes to web/adapter/webui/login.go.

17
18
19
20
21
22
23
24
25



26






27
28
29
30
31
32
33
17
18
19
20
21
22
23


24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40







-
-
+
+
+

+
+
+
+
+
+








	"zettelstore.de/z/auth"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetLoginHandler creates a new HTTP handler to display the HTML login view.
func (wui *WebUI) MakeGetLoginHandler() http.HandlerFunc {
// MakeGetLoginOutHandler creates a new HTTP handler to display the HTML login view,
// or to execute a logout.
func (wui *WebUI) MakeGetLoginOutHandler() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		query := r.URL.Query()
		if query.Has("logout") {
			wui.clearToken(r.Context(), w)
			redirectFound(w, r, wui.NewURLBuilder('/'))
			return
		}
		wui.renderLoginForm(wui.clearToken(r.Context(), w), w, false)
	}
}

func (wui *WebUI) renderLoginForm(ctx context.Context, w http.ResponseWriter, retry bool) {
	var base baseData
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), "Login", nil, &base)
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
67
68
69
70
71
72
73








74
75
76







-
-
-
-
-
-
-
-



		}
		if token == nil {
			wui.renderLoginForm(wui.clearToken(ctx, w), w, true)
			return
		}

		wui.setToken(w, token)
		redirectFound(w, r, wui.NewURLBuilder('/'))
	}
}

// MakeGetLogoutHandler creates a new HTTP handler to log out the current user
func (wui *WebUI) MakeGetLogoutHandler() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		wui.clearToken(r.Context(), w)
		redirectFound(w, r, wui.NewURLBuilder('/'))
	}
}

Changes to web/adapter/webui/rename_zettel.go.

38
39
40
41
42
43
44
45

46
47

48
49
50
51
52
53
54
38
39
40
41
42
43
44

45
46

47
48
49
50
51
52
53
54







-
+

-
+








		m, err := getMeta.Run(ctx, zid)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}

		if format, formatText := adapter.GetFormat(r, r.URL.Query(), api.EncoderHTML); format != api.EncoderHTML {
		if enc, encText := adapter.GetEncoding(r, r.URL.Query(), api.EncoderHTML); enc != api.EncoderHTML {
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Rename zettel %q not possible in format %q", zid.String(), formatText)))
				fmt.Sprintf("Rename zettel %q not possible in encoding %q", zid.String(), encText)))
			return
		}

		user := wui.getUser(ctx)
		var base baseData
		wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Rename Zettel "+zid.String(), user, &base)
		wui.renderTemplate(ctx, w, id.RenameTemplateZid, &base, struct {

Changes to web/adapter/webui/webui.go.

53
54
55
56
57
58
59

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

75
76
77
78
79
80
81
82
83
84

85
86

87
88
89
90
91
92
93


94
95
96
97
98
99
100
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

86


87

88
89
90
91
92

93
94
95
96
97
98
99
100
101







+















+









-
+
-
-
+
-





-
+
+







	cssUserURL    string
	homeURL       string
	listZettelURL string
	listRolesURL  string
	listTagsURL   string
	withAuth      bool
	loginURL      string
	logoutURL     string
	searchURL     string
}

type webuiBox interface {
	CanCreateZettel(ctx context.Context) bool
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
	CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool
	AllowRenameZettel(ctx context.Context, zid id.Zid) bool
	CanDeleteZettel(ctx context.Context, zid id.Zid) bool
}

// New creates a new WebUI struct.
func New(ab server.AuthBuilder, authz auth.AuthzManager, rtConfig config.Config, token auth.TokenManager,
	mgr box.Manager, pol auth.Policy) *WebUI {
	loginoutBase := ab.NewURLBuilder('i')
	wui := &WebUI{
		ab:       ab,
		rtConfig: rtConfig,
		authz:    authz,
		token:    token,
		box:      mgr,
		policy:   pol,

		tokenLifetime: kernel.Main.GetConfig(kernel.WebService, kernel.WebTokenLifetimeHTML).(time.Duration),
		cssBaseURL: ab.NewURLBuilder('z').SetZid(
		cssBaseURL:    ab.NewURLBuilder('z').SetZid(id.BaseCSSZid).String(),
			id.BaseCSSZid).AppendQuery("_format", "raw").AppendQuery("_part", "content").String(),
		cssUserURL: ab.NewURLBuilder('z').SetZid(
		cssUserURL:    ab.NewURLBuilder('z').SetZid(id.UserCSSZid).String(),
			id.UserCSSZid).AppendQuery("_format", "raw").AppendQuery("_part", "content").String(),
		homeURL:       ab.NewURLBuilder('/').String(),
		listZettelURL: ab.NewURLBuilder('h').String(),
		listRolesURL:  ab.NewURLBuilder('h').AppendQuery("_l", "r").String(),
		listTagsURL:   ab.NewURLBuilder('h').AppendQuery("_l", "t").String(),
		withAuth:      authz.WithAuth(),
		loginURL:      ab.NewURLBuilder('a').String(),
		loginURL:      loginoutBase.String(),
		logoutURL:     loginoutBase.AppendQuery("logout", "").String(),
		searchURL:     ab.NewURLBuilder('f').String(),
	}
	wui.observe(box.UpdateInfo{Box: mgr, Reason: box.OnReload, Zid: id.Invalid})
	mgr.RegisterObserver(wui.observe)
	return wui
}

170
171
172
173
174
175
176
177
178

179
180
181
182
183
184

185
186
187
188
189
190

191
192
193


194
195

196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215

216
217
218
219
220
221

222
223
224
225
226
227
228
171
172
173
174
175
176
177

178
179
180
181
182
183
184
185
186
187
188
189
190


191



192
193


194
195
196
197
198

199
200
201
202
203
204
205
206
207
208
209
210
211

212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227







-

+






+




-
-
+
-
-
-
+
+
-
-
+




-













-

+






+







	Title             string
	HomeURL           string
	WithUser          bool
	WithAuth          bool
	UserIsValid       bool
	UserZettelURL     string
	UserIdent         string
	UserLogoutURL     string
	LoginURL          string
	LogoutURL         string
	ListZettelURL     string
	ListRolesURL      string
	ListTagsURL       string
	HasNewZettelLinks bool
	NewZettelLinks    []simpleLink
	SearchURL         string
	QueryKeySearch    string
	Content           string
	FooterHTML        string
}

func (wui *WebUI) makeBaseData(
	ctx context.Context, lang, title string, user *meta.Meta, data *baseData) {
func (wui *WebUI) makeBaseData(ctx context.Context, lang, title string, user *meta.Meta, data *baseData) {
	var (
		userZettelURL string
		userIdent     string
	var userZettelURL string
	var userIdent string
		userLogoutURL string
	)

	userIsValid := user != nil
	if userIsValid {
		userZettelURL = wui.NewURLBuilder('h').SetZid(user.Zid).String()
		userIdent = user.GetDefault(meta.KeyUserID, "")
		userLogoutURL = wui.NewURLBuilder('a').SetZid(user.Zid).String()
	}
	newZettelLinks := wui.fetchNewTemplates(ctx, user)

	data.Lang = lang
	data.CSSBaseURL = wui.cssBaseURL
	data.CSSUserURL = wui.cssUserURL
	data.Title = title
	data.HomeURL = wui.homeURL
	data.WithAuth = wui.withAuth
	data.WithUser = data.WithAuth
	data.UserIsValid = userIsValid
	data.UserZettelURL = userZettelURL
	data.UserIdent = userIdent
	data.UserLogoutURL = userLogoutURL
	data.LoginURL = wui.loginURL
	data.LogoutURL = wui.logoutURL
	data.ListZettelURL = wui.listZettelURL
	data.ListRolesURL = wui.listRolesURL
	data.ListTagsURL = wui.listTagsURL
	data.HasNewZettelLinks = len(newZettelLinks) > 0
	data.NewZettelLinks = newZettelLinks
	data.SearchURL = wui.searchURL
	data.QueryKeySearch = api.QueryKeySearch
	data.FooterHTML = wui.rtConfig.GetFooterHTML()
}

// htmlAttrNewWindow returns HTML attribute string for opening a link in a new window.
// If hasURL is false an empty string is returned.
func htmlAttrNewWindow(hasURL bool) string {
	if hasURL {
252
253
254
255
256
257
258
259

260
261

262
263
264
265
266
267
268
251
252
253
254
255
256
257

258
259

260
261
262
263
264
265
266
267







-
+

-
+







		}
		if !wui.policy.CanRead(user, m) {
			continue
		}
		title := config.GetTitle(m, wui.rtConfig)
		astTitle := parser.ParseInlines(input.NewInput(title), meta.ValueSyntaxZmk)
		env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)}
		menuTitle, err := adapter.FormatInlines(astTitle, api.EncoderHTML, &env)
		menuTitle, err := encodeInlines(astTitle, api.EncoderHTML, &env)
		if err != nil {
			menuTitle, err = adapter.FormatInlines(astTitle, api.EncoderText, nil)
			menuTitle, err = encodeInlines(astTitle, api.EncoderText, nil)
			if err != nil {
				menuTitle = title
			}
		}
		result = append(result, simpleLink{
			Text: menuTitle,
			URL:  wui.NewURLBuilder('g').SetZid(m.Zid).String(),

Changes to web/server/impl/impl.go.

39
40
41
42
43
44
45
46
47


48
49
50


51
52
53
54
55
56
57
39
40
41
42
43
44
45


46
47
48


49
50
51
52
53
54
55
56
57







-
-
+
+

-
-
+
+







	srv.server.initializeHTTPServer(listenAddr, &srv.router)
	return &srv
}

func (srv *myServer) Handle(pattern string, handler http.Handler) {
	srv.router.Handle(pattern, handler)
}
func (srv *myServer) AddListRoute(key byte, httpMethod string, handler http.Handler) {
	srv.router.addListRoute(key, httpMethod, handler)
func (srv *myServer) AddListRoute(key byte, method server.Method, handler http.Handler) {
	srv.router.addListRoute(key, method, handler)
}
func (srv *myServer) AddZettelRoute(key byte, httpMethod string, handler http.Handler) {
	srv.router.addZettelRoute(key, httpMethod, handler)
func (srv *myServer) AddZettelRoute(key byte, method server.Method, handler http.Handler) {
	srv.router.addZettelRoute(key, method, handler)
}
func (srv *myServer) SetUserRetriever(ur server.UserRetriever) {
	srv.router.ur = ur
}
func (srv *myServer) GetUser(ctx context.Context) *meta.Meta {
	if data := srv.GetAuthData(ctx); data != nil {
		return data.User

Changes to web/server/impl/router.go.

12
13
14
15
16
17
18

19

20
21
22
23
24
25


26









27
28
29
30
31
32
33
12
13
14
15
16
17
18
19
20
21
22
23
24
25


26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44







+

+




-
-
+
+

+
+
+
+
+
+
+
+
+







package impl

import (
	"net/http"
	"regexp"
	"strings"

	"zettelstore.de/z/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/web/server"
)

type (
	methodHandler map[string]http.Handler
	routingTable  map[byte]methodHandler
	methodHandler [server.MethodLAST]http.Handler
	routingTable  [256]*methodHandler
)

var mapMethod = map[string]server.Method{
	http.MethodHead:   server.MethodHead,
	http.MethodGet:    server.MethodGet,
	http.MethodPost:   server.MethodPost,
	http.MethodPut:    server.MethodPut,
	http.MethodDelete: server.MethodDelete,
	api.MethodMove:    server.MethodMove,
}

// httpRouter handles all routing for zettelstore.
type httpRouter struct {
	urlPrefix   string
	auth        auth.TokenManager
	minKey      byte
	maxKey      byte
42
43
44
45
46
47
48
49
50
51
52
53

54
55
56
57
58
59
60
61
62
63
64
65
66
67
68



69
70
71
72
73
74




75
76
77
78
79
80
81


82
83
84
85
86


87
88
89
90
91
92
93
94







95
96
97
98
99
100
101
102
103
104
105
106
107
108
109

110
111



112
113
114



115
116
117
118
119
120
121
53
54
55
56
57
58
59


60
61

62
63
64
65
66
67
68
69
70
71
72
73
74



75
76
77
78
79




80
81
82
83
84
85
86
87
88


89
90
91
92
93


94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124

125
126

127
128
129
130


131
132
133
134
135
136
137
138
139
140







-
-


-
+












-
-
-
+
+
+


-
-
-
-
+
+
+
+





-
-
+
+



-
-
+
+








+
+
+
+
+
+
+














-
+

-
+
+
+

-
-
+
+
+







func (rt *httpRouter) initializeRouter(urlPrefix string, auth auth.TokenManager) {
	rt.urlPrefix = urlPrefix
	rt.auth = auth
	rt.minKey = 255
	rt.maxKey = 0
	rt.reURL = regexp.MustCompile("^$")
	rt.mux = http.NewServeMux()
	rt.listTable = make(routingTable)
	rt.zettelTable = make(routingTable)
}

func (rt *httpRouter) addRoute(key byte, httpMethod string, handler http.Handler, table routingTable) {
func (rt *httpRouter) addRoute(key byte, method server.Method, handler http.Handler, table *routingTable) {
	// Set minKey and maxKey; re-calculate regexp.
	if key < rt.minKey || rt.maxKey < key {
		if key < rt.minKey {
			rt.minKey = key
		}
		if rt.maxKey < key {
			rt.maxKey = key
		}
		rt.reURL = regexp.MustCompile(
			"^/(?:([" + string(rt.minKey) + "-" + string(rt.maxKey) + "])(?:/(?:([0-9]{14})/?)?)?)$")
	}

	mh, hasKey := table[key]
	if !hasKey {
		mh = make(methodHandler)
	mh := table[key]
	if mh == nil {
		mh = new(methodHandler)
		table[key] = mh
	}
	mh[httpMethod] = handler
	if httpMethod == http.MethodGet {
		if _, hasHead := table[key][http.MethodHead]; !hasHead {
			table[key][http.MethodHead] = handler
	mh[method] = handler
	if method == server.MethodGet {
		if handler := mh[server.MethodHead]; handler == nil {
			mh[server.MethodHead] = handler
		}
	}
}

// addListRoute adds a route for the given key and HTTP method to work with a list.
func (rt *httpRouter) addListRoute(key byte, httpMethod string, handler http.Handler) {
	rt.addRoute(key, httpMethod, handler, rt.listTable)
func (rt *httpRouter) addListRoute(key byte, method server.Method, handler http.Handler) {
	rt.addRoute(key, method, handler, &rt.listTable)
}

// addZettelRoute adds a route for the given key and HTTP method to work with a zettel.
func (rt *httpRouter) addZettelRoute(key byte, httpMethod string, handler http.Handler) {
	rt.addRoute(key, httpMethod, handler, rt.zettelTable)
func (rt *httpRouter) addZettelRoute(key byte, method server.Method, handler http.Handler) {
	rt.addRoute(key, method, handler, &rt.zettelTable)
}

// Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics.
func (rt *httpRouter) Handle(pattern string, handler http.Handler) {
	rt.mux.Handle(pattern, handler)
}

func (rt *httpRouter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	// Something may panic. Ensure a kernel log.
	defer func() {
		if r := recover(); r != nil {
			kernel.Main.LogRecover("Web", r)
		}
	}()

	if prefixLen := len(rt.urlPrefix); prefixLen > 1 {
		if len(r.URL.Path) < prefixLen || r.URL.Path[:prefixLen] != rt.urlPrefix {
			http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
			return
		}
		r.URL.Path = r.URL.Path[prefixLen-1:]
	}
	match := rt.reURL.FindStringSubmatch(r.URL.Path)
	if len(match) != 3 {
		rt.mux.ServeHTTP(w, rt.addUserContext(r))
		return
	}

	key := match[1][0]
	table := rt.zettelTable
	var mh *methodHandler
	if match[2] == "" {
		table = rt.listTable
		mh = rt.listTable[key]
	} else {
		mh = rt.zettelTable[key]
	}
	if mh, ok := table[key]; ok {
		if handler, ok := mh[r.Method]; ok {
	method, ok := mapMethod[r.Method]
	if ok && mh != nil {
		if handler := mh[method]; handler != nil {
			r.URL.Path = "/" + match[2]
			handler.ServeHTTP(w, rt.addUserContext(r))
			return
		}
	}
	http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed)
}

Changes to web/server/server.go.

21
22
23
24
25
26
27














28
29
30
31
32
33


34
35
36
37
38
39
40
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45


46
47
48
49
50
51
52
53
54







+
+
+
+
+
+
+
+
+
+
+
+
+
+




-
-
+
+







	"zettelstore.de/z/domain/meta"
)

// UserRetriever allows to retrieve user data based on a given zettel identifier.
type UserRetriever interface {
	GetUser(ctx context.Context, zid id.Zid, ident string) (*meta.Meta, error)
}

// Method enumerates the allowed HTTP methods.
type Method uint8

// Values for method type
const (
	MethodGet Method = iota
	MethodHead
	MethodPost
	MethodPut
	MethodMove
	MethodDelete
	MethodLAST // must always be the last one
)

// Router allows to state routes for various URL paths.
type Router interface {
	Handle(pattern string, handler http.Handler)
	AddListRoute(key byte, httpMethod string, handler http.Handler)
	AddZettelRoute(key byte, httpMethod string, handler http.Handler)
	AddListRoute(key byte, method Method, handler http.Handler)
	AddZettelRoute(key byte, method Method, handler http.Handler)
	SetUserRetriever(ur UserRetriever)
}

// Builder allows to build new URLs for the web service.
type Builder interface {
	GetURLPrefix() string
	NewURLBuilder(key byte) *api.URLBuilder

Changes to www/changes.wiki.

1
2



3
4





















































5
6
7
8
9
10
11
1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66


+
+
+

-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







<title>Change Log</title>

<a name="0_0_16"></a>
<h2>Changes for Version 0.0.16 (pending)</h2>

<a name="0_0_15"></a>
<h2>Changes for Version 0.0.15 (pending)</h2>
<h2>Changes for Version 0.0.15 (2021-09-17)</h2>
  *  Move again endpoint characters for authentication to make room for future
     features. WebUI authentication moves from <tt>/a</tt> to <tt>/i</tt>
     (login) and <tt>/i?logout</tt> (logout). API authentication moves from
     <tt>/v</tt> to </tt>/a</tt>. JSON-based basic zettel handling moves from
     <tt>/z</tt> to <tt>/j</tt> and <tt>/z/{ID}</tt> to <tt>/j/{ID}</tt>. Since
     the API client is updated too, this should not be a breaking change for
     most users.
     (minor: api, webui; possibly breaking)
  *  Add API endpoint <tt>/v/{ID}</tt> to retrieve an evaluated zettel in
     various encodings. Mostly replaces endpoint <tt>/z/{ID}</tt> for other
     encodings except &ldquo;json&rdquo; and &ldquo;raw&rdquo;. Endpoint
     <tt>/j/{ID}</tt> now only returns JSON data, endpoint <tt>/z/{ID}</tt> is
     used to retrieve plain zettel data (previously called &ldquo;raw&rdquo;).
     See documentation for details.
     (major: api; breaking)
  *  Metadata values of type <em>tag set</em> (the metadata with key
     <tt>tags</tt> is its most prominent example), are now compared in
     a case-insensitive manner. Tags that only differ in upper / lower case
     character are now treated identical. This might break your workflow, if
     you depend on case-sensitive comparison of tag values. Tag values are
     translated to their lower case equivalent before comparing them and when
     you edit a zettel through Zettelstore. If you just modify the zettel
     files, your tag values remain unchanged.
     (major; breaking)
  *  Endpoint <tt>/z/{ID}</tt> allows the same methods as endpoint
     <tt>/j/{ID}</tt>: <tt>GET</tt> retrieves zettel (see above), <tt>PUT</tt>
     updates a zettel, <tt>DELETE</tt> deletes a zettel, <tt>MOVE</tt> renames
     a zettel. In addtion, <tt>POST /z</tt> will create a new zettel. When
     zettel data must be given, the format is plain text, with metadata
     separated from content by an empty line. See documentation for more
     details.
     (major: api (plus WebUI for some details))
  *  Allows to transclude / expand the content of another zettel into a target
     zettel when the zettel is rendered. By using the syntax of embedding an
     image (which is some kind of expansion too), the first top-level paragraph
     of a zettel may be transcluded into the target zettel. Endless recursion
     is checked, as well as a possible &ldquo;transclusion bomb &rdquo;
     (similar to a XML bomb). See manual for details.
     (major: zettelmarkup)
  *  The endpoint <tt>/z</tt> allows to list zettel in a simpler format than
     endpoint <tt>/j</tt>: one line per zettel, and only zettel identifier plus
     zettel title.
     (minor: api)
  *  Folgezettel are now displayed with full title at the bottom of a page.
     (minor: webui)
  *  Add API endpoint <tt>/p/{ID}</tt> to retrieve a parsed, but not evaluated
     zettel in various encodings.
     (minor: api)
  *  Fix: do not list a shadowed zettel that matches the select criteria.
     (minor)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_0_14"></a>
<h2>Changes for Version 0.0.14 (2021-07-23)</h2>
  *  Rename &ldquo;place&rdquo; into &ldquo;box&rdquo;. This also affects the
     configuration keys to specify boxes <tt>box-uri<em>X</em></tt> (previously
     <tt>place-uri-<em>X</em></tt>. Older changes documented here are renamed
     too.

Changes to www/download.wiki.

1
2
3
4
5
6
7
8
9
10
11
12

13
14
15
16
17
18





19
20
21
22
23
24

25
26
1
2
3
4
5
6
7
8
9
10
11

12
13





14
15
16
17
18
19
20
21
22
23

24
25
26











-
+

-
-
-
-
-
+
+
+
+
+





-
+


<title>Download</title>
<h1>Download of Zettelstore Software</h1>
<h2>Foreword</h2>
  *  Zettelstore is free/libre open source software, licensed under EUPL-1.2-or-later.
  *  The software is provided as-is.
  *  There is no guarantee that it will not damage your system.
  *  However, it is in use by the main developer since March 2020 without any damage.
  *  It may be useful for you. It is useful for me.
  *  Take a look at the [https://zettelstore.de/manual/|manual] to know how to start and use it.

<h2>ZIP-ped Executables</h2>
Build: <code>v0.0.14</code> (2021-07-23).
Build: <code>v0.0.15</code> (2021-09-17).

  *  [/uv/zettelstore-0.0.14-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.0.14-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.0.14-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.0.14-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.0.14-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)
  *  [/uv/zettelstore-0.0.15-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.0.15-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.0.15-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.0.15-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.0.15-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)

Unzip the appropriate file, install and execute Zettelstore according to the manual.

<h2>Zettel for the manual</h2>
As a starter, you can download the zettel for the manual
[/uv/manual-0.0.13.zip|here]. Just unzip the contained files and put them into
[/uv/manual-0.0.15.zip|here]. Just unzip the contained files and put them into
your zettel folder or configure a file box to read the zettel directly from the
ZIP file.

Changes to www/index.wiki.

13
14
15
16
17
18
19
20

21
22
23
24
25
26





27
28
29
30
31
32
33
34
35
36
37
13
14
15
16
17
18
19

20
21





22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37







-
+

-
-
-
-
-
+
+
+
+
+











It is a live example of the zettelstore software, running in read-only mode.

The software, including the manual, is licensed under the
[/file?name=LICENSE.txt&ci=trunk|European Union Public License 1.2 (or later)].

[https://twitter.com/zettelstore|Stay tuned]&hellip;
<hr>
<h3>Latest Release: 0.0.14 (2021-07-23)</h3>
<h3>Latest Release: 0.0.15 (2021-09-17)</h3>
  *  [./download.wiki|Download]
  *  [./changes.wiki#0_0_14|Change summary]
  *  [/timeline?p=version-0.0.14&bt=version-0.0.13&y=ci|Check-ins for version 0.0.14],
     [/vdiff?to=version-0.0.14&from=version-0.0.13|content diff]
  *  [/timeline?df=version-0.0.14&y=ci|Check-ins derived from the 0.0.14 release],
     [/vdiff?from=version-0.0.14&to=trunk|content diff]
  *  [./changes.wiki#0_0_15|Change summary]
  *  [/timeline?p=version-0.0.15&bt=version-0.0.14&y=ci|Check-ins for version 0.0.15],
     [/vdiff?to=version-0.0.15&from=version-0.0.14|content diff]
  *  [/timeline?df=version-0.0.15&y=ci|Check-ins derived from the 0.0.15 release],
     [/vdiff?from=version-0.0.15&to=trunk|content diff]
  *  [./plan.wiki|Limitations and planned improvements]
  *  [/timeline?t=release|Timeline of all past releases]

<hr>
<h2>Build instructions</h2>
Just install [https://golang.org/dl/|Go] and some Go-based tools. Please read
the [./build.md|instructions] for details.

  *  [/dir?ci=trunk|Source code]
  *  [/download|Download the source code] as a tarball or a ZIP file
     (you must [/login|login] as user &quot;anonymous&quot;).

Changes to www/plan.wiki.

1
2
3
4
5
6
7
8
9
10
11
12

13
14
15
16
17
18
19
20
21
22
23
24


25
26
27
28
29
30
31
32
1
2
3
4
5
6
7
8
9
10
11

12
13
14
15
16
17


18
19
20
21
22
23
24
25
26
27
28
29


30











-
+





-
-





+
+





-
-

<title>Limitations and planned improvements</title>

Here is a list of some shortcomings of Zettelstore.
They are planned to be solved.

<h3>Serious limitations</h3>
  *  Content with binary data (e.g. a GIF, PNG, or JPG file) cannot be created
     nor modified via the standard web interface. As a workaround, you should
     put your file into the directory where your zettel are stored. Make sure
     that the file name starts with unique 14 digits that make up the zettel
     identifier.
  *  Automatic lists and transclusions are not supported in Zettelmarkup.
  *  Automatic lists and full transclusions are not supported in Zettelmarkup.
  *  &hellip;

<h3>Smaller limitations</h3>
  *  Quoted attribute values are not yet supported in Zettelmarkup:
     <code>{key="value with space"}</code>.
  *  The <tt>file</tt> sub-command currently does not support output format
     &ldquo;json&rdquo;.
  *  The horizontal tab character (<tt>U+0009</tt>) is not supported.
  *  Missing support for citation keys.
  *  Changing the content syntax is not reflected in file extension.
  *  File names with additional text besides the zettel identifier are not
     always preserved.
  *  A running Zettelstore does not always detect when a directory box is
     removed. In case, it expects some zettel that cannot be retrieved.
  *  &hellip;

<h3>Planned improvements</h3>
  *  Support for mathematical content is missing, e.g. <code>$$F(x) &=
     \\int^a_b \\frac{1}{3}x^3$$</code>.
  *  Render zettel in [https://pandoc.org|pandoc's] JSON version of
     their native AST to make pandoc an external renderer for Zettelstore.
  *  &hellip;