Zettelstore Client

Check-in Differences
Login

Check-in Differences

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Difference From v0.11.0 To trunk

2025-03-20
15:22
Moved ScanEntity back to zsx ... (Leaf check-in: 07bbd8cce6 user: stern tags: trunk)
15:13
Got zsc/input/ScanEntity ... (check-in: 95f53cd9aa user: stern tags: trunk)
2023-04-02
16:14
Add metadata key 'expire' ... (check-in: e11f223228 user: stern tags: trunk)
2023-03-27
14:37
Version 0.11.0 ... (check-in: 331cad470c user: stern tags: release, trunk, v0.11.0)
14:34
Version 0.11.0 ... (check-in: 712c21afc8 user: stern tags: release, trunk, v0.10.0)

Deleted .github/dependabot.yml.

1
2
3
4
5
6
7
8
9
10
11
12












-
-
-
-
-
-
-
-
-
-
-
-
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates

version: 2
updates:
  - package-ecosystem: "gomod" # See documentation for possible values
    directory: "/" # Location of package manifests
    schedule:
      interval: "daily"
    rebase-strategy: "disabled"

Changes to api/api.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

35
36
37
38
39
40
41
42
43
44
45
46

47
48
49

50
51
52
53
54
55
56

57
58
59
60

61
62
63
64
65




66
67
68
69
70
71
72
73
74

75
76
77
78
79




80
81
82
83
84
85
86
87
88
89
90
91








92
93

94
95
96

97
98
99
100
101
102


103
104
105
106
107
108
109
110
111
112
113


114
115
116
117
118
119
120
121
122
123
124
125
126
127
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16



17


18















19
20
21
22
23
24
25
26
27
28
29
30

31
32


33







34




35





36
37
38
39
40
41







42





43
44
45
46

47










48
49
50
51
52
53
54
55


56



57






58
59
60
61









62
63






















+
+
+





-
-
-
+
-
-

-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+











-
+

-
-
+
-
-
-
-
-
-
-
+
-
-
-
-
+
-
-
-
-
-
+
+
+
+


-
-
-
-
-
-
-
+
-
-
-
-
-
+
+
+
+
-

-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
-
-
+
-
-
-
+
-
-
-
-
-
-
+
+


-
-
-
-
-
-
-
-
-
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package api contains common definitions used for client and server.
package api

// ZettelID contains the identifier of a zettel. It is a string with 14 digits.
type ZettelID string

import "t73f.de/r/zsc/domain/id"
// InvalidZID is an invalif zettel identifier
const InvalidZID = ""

// IsValid returns true, if the idenfifier contains 14 digits.
func (zid ZettelID) IsValid() bool {
	if len(zid) != 14 {
		return false
	}
	for i := 0; i < 14; i++ {
		ch := zid[i]
		if ch < '0' || '9' < ch {
			return false
		}
	}
	return true
}

// ZettelMeta is a map containg the metadata of a zettel.
// ZettelMeta is a map containg the normalized metadata of a zettel.
type ZettelMeta map[string]string

// ZettelRights is an integer that encode access rights for a zettel.
type ZettelRights uint8

// Values for ZettelRights, can be or-ed
const (
	ZettelCanNone   ZettelRights = 1 << iota
	ZettelCanCreate              // Current user is allowed to create a new zettel
	ZettelCanRead                // Requesting user is allowed to read the zettel
	ZettelCanWrite               // Requesting user is allowed to update the zettel
	ZettelCanRename              // Requesting user is allowed to provide the zettel with a new identifier
	placeholdergo1               // Was assigned to rename right, which is now removed
	ZettelCanDelete              // Requesting user is allowed to delete the zettel
)

	ZettelMaxRight               // Sentinel value
// AuthJSON contains the result of an authentication call.
type AuthJSON struct {
	Token   string `json:"token"`
	Type    string `json:"token_type"`
	Expires int    `json:"expires_in"`
}

)
// ZidJSON contains the identifier data of a zettel.
type ZidJSON struct {
	ID ZettelID `json:"id"`
}


// MetaJSON contains the metadata of a zettel.
type MetaJSON struct {
	Meta   ZettelMeta   `json:"meta"`
	Rights ZettelRights `json:"rights"`
// MetaRights contains the metadata of a zettel, and its rights.
type MetaRights struct {
	Meta   ZettelMeta
	Rights ZettelRights
}

// ZidMetaJSON contains the identifier and the metadata of a zettel.
type ZidMetaJSON struct {
	ID     ZettelID     `json:"id"`
	Meta   ZettelMeta   `json:"meta"`
	Rights ZettelRights `json:"rights"`
}

// ZidMetaRights contains the identifier, the metadata of a zettel, and its rights.
// ZidMetaRelatedList contains identifier/metadata of a zettel and the same for related zettel
type ZidMetaRelatedList struct {
	ID     ZettelID      `json:"id"`
	Meta   ZettelMeta    `json:"meta"`
	Rights ZettelRights  `json:"rights"`
type ZidMetaRights struct {
	ID     id.Zid
	Meta   ZettelMeta
	Rights ZettelRights
	List   []ZidMetaJSON `json:"list"`
}

// ZettelDataJSON contains all data for a zettel.
type ZettelDataJSON struct {
	Meta     ZettelMeta `json:"meta"`
	Encoding string     `json:"encoding"`
	Content  string     `json:"content"`
}

// ZettelJSON contains all data for a zettel, the identifier, the metadata, and the content.
type ZettelJSON struct {

// ZettelData contains all data for a zettel.
//
//   - Meta is a map containing the metadata of the zettel.
//   - Rights is an integer specifying the access rights.
//   - Encoding is a string specifying the encoding of the zettel content.
//   - Content is the zettel content itself.
type ZettelData struct {
	ID       ZettelID     `json:"id"`
	Meta     ZettelMeta   `json:"meta"`
	Meta     ZettelMeta
	Encoding string       `json:"encoding"`
	Content  string       `json:"content"`
	Rights   ZettelRights `json:"rights"`
	Rights   ZettelRights
}

// ZettelContentJSON contains all elements to transfer the content of a zettel.
type ZettelContentJSON struct {
	Encoding string `json:"encoding"`
	Content  string `json:"content"`
	Encoding string
	Content  string // raw, uninterpreted zettel content
}

// ZettelListJSON contains data for a zettel list.
type ZettelListJSON struct {
	Query string        `json:"query"`
	Human string        `json:"human"`
	List  []ZidMetaJSON `json:"list"`
}

// MapMeta maps metadata keys to list of metadata.
type MapMeta map[string][]ZettelID
// Aggregate maps metadata keys to list of zettel identifier.
type Aggregate map[string][]id.Zid

// MapListJSON specifies the map of metadata key to list of metadata that contains the key.
type MapListJSON struct {
	Map MapMeta `json:"map"`
}

// VersionJSON contains version information.
type VersionJSON struct {
	Major int    `json:"major"`
	Minor int    `json:"minor"`
	Patch int    `json:"patch"`
	Info  string `json:"info"`
	Hash  string `json:"hash"`
}

Changes to api/const.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163

164
165
166

167
168
169
170
171
172
173
174
175
176


177
178
179
180




181
182
183


184
185
186
187
188
189
190

191
192
193
194
195

196
197
198
199
200
201
202
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
































































































































18
19


20
21
22
23
24
25
26
27
28


29
30

31
32
33
34
35


36




37
38
39
40


41
42




43
44
45
46
47


48
49
50
51
52
53
54

55
56
57
58
59
60

61
62
63
64
65
66
67
68








+
+
+






-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-


-
-









-
-


-



+

-
-
+
-
-
-
-




-
-
+
+
-
-
-
-
+
+
+
+

-
-
+
+





-

+




-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package api

import "fmt"

// Predefined Zettel Identifier
const (
	// System zettel
	ZidVersion              = ZettelID("00000000000001")
	ZidHost                 = ZettelID("00000000000002")
	ZidOperatingSystem      = ZettelID("00000000000003")
	ZidLicense              = ZettelID("00000000000004")
	ZidAuthors              = ZettelID("00000000000005")
	ZidDependencies         = ZettelID("00000000000006")
	ZidLog                  = ZettelID("00000000000007")
	ZidBoxManager           = ZettelID("00000000000020")
	ZidMetadataKey          = ZettelID("00000000000090")
	ZidParser               = ZettelID("00000000000092")
	ZidStartupConfiguration = ZettelID("00000000000096")
	ZidConfiguration        = ZettelID("00000000000100")

	// WebUI HTML templates are in the range 10000..19999
	ZidBaseTemplate    = ZettelID("00000000010100")
	ZidLoginTemplate   = ZettelID("00000000010200")
	ZidListTemplate    = ZettelID("00000000010300")
	ZidZettelTemplate  = ZettelID("00000000010401")
	ZidInfoTemplate    = ZettelID("00000000010402")
	ZidFormTemplate    = ZettelID("00000000010403")
	ZidRenameTemplate  = ZettelID("00000000010404")
	ZidDeleteTemplate  = ZettelID("00000000010405")
	ZidContextTemplate = ZettelID("00000000010406")
	ZidErrorTemplate   = ZettelID("00000000010700")

	// CSS-related zettel are in the range 20000..29999
	ZidBaseCSS    = ZettelID("00000000020001")
	ZidUserCSS    = ZettelID("00000000025001")
	ZidRoleCSSMap = ZettelID("00000000029000") // Maps roles to CSS zettel, which should be in the range 29001..29999.

	// WebUI JS zettel are in the range 30000..39999

	// WebUI image zettel are in the range 40000..49999
	ZidEmoji = ZettelID("00000000040001")

	// Range 90000...99999 is reserved for zettel templates
	ZidTOCNewTemplate    = ZettelID("00000000090000")
	ZidTemplateNewZettel = ZettelID("00000000090001")
	ZidTemplateNewUser   = ZettelID("00000000090002")

	ZidDefaultHome = ZettelID("00010000000000")
)

// LengthZid factors the constant length of a zettel identifier
const LengthZid = len(ZidDefaultHome)

// Values of the metadata key/value type.
const (
	MetaCredential   = "Credential"
	MetaEmpty        = "EString"
	MetaID           = "Identifier"
	MetaIDSet        = "IdentifierSet"
	MetaNumber       = "Number"
	MetaString       = "String"
	MetaTagSet       = "TagSet"
	MetaTimestamp    = "Timestamp"
	MetaURL          = "URL"
	MetaWord         = "Word"
	MetaWordSet      = "WordSet"
	MetaZettelmarkup = "Zettelmarkup"
)

// Predefined general Metadata keys
const (
	KeyID           = "id"
	KeyTitle        = "title"
	KeyRole         = "role"
	KeyTags         = "tags"
	KeySyntax       = "syntax"
	KeyAuthor       = "author"
	KeyBack         = "back"
	KeyBackward     = "backward"
	KeyBoxNumber    = "box-number"
	KeyCopyright    = "copyright"
	KeyCreated      = "created"
	KeyCredential   = "credential"
	KeyDead         = "dead"
	KeyFolge        = "folge"
	KeyForward      = "forward"
	KeyLang         = "lang"
	KeyLicense      = "license"
	KeyModified     = "modified"
	KeyPrecursor    = "precursor"
	KeyPredecessor  = "predecessor"
	KeyPublished    = "published"
	KeyQuery        = "query"
	KeyReadOnly     = "read-only"
	KeySuccessors   = "successors"
	KeySummary      = "summary"
	KeyURL          = "url"
	KeyUselessFiles = "useless-files"
	KeyUserID       = "user-id"
	KeyUserRole     = "user-role"
	KeyVisibility   = "visibility"
)

// Predefined Metadata values
const (
	ValueFalse             = "false"
	ValueTrue              = "true"
	ValueLangEN            = "en"
	ValueRoleConfiguration = "configuration"
	ValueRoleZettel        = "zettel"
	ValueSyntaxCSS         = "css"
	ValueSyntaxDraw        = "draw"
	ValueSyntaxGif         = "gif"
	ValueSyntaxHTML        = "html"
	ValueSyntaxMarkdown    = "markdown"
	ValueSyntaxMD          = "md"
	ValueSyntaxMustache    = "mustache"
	ValueSyntaxNone        = "none"
	ValueSyntaxSVG         = "svg"
	ValueSyntaxText        = "text"
	ValueSyntaxZmk         = "zmk"
	ValueUserRoleCreator   = "creator"
	ValueUserRoleOwner     = "owner"
	ValueUserRoleReader    = "reader"
	ValueUserRoleWriter    = "writer"
	ValueVisibilityCreator = "creator"
	ValueVisibilityExpert  = "expert"
	ValueVisibilityLogin   = "login"
	ValueVisibilityOwner   = "owner"
	ValueVisibilityPublic  = "public"
)

// Additional HTTP constants.
const (
	MethodMove = "MOVE" // HTTP method for renaming a zettel

	HeaderAccept      = "Accept"
	HeaderContentType = "Content-Type"
	HeaderDestination = "Destination"
	HeaderLocation    = "Location"
)

// Values for HTTP query parameter.
const (
	QueryKeyCommand   = "cmd"
	QueryKeyCost      = "cost"
	QueryKeyDir       = "dir"
	QueryKeyEncoding  = "enc"
	QueryKeyParseOnly = "parseonly"
	QueryKeyLimit     = "limit"
	QueryKeyPart      = "part"
	QueryKeyPhrase    = "phrase"
	QueryKeyQuery     = "q"
	QueryKeyRole      = "role"
	QueryKeySeed      = "_seed"
)

	QueryKeyTag       = "tag"
// Supported dir values.
const (
	DirBackward = "backward"
	DirForward  = "forward"
)

// Supported encoding values.
const (
	EncodingHTML  = "html"
	EncodingMD    = "md"
	EncodingHTML  = "html"  // Plain HTML
	EncodingMD    = "md"    // Markdown
	EncodingSexpr = "sexpr"
	EncodingSHTML = "shtml"
	EncodingText  = "text"
	EncodingZMK   = "zmk"
	EncodingSHTML = "shtml" // SxHTML
	EncodingSz    = "sz"    // Structure of zettel, encoded a an S-expression
	EncodingText  = "text"  // plain text content
	EncodingZMK   = "zmk"   // Zettelmarkup

	EncodingPlain = "plain"
	EncodingJson  = "json"
	EncodingPlain = "plain" // Plain zettel, no processing
	EncodingData  = "data"  // Plain zettel, metadata as S-Expression
)

var mapEncodingEnum = map[string]EncodingEnum{
	EncodingHTML:  EncoderHTML,
	EncodingMD:    EncoderMD,
	EncodingSexpr: EncoderSexpr,
	EncodingSHTML: EncoderSHTML,
	EncodingSz:    EncoderSz,
	EncodingText:  EncoderText,
	EncodingZMK:   EncoderZmk,

	EncodingPlain: EncoderPlain,
	EncodingJson:  EncoderJson,
	EncodingData:  EncoderData,
}
var mapEnumEncoding = map[EncodingEnum]string{}

func init() {
	for k, v := range mapEncodingEnum {
		mapEnumEncoding[v] = k
	}
214
215
216
217
218
219
220
221
222


223
224
225
226
227

228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248

249
250
251
252
253
254

255



















256
257
258
259
260
261
262
263
264
265
266
267



























268



80
81
82
83
84
85
86


87
88
89
90
91
92

93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

114
115
116
117
118
119

120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140












141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171







-
-
+
+




-
+




















-
+





-
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

+
+
+
type EncodingEnum uint8

// Values for EncoderEnum
const (
	EncoderUnknown EncodingEnum = iota
	EncoderHTML
	EncoderMD
	EncoderSexpr
	EncoderSHTML
	EncoderSHTML
	EncoderSz
	EncoderText
	EncoderZmk

	EncoderPlain
	EncoderJson
	EncoderData
)

// String representation of an encoder key.
func (e EncodingEnum) String() string {
	if f, ok := mapEnumEncoding[e]; ok {
		return f
	}
	return fmt.Sprintf("*Unknown*(%d)", e)
}

// Supported part values.
const (
	PartMeta    = "meta"
	PartContent = "content"
	PartZettel  = "zettel"
)

// Command to be executed atthe Zettelstore
type Command string

// Supported command values
// Supported command values.
const (
	CommandAuthenticated = Command("authenticated")
	CommandRefresh       = Command("refresh")
)

// Supported search operator representations
// Supported search operator representations.
const (
	BackwardDirective = "BACKWARD" // Backward-only context
	ContextDirective  = "CONTEXT"  // Context directive
	CostDirective     = "COST"     // Maximum cost of a context operation
	ForwardDirective  = "FORWARD"  // Forward-only context
	FullDirective     = "FULL"     // Include tags in context
	IdentDirective    = "IDENT"    // Use only specified zettel
	ItemsDirective    = "ITEMS"    // Select list elements in a zettel
	MaxDirective      = "MAX"      // Maximum number of context results
	MinDirective      = "MIN"      // Minimum number of context results
	LimitDirective    = "LIMIT"    // Maximum number of zettel
	OffsetDirective   = "OFFSET"   // Offset to start returned zettel list
	OrDirective       = "OR"       // Combine several search expression with an "or"
	OrderDirective    = "ORDER"    // Specify metadata keys for the order of returned list
	PhraseDirective   = "PHRASE"   // Only unlinked zettel with given phrase
	PickDirective     = "PICK"     // Pick some random zettel
	RandomDirective   = "RANDOM"   // Order zettel list randomly
	ReverseDirective  = "REVERSE"  // Reverse the order of a zettel list
	UnlinkedDirective = "UNLINKED" // Search for zettel that contain a phase(s) but do not link

	ActionSeparator        = "|"
	ExistOperator          = "?"
	ExistNotOperator       = "!?"
	SearchOperatorNot      = "!"
	SearchOperatorHas      = ":"
	SearchOperatorHasNot   = "!:"
	SearchOperatorPrefix   = ">"
	SearchOperatorNoPrefix = "!>"
	SearchOperatorSuffix   = "<"
	SearchOperatorNoSuffix = "!<"
	SearchOperatorMatch    = "~"
	SearchOperatorNoMatch  = "!~"
	ActionSeparator = "|" // Separates action list of previous elements of query expression

	KeysAction     = "KEYS"     // Provide metadata key used
	MinAction      = "MIN"      // Return only those values with a minimum amount of zettel
	MaxAction      = "MAX"      // Return only those values with a maximum amount of zettel
	NumberedAction = "NUMBERED" // Return a numbered list
	RedirectAction = "REDIRECT" // Return the first zettel in list
	ReIndexAction  = "REINDEX"  // Ensure that zettel is/are indexed.

	ExistOperator    = "?"  // Does zettel have metadata with given key?
	ExistNotOperator = "!?" // True id zettel does not have metadata with given key.

	SearchOperatorNot        = "!"
	SearchOperatorEqual      = "="  // True if values are equal
	SearchOperatorNotEqual   = "!=" // False if values are equal
	SearchOperatorHas        = ":"  // True if values are equal/included
	SearchOperatorHasNot     = "!:" // False if values are equal/included
	SearchOperatorPrefix     = "["  // True if value is prefix of the other
	SearchOperatorNoPrefix   = "![" // False if value is prefix of the other
	SearchOperatorSuffix     = "]"  // True if value is suffix of other
	SearchOperatorNoSuffix   = "!]" // False if value is suffix of other
	SearchOperatorMatch      = "~"  // True if value is included in other
	SearchOperatorNoMatch    = "!~" // False if value is included in other
	SearchOperatorLess       = "<"  // True if value is smaller than other
	SearchOperatorNotLess    = "!<" // False if value is smaller than other
	SearchOperatorGreater    = ">"  // True if value is greater than other
	SearchOperatorNotGreater = "!>" // False if value is greater than other
)

// QueryPrefix is the prefix that denotes a query expression within a reference.
const QueryPrefix = "query:"

Changes to api/urlbuilder.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15


16
17
18
19
20
21

22

23
24
25
26
27
28
29
30
31

32
33


34
35

36
37
38
39


40
41
42
43
44
45
46
47
48

49
50
51
52


53
54
55
56
57
58
59
60




61
62
63
64

65
66
67
68
69

70
71
72
73

74
75

76
77
78
79
80
81
82
83
84
85

86
87
88

89
90
91
92



93
94
95



96
97
98
99
100
101
102

103
104
105
106
107

108
109
110

111
112
113
114
115
116

117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16


17
18
19
20


21
22
23

24





25
26
27
28
29


30
31


32




33
34


35






36
37
38


39
40








41
42
43
44
45
46
47

48





49
50
51
52

53
54

55






56
57
58

59
60


61
62
63
64

65
66
67
68


69
70
71
72
73
74
75
76


77

78
79
80

81
82


83
84
85
86
87
88

89









































90








+
+
+





-
-
+
+


-
-


+
-
+
-
-
-
-
-




+
-
-
+
+
-
-
+
-
-
-
-
+
+
-
-

-
-
-
-
-
-
+


-
-
+
+
-
-
-
-
-
-
-
-
+
+
+
+



-
+
-
-
-
-
-
+



-
+

-
+
-
-
-
-
-
-



-
+

-
-
+



-
+
+
+

-
-
+
+
+





-
-
+
-



-
+

-
-
+





-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-

//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package api

import (
	"net/url"
	"strings"
	"t73f.de/r/webs/urlbuilder"
	"t73f.de/r/zsc/domain/id"
)

type urlQuery struct{ key, val string }

// URLBuilder should be used to create zettelstore URLs.
type URLBuilder struct {
	base   urlbuilder.URLBuilder
	prefix   string
	prefix string
	key      byte
	rawLocal string
	path     []string
	query    []urlQuery
	fragment string
}

// NewURLBuilder creates a new URL builder with the given prefix and key.
func NewURLBuilder(prefix string, key byte) *URLBuilder {
	for len(prefix) > 0 && prefix[len(prefix)-1] == '/' {
	return &URLBuilder{prefix: prefix, key: key}
}
		prefix = prefix[0 : len(prefix)-1]
	}

// Clone an URLBuilder
	result := URLBuilder{prefix: prefix}
func (ub *URLBuilder) Clone() *URLBuilder {
	cpy := new(URLBuilder)
	cpy.key = ub.key
	if len(ub.path) > 0 {
	if key != '/' {
		result.base.AddPath(string([]byte{key}))
		cpy.path = make([]string, 0, len(ub.path))
		cpy.path = append(cpy.path, ub.path...)
	}
	if len(ub.query) > 0 {
		cpy.query = make([]urlQuery, 0, len(ub.query))
		cpy.query = append(cpy.query, ub.query...)
	}
	cpy.fragment = ub.fragment
	return cpy
	return &result
}

// SetRawLocal sets everything that follows the prefix / key.
func (ub *URLBuilder) SetRawLocal(rawLocal string) *URLBuilder {
// Clone an URLBuilder.
func (ub *URLBuilder) Clone() *URLBuilder {
	for len(rawLocal) > 0 && rawLocal[0] == '/' {
		rawLocal = rawLocal[1:]
	}
	ub.rawLocal = rawLocal
	ub.path = nil
	ub.query = nil
	ub.fragment = ""
	return ub
	cpy := new(URLBuilder)
	ub.base.Copy(&cpy.base)
	cpy.prefix = ub.prefix
	return cpy
}

// SetZid sets the zettel identifier.
func (ub *URLBuilder) SetZid(zid ZettelID) *URLBuilder {
func (ub *URLBuilder) SetZid(zid id.Zid) *URLBuilder {
	if len(ub.path) > 0 {
		panic("Cannot add Zid")
	}
	ub.rawLocal = ""
	ub.path = append(ub.path, string(zid))
	ub.base.AddPath(zid.String())
	return ub
}

// AppendPath adds a new path element
// AppendPath adds a new path element.
func (ub *URLBuilder) AppendPath(p string) *URLBuilder {
	ub.rawLocal = ""
	ub.base.AddPath(p)
	for len(p) > 0 && p[0] == '/' {
		p = p[1:]
	}
	if p != "" {
		ub.path = append(ub.path, p)
	}
	return ub
}

// AppendKVQuery adds a new key/value query parameter
// AppendKVQuery adds a new key/value query parameter.
func (ub *URLBuilder) AppendKVQuery(key, value string) *URLBuilder {
	ub.rawLocal = ""
	ub.query = append(ub.query, urlQuery{key, value})
	ub.base.AddQuery(key, value)
	return ub
}

// AppendQuery adds a new query
// AppendQuery adds a new query.
//
// Basically the same as [URLBuilder.AppendKVQuery]([api.QueryKeyQuery], value)
func (ub *URLBuilder) AppendQuery(value string) *URLBuilder {
	ub.rawLocal = ""
	ub.query = append(ub.query, urlQuery{QueryKeyQuery, value})
	if value != "" {
		ub.base.AddQuery(QueryKeyQuery, value)
	}
	return ub
}

// ClearQuery removes all query parameters.
func (ub *URLBuilder) ClearQuery() *URLBuilder {
	ub.rawLocal = ""
	ub.query = nil
	ub.base.RemoveQueries()
	ub.fragment = ""
	return ub
}

// SetFragment stores the fragment
// SetFragment sets the fragment.
func (ub *URLBuilder) SetFragment(s string) *URLBuilder {
	ub.rawLocal = ""
	ub.fragment = s
	ub.base.SetFragment(s)
	return ub
}

// String produces a string value.
func (ub *URLBuilder) String() string {
	return ub.asString("&")
	return ub.prefix + ub.base.String()
}

// AttrString returns the string value of the URL suitable to be placed in a HTML attribute.
func (ub *URLBuilder) AttrString() string {
	return ub.asString("&amp;")
}

func (ub *URLBuilder) asString(qsep string) string {
	var sb strings.Builder

	sb.WriteString(ub.prefix)
	if ub.key != '/' {
		sb.WriteByte(ub.key)
	}
	if ub.rawLocal != "" {
		sb.WriteString(ub.rawLocal)
		return sb.String()
	}
	for i, p := range ub.path {
		if i > 0 || ub.key != '/' {
			sb.WriteByte('/')
		}
		sb.WriteString(url.PathEscape(p))
	}
	if len(ub.fragment) > 0 {
		sb.WriteByte('#')
		sb.WriteString(ub.fragment)
	}
	for i, q := range ub.query {
		if i == 0 {
			sb.WriteByte('?')
		} else {
			sb.WriteString(qsep)
		}
		sb.WriteString(q.key)
		if val := q.val; val != "" {
			sb.WriteByte('=')
			sb.WriteString(url.QueryEscape(val))
		}
	}
	return sb.String()
}

Deleted attrs/attrs.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125





























































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//-----------------------------------------------------------------------------

// Package attrs stores attributes of zettel parts.
package attrs

import (
	"strings"

	"zettelstore.de/c/maps"
)

// Attributes store additional information about some node types.
type Attributes map[string]string

// IsEmpty returns true if there are no attributes.
func (a Attributes) IsEmpty() bool { return len(a) == 0 }

// DefaultAttribute is the value of the key of the default attribute
const DefaultAttribute = "-"

// HasDefault returns true, if the default attribute "-" has been set.
func (a Attributes) HasDefault() bool {
	if a != nil {
		_, ok := a[DefaultAttribute]
		return ok
	}
	return false
}

// RemoveDefault removes the default attribute
func (a Attributes) RemoveDefault() Attributes {
	if a != nil {
		a.Remove(DefaultAttribute)
	}
	return a
}

// Keys returns the sorted list of keys.
func (a Attributes) Keys() []string { return maps.Keys(a) }

// Get returns the attribute value of the given key and a succes value.
func (a Attributes) Get(key string) (string, bool) {
	if a != nil {
		value, ok := a[key]
		return value, ok
	}
	return "", false
}

// Clone returns a duplicate of the attribute.
func (a Attributes) Clone() Attributes {
	if a == nil {
		return nil
	}
	attrs := make(map[string]string, len(a))
	for k, v := range a {
		attrs[k] = v
	}
	return attrs
}

// Set changes the attribute that a given key has now a given value.
func (a Attributes) Set(key, value string) Attributes {
	if a == nil {
		return map[string]string{key: value}
	}
	a[key] = value
	return a
}

// Remove the key from the attributes.
func (a Attributes) Remove(key string) Attributes {
	if a != nil {
		delete(a, key)
	}
	return a
}

// AddClass adds a value to the class attribute.
func (a Attributes) AddClass(class string) Attributes {
	if a == nil {
		return map[string]string{"class": class}
	}
	classes := a.GetClasses()
	for _, cls := range classes {
		if cls == class {
			return a
		}
	}
	classes = append(classes, class)
	a["class"] = strings.Join(classes, " ")
	return a
}

// GetClasses returns the class values as a string slice
func (a Attributes) GetClasses() []string {
	if a == nil {
		return nil
	}
	classes, ok := a["class"]
	if !ok {
		return nil
	}
	return strings.Fields(classes)
}

// HasClass returns true, if attributes contains the given class.
func (a Attributes) HasClass(s string) bool {
	if a == nil {
		return false
	}
	classes, found := a["class"]
	if !found {
		return false
	}
	return strings.Contains(" "+classes+" ", " "+s+" ")
}

Deleted attrs/attrs_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75











































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//-----------------------------------------------------------------------------

package attrs_test

import (
	"testing"

	"zettelstore.de/c/attrs"
)

func TestHasDefault(t *testing.T) {
	t.Parallel()
	attr := attrs.Attributes{}
	if attr.HasDefault() {
		t.Error("Should not have default attr")
	}
	attr = attrs.Attributes(map[string]string{"-": "value"})
	if !attr.HasDefault() {
		t.Error("Should have default attr")
	}
}

func TestAttrClone(t *testing.T) {
	t.Parallel()
	orig := attrs.Attributes{}
	clone := orig.Clone()
	if !clone.IsEmpty() {
		t.Error("Attrs must be empty")
	}

	orig = attrs.Attributes(map[string]string{"": "0", "-": "1", "a": "b"})
	clone = orig.Clone()
	if clone[""] != "0" || clone["-"] != "1" || clone["a"] != "b" || len(clone) != len(orig) {
		t.Error("Wrong cloned map")
	}
	clone["a"] = "c"
	if orig["a"] != "b" {
		t.Error("Aliased map")
	}
}

func TestHasClass(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		classes string
		class   string
		exp     bool
	}{
		{"", "", true},
		{"x", "", false},
		{"x", "x", true},
		{"x", "y", false},
		{"abc def ghi", "abc", true},
		{"abc def ghi", "def", true},
		{"abc def ghi", "ghi", true},
		{"ab de gi", "b", false},
		{"ab de gi", "d", false},
	}
	for _, tc := range testcases {
		var a attrs.Attributes
		a = a.Set("class", tc.classes)
		got := a.HasClass(tc.class)
		if tc.exp != got {
			t.Errorf("%q.HasClass(%q)=%v, but got %v", tc.classes, tc.class, tc.exp, got)
		}
	}
}

Changes to client/client.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28
29





30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72













73




74
75
76
77
78
79

80
81
82
83
84
85
86
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

18
19

20
21
22
23
24
25
26
27
28



29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

50
51
52
53
54
55
56
57
58



59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105








+
+
+






-


-
+








-
-
-
+
+
+
+
+
















-
+








-
-
-















+
+
+
+
+
+
+
+
+
+
+
+
+

+
+
+
+






+







//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package client provides a client for accessing the Zettelstore via its API.
package client

import (
	"bufio"
	"bytes"
	"context"
	"encoding/json"
	"fmt"
	"io"
	"net"
	"net/http"
	"net/url"
	"strconv"
	"strings"
	"time"

	"codeberg.org/t73fde/sxpf"
	"codeberg.org/t73fde/sxpf/reader"
	"zettelstore.de/c/api"
	"t73f.de/r/sx"
	"t73f.de/r/sx/sxreader"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsc/sexp"
)

// Client contains all data to execute requests.
type Client struct {
	base      string
	username  string
	password  string
	token     string
	tokenType string
	expires   time.Time
	client    http.Client
}

// Base returns the base part of the URLs that are used to communicate with a Zettelstore.
func (c *Client) Base() string { return c.base }

// NewClient create a new client.
// NewClient creates a new client with a given base URL to a Zettelstore.
func NewClient(u *url.URL) *Client {
	myURL := *u
	myURL.User = nil
	myURL.ForceQuery = false
	myURL.RawQuery = ""
	myURL.Fragment = ""
	myURL.RawFragment = ""
	base := myURL.String()
	if !strings.HasSuffix(base, "/") {
		base += "/"
	}
	c := Client{
		base: base,
		client: http.Client{
			Timeout: 10 * time.Second,
			Transport: &http.Transport{
				DialContext: (&net.Dialer{
					Timeout: 5 * time.Second, // TCP connect timeout
				}).DialContext,
				TLSHandshakeTimeout: 5 * time.Second,
			},
		},
	}
	return &c
}

// AllowRedirect will modify the client to not follow redirect status code when
// using the Zettelstore. The original behaviour can be restored by setting
// allow to false.
func (c *Client) AllowRedirect(allow bool) {
	if allow {
		c.client.CheckRedirect = func(*http.Request, []*http.Request) error {
			return http.ErrUseLastResponse
		}
	} else {
		c.client.CheckRedirect = nil
	}
}

// Error encapsulates the possible client call errors.
//
//   - StatusCode is the HTTP status code, e.g. 200
//   - Message is the HTTP message, e.g. "OK"
//   - Body is the HTTP body returned by a request.
type Error struct {
	StatusCode int
	Message    string
	Body       []byte
}

// Error returns the error as a string.
func (err *Error) Error() string {
	var body string
	if err.Body == nil {
		body = "nil"
	} else if bl := len(err.Body); bl == 0 {
		body = "empty"
	} else {
107
108
109
110
111
112
113






114

115
116
117
118
119
120
121
122
123
124
125
126
127
128

129
130
131
132
133
134
135
136





137
138
139
140
141
142
143
144
145
146
147
148
149
150
151


152
153
154
155
156
157
158
159
160
161
162
163
164
165

166
167
168
169
170
171






172
173
174




175
176
177



178
179
180
181
182
183
184
185
186
187
188
189
190
191


192
193
194

195
196
197
198
199
200
201
202


203
204

205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231

232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258

259
260
261
262
263
264

265
266

267
268
269
270



271
272

273
274
275


276
277
278
279

280
281
282
283
284
285
286
287
288
289
290
291

292
293
294
295
296
297
298
299
300
301
302
303

304
305
306
307
308
309
310
311

312
313
314
315
316
317
318
319
320
321


322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346

347
348
349
350
351
352
353
354
355



356
357
358

359
360
361
362



363
364
365
366

367
368
369
370
371

372
373

374
375

376
377
378
379
380
381
382

383
384

385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410


411
412
413
414
415
416
417
418
419
420
421
422

423
424
425
426
427
428
429
430
431
432
433


434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454

455
456
457

458
459
460
461


462
463
464
465
466
467
468

469
470
471
472
473
474
475
476
477
478
479
480
481
482
483

484
485

486

487
488

489


490
491


492
493
494
495

496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529








530
531
532
533

534
535
536
537
538
539
540
541


542
543

544
545
546
547


548
549
550
551

552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578



579
580
581
582

583
584
585
586
587
588
589




590
591
592


593
594
595
596

597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
126
127
128
129
130
131
132
133
134
135
136
137
138

139
140
141
142
143
144
145
146
147
148
149
150
151
152

153
154
155
156
157
158
159
160

161
162
163
164
165
166
167
168
169
170
171
172
173



174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192

193
194
195
196



197
198
199
200
201
202
203
204
205
206
207
208
209



210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230

231
232
233
234
235
236
237
238
239
240
241
242

243
244
245
246
247
248
249
250




















251



























252






253


254




255
256
257
258

259
260


261
262




263












264












265


266





267





268




269
270









271















272









273
274
275



276




277
278
279




280





281
282

283
284

285







286


287


























288
289
290









291

292
293










294
295





















296



297




298
299







300












301


302


303

304
305

306
307
308
309


310
311




312


































313
314
315
316
317
318
319
320
321
322
323

324
325
326
327
328
329
330


331
332
333

334
335
336


337
338
339
340
341

342

















343
344
345
346
347
348
349



350
351
352
353
354
355

356
357
358
359
360
361
362
363
364
365
366
367
368


369
370
371
372
373

374
375
376
377
378
379



































































+
+
+
+
+
+
-
+













-
+







-
+
+
+
+
+








-
-
-




+
+













-
+



-
-
-
+
+
+
+
+
+



+
+
+
+
-
-
-
+
+
+














+
+


-
+








+
+

-
+







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-
+
-
-
+
-
-
-
-
+
+
+

-
+

-
-
+
+
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-

-
-
-
-
-
+
-
-
-
-
-

-
-
-
-
+
+
-
-
-
-
-
-
-
-
-

-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
-
+
+
+
-
-
-
+
-
-
-
-
+
+
+
-
-
-
-
+
-
-
-
-
-
+

-
+

-
+
-
-
-
-
-
-
-
+
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+

-
-
-
-
-
-
-
-
-

-
+

-
-
-
-
-
-
-
-
-
-
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
+
-
-
-
-
+
+
-
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-

-
-
+
-
-
+
-
+

-
+

+
+
-
-
+
+
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+



-
+






-
-
+
+

-
+


-
-
+
+



-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







-
-
-
+
+
+



-
+







+
+
+
+

-
-
+
+



-
+





-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
	return &Error{
		StatusCode: resp.StatusCode,
		Message:    resp.Status[4:],
		Body:       body,
	}
}

// NewURLBuilder creates a new URL builder for the client with the given key.
//
// key is one of the defined lower case letters to specify an endpoint.
// See [Endpoints used by the API] for details.
//
// [Endpoints used by the API]: https://zettelstore.de/manual/h/00001012920000
func (c *Client) newURLBuilder(key byte) *api.URLBuilder {
func (c *Client) NewURLBuilder(key byte) *api.URLBuilder {
	return api.NewURLBuilder(c.base, key)
}
func (*Client) newRequest(ctx context.Context, method string, ub *api.URLBuilder, body io.Reader) (*http.Request, error) {
	return http.NewRequestWithContext(ctx, method, ub.String(), body)
}

func (c *Client) executeRequest(req *http.Request) (*http.Response, error) {
	if c.token != "" {
		req.Header.Add("Authorization", c.tokenType+" "+c.token)
	}
	resp, err := c.client.Do(req)
	if err != nil {
		if resp != nil && resp.Body != nil {
			resp.Body.Close()
			_ = resp.Body.Close()
		}
		return nil, err
	}
	return resp, err
}

func (c *Client) buildAndExecuteRequest(
	ctx context.Context, method string, ub *api.URLBuilder, body io.Reader, h http.Header) (*http.Response, error) {
	ctx context.Context,
	method string,
	ub *api.URLBuilder,
	body io.Reader,
) (*http.Response, error) {
	req, err := c.newRequest(ctx, method, ub, body)
	if err != nil {
		return nil, err
	}
	err = c.updateToken(ctx)
	if err != nil {
		return nil, err
	}
	for key, val := range h {
		req.Header[key] = append(req.Header[key], val...)
	}
	return c.executeRequest(req)
}

// SetAuth sets authentication data.
//
// username and password are the same values that are used to authenticate via the Web-UI.
func (c *Client) SetAuth(username, password string) {
	c.username = username
	c.password = password
	c.token = ""
	c.tokenType = ""
	c.expires = time.Time{}
}

func (c *Client) executeAuthRequest(req *http.Request) error {
	resp, err := c.executeRequest(req)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusOK {
		return statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var tinfo api.AuthJSON
	err = dec.Decode(&tinfo)
	rd := sxreader.MakeReader(resp.Body)
	obj, err := rd.Read()
	if err != nil {
		return err
	}
	vals, err := sexp.ParseList(obj, "ssi")
	if err != nil {
		return err
	}
	token := vals[1].(sx.String).GetValue()
	if len(token) < 4 {
		return fmt.Errorf("no valid token found: %q", token)
	}
	c.token = tinfo.Token
	c.tokenType = tinfo.Type
	c.expires = time.Now().Add(time.Duration(tinfo.Expires*10/9) * time.Second)
	c.token = token
	c.tokenType = vals[0].(sx.String).GetValue()
	c.expires = time.Now().Add(time.Duration(vals[2].(sx.Int64)*9/10) * time.Second)
	return nil
}

func (c *Client) updateToken(ctx context.Context) error {
	if c.username == "" {
		return nil
	}
	if time.Now().After(c.expires) {
		return c.Authenticate(ctx)
	}
	return c.RefreshToken(ctx)
}

// Authenticate sets a new token by sending user name and password.
//
// [Client.SetAuth] should be called before.
func (c *Client) Authenticate(ctx context.Context) error {
	authData := url.Values{"username": {c.username}, "password": {c.password}}
	req, err := c.newRequest(ctx, http.MethodPost, c.newURLBuilder('a'), strings.NewReader(authData.Encode()))
	req, err := c.newRequest(ctx, http.MethodPost, c.NewURLBuilder('a'), strings.NewReader(authData.Encode()))
	if err != nil {
		return err
	}
	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
	return c.executeAuthRequest(req)
}

// RefreshToken updates the access token
//
// [Client.SetAuth] should be called before.
func (c *Client) RefreshToken(ctx context.Context) error {
	req, err := c.newRequest(ctx, http.MethodPut, c.newURLBuilder('a'), nil)
	req, err := c.newRequest(ctx, http.MethodPut, c.NewURLBuilder('a'), nil)
	if err != nil {
		return err
	}
	return c.executeAuthRequest(req)
}

// CreateZettel creates a new zettel and returns its URL.
func (c *Client) CreateZettel(ctx context.Context, data []byte) (api.ZettelID, error) {
	ub := c.newURLBuilder('z')
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, bytes.NewBuffer(data), nil)
	if err != nil {
		return api.InvalidZID, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusCreated {
		return api.InvalidZID, statusToError(resp)
	}
	b, err := io.ReadAll(resp.Body)
	if err != nil {
		return api.InvalidZID, err
	}
	if zid := api.ZettelID(b); zid.IsValid() {
		return zid, nil
	}
	return api.InvalidZID, err
}

//
// CreateZettelJSON creates a new zettel and returns its URL.
func (c *Client) CreateZettelJSON(ctx context.Context, data *api.ZettelDataJSON) (api.ZettelID, error) {
	var buf bytes.Buffer
	if err := encodeZettelData(&buf, data); err != nil {
		return api.InvalidZID, err
	}
	ub := c.newURLBuilder('z').AppendKVQuery(api.QueryKeyEncoding, api.EncodingJson)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, &buf, nil)
	if err != nil {
		return api.InvalidZID, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusCreated {
		return api.InvalidZID, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var newZid api.ZidJSON
	err = dec.Decode(&newZid)
	if err != nil {
		return api.InvalidZID, err
	}
	if zid := newZid.ID; zid.IsValid() {
		return zid, nil
	}
	return api.InvalidZID, err
}

// data contains the zettel metadata and content, as it is stored in a file in a zettel box,
func encodeZettelData(buf *bytes.Buffer, data *api.ZettelDataJSON) error {
	enc := json.NewEncoder(buf)
	enc.SetEscapeHTML(false)
	return enc.Encode(&data)
}

// or as returned by [Client.GetZettel].
var bsLF = []byte{'\n'}

// Metadata is separated from zettel content by an empty line.
// ListZettel returns a list of all Zettel.
func (c *Client) ListZettel(ctx context.Context, query string) ([][]byte, error) {
	ub := c.newURLBuilder('z').AppendQuery(query)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
func (c *Client) CreateZettel(ctx context.Context, data []byte) (id.Zid, error) {
	ub := c.NewURLBuilder('z')
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, bytes.NewBuffer(data))
	if err != nil {
		return nil, err
		return id.Invalid, err
	}
	defer resp.Body.Close()
	switch resp.StatusCode {
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusCreated {
	case http.StatusOK:
	case http.StatusNoContent:
	default:
		return nil, statusToError(resp)
		return id.Invalid, statusToError(resp)
	}
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}
	lines := bytes.Split(data, bsLF)
	if len(lines[len(lines)-1]) == 0 {
		lines = lines[:len(lines)-1]
	}
	return lines, nil
}

	}
// ListZettelJSON returns a list of zettel.
func (c *Client) ListZettelJSON(ctx context.Context, query string) (string, string, []api.ZidMetaJSON, error) {
	ub := c.newURLBuilder('z').AppendKVQuery(api.QueryKeyEncoding, api.EncodingJson).AppendQuery(query)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return "", "", nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return "", "", nil, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	b, err := io.ReadAll(resp.Body)
	var zl api.ZettelListJSON
	err = dec.Decode(&zl)
	if err != nil {
		return "", "", nil, err
	}
	return zl.Query, zl.Human, zl.List, nil
}

		return id.Invalid, err
// GetZettel returns a zettel as a string.
func (c *Client) GetZettel(ctx context.Context, zid api.ZettelID, part string) ([]byte, error) {
	ub := c.newURLBuilder('z').SetZid(zid)
	if part != "" && part != api.PartContent {
		ub.AppendKVQuery(api.QueryKeyPart, part)
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	return id.Parse(string(b))
}
	defer resp.Body.Close()
	switch resp.StatusCode {
	case http.StatusOK:
	case http.StatusNoContent:
	default:
		return nil, statusToError(resp)
	}
	return io.ReadAll(resp.Body)
}

// GetZettelJSON returns a zettel as a JSON struct.
func (c *Client) GetZettelJSON(ctx context.Context, zid api.ZettelID) (*api.ZettelDataJSON, error) {
	ub := c.newURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, api.EncodingJson)
	ub.AppendKVQuery(api.QueryKeyPart, api.PartZettel)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var out api.ZettelDataJSON
// CreateZettelData creates a new zettel and returns its URL.
	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}
	return &out, nil
}

// GetParsedZettel return a parsed zettel in a defined encoding.
func (c *Client) GetParsedZettel(ctx context.Context, zid api.ZettelID, enc api.EncodingEnum) ([]byte, error) {
//
// data contains the zettel date, encoded as explicit struct.
func (c *Client) CreateZettelData(ctx context.Context, data api.ZettelData) (id.Zid, error) {
	return c.getZettelString(ctx, zid, enc, true)
}

	var buf bytes.Buffer
// GetEvaluatedZettel return an evaluated zettel in a defined encoding.
func (c *Client) GetEvaluatedZettel(ctx context.Context, zid api.ZettelID, enc api.EncodingEnum) ([]byte, error) {
	return c.getZettelString(ctx, zid, enc, false)
}
	if _, err := sx.Print(&buf, sexp.EncodeZettel(data)); err != nil {
		return id.Invalid, err
	}

func (c *Client) getZettelString(ctx context.Context, zid api.ZettelID, enc api.EncodingEnum, parseOnly bool) ([]byte, error) {
	ub := c.newURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, enc.String())
	ub := c.NewURLBuilder('z').AppendKVQuery(api.QueryKeyEncoding, api.EncodingData)
	ub.AppendKVQuery(api.QueryKeyPart, api.PartContent)
	if parseOnly {
		ub.AppendKVQuery(api.QueryKeyParseOnly, "")
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, &buf)
	if err != nil {
		return nil, err
		return id.Invalid, err
	}
	defer resp.Body.Close()
	defer func() { _ = resp.Body.Close() }()
	switch resp.StatusCode {
	case http.StatusOK:
	case http.StatusNoContent:
	default:
		return nil, statusToError(resp)
	}
	return io.ReadAll(resp.Body)
	rdr := sxreader.MakeReader(resp.Body)
}

	obj, err := rdr.Read()
// GetParsedSexpr returns an parsed zettel as a Sexpr-decoded data structure.
func (c *Client) GetParsedSexpr(ctx context.Context, zid api.ZettelID, part string, sf sxpf.SymbolFactory) (sxpf.Object, error) {
	return c.getSexpr(ctx, zid, part, true, sf)
}

// GetEvaluatedSexpr returns an evaluated zettel as a Sexpr-decoded data structure.
func (c *Client) GetEvaluatedSexpr(ctx context.Context, zid api.ZettelID, part string, sf sxpf.SymbolFactory) (sxpf.Object, error) {
	return c.getSexpr(ctx, zid, part, false, sf)
}

func (c *Client) getSexpr(ctx context.Context, zid api.ZettelID, part string, parseOnly bool, sf sxpf.SymbolFactory) (sxpf.Object, error) {
	ub := c.newURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, api.EncodingSexpr)
	if part != "" {
		ub.AppendKVQuery(api.QueryKeyPart, part)
	}
	if parseOnly {
		ub.AppendKVQuery(api.QueryKeyParseOnly, "")
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	if resp.StatusCode != http.StatusCreated {
		return id.Invalid, statusToError(resp)
	}
	return reader.MakeReader(bufio.NewReaderSize(resp.Body, 8), reader.WithSymbolFactory(sf)).Read()
}

// GetMeta returns the metadata of a zettel.
func (c *Client) GetMeta(ctx context.Context, zid api.ZettelID) (api.ZettelMeta, error) {
	ub := c.newURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, api.EncodingJson)
	ub.AppendKVQuery(api.QueryKeyPart, api.PartMeta)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
		return id.Invalid, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var out api.MetaJSON
	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}
	return makeZettelID(obj)
}
	return out.Meta, nil
}

// GetZettelOrder returns metadata of the given zettel and, more important,
// metadata of zettel that are referenced in a list within the first zettel.
func (c *Client) GetZettelOrder(ctx context.Context, zid api.ZettelID) (*api.ZidMetaRelatedList, error) {
	ub := c.newURLBuilder('o').SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var out api.ZidMetaRelatedList
	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}

	return &out, nil
}

func makeZettelID(obj sx.Object) (id.Zid, error) {
// ContextDirection specifies how the context should be calculated.
type ContextDirection uint8

// Allowed values for ContextDirection
	val, isInt64 := obj.(sx.Int64)
	if !isInt64 || val <= 0 {
const (
	_ ContextDirection = iota
	DirBoth
	DirBackward
	DirForward
)

		return id.Invalid, fmt.Errorf("invalid zettel ID: %v", val)
// GetZettelContext returns metadata of the given zettel and, more important,
// metadata of zettel that for the context of the first zettel.
func (c *Client) GetZettelContext(
	ctx context.Context, zid api.ZettelID, dir ContextDirection, cost, limit int) (
	*api.ZidMetaRelatedList, error,
) {
	ub := c.newURLBuilder('x').SetZid(zid)
	switch dir {
	case DirBackward:
		ub.AppendKVQuery(api.QueryKeyDir, api.DirBackward)
	case DirForward:
		ub.AppendKVQuery(api.QueryKeyDir, api.DirForward)
	}
	if cost > 0 {
		ub.AppendKVQuery(api.QueryKeyCost, strconv.Itoa(cost))
	sVal := strconv.FormatInt(int64(val), 10)
	}
	if limit > 0 {
	if len(sVal) < 14 {
		ub.AppendKVQuery(api.QueryKeyLimit, strconv.Itoa(limit))
		sVal = "00000000000000"[0:14-len(sVal)] + sVal
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	zid, err := id.Parse(sVal)
	if err != nil {
		return id.Invalid, fmt.Errorf("invalid zettel ID %v: %w", val, err)
	}
		return nil, err
	}
	return zid, nil
}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	}

	dec := json.NewDecoder(resp.Body)
	var out api.ZidMetaRelatedList
	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}
	return &out, nil
}

// GetUnlinkedReferences returns connections to other zettel, embedded material, externals URLs.
func (c *Client) GetUnlinkedReferences(
	ctx context.Context, zid api.ZettelID, query url.Values) (*api.ZidMetaRelatedList, error) {
	ub := c.newQueryURLBuilder('u', query).SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil, nil)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var out api.ZidMetaRelatedList
	err = dec.Decode(&out)
	if err != nil {
		return nil, err
	}
	return &out, nil
}

// UpdateZettel updates an existing zettel.
func (c *Client) UpdateZettel(ctx context.Context, zid api.ZettelID, data []byte) error {
	ub := c.newURLBuilder('z').SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPut, ub, bytes.NewBuffer(data), nil)
// UpdateZettel updates an existing zettel, specified by its zettel identifier.
//
// data contains the zettel metadata and content, as it is stored in a file in a zettel box,
// or as returned by [Client.GetZettel].
// Metadata is separated from zettel content by an empty line.
func (c *Client) UpdateZettel(ctx context.Context, zid id.Zid, data []byte) error {
	ub := c.NewURLBuilder('z').SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPut, ub, bytes.NewBuffer(data))
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusNoContent {
		return statusToError(resp)
	}
	return nil
}

// UpdateZettelJSON updates an existing zettel.
func (c *Client) UpdateZettelJSON(ctx context.Context, zid api.ZettelID, data *api.ZettelDataJSON) error {
// UpdateZettelData updates an existing zettel, specified by its zettel identifier.
func (c *Client) UpdateZettelData(ctx context.Context, zid id.Zid, data api.ZettelData) error {
	var buf bytes.Buffer
	if err := encodeZettelData(&buf, data); err != nil {
	if _, err := sx.Print(&buf, sexp.EncodeZettel(data)); err != nil {
		return err
	}
	ub := c.newURLBuilder('z').SetZid(zid).AppendKVQuery(api.QueryKeyEncoding, api.EncodingJson)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPut, ub, &buf, nil)
	ub := c.NewURLBuilder('z').SetZid(zid).AppendKVQuery(api.QueryKeyEncoding, api.EncodingData)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPut, ub, &buf)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusNoContent {
		return statusToError(resp)
	}
	return nil
}

// RenameZettel renames a zettel.
func (c *Client) RenameZettel(ctx context.Context, oldZid, newZid api.ZettelID) error {
	ub := c.newURLBuilder('z').SetZid(oldZid)
	h := http.Header{
		api.HeaderDestination: {c.newURLBuilder('z').SetZid(newZid).String()},
	}
	resp, err := c.buildAndExecuteRequest(ctx, api.MethodMove, ub, nil, h)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusNoContent {
		return statusToError(resp)
	}
	return nil
}

// DeleteZettel deletes a zettel with the given identifier.
func (c *Client) DeleteZettel(ctx context.Context, zid api.ZettelID) error {
	ub := c.newURLBuilder('z').SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodDelete, ub, nil, nil)
func (c *Client) DeleteZettel(ctx context.Context, zid id.Zid) error {
	ub := c.NewURLBuilder('z').SetZid(zid)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodDelete, ub, nil)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusNoContent {
		return statusToError(resp)
	}
	return nil
}

// ExecuteCommand will execute a given command at the Zettelstore.
//
// See [API commands] for a list of valid commands.
//
// [API commands]: https://zettelstore.de/manual/h/00001012080100
func (c *Client) ExecuteCommand(ctx context.Context, command api.Command) error {
	ub := c.newURLBuilder('x').AppendKVQuery(api.QueryKeyCommand, string(command))
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, nil, nil)
	ub := c.NewURLBuilder('x').AppendKVQuery(api.QueryKeyCommand, string(command))
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodPost, ub, nil)
	if err != nil {
		return err
	}
	defer resp.Body.Close()
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusNoContent {
		return statusToError(resp)
	}
	return nil
}

func (c *Client) newQueryURLBuilder(key byte, query url.Values) *api.URLBuilder {
	ub := c.newURLBuilder(key)
	for key, values := range query {
		if key == api.QueryKeyEncoding {
			continue
		}
		for _, val := range values {
			ub.AppendKVQuery(key, val)
		}
	}
	return ub
}

// QueryMapMeta returns a map of all metadata values with the given query action to the
// list of zettel IDs containing this value.
func (c *Client) QueryMapMeta(ctx context.Context, query string) (api.MapMeta, error) {
	err := c.updateToken(ctx)
	if err != nil {
		return nil, err
	}
	req, err := c.newRequest(ctx, http.MethodGet, c.newURLBuilder('z').AppendKVQuery(api.QueryKeyEncoding, api.EncodingJson).AppendQuery(query), nil)
	if err != nil {
		return nil, err
	}
	resp, err := c.executeRequest(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var mlj api.MapListJSON
	err = dec.Decode(&mlj)
	if err != nil {
		return nil, err
	}
	return mlj.Map, nil
}

// GetVersionJSON returns version information..
func (c *Client) GetVersionJSON(ctx context.Context) (api.VersionJSON, error) {
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, c.newURLBuilder('x'), nil, nil)
	if err != nil {
		return api.VersionJSON{}, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return api.VersionJSON{}, statusToError(resp)
	}
	dec := json.NewDecoder(resp.Body)
	var version api.VersionJSON
	err = dec.Decode(&version)
	if err != nil {
		return api.VersionJSON{}, err
	}
	return version, nil
}

Changes to client/client_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21
22
23



24
25
26
27
28

29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45
46
47
48

49
50
51
52
53

54
55
56
57
58
59
60
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22




23
24
25
26
27
28
29

30
31
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46
47
48
49

50
51




52
53
54
55
56
57
58
59








+
+
+











-
-
-
-
+
+
+




-
+








-
+










-
+

-
-
-
-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

package client_test

import (
	"context"
	"flag"
	"net/http"
	"net/url"
	"testing"

	"codeberg.org/t73fde/sxpf"
	"zettelstore.de/c/api"
	"zettelstore.de/c/client"
	"zettelstore.de/c/sexpr"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/client"
	"t73f.de/r/zsc/domain/id"
)

func TestZettelList(t *testing.T) {
	c := getClient()
	_, err := c.ListZettel(context.Background(), "")
	_, err := c.QueryZettel(context.Background(), "")
	if err != nil {
		t.Error(err)
		return
	}
}

func TestGetProtectedZettel(t *testing.T) {
	c := getClient()
	_, err := c.GetZettel(context.Background(), api.ZidStartupConfiguration, api.PartZettel)
	_, err := c.GetZettel(context.Background(), id.ZidStartupConfiguration, api.PartZettel)
	if err != nil {
		if cErr, ok := err.(*client.Error); ok && cErr.StatusCode == http.StatusForbidden {
			return
		} else {
			t.Error(err)
		}
		return
	}
}

func TestGetSexprZettel(t *testing.T) {
func TestGetSzZettel(t *testing.T) {
	c := getClient()
	sf := sxpf.MakeMappedFactory()
	var zetSyms sexpr.ZettelSymbols
	zetSyms.InitializeZettelSymbols(sf)
	value, err := c.GetEvaluatedSexpr(context.Background(), api.ZidDefaultHome, api.PartContent, sf)
	value, err := c.GetEvaluatedSz(context.Background(), id.ZidDefaultHome, api.PartContent)
	if err != nil {
		t.Error(err)
		return
	}
	if value.IsNil() {
		t.Error("No data")
	}

Added client/retrieve.go.











































































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package client

import (
	"bufio"
	"bytes"
	"context"
	"fmt"
	"io"
	"net/http"

	"t73f.de/r/sx"
	"t73f.de/r/sx/sxreader"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsc/sexp"
	"t73f.de/r/zsx"
)

var bsLF = []byte{'\n'}

// QueryZettel returns a list of all Zettel based on the given query.
//
// query is a search expression, as described in [Query the list of all zettel].
//
// The functions returns a slice of bytes slices, where each byte slice contains the
// zettel identifier within its first 14 bytes. The next byte is a space character,
// followed by the title of the zettel.
//
// [Query the list of all zettel]: https://zettelstore.de/manual/h/00001012051400
func (c *Client) QueryZettel(ctx context.Context, query string) ([][]byte, error) {
	ub := c.NewURLBuilder('z').AppendQuery(query)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return nil, err
	}
	defer func() { _ = resp.Body.Close() }()
	data, err := io.ReadAll(resp.Body)
	switch resp.StatusCode {
	case http.StatusOK:
	case http.StatusNoContent:
		return nil, nil
	default:
		return nil, statusToError(resp)
	}
	if err != nil {
		return nil, err
	}
	lines := bytes.Split(data, bsLF)
	if len(lines[len(lines)-1]) == 0 {
		lines = lines[:len(lines)-1]
	}
	return lines, nil
}

// QueryZettelData returns a list of zettel metadata.
//
// query is a search expression, as described in [Query the list of all zettel].
//
// The functions returns the normalized query and its human-readable representation as
// its first two result values.
//
// [Query the list of all zettel]: https://zettelstore.de/manual/h/00001012051400
func (c *Client) QueryZettelData(ctx context.Context, query string) (string, string, []api.ZidMetaRights, error) {
	ub := c.NewURLBuilder('z').AppendKVQuery(api.QueryKeyEncoding, api.EncodingData).AppendQuery(query)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return "", "", nil, err
	}
	defer func() { _ = resp.Body.Close() }()
	rdr := sxreader.MakeReader(resp.Body).SetListLimit(0) // No limit b/c number of zettel may be more than 100000. We must trust the server
	obj, err := rdr.Read()
	switch resp.StatusCode {
	case http.StatusOK:
	case http.StatusNoContent:
		return "", "", nil, nil
	default:
		return "", "", nil, statusToError(resp)
	}
	if err != nil {
		return "", "", nil, err
	}
	vals, err := sexp.ParseList(obj, "yppr")
	if err != nil {
		return "", "", nil, err
	}
	qVals, err := sexp.ParseList(vals[1], "ys")
	if err != nil {
		return "", "", nil, err
	}
	hVals, err := sexp.ParseList(vals[2], "ys")
	if err != nil {
		return "", "", nil, err
	}
	metaList, err := parseMetaList(vals[3].(*sx.Pair))
	return zsx.GoValue(qVals[1]), zsx.GoValue(hVals[1]), metaList, err
}

func parseMetaList(metaPair *sx.Pair) ([]api.ZidMetaRights, error) {
	var result []api.ZidMetaRights
	for node := metaPair; !sx.IsNil(node); {
		elem, isPair := sx.GetPair(node)
		if !isPair {
			return nil, fmt.Errorf("meta-list not a proper list: %v", metaPair.String())
		}
		node = elem.Tail()
		vals, err := sexp.ParseList(elem.Car(), "yppp")
		if err != nil {
			return nil, err
		}

		if errSym := sexp.CheckSymbol(vals[0], "zettel"); errSym != nil {
			return nil, errSym
		}

		idVals, err := sexp.ParseList(vals[1], "yi")
		if err != nil {
			return nil, err
		}
		if errSym := sexp.CheckSymbol(idVals[0], "id"); errSym != nil {
			return nil, errSym
		}
		zid, err := makeZettelID(idVals[1])
		if err != nil {
			return nil, err
		}

		meta, err := sexp.ParseMeta(vals[2].(*sx.Pair))
		if err != nil {
			return nil, err
		}

		rights, err := sexp.ParseRights(vals[3])
		if err != nil {
			return nil, err
		}

		result = append(result, api.ZidMetaRights{
			ID:     zid,
			Meta:   meta,
			Rights: rights,
		})
	}
	return result, nil
}

// QueryAggregate returns a aggregate as a result of a query.
// It is most often used in a query with an action, where the action is either
// a metadata key of type Word or of type TagSet.
//
// query is a search expression, as described in [Query the list of all zettel].
// It must contain an aggregate action.
//
// [Query the list of all zettel]: https://zettelstore.de/manual/h/00001012051400
func (c *Client) QueryAggregate(ctx context.Context, query string) (api.Aggregate, error) {
	lines, err := c.QueryZettel(ctx, query)
	if err != nil {
		return nil, err
	}
	if len(lines) == 0 {
		return nil, nil
	}
	agg := make(api.Aggregate, len(lines))
	for _, line := range lines {
		if fields := bytes.Fields(line); len(fields) > 1 {
			key := string(fields[0])
			for _, field := range fields[1:] {
				if zid, zidErr := id.Parse(string(field)); zidErr == nil {
					agg[key] = append(agg[key], zid)
				}
			}
		}
	}
	return agg, nil
}

// TagZettel returns the identifier of the tag zettel for a given tag.
//
// This method only works if c.AllowRedirect(true) was called.
func (c *Client) TagZettel(ctx context.Context, tag string) (id.Zid, error) {
	return c.fetchTagOrRoleZettel(ctx, api.QueryKeyTag, tag)
}

// RoleZettel returns the identifier of the tag zettel for a given role.
//
// This method only works if c.AllowRedirect(true) was called.
func (c *Client) RoleZettel(ctx context.Context, role string) (id.Zid, error) {
	return c.fetchTagOrRoleZettel(ctx, api.QueryKeyRole, role)
}

func (c *Client) fetchTagOrRoleZettel(ctx context.Context, key, val string) (id.Zid, error) {
	if c.client.CheckRedirect == nil {
		panic("client does not allow to track redirect")
	}
	ub := c.NewURLBuilder('z').AppendKVQuery(key, val)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return id.Invalid, err
	}
	defer func() { _ = resp.Body.Close() }()
	data, err := io.ReadAll(resp.Body)
	if err != nil {
		return id.Invalid, err
	}

	switch resp.StatusCode {
	case http.StatusNotFound:
		return id.Invalid, nil
	case http.StatusFound:
		return id.Parse(string(data))
	default:
		return id.Invalid, statusToError(resp)
	}
}

// GetZettel returns a zettel as a byte slice.
//
// part must be one of "meta", "content", or "zettel".
//
// The format of the byte slice is described in [Layout of a zettel].
//
// [Layout of a zettel]: https://zettelstore.de/manual/h/00001006000000
func (c *Client) GetZettel(ctx context.Context, zid id.Zid, part string) ([]byte, error) {
	ub := c.NewURLBuilder('z').SetZid(zid)
	if part != "" && part != api.PartContent {
		ub.AppendKVQuery(api.QueryKeyPart, part)
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return nil, err
	}
	defer func() { _ = resp.Body.Close() }()
	data, err := io.ReadAll(resp.Body)
	switch resp.StatusCode {
	case http.StatusOK:
	case http.StatusNoContent:
		return nil, nil
	default:
		return nil, statusToError(resp)
	}
	return data, err
}

// GetZettelData returns a zettel as a struct of its parts.
func (c *Client) GetZettelData(ctx context.Context, zid id.Zid) (api.ZettelData, error) {
	ub := c.NewURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, api.EncodingData)
	ub.AppendKVQuery(api.QueryKeyPart, api.PartZettel)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err == nil {
		defer func() { _ = resp.Body.Close() }()
		if resp.StatusCode != http.StatusOK {
			return api.ZettelData{}, statusToError(resp)
		}
		rdr := sxreader.MakeReader(resp.Body)
		obj, err2 := rdr.Read()
		if err2 == nil {
			return sexp.ParseZettel(obj)
		}
	}
	return api.ZettelData{}, err
}

// GetParsedZettel return a parsed zettel in a specified text-based encoding.
//
// A parsed zettel is just read from its box and is not processed any further.
//
// Valid encoding values are given as constants. They are described in more
// detail in [Encodings available via the API].
//
// [Encodings available via the API]: https://zettelstore.de/manual/h/00001012920500
func (c *Client) GetParsedZettel(ctx context.Context, zid id.Zid, enc api.EncodingEnum) ([]byte, error) {
	return c.getZettelString(ctx, zid, enc, true)
}

// GetEvaluatedZettel return an evaluated zettel in a specified text-based encoding.
//
// An evaluated zettel was parsed, and any transclusions etc. are resolved.
// This is the zettel representation you typically see on the Web UI.
//
// Valid encoding values are given as constants. They are described in more
// detail in [Encodings available via the API].
//
// [Encodings available via the API]: https://zettelstore.de/manual/h/00001012920500
func (c *Client) GetEvaluatedZettel(ctx context.Context, zid id.Zid, enc api.EncodingEnum) ([]byte, error) {
	return c.getZettelString(ctx, zid, enc, false)
}

func (c *Client) getZettelString(ctx context.Context, zid id.Zid, enc api.EncodingEnum, parseOnly bool) ([]byte, error) {
	ub := c.NewURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, enc.String())
	ub.AppendKVQuery(api.QueryKeyPart, api.PartContent)
	if parseOnly {
		ub.AppendKVQuery(api.QueryKeyParseOnly, "")
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return nil, err
	}
	defer func() { _ = resp.Body.Close() }()
	switch resp.StatusCode {
	case http.StatusOK:
	case http.StatusNoContent:
	default:
		return nil, statusToError(resp)
	}
	return io.ReadAll(resp.Body)
}

// GetParsedSz returns a part of an parsed zettel as a Sexpr-decoded data structure.
//
// A parsed zettel is just read from its box and is not processed any further.
//
// part must be one of "meta", "content", or "zettel".
//
// Basically, this function returns the sz encoding of a part of a zettel.
func (c *Client) GetParsedSz(ctx context.Context, zid id.Zid, part string) (sx.Object, error) {
	return c.getSz(ctx, zid, part, true)
}

// GetEvaluatedSz returns an evaluated zettel as a Sexpr-decoded data structure.
//
// An evaluated zettel was parsed, and any transclusions etc. are resolved.
// This is the zettel representation you typically see on the Web UI.
//
// part must be one of "meta", "content", or "zettel".
//
// Basically, this function returns the sz encoding of a part of a zettel.
func (c *Client) GetEvaluatedSz(ctx context.Context, zid id.Zid, part string) (sx.Object, error) {
	return c.getSz(ctx, zid, part, false)
}

func (c *Client) getSz(ctx context.Context, zid id.Zid, part string, parseOnly bool) (sx.Object, error) {
	ub := c.NewURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, api.EncodingSz)
	if part != "" {
		ub.AppendKVQuery(api.QueryKeyPart, part)
	}
	if parseOnly {
		ub.AppendKVQuery(api.QueryKeyParseOnly, "")
	}
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return nil, err
	}
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusOK {
		return nil, statusToError(resp)
	}
	return sxreader.MakeReader(bufio.NewReaderSize(resp.Body, 8)).Read()
}

// GetMetaData returns the metadata of a zettel.
func (c *Client) GetMetaData(ctx context.Context, zid id.Zid) (api.MetaRights, error) {
	ub := c.NewURLBuilder('z').SetZid(zid)
	ub.AppendKVQuery(api.QueryKeyEncoding, api.EncodingData)
	ub.AppendKVQuery(api.QueryKeyPart, api.PartMeta)
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return api.MetaRights{}, err
	}
	defer func() { _ = resp.Body.Close() }()
	rdr := sxreader.MakeReader(resp.Body)
	obj, err := rdr.Read()
	if resp.StatusCode != http.StatusOK {
		return api.MetaRights{}, statusToError(resp)
	}
	if err != nil {
		return api.MetaRights{}, err
	}
	vals, err := sexp.ParseList(obj, "ypp")
	if err != nil {
		return api.MetaRights{}, err
	}
	if errSym := sexp.CheckSymbol(vals[0], "list"); errSym != nil {
		return api.MetaRights{}, err
	}

	meta, err := sexp.ParseMeta(vals[1].(*sx.Pair))
	if err != nil {
		return api.MetaRights{}, err
	}

	rights, err := sexp.ParseRights(vals[2])
	if err != nil {
		return api.MetaRights{}, err
	}

	return api.MetaRights{
		Meta:   meta,
		Rights: rights,
	}, nil
}

// GetVersionInfo returns version information of the Zettelstore that is used.
func (c *Client) GetVersionInfo(ctx context.Context) (VersionInfo, error) {
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, c.NewURLBuilder('x'), nil)
	if err != nil {
		return VersionInfo{}, err
	}
	defer func() { _ = resp.Body.Close() }()
	if resp.StatusCode != http.StatusOK {
		return VersionInfo{}, statusToError(resp)
	}
	rdr := sxreader.MakeReader(resp.Body)
	obj, err := rdr.Read()
	if err == nil {
		if vals, errVals := sexp.ParseList(obj, "iiiss"); errVals == nil {
			return VersionInfo{
				Major: int(vals[0].(sx.Int64)),
				Minor: int(vals[1].(sx.Int64)),
				Patch: int(vals[2].(sx.Int64)),
				Info:  vals[3].(sx.String).GetValue(),
				Hash:  vals[4].(sx.String).GetValue(),
			}, nil
		}
	}
	return VersionInfo{}, err
}

// VersionInfo contains version information of the associated Zettelstore.
//
//   - Major is an integer containing the major software version of Zettelstore.
//     If its value is greater than zero, different major versions are not compatible.
//   - Minor is an integer specifying the minor software version for the given major version.
//     If the major version is greater than zero, minor versions are backward compatible.
//   - Patch is an integer that specifies a change within a minor version.
//     A version that have equal major and minor versions and differ in patch version are
//     always compatible, even if the major version equals zero.
//   - Info contains some optional text, i.e. it may be the empty string. Typically, Info
//     specifies a developer version by containing the string "dev".
//   - Hash contains the value of the source code version stored in the Zettelstore repository.
//     You can use it to reproduce bugs that occured, when source code was changed since
//     its introduction.
type VersionInfo struct {
	Major int
	Minor int
	Patch int
	Info  string
	Hash  string
}

// GetApplicationZid returns the zettel identifier used to configure a client
// application with the given name.
func (c *Client) GetApplicationZid(ctx context.Context, appname string) (id.Zid, error) {
	mr, err := c.GetMetaData(ctx, id.ZidAppDirectory)
	if err != nil {
		return id.Invalid, err
	}
	key := appname + "-zid"
	val, found := mr.Meta[key]
	if !found {
		return id.Invalid, fmt.Errorf("no application registered: %v", appname)
	}
	zid, err := id.Parse(val)
	if err == nil {
		return zid, nil
	}
	return id.Invalid, fmt.Errorf("invalid identifier for application %v: %v", appname, val)
}

// Get executes a GET request to the given URL and returns the read data.
func (c *Client) Get(ctx context.Context, ub *api.URLBuilder) ([]byte, error) {
	resp, err := c.buildAndExecuteRequest(ctx, http.MethodGet, ub, nil)
	if err != nil {
		return nil, err
	}
	defer func() { _ = resp.Body.Close() }()
	switch resp.StatusCode {
	case http.StatusOK:
	case http.StatusNoContent:
		return nil, nil
	default:
		return nil, statusToError(resp)
	}
	data, err := io.ReadAll(resp.Body)
	return data, err
}

Added domain/id/id.go.



























































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package id provides zettel specific types, constants, and functions about
// zettel identifier.
package id

import (
	"strconv"
	"time"
)

// Zid is the internal identifier of a zettel. Typically, it is a time stamp
// of the form "YYYYMMDDHHmmSS" converted to an unsigned integer.
type Zid uint64

// LengthZid factors the constant length of a zettel identifier
const LengthZid = 14

// Some important ZettelIDs.
const (
	Invalid = Zid(0) // Invalid is a Zid that will never be valid

	maxZid = 99999999999999
)

// Predefined zettel identifier.
//
// See [List of predefined zettel].
//
// [List of predefined zettel]: https://zettelstore.de/manual/h/00001005090000
const (
	// System zettel
	ZidVersion              = Zid(1)
	ZidHost                 = Zid(2)
	ZidOperatingSystem      = Zid(3)
	ZidLicense              = Zid(4)
	ZidAuthors              = Zid(5)
	ZidDependencies         = Zid(6)
	ZidLog                  = Zid(7)
	ZidMemory               = Zid(8)
	ZidSx                   = Zid(9)
	ZidHTTP                 = Zid(10)
	ZidAPI                  = Zid(11)
	ZidWebUI                = Zid(12)
	ZidConsole              = Zid(13)
	ZidBoxManager           = Zid(20)
	ZidZettel               = Zid(21)
	ZidIndex                = Zid(22)
	ZidQuery                = Zid(23)
	ZidMetadataKey          = Zid(90)
	ZidParser               = Zid(92)
	ZidStartupConfiguration = Zid(96)
	ZidConfiguration        = Zid(100)
	ZidDirectory            = Zid(101)

	// WebUI HTML templates are in the range 10000..19999
	ZidBaseTemplate   = Zid(10100)
	ZidLoginTemplate  = Zid(10200)
	ZidListTemplate   = Zid(10300)
	ZidZettelTemplate = Zid(10401)
	ZidInfoTemplate   = Zid(10402)
	ZidFormTemplate   = Zid(10403)
	ZidDeleteTemplate = Zid(10405)
	ZidErrorTemplate  = Zid(10700)

	// WebUI sxn code zettel are in the range 19000..19999
	ZidSxnStart = Zid(19000)
	ZidSxnBase  = Zid(19990)

	// CSS-related zettel are in the range 20000..29999
	ZidBaseCSS = Zid(20001)
	ZidUserCSS = Zid(25001)

	// WebUI JS zettel are in the range 30000..39999

	// WebUI image zettel are in the range 40000..49999
	ZidEmoji = Zid(40001)

	// Other sxn code zettel are in the range 50000..59999
	ZidSxnPrelude = Zid(59900)

	// Predefined Zettelmarkup zettel are in the range 60000..69999
	ZidRoleZettelZettel        = Zid(60010)
	ZidRoleConfigurationZettel = Zid(60020)
	ZidRoleRoleZettel          = Zid(60030)
	ZidRoleTagZettel           = Zid(60040)

	// Range 80000...89999 is reserved for web ui menus
	ZidTOCListsMenu = Zid(80001) // "Lists" menu

	// Range 90000...99999 is reserved for zettel templates
	ZidTOCNewTemplate    = Zid(90000)
	ZidTemplateNewZettel = Zid(90001)
	ZidTemplateNewRole   = Zid(90004)
	ZidTemplateNewTag    = Zid(90003)
	ZidTemplateNewUser   = Zid(90002)

	// Range 00000999999900...00000999999999 are predefined zettel to be searched by content.
	ZidAppDirectory = Zid(999999999)

	// Default Home Zettel
	ZidDefaultHome = Zid(10000000000)
)

// ParseUint interprets a string as a possible zettel identifier
// and returns its integer value.
func ParseUint(s string) (uint64, error) {
	res, err := strconv.ParseUint(s, 10, 47)
	if err != nil {
		return 0, err
	}
	if res == 0 || res > maxZid {
		return res, strconv.ErrRange
	}
	return res, nil
}

// Parse interprets a string as a zettel identification and
// returns its value.
func Parse(s string) (Zid, error) {
	if len(s) != LengthZid {
		return Invalid, strconv.ErrSyntax
	}
	res, err := ParseUint(s)
	if err != nil {
		return Invalid, err
	}
	return Zid(res), nil
}

// MustParse tries to interpret a string as a zettel identifier and returns
// its value or panics otherwise.
func MustParse(s string) Zid {
	zid, err := Parse(string(s))
	if err == nil {
		return zid
	}
	panic(err)
}

// String converts the zettel identification to a string of 14 digits.
// Only defined for valid ids.
func (zid Zid) String() string {
	var result [LengthZid]byte
	zid.toByteArray(&result)
	return string(result[:])
}

// Bytes converts the zettel identification to a byte slice of 14 digits.
// Only defined for valid ids.
func (zid Zid) Bytes() []byte {
	var result [LengthZid]byte
	zid.toByteArray(&result)
	return result[:]
}

// toByteArray converts the Zid into a fixed byte array, usable for printing.
//
// Based on idea by Daniel Lemire: "Converting integers to fix-digit representations quickly"
// https://lemire.me/blog/2021/11/18/converting-integers-to-fix-digit-representations-quickly/
func (zid Zid) toByteArray(result *[LengthZid]byte) {
	date := uint64(zid) / 1000000
	fullyear := date / 10000
	century, year := fullyear/100, fullyear%100
	monthday := date % 10000
	month, day := monthday/100, monthday%100
	time := uint64(zid) % 1000000
	hmtime, second := time/100, time%100
	hour, minute := hmtime/100, hmtime%100

	result[0] = byte(century/10) + '0'
	result[1] = byte(century%10) + '0'
	result[2] = byte(year/10) + '0'
	result[3] = byte(year%10) + '0'
	result[4] = byte(month/10) + '0'
	result[5] = byte(month%10) + '0'
	result[6] = byte(day/10) + '0'
	result[7] = byte(day%10) + '0'
	result[8] = byte(hour/10) + '0'
	result[9] = byte(hour%10) + '0'
	result[10] = byte(minute/10) + '0'
	result[11] = byte(minute%10) + '0'
	result[12] = byte(second/10) + '0'
	result[13] = byte(second%10) + '0'
}

// IsValid determines if zettel id is a valid one, e.g. consists of max. 14 digits.
func (zid Zid) IsValid() bool { return 0 < zid && zid <= maxZid }

// TimestampLayout to transform a date into a Zid and into other internal dates.
const TimestampLayout = "20060102150405"

// New returns a new zettel id based on the current time.
func New(withSeconds bool) Zid {
	now := time.Now().Local()
	var s string
	if withSeconds {
		s = now.Format(TimestampLayout)
	} else {
		s = now.Format("20060102150400")
	}
	res, err := Parse(s)
	if err != nil {
		panic(err)
	}
	return res
}

Added domain/id/id_test.go.





























































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package id_test provides unit tests for testing zettel id specific functions.
package id_test

import (
	"testing"

	"t73f.de/r/zsc/domain/id"
)

func TestIsValid(t *testing.T) {
	t.Parallel()
	validIDs := []string{
		"00000000000001",
		"00000000000020",
		"00000000000300",
		"00000000004000",
		"00000000050000",
		"00000000600000",
		"00000007000000",
		"00000080000000",
		"00000900000000",
		"00001000000000",
		"00020000000000",
		"00300000000000",
		"04000000000000",
		"50000000000000",
		"99999999999999",
		"00001007030200",
		"20200310195100",
		"12345678901234",
	}

	for i, sid := range validIDs {
		zid, err := id.Parse(sid)
		if err != nil {
			t.Errorf("i=%d: sid=%q is not valid, but should be. err=%v", i, sid, err)
		}
		s := zid.String()
		if s != sid {
			t.Errorf(
				"i=%d: zid=%v does not format to %q, but to %q", i, zid, sid, s)
		}
	}

	invalidIDs := []string{
		"", "0", "a",
		"00000000000000",
		"0000000000000a",
		"000000000000000",
		"20200310T195100",
		"+1234567890123",
	}

	for i, sid := range invalidIDs {
		if zid, err := id.Parse(sid); err == nil {
			t.Errorf("i=%d: sid=%q is valid (zid=%s), but should not be", i, sid, zid)
		}
	}
}

var sResult string // to disable compiler optimization in loop below

func BenchmarkString(b *testing.B) {
	var s string
	for b.Loop() {
		s = id.Zid(12345678901200).String()
	}
	sResult = s
}

var bResult []byte // to disable compiler optimization in loop below

func BenchmarkBytes(b *testing.B) {
	var bs []byte
	for b.Loop() {
		bs = id.Zid(12345678901200).Bytes()
	}
	bResult = bs
}

Added domain/id/idset/idset.go.






































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

// Package idset implements sets of zettel identifier.
package idset

import (
	"slices"
	"strings"

	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsc/domain/meta"
)

// Set is a set of zettel identifier
type Set struct {
	seq []id.Zid
}

// String returns a string representation of the set.
func (s *Set) String() string {
	return "{" + s.MetaString() + "}"
}

// MetaString returns a string representation of the set to be stored as metadata.
func (s *Set) MetaString() string {
	if s == nil || len(s.seq) == 0 {
		return ""
	}
	var sb strings.Builder
	for i, zid := range s.seq {
		if i > 0 {
			sb.WriteByte(' ')
		}
		sb.Write(zid.Bytes())
	}
	return sb.String()
}

// MetaValue returns a metadata value representation of the set.
func (s *Set) MetaValue() meta.Value { return meta.Value(s.MetaString()) }

// New returns a new set of identifier with the given initial values.
func New(zids ...id.Zid) *Set {
	switch l := len(zids); l {
	case 0:
		return &Set{seq: nil}
	case 1:
		return &Set{seq: []id.Zid{zids[0]}}
	default:
		result := Set{seq: make([]id.Zid, 0, l)}
		result.AddSlice(zids)
		return &result
	}
}

// NewCap returns a new set of identifier with the given capacity and initial values.
func NewCap(c int, zids ...id.Zid) *Set {
	result := Set{seq: make([]id.Zid, 0, max(c, len(zids)))}
	result.AddSlice(zids)
	return &result
}

// IsEmpty returns true, if the set conains no element.
func (s *Set) IsEmpty() bool {
	return s == nil || len(s.seq) == 0
}

// Length returns the number of elements in this set.
func (s *Set) Length() int {
	if s == nil {
		return 0
	}
	return len(s.seq)
}

// Clone returns a copy of the given set.
func (s *Set) Clone() *Set {
	if s == nil || len(s.seq) == 0 {
		return nil
	}
	return &Set{seq: slices.Clone(s.seq)}
}

// Add adds a Add to the set.
func (s *Set) Add(zid id.Zid) *Set {
	if s == nil {
		return New(zid)
	}
	s.add(zid)
	return s
}

// Contains return true if the set is non-nil and the set contains the given Zettel identifier.
func (s *Set) Contains(zid id.Zid) bool { return s != nil && s.contains(zid) }

// ContainsOrNil return true if the set is nil or if the set contains the given Zettel identifier.
func (s *Set) ContainsOrNil(zid id.Zid) bool { return s == nil || s.contains(zid) }

// AddSlice adds all identifier of the given slice to the set.
func (s *Set) AddSlice(sl []id.Zid) *Set {
	if s == nil {
		return New(sl...)
	}
	s.seq = slices.Grow(s.seq, len(sl))
	for _, zid := range sl {
		s.add(zid)
	}
	return s
}

// SafeSorted returns the set as a new sorted slice of zettel identifier.
func (s *Set) SafeSorted() []id.Zid {
	if s == nil {
		return nil
	}
	return slices.Clone(s.seq)
}

// IntersectOrSet removes all zettel identifier that are not in the other set.
// Both sets can be modified by this method. One of them is the set returned.
// It contains the intersection of both, if s is not nil.
//
// If s == nil, then the other set is always returned.
func (s *Set) IntersectOrSet(other *Set) *Set {
	if s == nil || other == nil {
		return other.Clone()
	}
	topos, spos, opos := 0, 0, 0
	for spos < len(s.seq) && opos < len(other.seq) {
		sz, oz := s.seq[spos], other.seq[opos]
		if sz < oz {
			spos++
			continue
		}
		if sz > oz {
			opos++
			continue
		}
		s.seq[topos] = sz
		topos++
		spos++
		opos++
	}
	s.seq = s.seq[:topos]
	return s
}

// IUnion adds the elements of set other to s.
func (s *Set) IUnion(other *Set) *Set {
	if other == nil || len(other.seq) == 0 {
		return s
	}
	// TODO: if other is large enough (and s is not too small) -> optimize by swapping and/or loop through both
	return s.AddSlice(other.seq)
}

// ISubstract removes all zettel identifier from 's' that are in the set 'other'.
func (s *Set) ISubstract(other *Set) {
	if s == nil || len(s.seq) == 0 || other == nil || len(other.seq) == 0 {
		return
	}
	topos, spos, opos := 0, 0, 0
	for spos < len(s.seq) && opos < len(other.seq) {
		sz, oz := s.seq[spos], other.seq[opos]
		if sz < oz {
			s.seq[topos] = sz
			topos++
			spos++
			continue
		}
		if sz == oz {
			spos++
		}
		opos++
	}
	for spos < len(s.seq) {
		s.seq[topos] = s.seq[spos]
		topos++
		spos++
	}
	s.seq = s.seq[:topos]
}

// Diff returns the difference sets between the two sets: the first difference
// set is the set of elements that are in other, but not in s; the second
// difference set is the set of element that are in s but not in other.
//
// in other words: the first result is the set of elements from other that must
// be added to s; the second result is the set of elements that must be removed
// from s, so that s would have the same elemest as other.
func (s *Set) Diff(other *Set) (newS, remS *Set) {
	if s == nil || len(s.seq) == 0 {
		return other.Clone(), nil
	}
	if other == nil || len(other.seq) == 0 {
		return nil, s.Clone()
	}
	seqS, seqO := s.seq, other.seq
	var newRefs, remRefs []id.Zid
	npos, opos := 0, 0
	for npos < len(seqO) && opos < len(seqS) {
		rn, ro := seqO[npos], seqS[opos]
		if rn == ro {
			npos++
			opos++
			continue
		}
		if rn < ro {
			newRefs = append(newRefs, rn)
			npos++
			continue
		}
		remRefs = append(remRefs, ro)
		opos++
	}
	if npos < len(seqO) {
		newRefs = append(newRefs, seqO[npos:]...)
	}
	if opos < len(seqS) {
		remRefs = append(remRefs, seqS[opos:]...)
	}
	return newFromSlice(newRefs), newFromSlice(remRefs)
}

// Remove the identifier from the set.
func (s *Set) Remove(zid id.Zid) *Set {
	if s == nil || len(s.seq) == 0 {
		return nil
	}
	if pos, found := s.find(zid); found {
		copy(s.seq[pos:], s.seq[pos+1:])
		s.seq = s.seq[:len(s.seq)-1]
	}
	if len(s.seq) == 0 {
		return nil
	}
	return s
}

// Equal returns true if the other set is equal to the given set.
func (s *Set) Equal(other *Set) bool {
	if s == nil {
		return other == nil
	}
	if other == nil {
		return false
	}
	return slices.Equal(s.seq, other.seq)
}

// ForEach calls the given function for each element of the set.
//
// Every element is bigger than the previous one.
func (s *Set) ForEach(fn func(zid id.Zid)) {
	if s != nil {
		for _, zid := range s.seq {
			fn(zid)
		}
	}
}

// Pop return one arbitrary element of the set.
func (s *Set) Pop() (id.Zid, bool) {
	if s != nil {
		if l := len(s.seq); l > 0 {
			zid := s.seq[l-1]
			s.seq = s.seq[:l-1]
			return zid, true
		}
	}
	return id.Invalid, false
}

// Optimize the amount of memory to store the set.
func (s *Set) Optimize() {
	if s != nil {
		s.seq = slices.Clip(s.seq)
	}
}

// ----- unchecked base operations

func newFromSlice(seq []id.Zid) *Set {
	if l := len(seq); l == 0 {
		return nil
	}
	return &Set{seq: seq}
}

func (s *Set) add(zid id.Zid) {
	if pos, found := s.find(zid); !found {
		s.seq = slices.Insert(s.seq, pos, zid)
	}
}

func (s *Set) contains(zid id.Zid) bool {
	_, found := s.find(zid)
	return found
}

func (s *Set) find(zid id.Zid) (int, bool) {
	hi := len(s.seq)
	for lo := 0; lo < hi; {
		m := lo + (hi-lo)/2
		if z := s.seq[m]; z == zid {
			return m, true
		} else if z < zid {
			lo = m + 1
		} else {
			hi = m
		}
	}
	return hi, false
}

Added domain/id/idset/idset_test.go.


















































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2021-present Detlef Stern
//-----------------------------------------------------------------------------

package idset_test

import (
	"slices"
	"testing"

	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsc/domain/id/idset"
)

func TestSetContainsOrNil(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		s   *idset.Set
		zid id.Zid
		exp bool
	}{
		{nil, id.Invalid, true},
		{nil, 14, true},
		{idset.New(), id.Invalid, false},
		{idset.New(), 1, false},
		{idset.New(), id.Invalid, false},
		{idset.New(1), 1, true},
	}
	for i, tc := range testcases {
		got := tc.s.ContainsOrNil(tc.zid)
		if got != tc.exp {
			t.Errorf("%d: %v.ContainsOrNil(%v) == %v, but got %v", i, tc.s, tc.zid, tc.exp, got)
		}
	}
}

func TestSetAdd(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		s1, s2 *idset.Set
		exp    []id.Zid
	}{
		{nil, nil, nil},
		{idset.New(), nil, nil},
		{idset.New(), idset.New(), nil},
		{nil, idset.New(1), []id.Zid{1}},
		{idset.New(1), nil, []id.Zid{1}},
		{idset.New(1), idset.New(), []id.Zid{1}},
		{idset.New(1), idset.New(2), []id.Zid{1, 2}},
		{idset.New(1), idset.New(1), []id.Zid{1}},
	}
	for i, tc := range testcases {
		sl1 := tc.s1.SafeSorted()
		sl2 := tc.s2.SafeSorted()
		got := tc.s1.IUnion(tc.s2).SafeSorted()
		if !slices.Equal(got, tc.exp) {
			t.Errorf("%d: %v.Add(%v) should be %v, but got %v", i, sl1, sl2, tc.exp, got)
		}
	}
}

func TestSetSafeSorted(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		set *idset.Set
		exp []id.Zid
	}{
		{nil, nil},
		{idset.New(), nil},
		{idset.New(9, 4, 6, 1, 7), []id.Zid{1, 4, 6, 7, 9}},
	}
	for i, tc := range testcases {
		got := tc.set.SafeSorted()
		if !slices.Equal(got, tc.exp) {
			t.Errorf("%d: %v.SafeSorted() should be %v, but got %v", i, tc.set, tc.exp, got)
		}
	}
}

func TestSetIntersectOrSet(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		s1, s2 *idset.Set
		exp    []id.Zid
	}{
		{nil, nil, nil},
		{idset.New(), nil, nil},
		{nil, idset.New(), nil},
		{idset.New(), idset.New(), nil},
		{idset.New(1), nil, nil},
		{nil, idset.New(1), []id.Zid{1}},
		{idset.New(1), idset.New(), nil},
		{idset.New(), idset.New(1), nil},
		{idset.New(1), idset.New(2), nil},
		{idset.New(2), idset.New(1), nil},
		{idset.New(1), idset.New(1), []id.Zid{1}},
	}
	for i, tc := range testcases {
		sl1 := tc.s1.SafeSorted()
		sl2 := tc.s2.SafeSorted()
		got := tc.s1.IntersectOrSet(tc.s2).SafeSorted()
		if !slices.Equal(got, tc.exp) {
			t.Errorf("%d: %v.IntersectOrSet(%v) should be %v, but got %v", i, sl1, sl2, tc.exp, got)
		}
	}
}

func TestSetIUnion(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		s1, s2 *idset.Set
		exp    *idset.Set
	}{
		{nil, nil, nil},
		{idset.New(), nil, nil},
		{nil, idset.New(), nil},
		{idset.New(), idset.New(), nil},
		{idset.New(1), nil, idset.New(1)},
		{nil, idset.New(1), idset.New(1)},
		{idset.New(1), idset.New(), idset.New(1)},
		{idset.New(), idset.New(1), idset.New(1)},
		{idset.New(1), idset.New(2), idset.New(1, 2)},
		{idset.New(2), idset.New(1), idset.New(2, 1)},
		{idset.New(1), idset.New(1), idset.New(1)},
		{idset.New(1, 2, 3), idset.New(2, 3, 4), idset.New(1, 2, 3, 4)},
	}
	for i, tc := range testcases {
		s1 := tc.s1.Clone()
		sl1 := s1.SafeSorted()
		sl2 := tc.s2.SafeSorted()
		got := s1.IUnion(tc.s2)
		if !got.Equal(tc.exp) {
			t.Errorf("%d: %v.IUnion(%v) should be %v, but got %v", i, sl1, sl2, tc.exp, got)
		}
	}
}

func TestSetISubtract(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		s1, s2 *idset.Set
		exp    []id.Zid
	}{
		{nil, nil, nil},
		{idset.New(), nil, nil},
		{nil, idset.New(), nil},
		{idset.New(), idset.New(), nil},
		{idset.New(1), nil, []id.Zid{1}},
		{nil, idset.New(1), nil},
		{idset.New(1), idset.New(), []id.Zid{1}},
		{idset.New(), idset.New(1), nil},
		{idset.New(1), idset.New(2), []id.Zid{1}},
		{idset.New(2), idset.New(1), []id.Zid{2}},
		{idset.New(1), idset.New(1), nil},
		{idset.New(1, 2, 3), idset.New(1), []id.Zid{2, 3}},
		{idset.New(1, 2, 3), idset.New(2), []id.Zid{1, 3}},
		{idset.New(1, 2, 3), idset.New(3), []id.Zid{1, 2}},
		{idset.New(1, 2, 3), idset.New(1, 2), []id.Zid{3}},
		{idset.New(1, 2, 3), idset.New(1, 3), []id.Zid{2}},
		{idset.New(1, 2, 3), idset.New(2, 3), []id.Zid{1}},
	}
	for i, tc := range testcases {
		s1 := tc.s1.Clone()
		sl1 := s1.SafeSorted()
		sl2 := tc.s2.SafeSorted()
		s1.ISubstract(tc.s2)
		got := s1.SafeSorted()
		if !slices.Equal(got, tc.exp) {
			t.Errorf("%d: %v.ISubstract(%v) should be %v, but got %v", i, sl1, sl2, tc.exp, got)
		}
	}
}

func TestSetDiff(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		in1, in2   *idset.Set
		exp1, exp2 *idset.Set
	}{
		{nil, nil, nil, nil},
		{idset.New(1), nil, nil, idset.New(1)},
		{nil, idset.New(1), idset.New(1), nil},
		{idset.New(1), idset.New(1), nil, nil},
		{idset.New(1, 2), idset.New(1), nil, idset.New(2)},
		{idset.New(1), idset.New(1, 2), idset.New(2), nil},
		{idset.New(1, 2), idset.New(1, 3), idset.New(3), idset.New(2)},
		{idset.New(1, 2, 3), idset.New(2, 3, 4), idset.New(4), idset.New(1)},
		{idset.New(2, 3, 4), idset.New(1, 2, 3), idset.New(1), idset.New(4)},
	}
	for i, tc := range testcases {
		gotN, gotO := tc.in1.Diff(tc.in2)
		if !tc.exp1.Equal(gotN) {
			t.Errorf("%d: expected %v, but got: %v", i, tc.exp1, gotN)
		}
		if !tc.exp2.Equal(gotO) {
			t.Errorf("%d: expected %v, but got: %v", i, tc.exp2, gotO)
		}
	}
}

func TestSetRemove(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		s1, s2 *idset.Set
		exp    []id.Zid
	}{
		{nil, nil, nil},
		{idset.New(), nil, nil},
		{idset.New(), idset.New(), nil},
		{idset.New(1), nil, []id.Zid{1}},
		{idset.New(1), idset.New(), []id.Zid{1}},
		{idset.New(1), idset.New(2), []id.Zid{1}},
		{idset.New(1), idset.New(1), []id.Zid{}},
	}
	for i, tc := range testcases {
		sl1 := tc.s1.SafeSorted()
		sl2 := tc.s2.SafeSorted()
		newS1 := idset.New(sl1...)
		newS1.ISubstract(tc.s2)
		got := newS1.SafeSorted()
		if !slices.Equal(got, tc.exp) {
			t.Errorf("%d: %v.Remove(%v) should be %v, but got %v", i, sl1, sl2, tc.exp, got)
		}
	}
}

func BenchmarkSet(b *testing.B) {
	s := idset.NewCap(b.N)
	for i := range b.N {
		s.Add(id.Zid(i))
	}
}

Added domain/meta/collection.go.



















































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

package meta

import (
	"slices"
	"strings"
)

// Arrangement stores metadata within its categories.
// Typecally a category might be a tag name, a role name, a syntax value.
type Arrangement map[string][]*Meta

// CreateArrangement by inspecting a given key and use the found
// value as a category.
func CreateArrangement(metaList []*Meta, key string) Arrangement {
	if len(metaList) == 0 {
		return nil
	}
	descr := Type(key)
	if descr == nil {
		return nil
	}
	if descr.IsSet {
		return createSetArrangement(metaList, key)
	}
	return createSimplearrangement(metaList, key)
}

func createSetArrangement(metaList []*Meta, key string) Arrangement {
	a := make(Arrangement)
	for _, m := range metaList {
		for val := range m.GetFields(key) {
			a[val] = append(a[val], m)
		}
	}
	return a
}

func createSimplearrangement(metaList []*Meta, key string) Arrangement {
	a := make(Arrangement)
	for _, m := range metaList {
		if val, ok := m.Get(key); ok && val != "" {
			a[string(val)] = append(a[string(val)], m)
		}
	}
	return a
}

// Counted returns the list of categories, together with the number of
// metadata for each category.
func (a Arrangement) Counted() CountedCategories {
	if len(a) == 0 {
		return nil
	}
	result := make(CountedCategories, 0, len(a))
	for cat, metas := range a {
		result = append(result, CountedCategory{Name: cat, Count: len(metas)})
	}
	return result
}

// CountedCategory contains of a name and the number how much this name occured
// somewhere.
type CountedCategory struct {
	Name  string
	Count int
}

// CountedCategories is the list of CountedCategories.
// Every name must occur only once.
type CountedCategories []CountedCategory

// SortByName sorts the list by the name attribute.
// Since each name must occur only once, two CountedCategories cannot have
// the same name.
func (ccs CountedCategories) SortByName() {
	slices.SortFunc(ccs, func(i, j CountedCategory) int { return strings.Compare(i.Name, j.Name) })
}

// SortByCount sorts the list by the count attribute, descending.
// If two counts are equal, elements are sorted by name.
func (ccs CountedCategories) SortByCount() {
	slices.SortFunc(ccs, func(i, j CountedCategory) int {
		iCount, jCount := i.Count, j.Count
		if iCount > jCount {
			return -1
		}
		if iCount == jCount {
			return strings.Compare(i.Name, j.Name)
		}
		return 1
	})
}

// Categories returns just the category names.
func (ccs CountedCategories) Categories() []string {
	result := make([]string, len(ccs))
	for i, cc := range ccs {
		result[i] = cc.Name
	}
	return result
}

Added domain/meta/meta.go.











































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package meta provides the zettel specific type 'meta'.
package meta

import (
	"iter"
	"maps"
	"regexp"
	"slices"
	"strings"
	"unicode"
	"unicode/utf8"

	"t73f.de/r/zero/set"
	"t73f.de/r/zsc/domain/id"
)

type keyUsage int

const (
	_             keyUsage = iota
	usageUser              // Key will be manipulated by the user
	usageComputed          // Key is computed by zettelstore
	usageProperty          // Key is computed and not stored by zettelstore
)

// DescriptionKey formally describes each supported metadata key.
type DescriptionKey struct {
	Name    string
	Type    *DescriptionType
	usage   keyUsage
	Inverse string
}

// IsComputed returns true, if metadata is computed and not set by the user.
func (kd *DescriptionKey) IsComputed() bool { return kd.usage >= usageComputed }

// IsProperty returns true, if metadata is a computed property.
func (kd *DescriptionKey) IsProperty() bool { return kd.usage >= usageProperty }

var registeredKeys = make(map[string]*DescriptionKey)

func registerKey(name string, t *DescriptionType, usage keyUsage, inverse string) {
	if _, ok := registeredKeys[name]; ok {
		panic("Key '" + name + "' already defined")
	}
	if inverse != "" {
		if t != TypeID && t != TypeIDSet {
			panic("Inversable key '" + name + "' is not identifier type, but " + t.String())
		}
		inv, ok := registeredKeys[inverse]
		if !ok {
			panic("Inverse Key '" + inverse + "' not found")
		}
		if !inv.IsComputed() {
			panic("Inverse Key '" + inverse + "' is not computed.")
		}
		if inv.Type != TypeIDSet {
			panic("Inverse Key '" + inverse + "' is not an identifier set, but " + inv.Type.String())
		}
	}
	registeredKeys[name] = &DescriptionKey{name, t, usage, inverse}
}

// IsComputed returns true, if key denotes a computed metadata key.
func IsComputed(name string) bool {
	if kd, ok := registeredKeys[name]; ok {
		return kd.IsComputed()
	}
	return false
}

// IsProperty returns true, if key denotes a property metadata value.
func IsProperty(name string) bool {
	if kd, ok := registeredKeys[name]; ok {
		return kd.IsProperty()
	}
	return false
}

// Inverse returns the name of the inverse key.
func Inverse(name string) string {
	if kd, ok := registeredKeys[name]; ok {
		return kd.Inverse
	}
	return ""
}

// GetDescription returns the key description object of the given key name.
func GetDescription(name string) DescriptionKey {
	if d, ok := registeredKeys[name]; ok {
		return *d
	}
	return DescriptionKey{Type: Type(name)}
}

// GetSortedKeyDescriptions delivers all metadata key descriptions as a slice, sorted by name.
func GetSortedKeyDescriptions() []*DescriptionKey {
	keys := slices.Sorted(maps.Keys(registeredKeys))
	result := make([]*DescriptionKey, 0, len(keys))
	for _, n := range keys {
		result = append(result, registeredKeys[n])
	}
	return result
}

// Key is the type of metadata keys.
type Key = string

// Predefined / supported metadata keys.
//
// See [Supported Metadata Keys].
//
// [Supported Metadata Keys]: https://zettelstore.de/manual/h/00001006020000
const (
	KeyID           = "id"
	KeyTitle        = "title"
	KeyRole         = "role"
	KeyTags         = "tags"
	KeySyntax       = "syntax"
	KeyAuthor       = "author"
	KeyBack         = "back"
	KeyBackward     = "backward"
	KeyBoxNumber    = "box-number"
	KeyCopyright    = "copyright"
	KeyCreated      = "created"
	KeyCredential   = "credential"
	KeyDead         = "dead"
	KeyExpire       = "expire"
	KeyFolge        = "folge"
	KeyFolgeRole    = "folge-role"
	KeyForward      = "forward"
	KeyLang         = "lang"
	KeyLicense      = "license"
	KeyModified     = "modified"
	KeyPrecursor    = "precursor"
	KeyPredecessor  = "predecessor"
	KeyPrequel      = "prequel"
	KeyPublished    = "published"
	KeyQuery        = "query"
	KeyReadOnly     = "read-only"
	KeySequel       = "sequel"
	KeySubordinates = "subordinates"
	KeySuccessors   = "successors"
	KeySummary      = "summary"
	KeySuperior     = "superior"
	KeyURL          = "url"
	KeyUselessFiles = "useless-files"
	KeyUserID       = "user-id"
	KeyUserRole     = "user-role"
	KeyVisibility   = "visibility"
)

// Supported keys.
func init() {
	registerKey(KeyID, TypeID, usageComputed, "")
	registerKey(KeyTitle, TypeEmpty, usageUser, "")
	registerKey(KeyRole, TypeWord, usageUser, "")
	registerKey(KeyTags, TypeTagSet, usageUser, "")
	registerKey(KeySyntax, TypeWord, usageUser, "")

	// Properties that are inverse keys
	registerKey(KeyFolge, TypeIDSet, usageProperty, "")
	registerKey(KeySequel, TypeIDSet, usageProperty, "")
	registerKey(KeySuccessors, TypeIDSet, usageProperty, "")
	registerKey(KeySubordinates, TypeIDSet, usageProperty, "")

	// Non-inverse keys
	registerKey(KeyAuthor, TypeString, usageUser, "")
	registerKey(KeyBack, TypeIDSet, usageProperty, "")
	registerKey(KeyBackward, TypeIDSet, usageProperty, "")
	registerKey(KeyBoxNumber, TypeNumber, usageProperty, "")
	registerKey(KeyCopyright, TypeString, usageUser, "")
	registerKey(KeyCreated, TypeTimestamp, usageComputed, "")
	registerKey(KeyCredential, TypeCredential, usageUser, "")
	registerKey(KeyDead, TypeIDSet, usageProperty, "")
	registerKey(KeyExpire, TypeTimestamp, usageUser, "")
	registerKey(KeyFolgeRole, TypeWord, usageUser, "")
	registerKey(KeyForward, TypeIDSet, usageProperty, "")
	registerKey(KeyLang, TypeWord, usageUser, "")
	registerKey(KeyLicense, TypeEmpty, usageUser, "")
	registerKey(KeyModified, TypeTimestamp, usageComputed, "")
	registerKey(KeyPrecursor, TypeIDSet, usageUser, KeyFolge)
	registerKey(KeyPredecessor, TypeID, usageUser, KeySuccessors)
	registerKey(KeyPrequel, TypeIDSet, usageUser, KeySequel)
	registerKey(KeyPublished, TypeTimestamp, usageProperty, "")
	registerKey(KeyQuery, TypeEmpty, usageUser, "")
	registerKey(KeyReadOnly, TypeWord, usageUser, "")
	registerKey(KeySummary, TypeString, usageUser, "")
	registerKey(KeySuperior, TypeIDSet, usageUser, KeySubordinates)
	registerKey(KeyURL, TypeURL, usageUser, "")
	registerKey(KeyUselessFiles, TypeString, usageProperty, "")
	registerKey(KeyUserID, TypeWord, usageUser, "")
	registerKey(KeyUserRole, TypeWord, usageUser, "")
	registerKey(KeyVisibility, TypeWord, usageUser, "")
}

// NewPrefix is the prefix for metadata keys in template zettel for creating new zettel.
const NewPrefix = "new-"

// Meta contains all meta-data of a zettel.
type Meta struct {
	Zid     id.Zid
	pairs   map[Key]Value
	YamlSep bool
}

// New creates a new chunk for storing metadata.
func New(zid id.Zid) *Meta {
	return &Meta{Zid: zid, pairs: make(map[Key]Value, 5)}
}

// NewWithData creates metadata object with given data.
func NewWithData(zid id.Zid, data map[string]string) *Meta {
	pairs := make(map[Key]Value, len(data))
	for k, v := range data {
		pairs[k] = Value(v)
	}
	return &Meta{Zid: zid, pairs: pairs}
}

// ByteSize returns the number of bytes stored for the metadata.
func (m *Meta) ByteSize() int {
	if m == nil {
		return 0
	}
	result := 6 // storage needed for Zid
	for k, v := range m.pairs {
		result += len(k) + len(v) + 1 // 1 because separator
	}
	return result
}

// Clone returns a new copy of the metadata.
func (m *Meta) Clone() *Meta {
	return &Meta{
		Zid:     m.Zid,
		pairs:   maps.Clone(m.pairs),
		YamlSep: m.YamlSep,
	}
}

// Map returns a copy of the meta data as a string map.
func (m *Meta) Map() map[string]string {
	pairs := make(map[string]string, len(m.pairs))
	for k, v := range m.pairs {
		pairs[k] = string(v)
	}
	return pairs
}

var reKey = regexp.MustCompile("^[0-9a-z][-0-9a-z]{0,254}$")

// KeyIsValid returns true, if the string is a valid metadata key.
func KeyIsValid(s string) bool { return reKey.MatchString(s) }

var firstKeys = []string{KeyTitle, KeyRole, KeyTags, KeySyntax}

// Set stores the given string value under the given key.
func (m *Meta) Set(key string, value Value) {
	if key != KeyID {
		m.pairs[key] = value.TrimSpace()
	}
}

// SetNonEmpty stores the given value under the given key, if the value is non-empty.
// An empty value will delete the previous association.
func (m *Meta) SetNonEmpty(key string, value Value) {
	if value == "" {
		delete(m.pairs, key) // TODO: key != KeyID
	} else {
		m.Set(key, value.TrimSpace())
	}
}

// Get retrieves the string value of a given key. The bool value signals,
// whether there was a value stored or not.
func (m *Meta) Get(key string) (Value, bool) {
	if m == nil {
		return "", false
	}
	if key == KeyID {
		return Value(m.Zid.String()), true
	}
	value, ok := m.pairs[key]
	return value, ok
}

// GetDefault retrieves the string value of the given key. If no value was
// stored, the given default value is returned.
func (m *Meta) GetDefault(key string, def Value) Value {
	if value, found := m.Get(key); found {
		return value
	}
	return def
}

// GetTitle returns the title of the metadata. It is the only key that has a
// defined default value: the string representation of the zettel identifier.
func (m *Meta) GetTitle() string {
	if title, found := m.Get(KeyTitle); found {
		return string(title)
	}
	return m.Zid.String()
}

// All returns an iterator over all key/value pairs, except the zettel identifier
// and computed values.
func (m *Meta) All() iter.Seq2[Key, Value] {
	return func(yield func(Key, Value) bool) {
		m.firstKeys()(yield)
		m.restKeys(notComputedKey)(yield)
	}
}

// Computed returns an iterator over all key/value pairs, except the zettel identifier.
func (m *Meta) Computed() iter.Seq2[Key, Value] {
	return func(yield func(Key, Value) bool) {
		m.firstKeys()(yield)
		m.restKeys(anyKey)(yield)
	}
}

// Rest returns an iterator over all key/value pairs, except the zettel identifier,
// the main keys, and computed values.
func (m *Meta) Rest() iter.Seq2[Key, Value] {
	return func(yield func(Key, Value) bool) {
		m.restKeys(notComputedKey)(yield)
	}
}

// ComputedRest returns an iterator over all key/value pairs, except the zettel identifier,
// and the main keys.
func (m *Meta) ComputedRest() iter.Seq2[Key, Value] {
	return func(yield func(Key, Value) bool) {
		m.restKeys(anyKey)(yield)
	}
}

func (m *Meta) firstKeys() iter.Seq2[Key, Value] {
	return func(yield func(Key, Value) bool) {
		for _, key := range firstKeys {
			if val, ok := m.pairs[key]; ok {
				if !yield(key, val) {
					return
				}
			}
		}
	}
}

func (m *Meta) restKeys(addKeyPred func(Key) bool) iter.Seq2[Key, Value] {
	return func(yield func(Key, Value) bool) {
		keys := slices.Sorted(maps.Keys(m.pairs))
		for _, key := range keys {
			if !slices.Contains(firstKeys, key) && addKeyPred(key) {
				if !yield(key, m.pairs[key]) {
					return
				}
			}
		}
	}
}

func notComputedKey(key string) bool { return !IsComputed(key) }
func anyKey(string) bool             { return true }

// Delete removes a key from the data.
func (m *Meta) Delete(key string) {
	if key != KeyID {
		delete(m.pairs, key)
	}
}

// Equal compares to metas for equality.
func (m *Meta) Equal(o *Meta, allowComputed bool) bool {
	if m == nil && o == nil {
		return true
	}
	if m == nil || o == nil || m.Zid != o.Zid {
		return false
	}
	tested := set.New[string]()
	for k, v := range m.pairs {
		tested.Add(k)
		if !equalValue(k, v, o, allowComputed) {
			return false
		}
	}
	for k, v := range o.pairs {
		if !tested.Contains(k) && !equalValue(k, v, m, allowComputed) {
			return false
		}
	}
	return true
}

func equalValue(key string, val Value, other *Meta, allowComputed bool) bool {
	if allowComputed || !IsComputed(key) {
		if valO, found := other.pairs[key]; !found || val != valO {
			return false
		}
	}
	return true
}

// Sanitize all metadata keys and values, so that they can be written safely into a file.
func (m *Meta) Sanitize() {
	if m == nil {
		return
	}
	for key, val := range m.pairs {
		newKey := RemoveNonGraphic(key)
		if key == newKey {
			m.pairs[key] = Value(RemoveNonGraphic(string(val)))
		} else {
			delete(m.pairs, key)
			m.pairs[newKey] = Value(RemoveNonGraphic(string(val)))
		}
	}
}

// RemoveNonGraphic changes the given string not to include non-graphical characters.
// It is needed to sanitize meta data.
func RemoveNonGraphic(s string) string {
	if s == "" {
		return ""
	}
	pos := 0
	var sb strings.Builder
	for pos < len(s) {
		nextPos := strings.IndexFunc(s[pos:], func(r rune) bool { return !unicode.IsGraphic(r) })
		if nextPos < 0 {
			break
		}
		sb.WriteString(s[pos:nextPos])
		sb.WriteByte(' ')
		_, size := utf8.DecodeRuneInString(s[nextPos:])
		pos = nextPos + size
	}
	if pos == 0 {
		return strings.TrimSpace(s)
	}
	sb.WriteString(s[pos:])
	return strings.TrimSpace(sb.String())
}

Added domain/meta/meta_test.go.











































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta

import (
	"iter"
	"slices"
	"strings"
	"testing"

	"t73f.de/r/zsc/domain/id"
)

const testID = id.Zid(98765432101234)

func TestKeyIsValid(t *testing.T) {
	t.Parallel()
	validKeys := []string{"0", "a", "0-", "title", "title-----", strings.Repeat("r", 255)}
	for _, key := range validKeys {
		if !KeyIsValid(key) {
			t.Errorf("Key %q wrongly identified as invalid key", key)
		}
	}
	invalidKeys := []string{"", "-", "-a", "Title", "a_b", strings.Repeat("e", 256)}
	for _, key := range invalidKeys {
		if KeyIsValid(key) {
			t.Errorf("Key %q wrongly identified as valid key", key)
		}
	}
}

func TestTitleHeader(t *testing.T) {
	t.Parallel()
	m := New(testID)
	if got, ok := m.Get(KeyTitle); ok && got != "" {
		t.Errorf("Title is not empty, but %q", got)
	}
	addToMeta(m, KeyTitle, " ")
	if got, ok := m.Get(KeyTitle); ok && got != "" {
		t.Errorf("Title is not empty, but %q", got)
	}
	const st = "A simple text"
	addToMeta(m, KeyTitle, " "+st+"  ")
	if got, ok := m.Get(KeyTitle); !ok || got != st {
		t.Errorf("Title is not %q, but %q", st, got)
	}
	addToMeta(m, KeyTitle, "  "+st+"\t")
	const exp = st + " " + st
	if got, ok := m.Get(KeyTitle); !ok || got != exp {
		t.Errorf("Title is not %q, but %q", exp, got)
	}

	m = New(testID)
	const at = "A Title"
	addToMeta(m, KeyTitle, at)
	addToMeta(m, KeyTitle, " ")
	if got, ok := m.Get(KeyTitle); !ok || got != at {
		t.Errorf("Title is not %q, but %q", at, got)
	}
}

func checkTags(t *testing.T, exp []string, m *Meta) {
	t.Helper()
	got := slices.Collect(m.GetFields(KeyTags))
	for i, tag := range exp {
		if i < len(got) {
			if tag != got[i] {
				t.Errorf("Pos=%d, expected %q, got %q", i, exp[i], got[i])
			}
		} else {
			t.Errorf("Expected %q, but is missing", exp[i])
		}
	}
	if len(exp) < len(got) {
		t.Errorf("Extra tags: %q", got[len(exp):])
	}
}

func TestTagsHeader(t *testing.T) {
	t.Parallel()
	m := New(testID)
	checkTags(t, []string{}, m)

	addToMeta(m, KeyTags, "")
	checkTags(t, []string{}, m)

	addToMeta(m, KeyTags, "  #t1 #t2  #t3 #t4  ")
	checkTags(t, []string{"#t1", "#t2", "#t3", "#t4"}, m)

	addToMeta(m, KeyTags, "#t5")
	checkTags(t, []string{"#t1", "#t2", "#t3", "#t4", "#t5"}, m)

	addToMeta(m, KeyTags, "t6")
	checkTags(t, []string{"#t1", "#t2", "#t3", "#t4", "#t5"}, m)
}

func TestSyntax(t *testing.T) {
	t.Parallel()
	m := New(testID)
	if got, ok := m.Get(KeySyntax); ok || got != "" {
		t.Errorf("Syntax is not %q, but %q", "", got)
	}
	addToMeta(m, KeySyntax, " ")
	if got, _ := m.Get(KeySyntax); got != "" {
		t.Errorf("Syntax is not %q, but %q", "", got)
	}
	addToMeta(m, KeySyntax, "MarkDown")
	const exp = "markdown"
	if got, ok := m.Get(KeySyntax); !ok || got != exp {
		t.Errorf("Syntax is not %q, but %q", exp, got)
	}
	addToMeta(m, KeySyntax, " ")
	if got, _ := m.Get(KeySyntax); got != "" {
		t.Errorf("Syntax is not %q, but %q", "", got)
	}
}

func checkHeader(t *testing.T, exp map[string]string, gotI iter.Seq2[Key, Value]) {
	t.Helper()
	got := make(map[string]string)
	gotI(func(key Key, val Value) bool {
		got[key] = string(val)
		if _, ok := exp[key]; !ok {
			t.Errorf("Key %q is not expected, but has value %q", key, val)
		}
		return true
	})
	for k, v := range exp {
		if gv, ok := got[k]; !ok || v != gv {
			if ok {
				t.Errorf("Key %q is not %q, but %q", k, v, got[k])
			} else {
				t.Errorf("Key %q missing, should have value %q", k, v)
			}
		}
	}
}

func TestDefaultHeader(t *testing.T) {
	t.Parallel()
	m := New(testID)
	addToMeta(m, "h1", "d1")
	addToMeta(m, "H2", "D2")
	addToMeta(m, "H1", "D1.1")
	exp := map[string]string{"h1": "d1 D1.1", "h2": "D2"}
	checkHeader(t, exp, m.All())
	addToMeta(m, "", "d0")
	checkHeader(t, exp, m.All())
	addToMeta(m, "h3", "")
	exp["h3"] = ""
	checkHeader(t, exp, m.All())
	addToMeta(m, "h3", "  ")
	checkHeader(t, exp, m.All())
	addToMeta(m, "h4", " ")
	exp["h4"] = ""
	checkHeader(t, exp, m.All())
}

func TestDelete(t *testing.T) {
	t.Parallel()
	m := New(testID)
	m.Set("key", "val")
	if got, ok := m.Get("key"); !ok || got != "val" {
		t.Errorf("Value != %q, got: %v/%q", "val", ok, got)
	}
	m.Set("key", "")
	if got, ok := m.Get("key"); !ok || got != "" {
		t.Errorf("Value != %q, got: %v/%q", "", ok, got)
	}
	m.Delete("key")
	if got, ok := m.Get("key"); ok || got != "" {
		t.Errorf("Value != %q, got: %v/%q", "", ok, got)
	}
}

func TestEqual(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		pairs1, pairs2 []string
		allowComputed  bool
		exp            bool
	}{
		{nil, nil, true, true},
		{nil, nil, false, true},
		{[]string{"a", "a"}, nil, false, false},
		{[]string{"a", "a"}, nil, true, false},
		{[]string{KeyFolge, "0"}, nil, true, false},
		{[]string{KeyFolge, "0"}, nil, false, true},
		{[]string{KeyFolge, "0"}, []string{KeyFolge, "0"}, true, true},
		{[]string{KeyFolge, "0"}, []string{KeyFolge, "0"}, false, true},
	}
	for i, tc := range testcases {
		m1 := pairs2meta(tc.pairs1)
		m2 := pairs2meta(tc.pairs2)
		got := m1.Equal(m2, tc.allowComputed)
		if tc.exp != got {
			t.Errorf("%d: %v =?= %v: expected=%v, but got=%v", i, tc.pairs1, tc.pairs2, tc.exp, got)
		}
		got = m2.Equal(m1, tc.allowComputed)
		if tc.exp != got {
			t.Errorf("%d: %v =!= %v: expected=%v, but got=%v", i, tc.pairs1, tc.pairs2, tc.exp, got)
		}
	}

	// Pathologic cases
	var m1, m2 *Meta
	if !m1.Equal(m2, true) {
		t.Error("Nil metas should be treated equal")
	}
	m1 = New(testID)
	if m1.Equal(m2, true) {
		t.Error("Empty meta should not be equal to nil")
	}
	if m2.Equal(m1, true) {
		t.Error("Nil meta should should not be equal to empty")
	}
	m2 = New(testID + 1)
	if m1.Equal(m2, true) {
		t.Error("Different ID should differentiate")
	}
	if m2.Equal(m1, true) {
		t.Error("Different ID should differentiate")
	}
}

func pairs2meta(pairs []string) *Meta {
	m := New(testID)
	for i := 0; i < len(pairs); i += 2 {
		m.Set(pairs[i], Value(pairs[i+1]))
	}
	return m
}

func TestRemoveNonGraphic(t *testing.T) {
	testCases := []struct {
		inp string
		exp string
	}{
		{"", ""},
		{" ", ""},
		{"a", "a"},
		{"a ", "a"},
		{"a b", "a b"},
		{"\n", ""},
		{"a\n", "a"},
		{"a\nb", "a b"},
		{"a\tb", "a b"},
	}
	for i, tc := range testCases {
		got := RemoveNonGraphic(tc.inp)
		if tc.exp != got {
			t.Errorf("%q/%d: expected %q, but got %q", tc.inp, i, tc.exp, got)
		}
	}
}

Added domain/meta/parse.go.










































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta

import (
	"iter"
	"slices"
	"strings"

	"t73f.de/r/zero/set"
	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsx/input"
)

// NewFromInput parses the meta data of a zettel.
func NewFromInput(zid id.Zid, inp *input.Input) *Meta {
	if inp.Ch == '-' && inp.PeekN(0) == '-' && inp.PeekN(1) == '-' {
		skipToEOL(inp)
		inp.EatEOL()
	}
	meta := New(zid)
	for {
		inp.SkipSpace()
		switch inp.Ch {
		case '\r':
			if inp.Peek() == '\n' {
				inp.Next()
			}
			fallthrough
		case '\n':
			inp.Next()
			return meta
		case input.EOS:
			return meta
		case '%':
			skipToEOL(inp)
			inp.EatEOL()
			continue
		}
		parseHeader(meta, inp)
		if inp.Ch == '-' && inp.PeekN(0) == '-' && inp.PeekN(1) == '-' {
			skipToEOL(inp)
			inp.EatEOL()
			meta.YamlSep = true
			return meta
		}
	}
}

func parseHeader(m *Meta, inp *input.Input) {
	pos := inp.Pos
	for isHeader(inp.Ch) {
		inp.Next()
	}
	key := inp.Src[pos:inp.Pos]
	inp.SkipSpace()
	if inp.Ch == ':' {
		inp.Next()
	}
	var val []byte
	for {
		inp.SkipSpace()
		pos = inp.Pos
		skipToEOL(inp)
		val = append(val, inp.Src[pos:inp.Pos]...)
		inp.EatEOL()
		if !inp.IsSpace() {
			break
		}
		val = append(val, ' ')
	}
	addToMeta(m, string(key), Value(val))
}

func skipToEOL(inp *input.Input) {
	for {
		switch inp.Ch {
		case '\n', '\r', input.EOS:
			return
		}
		inp.Next()
	}
}

// Return true iff rune is valid for header key.
func isHeader(ch rune) bool {
	return ('a' <= ch && ch <= 'z') ||
		('0' <= ch && ch <= '9') ||
		ch == '-' ||
		('A' <= ch && ch <= 'Z')
}

type predValidElem func(string) bool

func addToSet(set *set.Set[string], it iter.Seq[string], useElem predValidElem) {
	for e := range it {
		if len(e) > 0 && useElem(e) {
			set.Add(e)
		}
	}
}

func addSet(m *Meta, key string, val Value, useElem predValidElem) {
	newElems := val.Fields()
	oldElems := m.GetFields(key)

	s := set.New[string]()
	addToSet(s, newElems, useElem)
	if s.Length() == 0 {
		// Nothing to add. Maybe because of rejected elements.
		return
	}
	addToSet(s, oldElems, useElem)
	m.SetList(key, slices.Sorted(s.Values()))
}

func addData(m *Meta, k string, v Value) {
	if o, ok := m.Get(k); !ok || o == "" {
		m.Set(k, v)
	} else if v != "" {
		m.Set(k, o+" "+v)
	}
}

func addToMeta(m *Meta, key string, val Value) {
	v := val.TrimSpace()
	key = strings.ToLower(key)
	if !KeyIsValid(key) {
		return
	}
	switch key {
	case "", KeyID:
		// Empty key and 'id' key will be ignored
		return
	}

	switch Type(key) {
	case TypeTagSet:
		addSet(m, key, v.ToLower(), func(s string) bool { return s[0] == '#' && len(s) > 1 })
	case TypeWord:
		m.Set(key, v.ToLower())
	case TypeID:
		if _, err := id.Parse(string(v)); err == nil {
			m.Set(key, v)
		}
	case TypeIDSet:
		addSet(m, key, v, func(s string) bool {
			_, err := id.Parse(s)
			return err == nil
		})
	case TypeTimestamp:
		if _, ok := v.AsTime(); ok {
			m.Set(key, v)
		}
	default:
		addData(m, key, v)
	}
}

Added domain/meta/parse_test.go.





























































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta_test

import (
	"iter"
	"slices"
	"strings"
	"testing"

	"t73f.de/r/zsc/domain/meta"
	"t73f.de/r/zsx/input"
)

func parseMetaStr(src string) *meta.Meta {
	return meta.NewFromInput(testID, input.NewInput([]byte(src)))
}

func TestEmpty(t *testing.T) {
	t.Parallel()
	m := parseMetaStr("")
	if got, ok := m.Get(meta.KeySyntax); ok || got != "" {
		t.Errorf("Syntax is not %q, but %q", "", got)
	}
	if got := slices.Collect(m.GetDefault(meta.KeyTags, "").Fields()); len(got) > 0 {
		t.Errorf("Tags are not nil, but %v", got)
	}
}

func TestTitle(t *testing.T) {
	t.Parallel()
	td := []struct {
		s string
		e meta.Value
	}{
		{meta.KeyTitle + ": a title", "a title"},
		{meta.KeyTitle + ": a\n\t title", "a title"},
		{meta.KeyTitle + ": a\n\t title\r\n  x", "a title x"},
		{meta.KeyTitle + " AbC", "AbC"},
		{meta.KeyTitle + " AbC\n ded", "AbC ded"},
		{meta.KeyTitle + ": o\ntitle: p", "o p"},
		{meta.KeyTitle + ": O\n\ntitle: P", "O"},
		{meta.KeyTitle + ": b\r\ntitle: c", "b c"},
		{meta.KeyTitle + ": B\r\n\r\ntitle: C", "B"},
		{meta.KeyTitle + ": r\rtitle: q", "r q"},
		{meta.KeyTitle + ": R\r\rtitle: Q", "R"},
	}
	for i, tc := range td {
		m := parseMetaStr(tc.s)
		if got, ok := m.Get(meta.KeyTitle); !ok || got != tc.e {
			t.Log(m)
			t.Errorf("TC=%d: expected %q, got %q", i, tc.e, got)
		}
	}
}

func TestTags(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		src string
		exp string
	}{
		{"", ""},
		{meta.KeyTags + ":", ""},
		{meta.KeyTags + ": c", ""},
		{meta.KeyTags + ": #", ""},
		{meta.KeyTags + ": #c", "c"},
		{meta.KeyTags + ": #c #", "c"},
		{meta.KeyTags + ": #c #b", "b c"},
		{meta.KeyTags + ": #c # #", "c"},
		{meta.KeyTags + ": #c # #b", "b c"},
	}
	for i, tc := range testcases {
		m := parseMetaStr(tc.src)
		tagsString, found := m.Get(meta.KeyTags)
		if !found {
			if tc.exp != "" {
				t.Errorf("%d / %q: no %s found", i, tc.src, meta.KeyTags)
			}
			continue
		}
		tags := tagsString.AsTags()
		if tc.exp == "" && len(tags) > 0 {
			t.Errorf("%d / %q: expected no %s, but got %v", i, tc.src, meta.KeyTags, tags)
			continue
		}
		got := strings.Join(tags, " ")
		if tc.exp != got {
			t.Errorf("%d / %q: expected %q, got: %q", i, tc.src, tc.exp, got)
		}
	}
}

func TestNewFromInput(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		input string
		exp   []pair
	}{
		{"", []pair{}},
		{" a:b", []pair{{"a", "b"}}},
		{"%a:b", []pair{}},
		{"a:b\r\n\r\nc:d", []pair{{"a", "b"}}},
		{"a:b\r\n%c:d", []pair{{"a", "b"}}},
		{"% a:b\r\n c:d", []pair{{"c", "d"}}},
		{"---\r\na:b\r\n", []pair{{"a", "b"}}},
		{"---\r\na:b\r\n--\r\nc:d", []pair{{"a", "b"}, {"c", "d"}}},
		{"---\r\na:b\r\n---\r\nc:d", []pair{{"a", "b"}}},
		{"---\r\na:b\r\n----\r\nc:d", []pair{{"a", "b"}}},
		{"new-title:\nnew-url:", []pair{{"new-title", ""}, {"new-url", ""}}},
	}
	for i, tc := range testcases {
		meta := parseMetaStr(tc.input)
		if got := iter2pairs(meta.All()); !equalPairs(tc.exp, got) {
			t.Errorf("TC=%d: expected=%v, got=%v", i, tc.exp, got)
		}
	}

	// Test, whether input position is correct.
	inp := input.NewInput([]byte("---\na:b\n---\nX"))
	m := meta.NewFromInput(testID, inp)
	exp := []pair{{"a", "b"}}
	if got := iter2pairs(m.All()); !equalPairs(exp, got) {
		t.Errorf("Expected=%v, got=%v", exp, got)
	}
	expCh := 'X'
	if gotCh := inp.Ch; gotCh != expCh {
		t.Errorf("Expected=%v, got=%v", expCh, gotCh)
	}
}

type pair struct {
	key meta.Key
	val meta.Value
}

func iter2pairs(it iter.Seq2[meta.Key, meta.Value]) (result []pair) {
	it(func(key meta.Key, val meta.Value) bool {
		result = append(result, pair{key, val})
		return true
	})
	return result
}

func equalPairs(one, two []pair) bool {
	if len(one) != len(two) {
		return false
	}
	for i := range len(one) {
		if one[i].key != two[i].key || one[i].val != two[i].val {
			return false
		}
	}
	return true
}

func TestPrecursorIDSet(t *testing.T) {
	t.Parallel()
	var testdata = []struct {
		inp string
		exp meta.Value
	}{
		{"", ""},
		{"123", ""},
		{"12345678901234", "12345678901234"},
		{"123 12345678901234", "12345678901234"},
		{"12345678901234 123", "12345678901234"},
		{"01234567890123 123 12345678901234", "01234567890123 12345678901234"},
		{"12345678901234 01234567890123", "01234567890123 12345678901234"},
	}
	for i, tc := range testdata {
		m := parseMetaStr(meta.KeyPrecursor + ": " + tc.inp)
		if got, ok := m.Get(meta.KeyPrecursor); (!ok && tc.exp != "") || tc.exp != got {
			t.Errorf("TC=%d: expected %q, but got %q when parsing %q", i, tc.exp, got, tc.inp)
		}
	}
}

Added domain/meta/type.go.






















































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta

import (
	"iter"
	"strconv"
	"strings"
	"sync"
	"time"

	zeroiter "t73f.de/r/zero/iter"
	"t73f.de/r/zsc/domain/id"
)

// DescriptionType is a description of a specific key type.
type DescriptionType struct {
	Name  string
	IsSet bool
}

// String returns the string representation of the given type
func (t DescriptionType) String() string { return t.Name }

var registeredTypes = make(map[string]*DescriptionType)

func registerType(name string, isSet bool) *DescriptionType {
	if _, ok := registeredTypes[name]; ok {
		panic("Type '" + name + "' already registered")
	}
	t := &DescriptionType{name, isSet}
	registeredTypes[name] = t
	return t
}

// Values of the metadata key/value types.
//
// See [Supported Key Types].
//
// [Supported Key Types]: https://zettelstore.de/manual/h/00001006030000
const (
	MetaCredential = "Credential"
	MetaEmpty      = "EString"
	MetaID         = "Identifier"
	MetaIDSet      = "IdentifierSet"
	MetaNumber     = "Number"
	MetaString     = "String"
	MetaTagSet     = "TagSet"
	MetaTimestamp  = "Timestamp"
	MetaURL        = "URL"
	MetaWord       = "Word"
)

// Supported key types.
var (
	TypeCredential = registerType(MetaCredential, false)
	TypeEmpty      = registerType(MetaEmpty, false)
	TypeID         = registerType(MetaID, false)
	TypeIDSet      = registerType(MetaIDSet, true)
	TypeNumber     = registerType(MetaNumber, false)
	TypeString     = registerType(MetaString, false)
	TypeTagSet     = registerType(MetaTagSet, true)
	TypeTimestamp  = registerType(MetaTimestamp, false)
	TypeURL        = registerType(MetaURL, false)
	TypeWord       = registerType(MetaWord, false)
)

// Type returns a type hint for the given key. If no type hint is specified,
// TypeUnknown is returned.
func (*Meta) Type(key string) *DescriptionType {
	return Type(key)
}

// Some constants for key suffixes that determine a type.
const (
	SuffixKeyRole = "-role"
	SuffixKeyURL  = "-url"
)

var (
	cachedTypedKeys = make(map[string]*DescriptionType)
	mxTypedKey      sync.RWMutex
	suffixTypes     = map[string]*DescriptionType{
		"-date":       TypeTimestamp,
		"-number":     TypeNumber,
		SuffixKeyRole: TypeWord,
		"-time":       TypeTimestamp,
		SuffixKeyURL:  TypeURL,
		"-zettel":     TypeID,
		"-zid":        TypeID,
		"-zids":       TypeIDSet,
	}
)

// Type returns a type hint for the given key. If no type hint is specified,
// TypeEmpty is returned.
func Type(key string) *DescriptionType {
	if k, ok := registeredKeys[key]; ok {
		return k.Type
	}
	mxTypedKey.RLock()
	k, found := cachedTypedKeys[key]
	mxTypedKey.RUnlock()
	if found {
		return k
	}

	for suffix, t := range suffixTypes {
		if strings.HasSuffix(key, suffix) {
			mxTypedKey.Lock()
			defer mxTypedKey.Unlock()
			// Double check to avoid races
			if _, found = cachedTypedKeys[key]; !found {
				cachedTypedKeys[key] = t
			}
			return t
		}
	}
	return TypeEmpty
}

// SetList stores the given string list value under the given key.
func (m *Meta) SetList(key string, values []string) {
	if key != KeyID {
		for i, val := range values {
			values[i] = string(Value(val).TrimSpace())
		}
		m.pairs[key] = Value(strings.Join(values, " "))
	}
}

// SetWord stores the given word under the given key.
func (m *Meta) SetWord(key, word string) {
	for val := range Value(word).Elems() {
		m.Set(key, val)
		return
	}
}

// SetNow stores the current timestamp under the given key.
func (m *Meta) SetNow(key string) {
	m.Set(key, Value(time.Now().Local().Format(id.TimestampLayout)))
}

// GetBool returns the boolean value of the given key.
func (m *Meta) GetBool(key string) bool {
	if val, ok := m.Get(key); ok {
		return val.AsBool()
	}
	return false
}

// GetFields returns the metadata value as a sequence of string. The bool value
// signals, whether there was a value stored or not.
func (m *Meta) GetFields(key Key) iter.Seq[string] {
	if val, ok := m.Get(key); ok {
		return val.Fields()
	}
	return zeroiter.EmptySeq[string]()
}

// GetNumber retrieves the numeric value of a given key.
func (m *Meta) GetNumber(key string, def int64) int64 {
	if value, ok := m.Get(key); ok {
		if num, err := strconv.ParseInt(string(value), 10, 64); err == nil {
			return num
		}
	}
	return def
}

Added domain/meta/type_test.go.
















































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta_test

import (
	"strconv"
	"testing"
	"time"

	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsc/domain/meta"
)

func TestNow(t *testing.T) {
	t.Parallel()
	m := meta.New(id.Invalid)
	m.SetNow("key")
	val, ok := m.Get("key")
	if !ok {
		t.Error("Unable to get value of key")
	}
	if len(val) != 14 {
		t.Errorf("Value is not 14 digits long: %q", val)
	}
	if _, err := strconv.ParseInt(string(val), 10, 64); err != nil {
		t.Errorf("Unable to parse %q as an int64: %v", val, err)
	}
	if _, ok = val.AsTime(); !ok {
		t.Errorf("Unable to get time from value %q", val)
	}
}

func TestTimeValue(t *testing.T) {
	t.Parallel()
	testCases := []struct {
		value meta.Value
		valid bool
		exp   time.Time
	}{
		{"", false, time.Time{}},
		{"1", false, time.Time{}},
		{"00000000000000", false, time.Time{}},
		{"98765432109876", false, time.Time{}},
		{"20201221111905", true, time.Date(2020, time.December, 21, 11, 19, 5, 0, time.UTC)},
		{"2023", true, time.Date(2023, time.January, 1, 0, 0, 0, 0, time.UTC)},
		{"20231", false, time.Time{}},
		{"202310", true, time.Date(2023, time.October, 1, 0, 0, 0, 0, time.UTC)},
		{"2023103", false, time.Time{}},
		{"20231030", true, time.Date(2023, time.October, 30, 0, 0, 0, 0, time.UTC)},
		{"202310301", false, time.Time{}},
		{"2023103016", true, time.Date(2023, time.October, 30, 16, 0, 0, 0, time.UTC)},
		{"20231030165", false, time.Time{}},
		{"202310301654", true, time.Date(2023, time.October, 30, 16, 54, 0, 0, time.UTC)},
		{"2023103016541", false, time.Time{}},
		{"20231030165417", true, time.Date(2023, time.October, 30, 16, 54, 17, 0, time.UTC)},
		{"2023103916541700", false, time.Time{}},
	}
	for i, tc := range testCases {
		got, ok := tc.value.AsTime()
		if ok != tc.valid {
			t.Errorf("%d: parsing of %q should be %v, but got %v", i, tc.value, tc.valid, ok)
			continue
		}
		if got != tc.exp {
			t.Errorf("%d: parsing of %q should return %v, but got %v", i, tc.value, tc.exp, got)
		}
	}
}

Added domain/meta/values.go.


































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta

import (
	"fmt"
	"iter"
	"slices"
	"strings"
	"time"

	zeroiter "t73f.de/r/zero/iter"
	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsx/input"
)

// Value ist a single metadata value.
type Value string

// AsBool returns the value interpreted as a bool.
func (val Value) AsBool() bool {
	if len(val) > 0 {
		switch val[0] {
		case '0', 'f', 'F', 'n', 'N':
			return false
		}
	}
	return true
}

// AsTime returns the time value of the given value.
func (val Value) AsTime() (time.Time, bool) {
	if t, err := time.Parse(id.TimestampLayout, ExpandTimestamp(val)); err == nil {
		return t, true
	}
	return time.Time{}, false
}

// ExpandTimestamp makes a short-form timestamp larger.
func ExpandTimestamp(val Value) string {
	switch l := len(val); l {
	case 4: // YYYY
		return string(val) + "0101000000"
	case 6: // YYYYMM
		return string(val) + "01000000"
	case 8, 10, 12: // YYYYMMDD, YYYYMMDDhh, YYYYMMDDhhmm
		return string(val) + "000000"[:14-l]
	case 14: // YYYYMMDDhhmmss
		return string(val)
	default:
		if l > 14 {
			return string(val[:14])
		}
		return string(val)
	}
}

// Fields iterates over the value as a list/set of strings.
func (val Value) Fields() iter.Seq[string] {
	return strings.FieldsSeq(string(val))
}

// Elems iterates over the value as a list/set of values.
func (val Value) Elems() iter.Seq[Value] {
	return zeroiter.MapSeq(val.Fields(), func(s string) Value { return Value(s) })
}

// AsSlice transforms a value into a slice of strings.
func (val Value) AsSlice() []string {
	return strings.Fields(string(val))
}

// ToLower maps the value to lowercase runes.
func (val Value) ToLower() Value { return Value(strings.ToLower(string(val))) }

// TrimSpace removes all leading and remaining space from value
func (val Value) TrimSpace() Value {
	return Value(strings.TrimFunc(string(val), input.IsSpace))
}

// AsTags returns the value as a sequence of normalized tags.
func (val Value) AsTags() []string {
	return slices.Collect(zeroiter.MapSeq(
		val.Fields(),
		func(e string) string { return string(Value(e).ToLower().CleanTag()) }))
}

// CleanTag removes the number character ('#') from a tag value.
func (val Value) CleanTag() Value {
	if len(val) > 1 && val[0] == '#' {
		return val[1:]
	}
	return val
}

// NormalizeTag adds a missing prefix "#" to the tag
func (val Value) NormalizeTag() Value {
	if len(val) > 0 && val[0] == '#' {
		return val
	}
	return "#" + val
}

// Predefined metadata values.
const (
	ValueFalse             = "false"
	ValueTrue              = "true"
	ValueLangEN            = "en"            // Default for "lang"
	ValueRoleConfiguration = "configuration" // A role for internal zettel
	ValueRoleTag           = "tag"           // A role for tag zettel
	ValueRoleRole          = "role"          // A role for role zettel
	ValueRoleZettel        = "zettel"        // A role for zettel
	ValueSyntaxCSS         = "css"           // Syntax: CSS
	ValueSyntaxDraw        = "draw"          // Syntax: Drawing
	ValueSyntaxGif         = "gif"           // Syntax: GIF image
	ValueSyntaxHTML        = "html"          // Syntax: HTML
	ValueSyntaxJPEG        = "jpeg"          // Syntax: JPEG image
	ValueSyntaxJPG         = "jpg"           // Syntax: PEG image
	ValueSyntaxMarkdown    = "markdown"      // Syntax: Markdown / CommonMark
	ValueSyntaxMD          = "md"            // Syntax: Markdown / CommonMark
	ValueSyntaxNone        = "none"          // Syntax: no syntax / content, just metadata
	ValueSyntaxPlain       = "plain"         // Syntax: plain text
	ValueSyntaxPNG         = "png"           // Syntax: PNG image
	ValueSyntaxSVG         = "svg"           // Syntax: SVG
	ValueSyntaxSxn         = "sxn"           // Syntax: S-Expression
	ValueSyntaxText        = "text"          // Syntax: plain text
	ValueSyntaxTxt         = "txt"           // Syntax: plain text
	ValueSyntaxWebp        = "webp"          // Syntax: WEBP image
	ValueSyntaxZmk         = "zmk"           // Syntax: Zettelmarkup
	ValueUserRoleCreator   = "creator"
	ValueUserRoleOwner     = "owner"
	ValueUserRoleReader    = "reader"
	ValueUserRoleWriter    = "writer"
	ValueVisibilityCreator = "creator"
	ValueVisibilityExpert  = "expert"
	ValueVisibilityLogin   = "login"
	ValueVisibilityOwner   = "owner"
	ValueVisibilityPublic  = "public"
)

// DefaultSyntax is the default value for metadata 'syntax'.
const DefaultSyntax = ValueSyntaxPlain

// Visibility enumerates the variations of the 'visibility' meta key.
type Visibility int

// Supported values for visibility.
const (
	_ Visibility = iota
	VisibilityUnknown
	VisibilityPublic
	VisibilityCreator
	VisibilityLogin
	VisibilityOwner
	VisibilityExpert
)

var visMap = map[Value]Visibility{
	ValueVisibilityPublic:  VisibilityPublic,
	ValueVisibilityCreator: VisibilityCreator,
	ValueVisibilityLogin:   VisibilityLogin,
	ValueVisibilityOwner:   VisibilityOwner,
	ValueVisibilityExpert:  VisibilityExpert,
}
var revVisMap = map[Visibility]Value{}

func init() {
	for k, v := range visMap {
		revVisMap[v] = k
	}
}

// AsVisibility returns the visibility value of the given value string
func (val Value) AsVisibility() Visibility {
	if vis, ok := visMap[val]; ok {
		return vis
	}
	return VisibilityUnknown
}

func (v Visibility) String() string {
	if s, ok := revVisMap[v]; ok {
		return string(s)
	}
	return fmt.Sprintf("Unknown (%d)", v)
}

// UserRole enumerates the supported values of meta key 'user-role'.
type UserRole int

// Supported values for user roles.
const (
	_ UserRole = iota
	UserRoleUnknown
	UserRoleCreator
	UserRoleReader
	UserRoleWriter
	UserRoleOwner
)

var urMap = map[Value]UserRole{
	ValueUserRoleCreator: UserRoleCreator,
	ValueUserRoleReader:  UserRoleReader,
	ValueUserRoleWriter:  UserRoleWriter,
	ValueUserRoleOwner:   UserRoleOwner,
}

// AsUserRole role returns the user role of the given string.
func (val Value) AsUserRole() UserRole {
	if ur, ok := urMap[val]; ok {
		return ur
	}
	return UserRoleUnknown
}

Added domain/meta/write.go.




























































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta

import "io"

// Write writes metadata to a writer, excluding computed and propery values.
func (m *Meta) Write(w io.Writer) (int, error) {
	return m.doWrite(w, IsComputed)
}

// WriteComputed writes metadata to a writer, including computed values,
// but excluding property values.
func (m *Meta) WriteComputed(w io.Writer) (int, error) {
	return m.doWrite(w, IsProperty)
}

func (m *Meta) doWrite(w io.Writer, ignoreKeyPred func(string) bool) (length int, err error) {
	for key, val := range m.Computed() {
		if ignoreKeyPred(key) {
			continue
		}
		if err != nil {
			break
		}
		var l int
		l, err = io.WriteString(w, key)
		length += l
		if err == nil {
			l, err = w.Write(colonSpace)
			length += l
		}
		if err == nil {
			l, err = io.WriteString(w, string(val))
			length += l
		}
		if err == nil {
			l, err = w.Write(newline)
			length += l
		}
	}
	return length, err
}

var (
	colonSpace = []byte{':', ' '}
	newline    = []byte{'\n'}
)

Added domain/meta/write_test.go.




























































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore Client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package meta_test

import (
	"strings"
	"testing"

	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsc/domain/meta"
)

const testID = id.Zid(98765432101234)

func newMeta(title string, tags []string, syntax string) *meta.Meta {
	m := meta.New(testID)
	if title != "" {
		m.Set(meta.KeyTitle, meta.Value(title))
	}
	if tags != nil {
		m.Set(meta.KeyTags, meta.Value(strings.Join(tags, " ")))
	}
	if syntax != "" {
		m.Set(meta.KeySyntax, meta.Value(syntax))
	}
	return m
}
func assertWriteMeta(t *testing.T, m *meta.Meta, expected string) {
	t.Helper()
	var sb strings.Builder
	_, _ = m.Write(&sb)
	if got := sb.String(); got != expected {
		t.Errorf("\nExp: %q\ngot: %q", expected, got)
	}
}

func TestWriteMeta(t *testing.T) {
	t.Parallel()
	assertWriteMeta(t, newMeta("", nil, ""), "")

	m := newMeta("TITLE", []string{"#t1", "#t2"}, "syntax")
	assertWriteMeta(t, m, "title: TITLE\ntags: #t1 #t2\nsyntax: syntax\n")

	m = newMeta("TITLE", nil, "")
	m.Set("user", "zettel")
	m.Set("auth", "basic")
	assertWriteMeta(t, m, "title: TITLE\nauth: basic\nuser: zettel\n")
}

Changes to go.mod.

1

2
3

4
5
6
7





8

1
2

3
4
5


6
7
8
9
10
11
-
+

-
+


-
-
+
+
+
+
+

module zettelstore.de/c
module t73f.de/r/zsc

go 1.20
go 1.24

require (
	codeberg.org/t73fde/sxhtml v0.0.0-20230317170051-24321195e197
	codeberg.org/t73fde/sxpf v0.0.0-20230319111333-7de220f3b475
	t73f.de/r/sx v0.0.0-20250226205800-c12af029b6d3
	t73f.de/r/sxwebs v0.0.0-20250226210617-7bc3145c269b
	t73f.de/r/webs v0.0.0-20250311182734-f263a38b32d5
	t73f.de/r/zero v0.0.0-20250226205915-c4194684acb7
	t73f.de/r/zsx v0.0.0-20250320152042-d72698d2606a
)

Changes to go.sum.

1
2
3
4














1
2
3
4
5
6
7
8
9
10
-
-
-
-
+
+
+
+
+
+
+
+
+
+
codeberg.org/t73fde/sxhtml v0.0.0-20230317170051-24321195e197 h1:6kX7TY25agLFlHNvByO1Jc3GrBA7mu7aOa8tCOniUew=
codeberg.org/t73fde/sxhtml v0.0.0-20230317170051-24321195e197/go.mod h1:Dp3EwBSsE3TvdPw9QZ4Wm25ZragluVT2OayRFRiq6jk=
codeberg.org/t73fde/sxpf v0.0.0-20230319111333-7de220f3b475 h1:0OTzV3FYY/Y7YsaVaSzF4Wd17pXzdH6DaSvMeqteJc4=
codeberg.org/t73fde/sxpf v0.0.0-20230319111333-7de220f3b475/go.mod h1:iSbMygOmtRQYp8pryNKYzRuMibYDSR80smU2b6qm1bc=
t73f.de/r/sx v0.0.0-20250226205800-c12af029b6d3 h1:Jek4x1Qp59SWXI1enWVTeP1wxcVO96FuBpJBnnwOY98=
t73f.de/r/sx v0.0.0-20250226205800-c12af029b6d3/go.mod h1:hzg05uSCMk3D/DWaL0pdlowfL2aWQeGIfD1S04vV+Xg=
t73f.de/r/sxwebs v0.0.0-20250226210617-7bc3145c269b h1:X+9mMDd3fKML5SPcQk4n28oDGFUwqjDiSmQrH2LHZwI=
t73f.de/r/sxwebs v0.0.0-20250226210617-7bc3145c269b/go.mod h1:p+3JCSzNm9e+Yyub0ODRiLDeKaGVYWvBKYANZaAWYIA=
t73f.de/r/webs v0.0.0-20250311182734-f263a38b32d5 h1:nnKfs/2i9n3S5VjbSj98odcwZKGcL96qPSIUATT/2P8=
t73f.de/r/webs v0.0.0-20250311182734-f263a38b32d5/go.mod h1:zk92hSKB4iWyT290+163seNzu350TA9XLATC9kOldqo=
t73f.de/r/zero v0.0.0-20250226205915-c4194684acb7 h1:OuzHSfniY8UzLmo5zp1w23Kd9h7x9CSXP2jQ+kppeqU=
t73f.de/r/zero v0.0.0-20250226205915-c4194684acb7/go.mod h1:T1vFcHoymUQcr7+vENBkS1yryZRZ3YB8uRtnMy8yRBA=
t73f.de/r/zsx v0.0.0-20250320152042-d72698d2606a h1:+HLfeVMoYfTzeVYnOJYywKmgahE/pqiZrPuNZn53M2k=
t73f.de/r/zsx v0.0.0-20250320152042-d72698d2606a/go.mod h1:hZlFM0YBLIMgh2iEdS0rnuxLfO7inpWmc+wyDeRptT8=

Deleted maps/maps.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

























-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//-----------------------------------------------------------------------------

package maps

import "sort"

func Keys[T any](m map[string]T) []string {
	if len(m) == 0 {
		return nil
	}
	result := make([]string, 0, len(m))
	for k := range m {
		result = append(result, k)
	}
	sort.Strings(result)
	return result
}

Deleted maps/maps_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46














































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//-----------------------------------------------------------------------------

package maps_test

import (
	"testing"

	"zettelstore.de/c/maps"
)

func isSorted(seq []string) bool {
	for i := 1; i < len(seq); i++ {
		if seq[i] < seq[i-1] {
			return false
		}
	}
	return true
}

func TestKeys(t *testing.T) {
	testcases := []struct{ keys []string }{
		{nil}, {[]string{""}},
		{[]string{"z", "y", "a"}},
	}
	for i, tc := range testcases {
		m := make(map[string]struct{})
		for _, k := range tc.keys {
			m[k] = struct{}{}
		}
		got := maps.Keys(m)
		if len(got) != len(tc.keys) {
			t.Errorf("%d: wrong number of keys: exp %d, got %d", i, len(tc.keys), len(got))
		}
		if !isSorted(got) {
			t.Errorf("%d: keys not sorted: %v", i, got)
		}
	}
}

Added sexp/sexp.go.






































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2023-present Detlef Stern
//-----------------------------------------------------------------------------

// Package sexp contains helper function to work with s-expression in an alien
// environment.
package sexp

import (
	"errors"
	"fmt"
	"sort"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/api"
)

// EncodeZettel transforms zettel data into a sx object.
func EncodeZettel(zettel api.ZettelData) sx.Object {
	return sx.MakeList(
		sx.MakeSymbol("zettel"),
		meta2sz(zettel.Meta),
		sx.MakeList(sx.MakeSymbol("rights"), sx.Int64(int64(zettel.Rights))),
		sx.MakeList(sx.MakeSymbol("encoding"), sx.MakeString(zettel.Encoding)),
		sx.MakeList(sx.MakeSymbol("content"), sx.MakeString(zettel.Content)),
	)
}

// ParseZettel parses an object to contain all needed data for a zettel.
func ParseZettel(obj sx.Object) (api.ZettelData, error) {
	vals, err := ParseList(obj, "ypppp")
	if err != nil {
		return api.ZettelData{}, err
	}
	if errSym := CheckSymbol(vals[0], "zettel"); errSym != nil {
		return api.ZettelData{}, errSym
	}

	meta, err := ParseMeta(vals[1].(*sx.Pair))
	if err != nil {
		return api.ZettelData{}, err
	}

	rights, err := ParseRights(vals[2])
	if err != nil {
		return api.ZettelData{}, err
	}

	encVals, err := ParseList(vals[3], "ys")
	if err != nil {
		return api.ZettelData{}, err
	}
	if errSym := CheckSymbol(encVals[0], "encoding"); errSym != nil {
		return api.ZettelData{}, errSym
	}

	contentVals, err := ParseList(vals[4], "ys")
	if err != nil {
		return api.ZettelData{}, err
	}
	if errSym := CheckSymbol(contentVals[0], "content"); errSym != nil {
		return api.ZettelData{}, errSym
	}

	return api.ZettelData{
		Meta:     meta,
		Rights:   rights,
		Encoding: encVals[1].(sx.String).GetValue(),
		Content:  contentVals[1].(sx.String).GetValue(),
	}, nil
}

// EncodeMetaRights translates metadata/rights into a sx object.
func EncodeMetaRights(mr api.MetaRights) *sx.Pair {
	return sx.MakeList(
		sx.SymbolList,
		meta2sz(mr.Meta),
		sx.MakeList(sx.MakeSymbol("rights"), sx.Int64(int64(mr.Rights))),
	)
}

func meta2sz(m api.ZettelMeta) sx.Object {
	var result sx.ListBuilder
	result.Add(sx.MakeSymbol("meta"))
	keys := make([]string, 0, len(m))
	for k := range m {
		keys = append(keys, k)
	}
	sort.Strings(keys)
	for _, k := range keys {
		val := sx.MakeList(sx.MakeSymbol(k), sx.MakeString(m[k]))
		result.Add(val)
	}
	return result.List()
}

// ParseMeta translates the given list to metadata.
func ParseMeta(pair *sx.Pair) (api.ZettelMeta, error) {
	if err := CheckSymbol(pair.Car(), "meta"); err != nil {
		return nil, err
	}
	res := api.ZettelMeta{}
	for obj := range pair.Tail().Values() {
		mVals, err := ParseList(obj, "ys")
		if err != nil {
			return nil, err
		}
		res[(mVals[0].(*sx.Symbol)).GetValue()] = mVals[1].(sx.String).GetValue()
	}
	return res, nil
}

// ParseRights returns the rights values of the given object.
func ParseRights(obj sx.Object) (api.ZettelRights, error) {
	rVals, err := ParseList(obj, "yi")
	if err != nil {
		return api.ZettelMaxRight, err
	}
	if errSym := CheckSymbol(rVals[0], "rights"); errSym != nil {
		return api.ZettelMaxRight, errSym
	}
	i64 := int64(rVals[1].(sx.Int64))
	if i64 < 0 && i64 >= int64(api.ZettelMaxRight) {
		return api.ZettelMaxRight, fmt.Errorf("invalid zettel right value: %v", i64)
	}
	return api.ZettelRights(i64), nil
}

// ParseList parses the given object as a proper list, based on a type specification.
//
// 'b' expects a boolean, 'i' an int64, 'o' any object, 'p' a pair, 's' a string,
// and 'y' expects a symbol. A 'r' as the last type spracification matches all
// remaining values, including a non existent object.
func ParseList(obj sx.Object, spec string) (sx.Vector, error) {
	pair, isPair := sx.GetPair(obj)
	if !isPair {
		return nil, fmt.Errorf("not a list: %T/%v", obj, obj)
	}
	if pair == nil {
		if spec == "r" {
			return sx.Vector{sx.Nil()}, nil
		}
		if spec == "" {
			return nil, nil
		}
		return nil, ErrElementsMissing
	}

	specLen := len(spec)
	result := make(sx.Vector, 0, specLen)
	node, i := pair, 0
loop:
	for ; node != nil; i++ {
		if i >= specLen {
			return nil, ErrNoSpec
		}
		var val sx.Object
		var ok bool
		car := node.Car()
		switch spec[i] {
		case 'b':
			val, ok = sx.MakeBoolean(!sx.IsNil(car)), true
		case 'i':
			val, ok = car.(sx.Int64)
		case 'o':
			val, ok = car, true
		case 'p':
			val, ok = sx.GetPair(car)
		case 'r':
			if i < specLen-1 {
				return nil, fmt.Errorf("spec 'r' must be the last: %v", spec)
			}
			result = append(result, node)
			i++
			break loop
		case 's':
			val, ok = sx.GetString(car)
		case 'y':
			val, ok = sx.GetSymbol(car)
		default:
			return nil, fmt.Errorf("unknown spec '%c'", spec[i])
		}
		if !ok {
			return nil, fmt.Errorf("does not match spec '%v': %v", spec[i], car)
		}
		result = append(result, val)
		next, isNextPair := sx.GetPair(node.Cdr())
		if !isNextPair {
			return nil, sx.ErrImproper{Pair: pair}
		}
		node = next
	}
	if i < specLen {
		if lastSpec := specLen - 1; i < lastSpec || spec[lastSpec] != 'r' {
			return nil, ErrElementsMissing
		}
		result = append(result, sx.Nil())
	}
	return result, nil
}

// ErrElementsMissing is returned,
// if ParseList is called with a list smaller than the number of type specifications.
var ErrElementsMissing = errors.New("spec contains more data")

// ErrNoSpec is returned,
// if ParseList if called with a list greater than the number of type specifications.
var ErrNoSpec = errors.New("no spec for elements")

// CheckSymbol ensures that the given object is a symbol with the given name.
func CheckSymbol(obj sx.Object, name string) error {
	sym, isSymbol := sx.GetSymbol(obj)
	if !isSymbol {
		return fmt.Errorf("object %v/%T is not a symbol", obj, obj)
	}
	if got := sym.GetValue(); got != name {
		return fmt.Errorf("symbol %q expected, but got: %q", name, got)
	}
	return nil
}

Added sexp/sexp_test.go.


























































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2023-present Detlef Stern
//-----------------------------------------------------------------------------

package sexp_test

import (
	"testing"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/sexp"
)

func TestParseObject(t *testing.T) {
	if elems, err := sexp.ParseList(sx.MakeString("a"), "s"); err == nil {
		t.Error("expected an error, but got: ", elems)
	}
	if elems, err := sexp.ParseList(sx.Nil(), ""); err != nil {
		t.Error(err)
	} else if len(elems) != 0 {
		t.Error("Must be empty, but got:", elems)
	}
	if elems, err := sexp.ParseList(sx.Nil(), "b"); err == nil {
		t.Error("expected error, but got: ", elems)
	}

	if elems, err := sexp.ParseList(sx.MakeList(sx.MakeString("a")), "ss"); err == nil {
		t.Error("expected error, but got: ", elems)
	}
	if elems, err := sexp.ParseList(sx.MakeList(sx.MakeString("a")), ""); err == nil {
		t.Error("expected error, but got: ", elems)
	}
	if _, err := sexp.ParseList(sx.MakeList(sx.MakeString("a")), "b"); err != nil {
		t.Error("expected [1], but got error: ", err)
	}
	if elems, err := sexp.ParseList(sx.Cons(sx.Nil(), sx.MakeString("a")), "ps"); err == nil {
		t.Error("expected error, but got: ", elems)
	}

	if elems, err := sexp.ParseList(sx.MakeList(sx.MakeString("a")), "s"); err != nil {
		t.Error(err)
	} else if len(elems) != 1 {
		t.Error("length == 1, but got: ", elems)
	} else {
		_ = elems[0].(sx.String)
	}

	if elems, err := sexp.ParseList(sx.Nil(), "r"); err != nil {
		t.Error(err)
	} else if len(elems) != 1 {
		t.Error("length == 1, but got: ", elems)
	} else if !sx.IsNil(elems[0]) {
		t.Error("must be nil, but got:", elems[0])
	}
	if elems, err := sexp.ParseList(sx.MakeList(sx.MakeString("a")), "r"); err != nil {
		t.Error(err)
	} else if len(elems) != 1 {
		t.Error("length == 1, but got: ", elems)
	} else if !elems[0].IsEqual(sx.MakeList(sx.MakeString("a"))) {
		t.Error("must be (\"a\"), but got:", elems[0])
	}
	if elems, err := sexp.ParseList(sx.MakeList(sx.MakeString("a")), "sr"); err != nil {
		t.Error(err)
	} else if len(elems) != 2 {
		t.Error("length == 2, but got: ", elems)
	} else if !elems[0].IsEqual(sx.MakeString("a")) {
		t.Error("0-th must be \"a\", but got:", elems[0])
	} else if !sx.IsNil(elems[1]) {
		t.Error("must be nil, but got:", elems[1])
	}
	if elems, err := sexp.ParseList(sx.MakeList(sx.MakeString("a"), sx.MakeString("b"), sx.MakeString("c")), "sr"); err != nil {
		t.Error(err)
	} else if len(elems) != 2 {
		t.Error("length == 2, but got: ", elems)
	} else if !elems[0].IsEqual(sx.MakeString("a")) {
		t.Error("0-th must be \"a\", but got:", elems[0])
	} else if !elems[1].IsEqual(sx.MakeList(sx.MakeString("b"), sx.MakeString("c"))) {
		t.Error("must be nil, but got:", elems[1])
	}
}

Deleted sexpr/const.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297









































































































































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//-----------------------------------------------------------------------------

package sexpr

import "codeberg.org/t73fde/sxpf"

// Various constants for Zettel data. Some of them are technically variables.

const (
	// Symbols for Metanodes
	NameSymBlock  = "BLOCK"
	NameSymInline = "INLINE"
	NameSymList   = "LIST"
	NameSymMeta   = "META"
	NameSymQuote  = "quote"

	// Symbols for Zettel node types.
	NameSymBLOB            = "BLOB"
	NameSymCell            = "CELL"
	NameSymCellCenter      = "CELL-CENTER"
	NameSymCellLeft        = "CELL-LEFT"
	NameSymCellRight       = "CELL-RIGHT"
	NameSymCite            = "CITE"
	NameSymDescription     = "DESCRIPTION"
	NameSymEmbed           = "EMBED"
	NameSymEmbedBLOB       = "EMBED-BLOB"
	NameSymEndnote         = "ENDNOTE"
	NameSymFormatEmph      = "FORMAT-EMPH"
	NameSymFormatDelete    = "FORMAT-DELETE"
	NameSymFormatInsert    = "FORMAT-INSERT"
	NameSymFormatQuote     = "FORMAT-QUOTE"
	NameSymFormatSpan      = "FORMAT-SPAN"
	NameSymFormatSub       = "FORMAT-SUB"
	NameSymFormatSuper     = "FORMAT-SUPER"
	NameSymFormatStrong    = "FORMAT-STRONG"
	NameSymHard            = "HARD"
	NameSymHeading         = "HEADING"
	NameSymLinkInvalid     = "LINK-INVALID"
	NameSymLinkZettel      = "LINK-ZETTEL"
	NameSymLinkSelf        = "LINK-SELF"
	NameSymLinkFound       = "LINK-FOUND"
	NameSymLinkBroken      = "LINK-BROKEN"
	NameSymLinkHosted      = "LINK-HOSTED"
	NameSymLinkBased       = "LINK-BASED"
	NameSymLinkQuery       = "LINK-QUERY"
	NameSymLinkExternal    = "LINK-EXTERNAL"
	NameSymListOrdered     = "ORDERED"
	NameSymListUnordered   = "UNORDERED"
	NameSymListQuote       = "QUOTATION"
	NameSymLiteralProg     = "LITERAL-CODE"
	NameSymLiteralComment  = "LITERAL-COMMENT"
	NameSymLiteralHTML     = "LITERAL-HTML"
	NameSymLiteralInput    = "LITERAL-INPUT"
	NameSymLiteralMath     = "LITERAL-MATH"
	NameSymLiteralOutput   = "LITERAL-OUTPUT"
	NameSymLiteralZettel   = "LITERAL-ZETTEL"
	NameSymMark            = "MARK"
	NameSymPara            = "PARA"
	NameSymRegionBlock     = "REGION-BLOCK"
	NameSymRegionQuote     = "REGION-QUOTE"
	NameSymRegionVerse     = "REGION-VERSE"
	NameSymSoft            = "SOFT"
	NameSymSpace           = "SPACE"
	NameSymTable           = "TABLE"
	NameSymText            = "TEXT"
	NameSymThematic        = "THEMATIC"
	NameSymTransclude      = "TRANSCLUDE"
	NameSymUnknown         = "UNKNOWN-NODE"
	NameSymVerbatimComment = "VERBATIM-COMMENT"
	NameSymVerbatimEval    = "VERBATIM-EVAL"
	NameSymVerbatimHTML    = "VERBATIM-HTML"
	NameSymVerbatimMath    = "VERBATIM-MATH"
	NameSymVerbatimProg    = "VERBATIM-CODE"
	NameSymVerbatimZettel  = "VERBATIM-ZETTEL"

	// Constant symbols for reference states.
	NameSymRefStateInvalid  = "INVALID"
	NameSymRefStateZettel   = "ZETTEL"
	NameSymRefStateSelf     = "SELF"
	NameSymRefStateFound    = "FOUND"
	NameSymRefStateBroken   = "BROKEN"
	NameSymRefStateHosted   = "HOSTED"
	NameSymRefStateBased    = "BASED"
	NameSymRefStateQuery    = "QUERY"
	NameSymRefStateExternal = "EXTERNAL"

	// Symbols for metadata types.
	NameSymTypeCredential   = "CREDENTIAL"
	NameSymTypeEmpty        = "EMPTY-STRING"
	NameSymTypeID           = "ZID"
	NameSymTypeIDSet        = "ZID-SET"
	NameSymTypeNumber       = "NUMBER"
	NameSymTypeString       = "STRING"
	NameSymTypeTagSet       = "TAG-SET"
	NameSymTypeTimestamp    = "TIMESTAMP"
	NameSymTypeURL          = "URL"
	NameSymTypeWord         = "WORD"
	NameSymTypeWordSet      = "WORD-SET"
	NameSymTypeZettelmarkup = "ZETTELMARKUP"
)

// ZettelSymbols collect all symbols needed to represent zettel data.
type ZettelSymbols struct {
	// Symbols for Metanodes
	SymBlock  *sxpf.Symbol
	SymInline *sxpf.Symbol
	SymList   *sxpf.Symbol
	SymMeta   *sxpf.Symbol
	SymQuote  *sxpf.Symbol

	// Symbols for Zettel node types.
	SymBLOB            *sxpf.Symbol
	SymCell            *sxpf.Symbol
	SymCellCenter      *sxpf.Symbol
	SymCellLeft        *sxpf.Symbol
	SymCellRight       *sxpf.Symbol
	SymCite            *sxpf.Symbol
	SymDescription     *sxpf.Symbol
	SymEmbed           *sxpf.Symbol
	SymEmbedBLOB       *sxpf.Symbol
	SymEndnote         *sxpf.Symbol
	SymFormatEmph      *sxpf.Symbol
	SymFormatDelete    *sxpf.Symbol
	SymFormatInsert    *sxpf.Symbol
	SymFormatQuote     *sxpf.Symbol
	SymFormatSpan      *sxpf.Symbol
	SymFormatSub       *sxpf.Symbol
	SymFormatSuper     *sxpf.Symbol
	SymFormatStrong    *sxpf.Symbol
	SymHard            *sxpf.Symbol
	SymHeading         *sxpf.Symbol
	SymLinkInvalid     *sxpf.Symbol
	SymLinkZettel      *sxpf.Symbol
	SymLinkSelf        *sxpf.Symbol
	SymLinkFound       *sxpf.Symbol
	SymLinkBroken      *sxpf.Symbol
	SymLinkHosted      *sxpf.Symbol
	SymLinkBased       *sxpf.Symbol
	SymLinkQuery       *sxpf.Symbol
	SymLinkExternal    *sxpf.Symbol
	SymListOrdered     *sxpf.Symbol
	SymListUnordered   *sxpf.Symbol
	SymListQuote       *sxpf.Symbol
	SymLiteralProg     *sxpf.Symbol
	SymLiteralComment  *sxpf.Symbol
	SymLiteralHTML     *sxpf.Symbol
	SymLiteralInput    *sxpf.Symbol
	SymLiteralMath     *sxpf.Symbol
	SymLiteralOutput   *sxpf.Symbol
	SymLiteralZettel   *sxpf.Symbol
	SymMark            *sxpf.Symbol
	SymPara            *sxpf.Symbol
	SymRegionBlock     *sxpf.Symbol
	SymRegionQuote     *sxpf.Symbol
	SymRegionVerse     *sxpf.Symbol
	SymSoft            *sxpf.Symbol
	SymSpace           *sxpf.Symbol
	SymTable           *sxpf.Symbol
	SymText            *sxpf.Symbol
	SymThematic        *sxpf.Symbol
	SymTransclude      *sxpf.Symbol
	SymUnknown         *sxpf.Symbol
	SymVerbatimComment *sxpf.Symbol
	SymVerbatimEval    *sxpf.Symbol
	SymVerbatimHTML    *sxpf.Symbol
	SymVerbatimMath    *sxpf.Symbol
	SymVerbatimProg    *sxpf.Symbol
	SymVerbatimZettel  *sxpf.Symbol

	// Constant symbols for reference states.

	SymRefStateInvalid  *sxpf.Symbol
	SymRefStateZettel   *sxpf.Symbol
	SymRefStateSelf     *sxpf.Symbol
	SymRefStateFound    *sxpf.Symbol
	SymRefStateBroken   *sxpf.Symbol
	SymRefStateHosted   *sxpf.Symbol
	SymRefStateBased    *sxpf.Symbol
	SymRefStateQuery    *sxpf.Symbol
	SymRefStateExternal *sxpf.Symbol

	// Symbols for metadata types

	SymTypeCredential   *sxpf.Symbol
	SymTypeEmpty        *sxpf.Symbol
	SymTypeID           *sxpf.Symbol
	SymTypeIDSet        *sxpf.Symbol
	SymTypeNumber       *sxpf.Symbol
	SymTypeString       *sxpf.Symbol
	SymTypeTagSet       *sxpf.Symbol
	SymTypeTimestamp    *sxpf.Symbol
	SymTypeURL          *sxpf.Symbol
	SymTypeWord         *sxpf.Symbol
	SymTypeWordSet      *sxpf.Symbol
	SymTypeZettelmarkup *sxpf.Symbol
}

func (zs *ZettelSymbols) InitializeZettelSymbols(sf sxpf.SymbolFactory) {
	// Symbols for Metanodes
	zs.SymBlock = sf.MustMake(NameSymBlock)
	zs.SymInline = sf.MustMake(NameSymInline)
	zs.SymList = sf.MustMake(NameSymList)
	zs.SymMeta = sf.MustMake(NameSymMeta)
	zs.SymQuote = sf.MustMake(NameSymQuote)

	// Symbols for Zettel node types.
	zs.SymBLOB = sf.MustMake(NameSymBLOB)
	zs.SymCell = sf.MustMake(NameSymCell)
	zs.SymCellCenter = sf.MustMake(NameSymCellCenter)
	zs.SymCellLeft = sf.MustMake(NameSymCellLeft)
	zs.SymCellRight = sf.MustMake(NameSymCellRight)
	zs.SymCite = sf.MustMake(NameSymCite)
	zs.SymDescription = sf.MustMake(NameSymDescription)
	zs.SymEmbed = sf.MustMake(NameSymEmbed)
	zs.SymEmbedBLOB = sf.MustMake(NameSymEmbedBLOB)
	zs.SymEndnote = sf.MustMake(NameSymEndnote)
	zs.SymFormatEmph = sf.MustMake(NameSymFormatEmph)
	zs.SymFormatDelete = sf.MustMake(NameSymFormatDelete)
	zs.SymFormatInsert = sf.MustMake(NameSymFormatInsert)
	zs.SymFormatQuote = sf.MustMake(NameSymFormatQuote)
	zs.SymFormatSpan = sf.MustMake(NameSymFormatSpan)
	zs.SymFormatSub = sf.MustMake(NameSymFormatSub)
	zs.SymFormatSuper = sf.MustMake(NameSymFormatSuper)
	zs.SymFormatStrong = sf.MustMake(NameSymFormatStrong)
	zs.SymHard = sf.MustMake(NameSymHard)
	zs.SymHeading = sf.MustMake(NameSymHeading)
	zs.SymLinkInvalid = sf.MustMake(NameSymLinkInvalid)
	zs.SymLinkZettel = sf.MustMake(NameSymLinkZettel)
	zs.SymLinkSelf = sf.MustMake(NameSymLinkSelf)
	zs.SymLinkFound = sf.MustMake(NameSymLinkFound)
	zs.SymLinkBroken = sf.MustMake(NameSymLinkBroken)
	zs.SymLinkHosted = sf.MustMake(NameSymLinkHosted)
	zs.SymLinkBased = sf.MustMake(NameSymLinkBased)
	zs.SymLinkQuery = sf.MustMake(NameSymLinkQuery)
	zs.SymLinkExternal = sf.MustMake(NameSymLinkExternal)
	zs.SymListOrdered = sf.MustMake(NameSymListOrdered)
	zs.SymListUnordered = sf.MustMake(NameSymListUnordered)
	zs.SymListQuote = sf.MustMake(NameSymListQuote)
	zs.SymLiteralProg = sf.MustMake(NameSymLiteralProg)
	zs.SymLiteralComment = sf.MustMake(NameSymLiteralComment)
	zs.SymLiteralHTML = sf.MustMake(NameSymLiteralHTML)
	zs.SymLiteralInput = sf.MustMake(NameSymLiteralInput)
	zs.SymLiteralMath = sf.MustMake(NameSymLiteralMath)
	zs.SymLiteralOutput = sf.MustMake(NameSymLiteralOutput)
	zs.SymLiteralZettel = sf.MustMake(NameSymLiteralZettel)
	zs.SymMark = sf.MustMake(NameSymMark)
	zs.SymPara = sf.MustMake(NameSymPara)
	zs.SymRegionBlock = sf.MustMake(NameSymRegionBlock)
	zs.SymRegionQuote = sf.MustMake(NameSymRegionQuote)
	zs.SymRegionVerse = sf.MustMake(NameSymRegionVerse)
	zs.SymSoft = sf.MustMake(NameSymSoft)
	zs.SymSpace = sf.MustMake(NameSymSpace)
	zs.SymTable = sf.MustMake(NameSymTable)
	zs.SymText = sf.MustMake(NameSymText)
	zs.SymThematic = sf.MustMake(NameSymThematic)
	zs.SymTransclude = sf.MustMake(NameSymTransclude)
	zs.SymUnknown = sf.MustMake(NameSymUnknown)
	zs.SymVerbatimComment = sf.MustMake(NameSymVerbatimComment)
	zs.SymVerbatimEval = sf.MustMake(NameSymVerbatimEval)
	zs.SymVerbatimHTML = sf.MustMake(NameSymVerbatimHTML)
	zs.SymVerbatimMath = sf.MustMake(NameSymVerbatimMath)
	zs.SymVerbatimProg = sf.MustMake(NameSymVerbatimProg)
	zs.SymVerbatimZettel = sf.MustMake(NameSymVerbatimZettel)

	// Constant symbols for reference states.
	zs.SymRefStateInvalid = sf.MustMake(NameSymRefStateInvalid)
	zs.SymRefStateZettel = sf.MustMake(NameSymRefStateZettel)
	zs.SymRefStateSelf = sf.MustMake(NameSymRefStateSelf)
	zs.SymRefStateFound = sf.MustMake(NameSymRefStateFound)
	zs.SymRefStateBroken = sf.MustMake(NameSymRefStateBroken)
	zs.SymRefStateHosted = sf.MustMake(NameSymRefStateHosted)
	zs.SymRefStateBased = sf.MustMake(NameSymRefStateBased)
	zs.SymRefStateQuery = sf.MustMake(NameSymRefStateQuery)
	zs.SymRefStateExternal = sf.MustMake(NameSymRefStateExternal)

	// Symbols for metadata types.
	zs.SymTypeCredential = sf.MustMake(NameSymTypeCredential)
	zs.SymTypeEmpty = sf.MustMake(NameSymTypeEmpty)
	zs.SymTypeID = sf.MustMake(NameSymTypeID)
	zs.SymTypeIDSet = sf.MustMake(NameSymTypeIDSet)
	zs.SymTypeNumber = sf.MustMake(NameSymTypeNumber)
	zs.SymTypeString = sf.MustMake(NameSymTypeString)
	zs.SymTypeTagSet = sf.MustMake(NameSymTypeTagSet)
	zs.SymTypeTimestamp = sf.MustMake(NameSymTypeTimestamp)
	zs.SymTypeURL = sf.MustMake(NameSymTypeURL)
	zs.SymTypeWord = sf.MustMake(NameSymTypeWord)
	zs.SymTypeWordSet = sf.MustMake(NameSymTypeWordSet)
	zs.SymTypeZettelmarkup = sf.MustMake(NameSymTypeZettelmarkup)
}

Deleted sexpr/const_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


























-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//-----------------------------------------------------------------------------

package sexpr_test

import (
	"testing"

	"codeberg.org/t73fde/sxpf"
	"zettelstore.de/c/sexpr"
)

func BenchmarkInitializeZettelSymbols(b *testing.B) {
	sf := sxpf.MakeMappedFactory()
	for i := 0; i < b.N; i++ {
		var zs sexpr.ZettelSymbols
		zs.InitializeZettelSymbols(sf)
	}
}

Deleted sexpr/sexpr.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126






























































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//-----------------------------------------------------------------------------

package sexpr

import (
	"codeberg.org/t73fde/sxpf"
	"zettelstore.de/c/attrs"
)

// GetAttributes traverses a s-expression list and returns an attribute structure.
func GetAttributes(seq *sxpf.List) (result attrs.Attributes) {
	for elem := seq; elem != nil; elem = elem.Tail() {
		p, ok := elem.Car().(*sxpf.List)
		if !ok || p == nil {
			continue
		}
		key := p.Car()
		if !sxpf.IsAtom(key) {
			continue
		}
		val := p.Cdr()
		if tail, ok2 := val.(*sxpf.List); ok2 {
			val = tail.Car()
		}
		if !sxpf.IsAtom(val) {
			continue
		}
		result = result.Set(key.String(), val.String())
	}
	return result
}

// GetMetaContent returns the metadata and the content of a sexpr encoded zettel.
func GetMetaContent(zettel sxpf.Object) (Meta, *sxpf.List) {
	if pair, ok := zettel.(*sxpf.List); ok {
		m := pair.Car()
		if s := pair.Tail(); s != nil {
			if content, ok2 := s.Car().(*sxpf.List); ok2 {
				return MakeMeta(m), content
			}
		}
		return MakeMeta(m), nil
	}
	return nil, nil
}

type Meta map[string]MetaValue
type MetaValue struct {
	Type  string
	Key   string
	Value sxpf.Object
}

func MakeMeta(val sxpf.Object) Meta {
	if result := doMakeMeta(val); len(result) > 0 {
		return result
	}
	return nil
}
func doMakeMeta(val sxpf.Object) Meta {
	result := make(map[string]MetaValue)
	for {
		if sxpf.IsNil(val) {
			return result
		}
		lst, ok := val.(*sxpf.List)
		if !ok {
			return result
		}
		if mv, ok2 := makeMetaValue(lst); ok2 {
			result[mv.Key] = mv
		}
		val = lst.Cdr()
	}
}
func makeMetaValue(pair *sxpf.List) (MetaValue, bool) {
	var result MetaValue
	typePair, ok := pair.Car().(*sxpf.List)
	if !ok {
		return result, false
	}
	typeVal, ok := typePair.Car().(*sxpf.Symbol)
	if !ok {
		return result, false
	}
	keyPair, ok := typePair.Cdr().(*sxpf.List)
	if !ok {
		return result, false
	}
	keyStr, ok := keyPair.Car().(sxpf.String)
	if !ok {
		return result, false
	}
	valPair, ok := keyPair.Cdr().(*sxpf.List)
	if !ok {
		return result, false
	}
	result.Type = typeVal.CanonicalName()
	result.Key = keyStr.String()
	result.Value = valPair.Car()
	return result, true
}

func (m Meta) GetString(key string) string {
	if v, found := m[key]; found {
		return v.Value.String()
	}
	return ""
}

func (m Meta) GetList(key string) *sxpf.List {
	if mv, found := m[key]; found {
		if seq, ok := mv.Value.(*sxpf.List); ok {
			return seq
		}
	}
	return nil
}

Added shtml/const.go.




















































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2024-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2024-present Detlef Stern
//-----------------------------------------------------------------------------

package shtml

import "t73f.de/r/sx"

// Symbols for HTML header tags
var (
	SymBody   = sx.MakeSymbol("body")
	SymHead   = sx.MakeSymbol("head")
	SymHTML   = sx.MakeSymbol("html")
	SymMeta   = sx.MakeSymbol("meta")
	SymScript = sx.MakeSymbol("script")
	SymTitle  = sx.MakeSymbol("title")
)

// Symbols for HTML body tags
var (
	SymA          = sx.MakeSymbol("a")
	SymASIDE      = sx.MakeSymbol("aside")
	symBLOCKQUOTE = sx.MakeSymbol("blockquote")
	symBR         = sx.MakeSymbol("br")
	symCITE       = sx.MakeSymbol("cite")
	symCODE       = sx.MakeSymbol("code")
	symDD         = sx.MakeSymbol("dd")
	symDEL        = sx.MakeSymbol("del")
	SymDIV        = sx.MakeSymbol("div")
	symDL         = sx.MakeSymbol("dl")
	symDT         = sx.MakeSymbol("dt")
	symEM         = sx.MakeSymbol("em")
	SymEMBED      = sx.MakeSymbol("embed")
	SymFIGURE     = sx.MakeSymbol("figure")
	SymH1         = sx.MakeSymbol("h1")
	SymH2         = sx.MakeSymbol("h2")
	SymHR         = sx.MakeSymbol("hr")
	SymIMG        = sx.MakeSymbol("img")
	symINS        = sx.MakeSymbol("ins")
	symKBD        = sx.MakeSymbol("kbd")
	SymLI         = sx.MakeSymbol("li")
	symMARK       = sx.MakeSymbol("mark")
	SymOL         = sx.MakeSymbol("ol")
	SymP          = sx.MakeSymbol("p")
	symPRE        = sx.MakeSymbol("pre")
	symSAMP       = sx.MakeSymbol("samp")
	SymSPAN       = sx.MakeSymbol("span")
	SymSTRONG     = sx.MakeSymbol("strong")
	symSUB        = sx.MakeSymbol("sub")
	symSUP        = sx.MakeSymbol("sup")
	symTABLE      = sx.MakeSymbol("table")
	symTBODY      = sx.MakeSymbol("tbody")
	symTHEAD      = sx.MakeSymbol("thead")
	symTD         = sx.MakeSymbol("td")
	symTH         = sx.MakeSymbol("th")
	symTR         = sx.MakeSymbol("tr")
	SymUL         = sx.MakeSymbol("ul")
)

// Symbols for HTML attribute keys
var (
	SymAttrClass  = sx.MakeSymbol("class")
	SymAttrHref   = sx.MakeSymbol("href")
	SymAttrID     = sx.MakeSymbol("id")
	SymAttrLang   = sx.MakeSymbol("lang")
	SymAttrOpen   = sx.MakeSymbol("open")
	SymAttrRel    = sx.MakeSymbol("rel")
	SymAttrRole   = sx.MakeSymbol("role")
	SymAttrSrc    = sx.MakeSymbol("src")
	SymAttrTarget = sx.MakeSymbol("target")
	SymAttrTitle  = sx.MakeSymbol("title")
	SymAttrType   = sx.MakeSymbol("type")
	SymAttrValue  = sx.MakeSymbol("value")
)

Added shtml/lang.go.











































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2023-present Detlef Stern
//-----------------------------------------------------------------------------

package shtml

import (
	"strings"

	"t73f.de/r/zsc/domain/meta"
)

// LangStack is a stack to store the nesting of "lang" attribute values.
// It is used to generate typographically correct quotes.
type LangStack []string

// NewLangStack creates a new language stack.
func NewLangStack(lang string) LangStack {
	ls := make([]string, 1, 16)
	ls[0] = lang
	return ls
}

// Reset restores the language stack to its initial value.
func (ls *LangStack) Reset() {
	*ls = (*ls)[0:1]
}

// Push adds a new language value.
func (ls *LangStack) Push(lang string) {
	*ls = append(*ls, lang)
}

// Pop removes the topmost language value.
func (ls *LangStack) Pop() {
	*ls = (*ls)[0 : len(*ls)-1]
}

// Top returns the topmost language value.
func (ls *LangStack) Top() string {
	return (*ls)[len(*ls)-1]
}

// Dup duplicates the topmost language value.
func (ls *LangStack) Dup() {
	*ls = append(*ls, (*ls)[len(*ls)-1])
}

// QuoteInfo contains language specific data about quotes.
type QuoteInfo struct {
	primLeft, primRight string
	secLeft, secRight   string
	nbsp                bool
}

// GetPrimary returns the primary left and right quote entity.
func (qi *QuoteInfo) GetPrimary() (string, string) {
	return qi.primLeft, qi.primRight
}

// GetSecondary returns the secondary left and right quote entity.
func (qi *QuoteInfo) GetSecondary() (string, string) {
	return qi.secLeft, qi.secRight
}

// GetQuotes returns quotes based on a nesting level.
func (qi *QuoteInfo) GetQuotes(level uint) (string, string) {
	if level%2 == 0 {
		return qi.GetPrimary()
	}
	return qi.GetSecondary()
}

// GetNBSp returns true, if there must be a non-breaking space between the
// quote entities and the quoted text.
func (qi *QuoteInfo) GetNBSp() bool { return qi.nbsp }

var langQuotes = map[string]*QuoteInfo{
	"":               {"&quot;", "&quot;", "&quot;", "&quot;", false},
	meta.ValueLangEN: {"&ldquo;", "&rdquo;", "&lsquo;", "&rsquo;", false},
	"de":             {"&bdquo;", "&ldquo;", "&sbquo;", "&lsquo;", false},
	"fr":             {"&laquo;", "&raquo;", "&lsaquo;", "&rsaquo;", true},
}

// GetQuoteInfo returns language specific data about quotes.
func GetQuoteInfo(lang string) *QuoteInfo {
	langFields := strings.FieldsFunc(lang, func(r rune) bool { return r == '-' || r == '_' })
	for len(langFields) > 0 {
		langSup := strings.Join(langFields, "-")
		quotes, ok := langQuotes[langSup]
		if ok {
			return quotes
		}
		langFields = langFields[0 : len(langFields)-1]
	}
	return langQuotes[""]
}

Changes to shtml/shtml.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19
20
21


22
23
24
25
26
27




28
29
30

31
32

33
34
35
36
37
38
39
40
41
42
43
44
45
46



47
48
49
50
51
52
53
54



55
56
57
58
59
60
61
62
63
64
65
66
67
68









69
70
71
72
73

74
75
76

77
78
79
80
81
82
83
84
85
86
87
88


89
90

91
92

93
94
95
96
97


98
99
100
101

102
103

104
105
106
107
108
109
110







111
112
113
114

115
116
117


118
119
120

121
122
123
124

125
126
127
128

129
130

131
132
133
134



135
136
137


138
139
140
141
142
143

144
145
146
147
148
149


150
151
152
153
154







155

156
157



158
159
160
161
162


163
164
165
166
167







168
169
170
171
172
173
174
175





176
177
178
179
180
181
182
183
184
185
186
187





























188
189
190
191
192
193
194
195
196















197
198
199

200
201

202
203
204
205
206
207



208







209
210



211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227

































228
229
230







231
232


233
234
235
236
237
238





239
240
241
242
243
244
245
246
247
248










249
250

251
252

253
254
255
256
257
258
259


260
261

262
263
264


265
266
267

268
269


270
271
272
273
274
275
276
277






278
279
280


281
282
283


284
285


286
287
288
289
290
291





292
293
294

295
296

297
298

299
300

301
302
303
304
305
306





307
308
309

310
311
312
313
314
315






316
317
318
319
320

321
322
323
324

325
326
327
328
329



330
331
332



333
334
335

336
337
338
339
340
341
342
343
344






















345
346

347
348
349

350
351
352
353
354
355




356
357
358
359


360
361
362
363



364
365
366
367
368
369



370
371
372
373
374
375

376
377

378
379
380
381
382
383
384
385







386
387

388
389

390
391
392
393
394
395

396
397



398
399










400



401
402
403



404
405
406
407



408
409

410
411

412
413
414


415
416
417


418
419
420

421
422
423
424
425
426
427
428






429
430
431
432
433
434
435
436
437
438
439






440
441
442

443
444
445
446

447
448
449
450
451
452
453
454
455
456
457
458
















459
460
461
462
463
464
465
466
467





468

469
470

471
472
473
474



475
476
477
478
479





480

481
482
483
484
485





486
487
488
489
490

491

492
493

494
495
496
497
498
499
500
501
502
503








504
505
506

507
508
509
510

511
512
513


514
515
516
517


518
519
520
521
522
523




524
525
526
527
528
529
530
531
532
533
534

























535
536


537
538

539
540

541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578

579
580
581
582
583

584
585
586
587


588
589
590
591
592
593
594








595
596

597
598
599
600
601








602
603
604
605
606
607
608





609
610
611
612



613
614
615
616
617


618
619
620
621
622

623
624
625

626
627
628
629
630
631





632
633
634

635
636
637
638
639
640







641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658











659






660

661
662
663
664
665
666
667
668



669
670
671
672
673
674
675
676









677
678
679
680





681
682
683
684
685
686

687
688
689
690


691
692
693


694
695

696
697
698


699
700
701
702
703





704
705
706

707
708

709
710
711
712
713
714
715
716
717
718
719
720
721
722



















723
724
725
726







727
728
729
730


731
732
733




734
735

736


737
738

739
740
741

742
743
744
745
746

747
748

749
750
751
752
753

754
755
756
757
758
759






760
761
762














































































































































763
764
765
766

767
768
769


770
771
772
773


774
775
776
777
778
779
780
781
782
783
784
785
786






787
788
789
790
791
792
793
794
795
796
797

798
799

800
801
802
803
804
805
806
807
808
809

810
811
812
813
814
815

816
817
818
819
820
821
822
823
824
825


826
827
828


829
830
831
832
833
834
835






836
837

838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853

854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22


23
24






25
26
27
28
29
30

31
32

33


34
35

36








37
38
39



40
41



42
43
44





45








46
47
48
49
50
51
52
53
54


55
56

57
58
59

60
61











62
63
64

65
66

67
68
69
70


71
72
73
74
75

76
77

78
79
80





81
82
83
84
85
86
87




88



89
90



91




92




93


94




95
96
97
98


99
100

101




102






103
104





105
106
107
108
109
110
111
112
113


114
115
116
117
118
119


120
121
122
123



124
125
126
127
128
129
130
131







132
133
134
135
136
137











138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166









167
168
169
170
171
172
173
174
175
176
177
178
179
180
181



182


183






184
185
186
187
188
189
190
191
192
193
194


195
196
197

















198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230



231
232
233
234
235
236
237


238
239






240
241
242
243
244
245









246
247
248
249
250
251
252
253
254
255
256

257
258

259
260
261
262
263
264


265
266
267

268
269


270
271

272
273
274


275
276








277
278
279
280
281
282
283


284
285
286


287
288
289

290
291
292





293
294
295
296
297
298
299

300
301

302
303

304
305

306
307





308
309
310
311
312
313
314

315
316





317
318
319
320
321
322
323




324




325





326
327
328



329
330
331
332
333

334
335
336







337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358


359
360
361

362
363
364




365
366
367
368
369
370


371
372




373
374
375
376
377
378



379
380
381
382
383
384
385
386

387
388

389
390







391
392
393
394
395
396
397


398


399



400
401

402
403
404
405
406
407


408
409
410
411
412
413
414
415
416
417
418
419
420
421



422
423
424
425



426
427
428
429

430
431

432
433


434
435



436
437
438


439








440
441
442
443
444
445











446
447
448
449
450
451
452
453

454
455
456
457

458












459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476







477
478
479
480
481

482
483

484
485
486


487
488
489





490
491
492
493
494
495
496
497
498



499
500
501
502
503
504
505
506
507
508
509

510
511

512
513









514
515
516
517
518
519
520
521
522
523

524
525
526
527

528
529


530
531
532
533


534
535
536
537




538
539
540
541











542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566


567
568
569

570
571

572
573





































574





575




576
577







578
579
580
581
582
583
584
585
586

587





588
589
590
591
592
593
594
595
596
597
598
599
600


601
602
603
604
605




606
607
608
609
610
611


612
613
614
615
616
617

618
619


620






621
622
623
624
625
626
627

628
629





630
631
632
633
634
635
636
637
638
639
640














641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658

659








660
661
662
663







664
665
666
667
668
669
670
671
672
673



674
675
676
677
678






679
680



681
682
683


684
685
686

687



688
689





690
691
692
693
694
695
696

697
698

699
700
701
702
703










704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722




723
724
725
726
727
728
729



730
731
732



733
734
735
736

737
738

739
740
741

742
743
744

745
746
747
748
749

750
751

752
753
754
755


756
757
758
759
760
761
762
763
764
765
766
767
768



769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913

914



915
916
917



918
919













920
921
922
923
924
925











926


927










928






929










930
931



932
933







934
935
936
937
938
939


940



941












942


























943
944
945
946
947
948
949








+
+
+











-
-
+
+
-
-
-
-
-
-
+
+
+
+


-
+

-
+
-
-


-

-
-
-
-
-
-
-
-
+
+
+
-
-
-


-
-
-
+
+
+
-
-
-
-
-

-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
-
-


-
+


-
+

-
-
-
-
-
-
-
-
-
-
-
+
+

-
+

-
+



-
-
+
+



-
+

-
+


-
-
-
-
-
+
+
+
+
+
+
+
-
-
-
-
+
-
-
-
+
+
-
-
-
+
-
-
-
-
+
-
-
-
-
+
-
-
+
-
-
-
-
+
+
+

-
-
+
+
-

-
-
-
-
+
-
-
-
-
-
-
+
+
-
-
-
-
-
+
+
+
+
+
+
+

+
-
-
+
+
+



-
-
+
+


-
-
-
+
+
+
+
+
+
+

-
-
-
-
-
-
-
+
+
+
+
+

-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
+
-
-
+
-
-
-
-
-
-
+
+
+

+
+
+
+
+
+
+
-
-
+
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
+
+
+
+
+
+
+
-
-
+
+
-
-
-
-
-
-
+
+
+
+
+

-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+

-
+

-
+





-
-
+
+

-
+

-
-
+
+
-


+
-
-
+
+
-
-
-
-
-
-
-
-
+
+
+
+
+
+

-
-
+
+

-
-
+
+

-
+
+

-
-
-
-
-
+
+
+
+
+


-
+

-
+

-
+

-
+

-
-
-
-
-
+
+
+
+
+


-
+

-
-
-
-
-
+
+
+
+
+
+

-
-
-
-
+
-
-
-
-
+
-
-
-
-
-
+
+
+
-
-
-
+
+
+


-
+


-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+


-
+


-
-
-
-
+
+
+
+


-
-
+
+
-
-
-
-
+
+
+



-
-
-
+
+
+





-
+

-
+

-
-
-
-
-
-
-
+
+
+
+
+
+
+
-
-
+
-
-
+
-
-
-


-
+


+
+
+
-
-
+
+
+
+
+
+
+
+
+
+

+
+
+
-
-
-
+
+
+

-
-
-
+
+
+

-
+

-
+

-
-
+
+
-
-
-
+
+

-
-
+
-
-
-
-
-
-
-
-
+
+
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+


-
+



-
+
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+


-
-
-
-
-
-
-
+
+
+
+
+
-
+

-
+


-
-
+
+
+
-
-
-
-
-
+
+
+
+
+

+


-
-
-
+
+
+
+
+





+
-
+

-
+

-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+


-
+



-
+

-
-
+
+


-
-
+
+


-
-
-
-
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+

-
+

-
+

-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
+
-
-
-
-
+
+
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+

-
+
-
-
-
-
-
+
+
+
+
+
+
+
+





-
-
+
+
+
+
+
-
-
-
-
+
+
+



-
-
+
+




-
+

-
-
+
-
-
-
-
-
-
+
+
+
+
+


-
+

-
-
-
-
-
+
+
+
+
+
+
+




-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+

+
+
+
+
+
+
-
+
-
-
-
-
-
-
-
-
+
+
+

-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+

-
-
-
+
+
+
+
+
-
-
-
-
-
-
+

-
-
-
+
+

-
-
+
+

-
+
-
-
-
+
+
-
-
-
-
-
+
+
+
+
+


-
+

-
+




-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
+
+
+
+
+
-
-
-

+
+
-
-
-
+
+
+
+
-

+
-
+
+

-
+


-
+




-
+

-
+



-
-
+






+
+
+
+
+
+
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+



-
+
-
-
-
+
+

-
-
-
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
-
-
-
-
-
-
-
-
-
-
-
+
-
-
+
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
+
+
-
-
-
+
+
-
-
-
-
-
-
-
+
+
+
+
+
+
-
-
+
-
-
-

-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







//-----------------------------------------------------------------------------
// Copyright (c) 2023-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2023-present Detlef Stern
//-----------------------------------------------------------------------------

// Package shtml transforms a s-expr encoded zettel AST into a s-expr representation of HTML.
package shtml

import (
	"fmt"
	"net/url"
	"strconv"
	"strings"

	"codeberg.org/t73fde/sxhtml"
	"codeberg.org/t73fde/sxpf"
	"t73f.de/r/sx"
	"t73f.de/r/sxwebs/sxhtml"
	"codeberg.org/t73fde/sxpf/builtins/quote"
	"codeberg.org/t73fde/sxpf/eval"
	"zettelstore.de/c/api"
	"zettelstore.de/c/attrs"
	"zettelstore.de/c/sexpr"
	"zettelstore.de/c/text"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/domain/meta"
	"t73f.de/r/zsc/sz"
	"t73f.de/r/zsx"
)

// Transformer will transform a s-expression that encodes the zettel AST into an s-expression
// Evaluator will transform a s-expression that encodes the zettel AST into an s-expression
// that represents HTML.
type Transformer struct {
type Evaluator struct {
	sf            sxpf.SymbolFactory
	rebinder      RebindProc
	headingOffset int64
	unique        string
	endnotes      []endnoteInfo
	noLinks       bool // true iff output must not include links
	symAttr       *sxpf.Symbol
	symClass      *sxpf.Symbol
	symMeta       *sxpf.Symbol
	symA          *sxpf.Symbol
	symSpan       *sxpf.Symbol
}

type endnoteInfo struct {

	fns     map[string]EvalFn
	minArgs map[string]int
	noteAST *sxpf.List // Endnote as AST
	noteHx  *sxpf.List // Endnote as SxHTML
	attrs   *sxpf.List // attrs a-list
}

// NewTransformer creates a new transformer object.
func NewTransformer(headingOffset int, sf sxpf.SymbolFactory) *Transformer {
	if sf == nil {
// NewEvaluator creates a new Evaluator object.
func NewEvaluator(headingOffset int) *Evaluator {
	ev := &Evaluator{
		sf = sxpf.MakeMappedFactory()
	}
	return &Transformer{
		sf:            sf,
		rebinder:      nil,
		headingOffset: int64(headingOffset),
		symAttr:       sf.MustMake(sxhtml.NameSymAttr),
		symClass:      sf.MustMake("class"),
		symMeta:       sf.MustMake("meta"),
		symA:          sf.MustMake("a"),
		symSpan:       sf.MustMake("span"),
	}
}


		fns:     make(map[string]EvalFn, 128),
		minArgs: make(map[string]int, 128),
	}
	ev.bindMetadata()
	ev.bindBlocks()
	ev.bindInlines()
	return ev
}
// SymbolFactory returns the symbol factory to create HTML symbols.
func (tr *Transformer) SymbolFactory() sxpf.SymbolFactory { return tr.sf }

// SetUnique sets a prefix to make several HTML ids unique.
func (tr *Transformer) SetUnique(s string) { tr.unique = s }
func (ev *Evaluator) SetUnique(s string) { ev.unique = s }

// IsValidName returns true, if name is a valid symbol name.
func (tr *Transformer) IsValidName(s string) bool { return tr.sf.IsValidName(s) }
func isValidName(s string) bool { return s != "" }

// Make a new HTML symbol.
func (tr *Transformer) Make(s string) *sxpf.Symbol { return tr.sf.MustMake(s) }

// RebindProc is a procedure which is called every time before a tranformation takes place.
type RebindProc func(*TransformEnv)

// SetRebinder sets the rebinder procedure.
func (tr *Transformer) SetRebinder(rb RebindProc) { tr.rebinder = rb }

// TransformAttrbute transforms the given attributes into a HTML s-expression.
func (tr *Transformer) TransformAttrbute(a attrs.Attributes) *sxpf.List {
// EvaluateAttributes transforms the given attributes into a HTML s-expression.
func EvaluateAttributes(a zsx.Attributes) *sx.Pair {
	if len(a) == 0 {
		return sxpf.Nil()
		return nil
	}
	plist := sxpf.Nil()
	plist := sx.Nil()
	keys := a.Keys()
	for i := len(keys) - 1; i >= 0; i-- {
		key := keys[i]
		if key != attrs.DefaultAttribute && tr.IsValidName(key) {
			plist = plist.Cons(sxpf.Cons(tr.Make(key), sxpf.MakeString(a[key])))
		if key != zsx.DefaultAttribute && isValidName(key) {
			plist = plist.Cons(sx.Cons(sx.MakeSymbol(key), sx.MakeString(a[key])))
		}
	}
	if plist == nil {
		return sxpf.Nil()
		return nil
	}
	return plist.Cons(tr.symAttr)
	return plist.Cons(sxhtml.SymAttr)
}

// TransformMeta creates a HTML meta s-expression
func (tr *Transformer) TransformMeta(a attrs.Attributes) *sxpf.List {
	return sxpf.Nil().Cons(tr.TransformAttrbute(a)).Cons(tr.symMeta)
}

// Evaluate a metadata s-expression into a list of HTML s-expressions.
func (ev *Evaluator) Evaluate(lst *sx.Pair, env *Environment) (*sx.Pair, error) {
	result := ev.Eval(lst, env)
	if err := env.err; err != nil {
		return nil, err
	}
	pair, isPair := sx.GetPair(result)
// Transform an AST s-expression into a list of HTML s-expressions.
func (tr *Transformer) Transform(lst *sxpf.List) (*sxpf.List, error) {
	astSF := sxpf.FindSymbolFactory(lst)
	if astSF != nil {
	if !isPair {
		if astSF == tr.sf {
			panic("Invalid AST SymbolFactory")
		}
		return nil, fmt.Errorf("evaluation does not result in a pair, but %T/%v", result, result)
	}
	} else {
		astSF = sxpf.MakeMappedFactory()
	}

	astEnv := sxpf.MakeRootEnvironment()
	engine := eval.MakeEngine(astSF, astEnv, eval.MakeDefaultParser(), eval.MakeSimpleExecutor())
	quote.InstallQuote(engine, sexpr.NameSymQuote, nil, 0)
	te := TransformEnv{
	for i := 0; i < len(env.endnotes); i++ {
		tr:      tr,
		astSF:   astSF,
		astEnv:  astEnv,
		err:     nil,
		// May extend tr.endnotes -> do not use for i := range len(...)!!!
		textEnc: text.NewEncoder(astSF),
	}

	te.initialize()
	if rb := tr.rebinder; rb != nil {
		rb(&te)
	}
		if env.endnotes[i].noteHx != nil {
			continue
		}

	val, err := engine.Eval(te.astEnv, lst)
	if err != nil {
		noteHx, _ := ev.EvaluateList(env.endnotes[i].noteAST, env)
		env.endnotes[i].noteHx = noteHx
		return sxpf.Nil(), err
	}
	res, ok := val.(*sxpf.List)
	if !ok {
		panic("Result is not a list")
	}

	for i := 0; i < len(tr.endnotes); i++ {
		// May extend tr.endnotes
		val, err = engine.Eval(te.astEnv, tr.endnotes[i].noteAST)
		if err != nil {
			return res, err
		}
	return pair, nil
}
		en, ok2 := val.(*sxpf.List)
		if !ok2 {
			panic("Endnote is not a list")
		}
		tr.endnotes[i].noteHx = en

// EvaluateList will evaluate all list elements separately and returns them as a sx.Pair list
func (ev *Evaluator) EvaluateList(lst sx.Vector, env *Environment) (*sx.Pair, error) {
	var result sx.ListBuilder
	for _, elem := range lst {
		p := ev.Eval(elem, env)
		result.Add(p)
	}
	if err := env.err; err != nil {
	return res, err

		return nil, err
	}
	return result.List(), nil
}

// Endnotes returns a SHTML object with all collected endnotes.
func (tr *Transformer) Endnotes() *sxpf.List {
	if len(tr.endnotes) == 0 {
func Endnotes(env *Environment) *sx.Pair {
	if env.err != nil || len(env.endnotes) == 0 {
		return nil
	}
	result := sxpf.Nil().Cons(tr.Make("ol"))
	currResult := result.AppendBang(sxpf.Nil().Cons(sxpf.Cons(tr.symClass, sxpf.MakeString("zs-endnotes"))).Cons(tr.symAttr))
	for i, fni := range tr.endnotes {

	var result sx.ListBuilder
	result.AddN(
		SymOL,
		sx.Nil().Cons(sx.Cons(SymAttrClass, sx.MakeString("zs-endnotes"))).Cons(sxhtml.SymAttr),
	)
	for i, fni := range env.endnotes {
		noteNum := strconv.Itoa(i + 1)
		noteID := tr.unique + noteNum

		attrs := fni.attrs.Cons(sxpf.Cons(tr.symClass, sxpf.MakeString("zs-endnote"))).
			Cons(sxpf.Cons(tr.Make("value"), sxpf.MakeString(noteNum))).
			Cons(sxpf.Cons(tr.Make("id"), sxpf.MakeString("fn:"+noteID))).
			Cons(sxpf.Cons(tr.Make("role"), sxpf.MakeString("doc-endnote"))).
			Cons(tr.symAttr)
		attrs := fni.attrs.Cons(sx.Cons(SymAttrClass, sx.MakeString("zs-endnote"))).
			Cons(sx.Cons(SymAttrValue, sx.MakeString(noteNum))).
			Cons(sx.Cons(SymAttrID, sx.MakeString("fn:"+fni.noteID))).
			Cons(sx.Cons(SymAttrRole, sx.MakeString("doc-endnote"))).
			Cons(sxhtml.SymAttr)

		backref := sxpf.Nil().Cons(sxpf.MakeString("\u21a9\ufe0e")).
			Cons(sxpf.Nil().
				Cons(sxpf.Cons(tr.symClass, sxpf.MakeString("zs-endnote-backref"))).
				Cons(sxpf.Cons(tr.Make("href"), sxpf.MakeString("#fnref:"+noteID))).
				Cons(sxpf.Cons(tr.Make("role"), sxpf.MakeString("doc-backlink"))).
				Cons(tr.symAttr)).
			Cons(tr.symA)

		li := sxpf.Nil().Cons(tr.Make("li"))
		li.AppendBang(attrs).
			ExtendBang(fni.noteHx).
		backref := sx.Nil().Cons(sx.MakeString("\u21a9\ufe0e")).
			Cons(sx.Nil().
				Cons(sx.Cons(SymAttrClass, sx.MakeString("zs-endnote-backref"))).
				Cons(sx.Cons(SymAttrHref, sx.MakeString("#fnref:"+fni.noteID))).
				Cons(sx.Cons(SymAttrRole, sx.MakeString("doc-backlink"))).
				Cons(sxhtml.SymAttr)).
			Cons(SymA)

		var li sx.ListBuilder
		li.AddN(SymLI, attrs)
		li.ExtendBang(fni.noteHx)
		li.AddN(sx.MakeString(" "), backref)
		result.Add(li.List())
	}
	return result.List()
}

// Environment where sz objects are evaluated to shtml objects
type Environment struct {
	err          error
	langStack    LangStack
	endnotes     []endnoteInfo
	quoteNesting uint
}
type endnoteInfo struct {
	noteID  string    // link id
	noteAST sx.Vector // Endnote as list of AST inline elements
	attrs   *sx.Pair  // attrs a-list
	noteHx  *sx.Pair  // Endnote as SxHTML
			AppendBang(sxpf.MakeString(" ")).AppendBang(backref)
		currResult = currResult.AppendBang(li)
	}
	tr.endnotes = nil
	return result
}

// TransformEnv is the environment where the actual transformation takes places.
type TransformEnv struct {
}

// MakeEnvironment builds a new evaluation environment.
func MakeEnvironment(lang string) Environment {
	return Environment{
		err:          nil,
		langStack:    NewLangStack(lang),
		endnotes:     nil,
		quoteNesting: 0,
	}
}

// GetError returns the last error found.
func (env *Environment) GetError() error { return env.err }

	tr          *Transformer
	astSF       sxpf.SymbolFactory
	astEnv      sxpf.Environment
// Reset the environment.
	err         error
	textEnc     *text.Encoder
func (env *Environment) Reset() {
	symNoEscape *sxpf.Symbol
	symAttr     *sxpf.Symbol
	symMeta     *sxpf.Symbol
	symA        *sxpf.Symbol
	symSpan     *sxpf.Symbol
	symP        *sxpf.Symbol
	env.langStack.Reset()
	env.endnotes = nil
	env.quoteNesting = 0
}

// pushAttribute adds the current attributes to the environment.
func (env *Environment) pushAttributes(a zsx.Attributes) {
	if value, ok := a.Get("lang"); ok {
		env.langStack.Push(value)
	} else {
		env.langStack.Dup()

func (te *TransformEnv) initialize() {
	}
}

	te.symNoEscape = te.Make(sxhtml.NameSymNoEscape)
	te.symAttr = te.tr.symAttr
	te.symMeta = te.tr.symMeta
	te.symA = te.tr.symA
	te.symSpan = te.tr.symSpan
	te.symP = te.Make("p")

	te.bind(sexpr.NameSymList, 0, listArgs)
	te.bindMetadata()
	te.bindBlocks()
	te.bindInlines()
}

func listArgs(args *sxpf.List) sxpf.Object { return args }

func (te *TransformEnv) bindMetadata() {
	te.bind(sexpr.NameSymMeta, 0, listArgs)
// popAttributes removes the current attributes from the envrionment.
func (env *Environment) popAttributes() { env.langStack.Pop() }

// getLanguage returns the current language.
func (env *Environment) getLanguage() string { return env.langStack.Top() }

func (env *Environment) getQuotes() (string, string, bool) {
	qi := GetQuoteInfo(env.getLanguage())
	leftQ, rightQ := qi.GetQuotes(env.quoteNesting)
	return leftQ, rightQ, qi.GetNBSp()
}

// EvalFn is a function to be called for evaluation.
type EvalFn func(sx.Vector, *Environment) sx.Object

func (ev *Evaluator) bind(sym *sx.Symbol, minArgs int, fn EvalFn) {
	symVal := sym.GetValue()
	ev.fns[symVal] = fn
	if minArgs > 0 {
		ev.minArgs[symVal] = minArgs
	}
}

// ResolveBinding returns the function bound to the given name.
func (ev *Evaluator) ResolveBinding(sym *sx.Symbol) EvalFn {
	if fn, found := ev.fns[sym.GetValue()]; found {
		return fn
	}
	return nil
}

// Rebind overwrites a binding, but leaves the minimum number of arguments intact.
func (ev *Evaluator) Rebind(sym *sx.Symbol, fn EvalFn) {
	te.bind(sexpr.NameSymTypeZettelmarkup, 2, func(args *sxpf.List) sxpf.Object {
		a := make(attrs.Attributes, 2).
			Set("name", te.getString(args).String()).
	symVal := sym.GetValue()
	if _, found := ev.fns[symVal]; !found {
		panic(sym)
	}
	ev.fns[symVal] = fn
}

			Set("content", te.textEnc.Encode(te.getList(args.Tail())))
		return te.transformMeta(a)
func (ev *Evaluator) bindMetadata() {
	ev.bind(sz.SymMeta, 0, ev.evalList)
	})
	metaString := func(args *sxpf.List) sxpf.Object {
		a := make(attrs.Attributes, 2).
			Set("name", te.getString(args).String()).
			Set("content", te.getString(args.Tail()).String())
		return te.transformMeta(a)
	evalMetaString := func(args sx.Vector, env *Environment) sx.Object {
		a := make(zsx.Attributes, 2).
			Set("name", getSymbol(args[0], env).GetValue()).
			Set("content", getString(args[1], env).GetValue())
		return ev.EvaluateMeta(a)
	}
	te.bind(sexpr.NameSymTypeCredential, 2, metaString)
	te.bind(sexpr.NameSymTypeEmpty, 2, metaString)
	te.bind(sexpr.NameSymTypeID, 2, metaString)
	te.bind(sexpr.NameSymTypeNumber, 2, metaString)
	te.bind(sexpr.NameSymTypeString, 2, metaString)
	te.bind(sexpr.NameSymTypeTimestamp, 2, metaString)
	te.bind(sexpr.NameSymTypeURL, 2, metaString)
	te.bind(sexpr.NameSymTypeWord, 2, metaString)
	metaSet := func(args *sxpf.List) sxpf.Object {
	ev.bind(sz.SymTypeCredential, 2, evalMetaString)
	ev.bind(sz.SymTypeEmpty, 2, evalMetaString)
	ev.bind(sz.SymTypeID, 2, evalMetaString)
	ev.bind(sz.SymTypeNumber, 2, evalMetaString)
	ev.bind(sz.SymTypeString, 2, evalMetaString)
	ev.bind(sz.SymTypeTimestamp, 2, evalMetaString)
	ev.bind(sz.SymTypeURL, 2, evalMetaString)
	ev.bind(sz.SymTypeWord, 2, evalMetaString)

	evalMetaSet := func(args sx.Vector, env *Environment) sx.Object {
		var sb strings.Builder
		for elem := te.getList(args.Tail()); elem != nil; elem = elem.Tail() {
		for obj := range getList(args[1], env).Values() {
			sb.WriteByte(' ')
			sb.WriteString(te.getString(elem).String())
			sb.WriteString(getString(obj, env).GetValue())
		}
		s := sb.String()
		if len(s) > 0 {
			s = s[1:]
		}
		a := make(attrs.Attributes, 2).
			Set("name", te.getString(args).String()).
		a := make(zsx.Attributes, 2).
			Set("name", getSymbol(args[0], env).GetValue()).
			Set("content", s)
		return te.transformMeta(a)
		return ev.EvaluateMeta(a)
	}
	te.bind(sexpr.NameSymTypeIDSet, 2, metaSet)
	te.bind(sexpr.NameSymTypeTagSet, 2, metaSet)
	ev.bind(sz.SymTypeIDSet, 2, evalMetaSet)
	ev.bind(sz.SymTypeTagSet, 2, evalMetaSet)
	te.bind(sexpr.NameSymTypeWordSet, 2, metaSet)
}

// EvaluateMeta returns HTML meta object for an attribute.
func (te *TransformEnv) bindBlocks() {
	te.bind(sexpr.NameSymBlock, 0, listArgs)
func (ev *Evaluator) EvaluateMeta(a zsx.Attributes) *sx.Pair {
	return sx.Nil().Cons(EvaluateAttributes(a)).Cons(SymMeta)
	te.bind(sexpr.NameSymPara, 0, func(args *sxpf.List) sxpf.Object {
		for ; args != nil; args = args.Tail() {
			lst, ok := sxpf.GetList(args.Car())
			if !ok || lst != nil {
				break
			}
		}
		return args.Cons(te.symP)
}

func (ev *Evaluator) bindBlocks() {
	ev.bind(zsx.SymBlock, 0, ev.evalList)
	ev.bind(zsx.SymPara, 0, func(args sx.Vector, env *Environment) sx.Object {
		return ev.evalSlice(args, env).Cons(SymP)
	})
	te.bind(sexpr.NameSymHeading, 5, func(args *sxpf.List) sxpf.Object {
		nLevel := te.getInt64(args)
	ev.bind(zsx.SymHeading, 5, func(args sx.Vector, env *Environment) sx.Object {
		nLevel := getInt64(args[0], env)
		if nLevel <= 0 {
			te.err = fmt.Errorf("%v is a negative level", nLevel)
			return sxpf.Nil()
			env.err = fmt.Errorf("%v is a negative heading level", nLevel)
			return sx.Nil()
		}
		level := strconv.FormatInt(nLevel+te.tr.headingOffset, 10)
		level := strconv.FormatInt(nLevel+ev.headingOffset, 10)
		headingSymbol := sx.MakeSymbol("h" + level)

		argAttr := args.Tail()
		a := te.getAttributes(argAttr)
		argFragment := argAttr.Tail().Tail()
		if fragment := te.getString(argFragment).String(); fragment != "" {
			a = a.Set("id", te.tr.unique+fragment)
		a := GetAttributes(args[1], env)
		env.pushAttributes(a)
		defer env.popAttributes()
		if fragment := getString(args[3], env).GetValue(); fragment != "" {
			a = a.Set("id", ev.unique+fragment)
		}

		if result, ok := sxpf.GetList(argFragment.Tail().Car()); ok && result != nil {
		if result, _ := ev.EvaluateList(args[4:], env); result != nil {
			if len(a) > 0 {
				result = result.Cons(te.transformAttribute(a))
				result = result.Cons(EvaluateAttributes(a))
			}
			return result.Cons(te.Make("h" + level))
			return result.Cons(headingSymbol)
		}
		return sxpf.MakeList(te.Make("h"+level), sxpf.MakeString("<MISSING TEXT>"))
		return sx.MakeList(headingSymbol, sx.MakeString("<MISSING TEXT>"))
	})
	te.bind(sexpr.NameSymThematic, 0, func(args *sxpf.List) sxpf.Object {
		result := sxpf.Nil()
		if args != nil {
			if attrList := te.getList(args); attrList != nil {
				result = result.Cons(te.transformAttribute(sexpr.GetAttributes(attrList)))
	ev.bind(zsx.SymThematic, 0, func(args sx.Vector, env *Environment) sx.Object {
		result := sx.Nil()
		if len(args) > 0 {
			if attrList := getList(args[0], env); attrList != nil {
				result = result.Cons(EvaluateAttributes(zsx.GetAttributes(attrList)))
			}
		}
		return result.Cons(te.Make("hr"))
		return result.Cons(SymHR)
	})
	te.bind(sexpr.NameSymListOrdered, 0, te.makeListFn("ol"))
	te.bind(sexpr.NameSymListUnordered, 0, te.makeListFn("ul"))
	te.bind(sexpr.NameSymDescription, 0, func(args *sxpf.List) sxpf.Object {
		if args == nil {
			return sxpf.Nil()

	ev.bind(zsx.SymListOrdered, 1, ev.makeListFn(SymOL))
	ev.bind(zsx.SymListUnordered, 1, ev.makeListFn(SymUL))
	ev.bind(zsx.SymListQuote, 1, func(args sx.Vector, env *Environment) sx.Object {
		if len(args) == 1 {
			return sx.Nil()
		}
		items := sxpf.Nil().Cons(te.Make("dl"))
		curItem := items
		for elem := args; elem != nil; elem = elem.Tail() {
			term := te.getList(elem)
		var result sx.ListBuilder
			curItem = curItem.AppendBang(term.Cons(te.Make("dt")))
			elem = elem.Tail()
			if elem == nil {
				break
		result.Add(symBLOCKQUOTE)
			}
			ddBlock := te.getList(elem)
			if ddBlock == nil {
				break
			}
		if attrs := EvaluateAttributes(GetAttributes(args[0], env)); attrs != nil {
			result.Add(attrs)
		}
			for ddlst := ddBlock; ddlst != nil; ddlst = ddlst.Tail() {
				dditem := te.getList(ddlst)
				curItem = curItem.AppendBang(dditem.Cons(te.Make("dd")))
		for _, elem := range args[1:] {
			if quote, isPair := sx.GetPair(ev.Eval(elem, env)); isPair {
				result.Add(quote.Cons(sxhtml.SymListSplice))
			}
		}
		return items
		return result.List()
	})

	te.bind(sexpr.NameSymListQuote, 0, func(args *sxpf.List) sxpf.Object {
		if args == nil {
			return sxpf.Nil()
		}
		result := sxpf.Nil().Cons(te.Make("blockquote"))
		currResult := result
		for elem := args; elem != nil; elem = elem.Tail() {
	ev.bind(zsx.SymDescription, 1, func(args sx.Vector, env *Environment) sx.Object {
		if len(args) == 1 {
			return sx.Nil()
		}
		var result sx.ListBuilder
		result.Add(symDL)
		if attrs := EvaluateAttributes(GetAttributes(args[0], env)); attrs != nil {
			result.Add(attrs)
		}
		for pos := 1; pos < len(args); pos++ {
			term := ev.evalDescriptionTerm(getList(args[pos], env), env)
			result.Add(term.Cons(symDT))
			pos++
			if pos >= len(args) {
				break
			}
			ddBlock := getList(ev.Eval(args[pos], env), env)
			if ddBlock == nil {
				continue
			}
			for ddlst := range ddBlock.Values() {
				dditem := getList(ddlst, env)
			if quote, ok := elem.Car().(*sxpf.List); ok {
				currResult = currResult.AppendBang(quote.Cons(te.symP))
				result.Add(dditem.Cons(symDD))
			}
		}
		return result
		return result.List()
	})

	te.bind(sexpr.NameSymTable, 1, func(args *sxpf.List) sxpf.Object {
		thead := sxpf.Nil()
		if header := te.getList(args); header != nil {
			thead = sxpf.Nil().Cons(te.transformTableRow(header)).Cons(te.Make("thead"))
	ev.bind(zsx.SymTable, 1, func(args sx.Vector, env *Environment) sx.Object {
		thead := sx.Nil()
		if header := getList(args[0], env); !sx.IsNil(header) {
			thead = sx.Nil().Cons(ev.evalTableRow(symTH, header, env)).Cons(symTHEAD)
		}

		tbody := sxpf.Nil()
		if argBody := args.Tail(); argBody != nil {
		var tbody sx.ListBuilder
		if len(args) > 1 {
			tbody = sxpf.Nil().Cons(te.Make("tbody"))
			curBody := tbody
			for row := argBody; row != nil; row = row.Tail() {
				curBody = curBody.AppendBang(te.transformTableRow(te.getList(row)))
			tbody.Add(symTBODY)
			for _, row := range args[1:] {
				tbody.Add(ev.evalTableRow(symTD, getList(row, env), env))
			}
		}

		table := sxpf.Nil()
		if tbody != nil {
			table = table.Cons(tbody)
		table := sx.Nil()
		if !tbody.IsEmpty() {
			table = table.Cons(tbody.List())
		}
		if thead != nil {
			table = table.Cons(thead)
		}
		if table == nil {
			return sxpf.Nil()
			return sx.Nil()
		}
		return table.Cons(te.Make("table"))
		return table.Cons(symTABLE)
	})
	te.bind(sexpr.NameSymCell, 0, te.makeCellFn(""))
	te.bind(sexpr.NameSymCellCenter, 0, te.makeCellFn("center"))
	te.bind(sexpr.NameSymCellLeft, 0, te.makeCellFn("left"))
	te.bind(sexpr.NameSymCellRight, 0, te.makeCellFn("right"))

	te.bind(sexpr.NameSymRegionBlock, 2, te.makeRegionFn(te.Make("div"), true))
	te.bind(sexpr.NameSymRegionQuote, 2, te.makeRegionFn(te.Make("blockquote"), false))
	ev.bind(zsx.SymCell, 1, func(args sx.Vector, env *Environment) sx.Object {
		tdata := ev.evalSlice(args[1:], env)
		pattrs := getList(args[0], env)
		if alignPairs := pattrs.Assoc(zsx.SymAttrAlign); alignPairs != nil {
			if salign, isString := sx.GetString(alignPairs.Cdr()); isString {
				a := zsx.GetAttributes(pattrs.RemoveAssoc(zsx.SymAttrAlign))
				// Since in Sz there are attributes of align:center|left|right, we can reuse the values.
	te.bind(sexpr.NameSymRegionVerse, 2, te.makeRegionFn(te.Make("div"), false))

				a = a.AddClass(salign.GetValue())
	te.bind(sexpr.NameSymVerbatimComment, 1, func(args *sxpf.List) sxpf.Object {
		if te.getAttributes(args).HasDefault() {
				tdata = tdata.Cons(EvaluateAttributes(a))
			if s := te.getString(args.Tail()); s != "" {
				t := sxpf.MakeString(s.String())
				return sxpf.Nil().Cons(t).Cons(te.Make(sxhtml.NameSymBlockComment))
			}
		}
		return nil
		return tdata
	})

	ev.bind(zsx.SymRegionBlock, 2, ev.makeRegionFn(SymDIV, true))
	ev.bind(zsx.SymRegionQuote, 2, ev.makeRegionFn(symBLOCKQUOTE, false))
	ev.bind(zsx.SymRegionVerse, 2, ev.makeRegionFn(SymDIV, false))
	te.bind(sexpr.NameSymVerbatimEval, 2, func(args *sxpf.List) sxpf.Object {
		return te.transformVerbatim(te.getAttributes(args).AddClass("zs-eval"), te.getString(args.Tail()))

	ev.bind(zsx.SymVerbatimComment, 1, func(args sx.Vector, env *Environment) sx.Object {
		if GetAttributes(args[0], env).HasDefault() {
			if len(args) > 1 {
				if s := getString(args[1], env); s.GetValue() != "" {
					return sx.Nil().Cons(s).Cons(sxhtml.SymBlockComment)
				}
			}
		}
		return nil
	})
	ev.bind(zsx.SymVerbatimEval, 2, func(args sx.Vector, env *Environment) sx.Object {
		return evalVerbatim(GetAttributes(args[0], env).AddClass("zs-eval"), getString(args[1], env))
	})
	te.bind(sexpr.NameSymVerbatimHTML, 2, te.transformHTML)
	te.bind(sexpr.NameSymVerbatimMath, 2, func(args *sxpf.List) sxpf.Object {
		return te.transformVerbatim(te.getAttributes(args).AddClass("zs-math"), te.getString(args.Tail()))
	ev.bind(zsx.SymVerbatimHTML, 2, ev.evalHTML)
	ev.bind(zsx.SymVerbatimMath, 2, func(args sx.Vector, env *Environment) sx.Object {
		return evalVerbatim(GetAttributes(args[0], env).AddClass("zs-math"), getString(args[1], env))
	})
	te.bind(sexpr.NameSymVerbatimProg, 2, func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args)
		content := te.getString(args.Tail())
	ev.bind(zsx.SymVerbatimCode, 2, func(args sx.Vector, env *Environment) sx.Object {
		a := GetAttributes(args[0], env)
		content := getString(args[1], env)
		if a.HasDefault() {
			content = sxpf.MakeString(visibleReplacer.Replace(content.String()))
			content = sx.MakeString(visibleReplacer.Replace(content.GetValue()))
		}
		return te.transformVerbatim(a, content)
		return evalVerbatim(a, content)
	})
	te.bind(sexpr.NameSymVerbatimZettel, 0, func(*sxpf.List) sxpf.Object { return sxpf.Nil() })

	ev.bind(zsx.SymVerbatimZettel, 0, nilFn)
	ev.bind(zsx.SymBLOB, 4, func(args sx.Vector, env *Environment) sx.Object {
	te.bind(sexpr.NameSymBLOB, 3, func(args *sxpf.List) sxpf.Object {
		argSyntax := args.Tail()
		return te.transformBLOB(te.getList(args), te.getString(argSyntax), te.getString(argSyntax.Tail()))
		a := GetAttributes(args[0], env)
		return evalBLOB(a, getList(args[1], env), getString(args[2], env), getString(args[3], env))
	})

	te.bind(sexpr.NameSymTransclude, 2, func(args *sxpf.List) sxpf.Object {
	ev.bind(zsx.SymTransclude, 2, func(args sx.Vector, env *Environment) sx.Object {
		ref, ok := args.Tail().Car().(*sxpf.List)
		if !ok {
			return sxpf.Nil()
		}
		refKind := ref.Car()
		if sxpf.IsNil(refKind) {
			return sxpf.Nil()
		}
		if refSym, refValue := GetReference(args[1], env); refSym != nil {
			if refSym.IsEqualSymbol(zsx.SymRefStateExternal) {
				a := GetAttributes(args[0], env).Set("src", refValue).AddClass("external")
				// TODO: if len(args) > 2, add "alt" attr based on args[2:], as in SymEmbed
				return sx.Nil().Cons(sx.Nil().Cons(EvaluateAttributes(a)).Cons(SymIMG)).Cons(SymP)
			}
		if refValue := te.getString(ref.Tail()); refValue != "" {
			if te.astSF.MustMake(sexpr.NameSymRefStateExternal).IsEqual(refKind) {
				a := te.getAttributes(args).Set("src", refValue.String()).AddClass("external")
				return sxpf.Nil().Cons(sxpf.Nil().Cons(te.transformAttribute(a)).Cons(te.Make("img"))).Cons(te.symP)
			}
			return sxpf.MakeList(
				te.Make(sxhtml.NameSymInlineComment),
				sxpf.MakeString("transclude"),
				refKind,
				sxpf.MakeString("->"),
				refValue,
			return sx.MakeList(
				sxhtml.SymInlineComment,
				sx.MakeString("transclude"),
				refSym,
				sx.MakeString("->"),
				sx.MakeString(refValue),
			)
		}
		return args
		return ev.evalSlice(args, env)
	})
}

func (te *TransformEnv) makeListFn(tag string) transformFn {
func (ev *Evaluator) makeListFn(sym *sx.Symbol) EvalFn {
	sym := te.Make(tag)
	return func(args *sxpf.List) sxpf.Object {
		result := sxpf.Nil().Cons(sym)
		last := result
		for elem := args; elem != nil; elem = elem.Tail() {
			item := sxpf.Nil().Cons(te.Make("li"))
			if res, ok := elem.Car().(*sxpf.List); ok {
				item.ExtendBang(res)
			}
			last = last.AppendBang(item)
		}
		return result
	return func(args sx.Vector, env *Environment) sx.Object {
		var result sx.ListBuilder
		result.Add(sym)
		if attrs := EvaluateAttributes(GetAttributes(args[0], env)); attrs != nil {
			result.Add(attrs)
		}
		if len(args) > 1 {
			for _, elem := range args[1:] {
				item := sx.Nil().Cons(SymLI)
				if res, isPair := sx.GetPair(ev.Eval(elem, env)); isPair {
					item.ExtendBang(res)
				}
				result.Add(item)
			}
		}
		return result.List()
	}
}
func (te *TransformEnv) transformTableRow(cells *sxpf.List) *sxpf.List {
	row := sxpf.Nil().Cons(te.Make("tr"))
	if cells == nil {
		return sxpf.Nil()
	}
	curRow := row
	for cell := cells; cell != nil; cell = cell.Tail() {

func (ev *Evaluator) evalDescriptionTerm(term *sx.Pair, env *Environment) *sx.Pair {
	var result sx.ListBuilder
	for obj := range term.Values() {
		elem := ev.Eval(obj, env)
		curRow = curRow.AppendBang(cell.Car())
		result.Add(elem)
	}
	return row
	return result.List()
}

func (te *TransformEnv) makeCellFn(align string) transformFn {
	return func(args *sxpf.List) sxpf.Object {
func (ev *Evaluator) evalTableRow(sym *sx.Symbol, pairs *sx.Pair, env *Environment) *sx.Pair {
	if pairs == nil {
		return nil
		tdata := args
		if align != "" {
			tdata = tdata.Cons(te.transformAttribute(attrs.Attributes{"class": align}))
		}
		return tdata.Cons(te.Make("td"))
	}
	var row sx.ListBuilder
	row.Add(symTR)
	for obj := range pairs.Values() {
		row.Add(sx.Cons(sym, ev.Eval(obj, env)))
	}
	return row.List()
}

func (te *TransformEnv) makeRegionFn(sym *sxpf.Symbol, genericToClass bool) transformFn {
	return func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args)
func (ev *Evaluator) makeRegionFn(sym *sx.Symbol, genericToClass bool) EvalFn {
	return func(args sx.Vector, env *Environment) sx.Object {
		a := GetAttributes(args[0], env)
		env.pushAttributes(a)
		defer env.popAttributes()
		if genericToClass {
			if val, found := a.Get(""); found {
				a = a.Remove("").AddClass(val)
			}
		}
		var result sx.ListBuilder
		result := sxpf.Nil()
		result.Add(sym)
		if len(a) > 0 {
			result = result.Cons(te.transformAttribute(a))
			result.Add(EvaluateAttributes(a))
		}
		result = result.Cons(sym)
		currResult := result.Last()
		blockArg := args.Tail()
		if region, ok := blockArg.Car().(*sxpf.List); ok {
			currResult = currResult.ExtendBang(region)
		}
		if citeArg := blockArg.Tail(); citeArg != nil {
			if cite, ok := citeArg.Car().(*sxpf.List); ok && cite != nil {
				currResult.AppendBang(cite.Cons(te.Make("cite")))
		if region, isPair := sx.GetPair(args[1]); isPair {
			if evalRegion := ev.EvalPairList(region, env); evalRegion != nil {
				result.ExtendBang(evalRegion)
			}
		}
		if len(args) > 2 {
			if cite, _ := ev.EvaluateList(args[2:], env); cite != nil {
				result.Add(cite.Cons(symCITE))
			}
		}
		return result
		return result.List()
	}
}

func (te *TransformEnv) transformVerbatim(a attrs.Attributes, s sxpf.String) sxpf.Object {
func evalVerbatim(a zsx.Attributes, s sx.String) sx.Object {
	a = setProgLang(a)
	code := sxpf.Nil().Cons(s)
	if al := te.transformAttribute(a); al != nil {
	code := sx.Nil().Cons(s)
	if al := EvaluateAttributes(a); al != nil {
		code = code.Cons(al)
	}
	code = code.Cons(te.Make("code"))
	return sxpf.Nil().Cons(code).Cons(te.Make("pre"))
	code = code.Cons(symCODE)
	return sx.Nil().Cons(code).Cons(symPRE)
}

func (te *TransformEnv) bindInlines() {
	te.bind(sexpr.NameSymInline, 0, listArgs)
	te.bind(sexpr.NameSymText, 1, func(args *sxpf.List) sxpf.Object { return te.getString(args) })
	te.bind(sexpr.NameSymSpace, 0, func(args *sxpf.List) sxpf.Object {
func (ev *Evaluator) bindInlines() {
	ev.bind(zsx.SymInline, 0, ev.evalList)
	ev.bind(zsx.SymText, 1, func(args sx.Vector, env *Environment) sx.Object { return getString(args[0], env) })
	ev.bind(zsx.SymSoft, 0, func(sx.Vector, *Environment) sx.Object { return sx.MakeString(" ") })
		if args.IsNil() {
			return sxpf.MakeString(" ")
		}
		return te.getString(args)
	})
	te.bind(sexpr.NameSymSoft, 0, func(*sxpf.List) sxpf.Object { return sxpf.MakeString(" ") })
	brSym := te.Make("br")
	te.bind(sexpr.NameSymHard, 0, func(*sxpf.List) sxpf.Object { return sxpf.Nil().Cons(brSym) })

	te.bind(sexpr.NameSymLinkInvalid, 2, func(args *sxpf.List) sxpf.Object {
		// a := te.getAttributes(args)
	ev.bind(zsx.SymHard, 0, func(sx.Vector, *Environment) sx.Object { return sx.Nil().Cons(symBR) })

	ev.bind(zsx.SymLink, 2, func(args sx.Vector, env *Environment) sx.Object {
		a := GetAttributes(args[0], env)
		env.pushAttributes(a)
		defer env.popAttributes()
		refSym, refValue := GetReference(args[1], env)
		switch refSym {
		case sz.SymRefStateZettel, zsx.SymRefStateSelf, sz.SymRefStateFound, zsx.SymRefStateHosted, sz.SymRefStateBased:
			return ev.evalLink(a.Set("href", refValue), refValue, args[2:], env)

		case zsx.SymRefStateExternal:
			return ev.evalLink(a.Set("href", refValue).Add("rel", "external"), refValue, args[2:], env)

		case sz.SymRefStateQuery:
			query := "?" + api.QueryKeyQuery + "=" + url.QueryEscape(refValue)
			return ev.evalLink(a.Set("href", query), refValue, args[2:], env)

		case sz.SymRefStateBroken:
			return ev.evalLink(a.AddClass("broken"), refValue, args[2:], env)
		}

		// sz.SymRefStateInvalid or unknown
		var inline *sx.Pair
		if len(args) > 2 {
		refArg := args.Tail()
		inline := refArg.Tail()
			inline = ev.evalSlice(args[2:], env)
		}
		if inline == nil {
			inline = sxpf.Nil().Cons(refArg.Car())
			inline = sx.Nil().Cons(sx.MakeString(refValue))
		}
		return inline.Cons(te.symSpan)
		return inline.Cons(SymSPAN)
	})
	transformHREF := func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args)
		refValue := te.getString(args.Tail())
		return te.transformLink(a.Set("href", refValue.String()), refValue, args.Tail().Tail())
	}
	te.bind(sexpr.NameSymLinkZettel, 2, transformHREF)
	te.bind(sexpr.NameSymLinkSelf, 2, transformHREF)
	te.bind(sexpr.NameSymLinkFound, 2, transformHREF)
	te.bind(sexpr.NameSymLinkBroken, 2, func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args)
		refValue := te.getString(args.Tail())
		return te.transformLink(a.AddClass("broken"), refValue, args.Tail().Tail())
	})
	te.bind(sexpr.NameSymLinkHosted, 2, transformHREF)
	te.bind(sexpr.NameSymLinkBased, 2, transformHREF)
	te.bind(sexpr.NameSymLinkQuery, 2, func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args)
		refValue := te.getString(args.Tail())
		query := "?" + api.QueryKeyQuery + "=" + url.QueryEscape(refValue.String())
		return te.transformLink(a.Set("href", query), refValue, args.Tail().Tail())
	})
	te.bind(sexpr.NameSymLinkExternal, 2, func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args)
		refValue := te.getString(args.Tail())
		return te.transformLink(a.Set("href", refValue.String()).AddClass("external"), refValue, args.Tail().Tail())
	})

	te.bind(sexpr.NameSymEmbed, 3, func(args *sxpf.List) sxpf.Object {
		argRef := args.Tail()
		ref := te.getList(argRef)
		syntax := te.getString(argRef.Tail())
		if syntax == api.ValueSyntaxSVG {
			embedAttr := sxpf.MakeList(
				te.symAttr,
				sxpf.Cons(te.Make("type"), sxpf.MakeString("image/svg+xml")),
				sxpf.Cons(te.Make("src"), sxpf.MakeString("/"+te.getString(ref.Tail()).String()+".svg")),
			)

			return sxpf.MakeList(
				te.Make("figure"),
				sxpf.MakeList(
					te.Make("embed"),
					embedAttr,
	ev.bind(zsx.SymEmbed, 3, func(args sx.Vector, env *Environment) sx.Object {
				),
			)
		}
		a := te.getAttributes(args)
		_, refValue := GetReference(args[1], env)
		a := GetAttributes(args[0], env).Set("src", refValue)
		a = a.Set("src", string(te.getString(ref.Tail())))
		var sb strings.Builder
		te.flattenText(&sb, ref.Tail().Tail().Tail())
		if d := sb.String(); d != "" {
			a = a.Set("alt", d)
		}
		return sxpf.MakeList(te.Make("img"), te.transformAttribute(a))
		if len(args) > 3 {
			var sb strings.Builder
			flattenText(&sb, sx.MakeList(args[3:]...))
			if d := sb.String(); d != "" {
				a = a.Set("alt", d)
			}
		}
		return sx.MakeList(SymIMG, EvaluateAttributes(a))
	})
	te.bind(sexpr.NameSymEmbedBLOB, 3, func(args *sxpf.List) sxpf.Object {
	ev.bind(zsx.SymEmbedBLOB, 3, func(args sx.Vector, env *Environment) sx.Object {
		argSyntax := args.Tail()
		a, syntax, data := te.getAttributes(args), te.getString(argSyntax), te.getString(argSyntax.Tail())
		summary, _ := a.Get(api.KeySummary)
		return te.transformBLOB(
			sxpf.MakeList(te.astSF.MustMake(sexpr.NameSymInline), sxpf.MakeString(summary)),
		a, syntax, data := GetAttributes(args[0], env), getString(args[1], env), getString(args[2], env)
		summary, hasSummary := a.Get(meta.KeySummary)
		if !hasSummary {
			summary = ""
		}
		return evalBLOB(
			a,
			sx.MakeList(sxhtml.SymListSplice, sx.MakeString(summary)),
			syntax,
			data,
		)
	})

	te.bind(sexpr.NameSymCite, 2, func(args *sxpf.List) sxpf.Object {
		result := sxpf.Nil()
	ev.bind(zsx.SymCite, 2, func(args sx.Vector, env *Environment) sx.Object {
		a := GetAttributes(args[0], env)
		env.pushAttributes(a)
		defer env.popAttributes()
		result := sx.Nil()
		argKey := args.Tail()
		if key := te.getString(argKey); key != "" {
			if text := argKey.Tail(); text != nil {
				result = text.Cons(sxpf.MakeString(", "))
		if key := getString(args[1], env); key.GetValue() != "" {
			if len(args) > 2 {
				result = ev.evalSlice(args[2:], env).Cons(sx.MakeString(", "))
			}
			result = result.Cons(key)
		}
		if a := te.getAttributes(args); len(a) > 0 {
			result = result.Cons(te.transformAttribute(a))
		if len(a) > 0 {
			result = result.Cons(EvaluateAttributes(a))
		}
		if result == nil {
			return nil
		}
		return result.Cons(te.symSpan)
		return result.Cons(SymSPAN)
	})

	te.bind(sexpr.NameSymMark, 3, func(args *sxpf.List) sxpf.Object {
	ev.bind(zsx.SymMark, 3, func(args sx.Vector, env *Environment) sx.Object {
		argFragment := args.Tail().Tail()
		result := argFragment.Tail()
		if !te.tr.noLinks {
			if fragment := te.getString(argFragment); fragment != "" {
				a := attrs.Attributes{"id": fragment.String() + te.tr.unique}
				return result.Cons(te.transformAttribute(a)).Cons(te.symA)
		result := ev.evalSlice(args[3:], env)
		if !ev.noLinks {
			if fragment := getString(args[2], env).GetValue(); fragment != "" {
				a := zsx.Attributes{"id": fragment + ev.unique}
				return result.Cons(EvaluateAttributes(a)).Cons(SymA)
			}
		}
		return result.Cons(te.symSpan)
		return result.Cons(SymSPAN)
	})

	te.bind(sexpr.NameSymEndnote, 1, func(args *sxpf.List) sxpf.Object {
		attrPlist := sxpf.Nil()
		if a := te.getAttributes(args); len(a) > 0 {
			if attrs := te.transformAttribute(a); attrs != nil {
	ev.bind(zsx.SymEndnote, 1, func(args sx.Vector, env *Environment) sx.Object {
		a := GetAttributes(args[0], env)
		env.pushAttributes(a)
		defer env.popAttributes()
		attrPlist := sx.Nil()
		if len(a) > 0 {
			if attrs := EvaluateAttributes(a); attrs != nil {
				attrPlist = attrs.Tail()
			}
		}

		text, ok := args.Tail().Car().(*sxpf.List)
		if !ok {
			return sxpf.Nil()
		}
		te.tr.endnotes = append(te.tr.endnotes, endnoteInfo{noteAST: text, noteHx: nil, attrs: attrPlist})
		noteNum := strconv.Itoa(len(te.tr.endnotes))
		noteID := te.tr.unique + noteNum
		hrefAttr := sxpf.Nil().Cons(sxpf.Cons(te.Make("role"), sxpf.MakeString("doc-noteref"))).
			Cons(sxpf.Cons(te.Make("href"), sxpf.MakeString("#fn:"+noteID))).
			Cons(sxpf.Cons(te.tr.symClass, sxpf.MakeString("zs-noteref"))).
			Cons(te.symAttr)
		href := sxpf.Nil().Cons(sxpf.MakeString(noteNum)).Cons(hrefAttr).Cons(te.symA)
		supAttr := sxpf.Nil().Cons(sxpf.Cons(te.Make("id"), sxpf.MakeString("fnref:"+noteID))).Cons(te.symAttr)
		return sxpf.Nil().Cons(href).Cons(supAttr).Cons(te.Make("sup"))
		noteNum := strconv.Itoa(len(env.endnotes) + 1)
		noteID := ev.unique + noteNum
		env.endnotes = append(env.endnotes, endnoteInfo{
			noteID: noteID, noteAST: args[1:], noteHx: nil, attrs: attrPlist})
		hrefAttr := sx.Nil().Cons(sx.Cons(SymAttrRole, sx.MakeString("doc-noteref"))).
			Cons(sx.Cons(SymAttrHref, sx.MakeString("#fn:"+noteID))).
			Cons(sx.Cons(SymAttrClass, sx.MakeString("zs-noteref"))).
			Cons(sxhtml.SymAttr)
		href := sx.Nil().Cons(sx.MakeString(noteNum)).Cons(hrefAttr).Cons(SymA)
		supAttr := sx.Nil().Cons(sx.Cons(SymAttrID, sx.MakeString("fnref:"+noteID))).Cons(sxhtml.SymAttr)
		return sx.Nil().Cons(href).Cons(supAttr).Cons(symSUP)
	})

	ev.bind(zsx.SymFormatDelete, 1, ev.makeFormatFn(symDEL))
	ev.bind(zsx.SymFormatEmph, 1, ev.makeFormatFn(symEM))
	ev.bind(zsx.SymFormatInsert, 1, ev.makeFormatFn(symINS))
	ev.bind(zsx.SymFormatMark, 1, ev.makeFormatFn(symMARK))
	ev.bind(zsx.SymFormatQuote, 1, ev.evalQuote)

	ev.bind(zsx.SymFormatSpan, 1, ev.makeFormatFn(SymSPAN))
	te.bind(sexpr.NameSymFormatDelete, 1, te.makeFormatFn("del"))
	te.bind(sexpr.NameSymFormatEmph, 1, te.makeFormatFn("em"))
	te.bind(sexpr.NameSymFormatInsert, 1, te.makeFormatFn("ins"))
	te.bind(sexpr.NameSymFormatQuote, 1, te.transformQuote)
	te.bind(sexpr.NameSymFormatSpan, 1, te.makeFormatFn("span"))
	te.bind(sexpr.NameSymFormatStrong, 1, te.makeFormatFn("strong"))
	te.bind(sexpr.NameSymFormatSub, 1, te.makeFormatFn("sub"))
	te.bind(sexpr.NameSymFormatSuper, 1, te.makeFormatFn("sup"))
	ev.bind(zsx.SymFormatStrong, 1, ev.makeFormatFn(SymSTRONG))
	ev.bind(zsx.SymFormatSub, 1, ev.makeFormatFn(symSUB))
	ev.bind(zsx.SymFormatSuper, 1, ev.makeFormatFn(symSUP))

	te.bind(sexpr.NameSymLiteralComment, 1, func(args *sxpf.List) sxpf.Object {
		if te.getAttributes(args).HasDefault() {
			if s := te.getString(args.Tail()); s != "" {
				return sxpf.Nil().Cons(s).Cons(te.Make(sxhtml.NameSymInlineComment))
			}
		}
		return sxpf.Nil()
	ev.bind(zsx.SymLiteralComment, 1, func(args sx.Vector, env *Environment) sx.Object {
		if GetAttributes(args[0], env).HasDefault() {
			if len(args) > 1 {
				if s := getString(ev.Eval(args[1], env), env); s.GetValue() != "" {
					return sx.Nil().Cons(s).Cons(sxhtml.SymInlineComment)
				}
			}
		}
		return sx.Nil()
	})
	te.bind(sexpr.NameSymLiteralHTML, 2, te.transformHTML)
	kbdSym := te.Make("kbd")
	te.bind(sexpr.NameSymLiteralInput, 2, func(args *sxpf.List) sxpf.Object {
	ev.bind(zsx.SymLiteralInput, 2, func(args sx.Vector, env *Environment) sx.Object {
		return evalLiteral(args, nil, symKBD, env)
	})
	ev.bind(zsx.SymLiteralMath, 2, func(args sx.Vector, env *Environment) sx.Object {
		a := GetAttributes(args[0], env).AddClass("zs-math")
		return te.transformLiteral(args, nil, kbdSym)
	})
	codeSym := te.Make("code")
	te.bind(sexpr.NameSymLiteralMath, 2, func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args).AddClass("zs-math")
		return te.transformLiteral(args, a, codeSym)
		return evalLiteral(args, a, symCODE, env)
	})
	sampSym := te.Make("samp")
	te.bind(sexpr.NameSymLiteralOutput, 2, func(args *sxpf.List) sxpf.Object {
		return te.transformLiteral(args, nil, sampSym)
	ev.bind(zsx.SymLiteralOutput, 2, func(args sx.Vector, env *Environment) sx.Object {
		return evalLiteral(args, nil, symSAMP, env)
	})
	te.bind(sexpr.NameSymLiteralProg, 2, func(args *sxpf.List) sxpf.Object {
		return te.transformLiteral(args, nil, codeSym)
	ev.bind(zsx.SymLiteralCode, 2, func(args sx.Vector, env *Environment) sx.Object {
		return evalLiteral(args, nil, symCODE, env)
	})

}
	te.bind(sexpr.NameSymLiteralZettel, 0, func(*sxpf.List) sxpf.Object { return sxpf.Nil() })
}


func (ev *Evaluator) makeFormatFn(sym *sx.Symbol) EvalFn {
func (te *TransformEnv) makeFormatFn(tag string) transformFn {
	sym := te.Make(tag)
	return func(args *sxpf.List) sxpf.Object {
		a := te.getAttributes(args)
		if val, found := a.Get(""); found {
	return func(args sx.Vector, env *Environment) sx.Object {
		a := GetAttributes(args[0], env)
		env.pushAttributes(a)
		defer env.popAttributes()
		if val, hasClass := a.Get(""); hasClass {
			a = a.Remove("").AddClass(val)
		}
		res := args.Tail()
		res := ev.evalSlice(args[1:], env)
		if len(a) > 0 {
			res = res.Cons(te.transformAttribute(a))
			res = res.Cons(EvaluateAttributes(a))
		}
		return res.Cons(sym)
	}
}
func (te *TransformEnv) transformQuote(args *sxpf.List) sxpf.Object {
	const langAttr = "lang"
	a := te.getAttributes(args)
	langVal, found := a.Get(langAttr)
	if found {
		a = a.Remove(langAttr)
	}
	if val, found2 := a.Get(""); found2 {
		a = a.Remove("").AddClass(val)
	}

func (ev *Evaluator) evalQuote(args sx.Vector, env *Environment) sx.Object {
	a := GetAttributes(args[0], env)
	env.pushAttributes(a)
	defer env.popAttributes()

	if val, hasClass := a.Get(""); hasClass {
		a = a.Remove("").AddClass(val)
	}
	leftQ, rightQ, withNbsp := env.getQuotes()

	env.quoteNesting++
	res := ev.evalSlice(args[1:], env)
	env.quoteNesting--

	lastPair := res.LastPair()
	if lastPair.IsNil() {
		res = sx.Cons(sx.MakeList(sxhtml.SymNoEscape, sx.MakeString(leftQ), sx.MakeString(rightQ)), sx.Nil())
	} else {
	res := args.Tail()
	if len(a) > 0 {
		res = res.Cons(te.transformAttribute(a))
	}
		if withNbsp {
			lastPair.AppendBang(sx.MakeList(sxhtml.SymNoEscape, sx.MakeString("&nbsp;"), sx.MakeString(rightQ)))
			res = res.Cons(sx.MakeList(sxhtml.SymNoEscape, sx.MakeString(leftQ), sx.MakeString("&nbsp;")))
		} else {
			lastPair.AppendBang(sx.MakeList(sxhtml.SymNoEscape, sx.MakeString(rightQ)))
			res = res.Cons(sx.MakeList(sxhtml.SymNoEscape, sx.MakeString(leftQ)))
		}
	res = res.Cons(te.Make("q"))
	if found {
		res = sxpf.Nil().Cons(res).Cons(te.transformAttribute(attrs.Attributes{}.Set(langAttr, langVal))).Cons(te.symSpan)
	}
	if len(a) > 0 {
		res = res.Cons(EvaluateAttributes(a))
	return res
}

		return res.Cons(SymSPAN)
	}
	return res.Cons(sxhtml.SymListSplice)
}
var visibleReplacer = strings.NewReplacer(" ", "\u2423")

var visibleReplacer = strings.NewReplacer(" ", "\u2423")
func (te *TransformEnv) transformLiteral(args *sxpf.List, a attrs.Attributes, sym *sxpf.Symbol) sxpf.Object {

func evalLiteral(args sx.Vector, a zsx.Attributes, sym *sx.Symbol, env *Environment) sx.Object {
	if a == nil {
		a = te.getAttributes(args)
		a = GetAttributes(args[0], env)
	}
	a = setProgLang(a)
	literal := te.getString(args.Tail()).String()
	literal := getString(args[1], env).GetValue()
	if a.HasDefault() {
		a = a.RemoveDefault()
		literal = visibleReplacer.Replace(literal)
	}
	res := sxpf.Nil().Cons(sxpf.MakeString(literal))
	res := sx.Nil().Cons(sx.MakeString(literal))
	if len(a) > 0 {
		res = res.Cons(te.transformAttribute(a))
		res = res.Cons(EvaluateAttributes(a))
	}
	return res.Cons(sym)
}

func setProgLang(a attrs.Attributes) attrs.Attributes {
func setProgLang(a zsx.Attributes) zsx.Attributes {
	if val, found := a.Get(""); found {
		a = a.AddClass("language-" + val).Remove("")
	}
	return a
}

func (ev *Evaluator) evalHTML(args sx.Vector, env *Environment) sx.Object {
	if s := getString(ev.Eval(args[1], env), env); s.GetValue() != "" && IsSafe(s.GetValue()) {
		return sx.Nil().Cons(s).Cons(sxhtml.SymNoEscape)
	}
	return nil
}
func (te *TransformEnv) transformHTML(args *sxpf.List) sxpf.Object {
	if s := te.getString(args.Tail()); s != "" && IsSafe(s.String()) {
		return sxpf.Nil().Cons(s).Cons(te.symNoEscape)

func evalBLOB(a zsx.Attributes, description *sx.Pair, syntax, data sx.String) sx.Object {
	if data.GetValue() == "" {
		return sx.Nil()
	}
	switch syntax.GetValue() {
	case "":
		return sx.Nil()
	case meta.ValueSyntaxSVG:
		return sx.Nil().Cons(sx.Nil().Cons(data).Cons(sxhtml.SymNoEscape)).Cons(SymP)
	default:
		a = a.Add("src", "data:image/"+syntax.GetValue()+";base64,"+data.GetValue())
		var sb strings.Builder
		flattenText(&sb, description)
		if d := sb.String(); d != "" {
			a = a.Add("alt", d)
		}
		return sx.Nil().Cons(sx.Nil().Cons(EvaluateAttributes(a)).Cons(SymIMG)).Cons(SymP)
	}
}

func flattenText(sb *strings.Builder, lst *sx.Pair) {
	for elem := range lst.Values() {
		switch obj := elem.(type) {
		case sx.String:
			sb.WriteString(obj.GetValue())
		case *sx.Pair:
			flattenText(sb, obj)
		}
	}
}

func (ev *Evaluator) evalList(args sx.Vector, env *Environment) sx.Object {
	return ev.evalSlice(args, env)
}
func nilFn(sx.Vector, *Environment) sx.Object { return sx.Nil() }

// Eval evaluates an object in an environment.
func (ev *Evaluator) Eval(obj sx.Object, env *Environment) sx.Object {
	if env.err != nil {
		return sx.Nil()
	}
	if sx.IsNil(obj) {
		return obj
	}
	lst, isLst := sx.GetPair(obj)
	if !isLst {
		return obj
	}
	sym, found := sx.GetSymbol(lst.Car())
	if !found {
		env.err = fmt.Errorf("symbol expected, but got %T/%v", lst.Car(), lst.Car())
		return sx.Nil()
	}
	symVal := sym.GetValue()
	fn, found := ev.fns[symVal]
	if !found {
		env.err = fmt.Errorf("symbol %q not bound", sym)
		return sx.Nil()
	}
	var args sx.Vector
	for cdr := lst.Cdr(); !sx.IsNil(cdr); {
		pair, isPair := sx.GetPair(cdr)
		if !isPair {
			break
		}
		args = append(args, pair.Car())
		cdr = pair.Cdr()
	}
	if minArgs, hasMinArgs := ev.minArgs[symVal]; hasMinArgs {
		if minArgs > len(args) {
			env.err = fmt.Errorf("%v needs at least %d arguments, but got only %d", sym, minArgs, len(args))
			return sx.Nil()
		}
	}
	result := fn(args, env)
	if env.err != nil {
		return sx.Nil()
	}
	return result
}

func (ev *Evaluator) evalSlice(args sx.Vector, env *Environment) *sx.Pair {
	var result sx.ListBuilder
	for _, arg := range args {
		elem := ev.Eval(arg, env)
		result.Add(elem)
	}
	if env.err == nil {
		return result.List()
	}
	return nil
}

// EvalPairList evaluates a list of lists.
func (ev *Evaluator) EvalPairList(pair *sx.Pair, env *Environment) *sx.Pair {
	var result sx.ListBuilder
	for obj := range pair.Values() {
		elem := ev.Eval(obj, env)
		result.Add(elem)
	}
	if env.err == nil {
		return result.List()
	}
	return nil
}

func (ev *Evaluator) evalLink(a zsx.Attributes, refValue string, inline sx.Vector, env *Environment) sx.Object {
	result := ev.evalSlice(inline, env)
	if len(inline) == 0 {
		result = sx.Nil().Cons(sx.MakeString(refValue))
	}
	if ev.noLinks {
		return result.Cons(SymSPAN)
	}
	return result.Cons(EvaluateAttributes(a)).Cons(SymA)
}

func getSymbol(obj sx.Object, env *Environment) *sx.Symbol {
	if env.err == nil {
		if sym, ok := sx.GetSymbol(obj); ok {
			return sym
		}
		env.err = fmt.Errorf("%v/%T is not a symbol", obj, obj)
	}
	return sx.MakeSymbol("???")
}
func getString(val sx.Object, env *Environment) sx.String {
	if env.err == nil {
		if s, ok := sx.GetString(val); ok {
			return s
		}
		env.err = fmt.Errorf("%v/%T is not a string", val, val)
	}
	return sx.MakeString("")
}
func getList(val sx.Object, env *Environment) *sx.Pair {
	if env.err == nil {
		if res, isPair := sx.GetPair(val); isPair {
			return res
		}
		env.err = fmt.Errorf("%v/%T is not a list", val, val)
	}
	return nil
}

func getInt64(val sx.Object, env *Environment) int64 {
func (te *TransformEnv) transformBLOB(description *sxpf.List, syntax, data sxpf.String) sxpf.Object {
	if data == "" {
		return sxpf.Nil()
	if env.err != nil {
		return -1017
	}
	switch syntax {
	case "":
		return sxpf.Nil()
	if num, ok := sx.GetNumber(val); ok {
		return int64(num.(sx.Int64))
	case api.ValueSyntaxSVG:
		return sxpf.Nil().Cons(sxpf.Nil().Cons(data).Cons(te.symNoEscape)).Cons(te.symP)
	default:
		imgAttr := sxpf.Nil().Cons(sxpf.Cons(te.Make("src"), sxpf.MakeString("data:image/"+syntax.String()+";base64,"+data.String())))
		var sb strings.Builder
		te.flattenText(&sb, description)
		if d := sb.String(); d != "" {
			imgAttr = imgAttr.Cons(sxpf.Cons(te.Make("alt"), sxpf.MakeString(d)))
		}
		return sxpf.Nil().Cons(sxpf.Nil().Cons(imgAttr.Cons(te.symAttr)).Cons(te.Make("img"))).Cons(te.symP)
	}
}

	}
	env.err = fmt.Errorf("%v/%T is not a number", val, val)
	return -1017
}

// GetAttributes evaluates the given arg in the given environment and returns
func (te *TransformEnv) flattenText(sb *strings.Builder, lst *sxpf.List) {
	for elem := lst; elem != nil; elem = elem.Tail() {
		switch obj := elem.Car().(type) {
		case sxpf.String:
			sb.WriteString(obj.String())
		case *sxpf.List:
			te.flattenText(sb, obj)
		}
	}
}

// the contained attributes.
type transformFn func(*sxpf.List) sxpf.Object

func GetAttributes(arg sx.Object, env *Environment) zsx.Attributes {
func (te *TransformEnv) bind(name string, minArity int, fn transformFn) {
	te.astEnv.Bind(te.astSF.MustMake(name), eval.MakeBuiltin(name, func(_ sxpf.Environment, args *sxpf.List) (sxpf.Object, error) {
		if nArgs := args.Length(); nArgs < minArity {
			return sxpf.Nil(), fmt.Errorf("not enough arguments (%d) for form %v (%d)", nArgs, name, minArity)
		}
		res := fn(args)
		return res, te.err
	}))
}

	return zsx.GetAttributes(getList(arg, env))
func (te *TransformEnv) Rebind(name string, fn func(sxpf.Environment, *sxpf.List, sxpf.Callable) sxpf.Object) {
	sym := te.astSF.MustMake(name)
	obj, found := te.astEnv.Lookup(sym)
	if !found {
		panic(sym.String())
	}
}
	preFn, ok := obj.(sxpf.Callable)
	if !ok {
		panic(sym.String())
	}
	te.astEnv.Bind(sym, eval.MakeBuiltin(name, func(env sxpf.Environment, args *sxpf.List) (sxpf.Object, error) {
		res := fn(env, args, preFn)
		return res, te.err
	}))
}


// GetReference returns the reference symbol and the reference value of a reference pair.
func (te *TransformEnv) Make(name string) *sxpf.Symbol { return te.tr.Make(name) }
func (te *TransformEnv) getString(lst *sxpf.List) sxpf.String {
	if te.err != nil {
func GetReference(val sx.Object, env *Environment) (*sx.Symbol, string) {
	if env.err == nil {
		return ""
	}
	val := lst.Car()
	if s, ok := val.(sxpf.String); ok {
		return s
	}
	te.err = fmt.Errorf("%v/%T is not a string", val, val)
		if p := getList(val, env); env.err == nil {
			sym, val := sz.GetReference(p)
			if sym != nil {
				return sym, val
			}
			env.err = fmt.Errorf("%v/%T is not a reference", val, val)
	return ""
}
		}
func (te *TransformEnv) getInt64(lst *sxpf.List) int64 {
	if te.err != nil {
		return -1017
	}
	val := lst.Car()
	if num, ok := val.(*sxpf.Number); ok {
		return num.GetInt64()
	}
	te.err = fmt.Errorf("%v/%T is not a number", val, val)
	return -1017
}
func (te *TransformEnv) getList(lst *sxpf.List) *sxpf.List {
	if te.err == nil {
		val := lst.Car()
		if res, ok := val.(*sxpf.List); ok {
			return res
	return nil, ""
		}
		te.err = fmt.Errorf("%v/%T is not a list", val, val)
	}
	return sxpf.Nil()
}
func (te *TransformEnv) getAttributes(args *sxpf.List) attrs.Attributes {
	return sexpr.GetAttributes(te.getList(args))
}

func (te *TransformEnv) transformLink(a attrs.Attributes, refValue sxpf.String, inline *sxpf.List) sxpf.Object {
	result := inline
	if inline.IsNil() {
		result = sxpf.Nil().Cons(refValue)
	}
	if te.tr.noLinks {
		return result.Cons(te.symSpan)
	}
	return result.Cons(te.transformAttribute(a)).Cons(te.symA)
}

func (te *TransformEnv) transformAttribute(a attrs.Attributes) *sxpf.List {
	return te.tr.TransformAttrbute(a)
}

func (te *TransformEnv) transformMeta(a attrs.Attributes) *sxpf.List {
	return te.tr.TransformMeta(a)
}

var unsafeSnippets = []string{
	"<script", "</script",
	"<iframe", "</iframe",
}

Added sz/const.go.












































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

package sz

import "t73f.de/r/sx"

// Various constants for Zettel data. They are technically variables.
// These are only Zettelstore-specific symbols. The more general symbols are
// defined in t73f.de/r/zsx
var (
	// Symbols for Metanodes
	SymMeta = sx.MakeSymbol("META")

	// Constant symbols for reference states.
	SymRefStateZettel = sx.MakeSymbol("ZETTEL")
	SymRefStateFound  = sx.MakeSymbol("FOUND")
	SymRefStateBroken = sx.MakeSymbol("BROKEN")
	SymRefStateBased  = sx.MakeSymbol("BASED")
	SymRefStateQuery  = sx.MakeSymbol("QUERY")

	// Symbols for metadata types.
	SymTypeCredential = sx.MakeSymbol("CREDENTIAL")
	SymTypeEmpty      = sx.MakeSymbol("EMPTY-STRING")
	SymTypeID         = sx.MakeSymbol("ZID")
	SymTypeIDSet      = sx.MakeSymbol("ZID-SET")
	SymTypeNumber     = sx.MakeSymbol("NUMBER")
	SymTypeString     = sx.MakeSymbol("STRING")
	SymTypeTagSet     = sx.MakeSymbol("TAG-SET")
	SymTypeTimestamp  = sx.MakeSymbol("TIMESTAMP")
	SymTypeURL        = sx.MakeSymbol("URL")
	SymTypeWord       = sx.MakeSymbol("WORD")
)

Added sz/parser.go.














































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2024-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2024-present Detlef Stern
//-----------------------------------------------------------------------------

package sz

import (
	"t73f.de/r/sx"
	"t73f.de/r/zsc/domain/meta"
	"t73f.de/r/zsx"
	"t73f.de/r/zsx/input"
)

// --- Contains some simple parsers

// ---- Syntax: none

// ParseNoneBlocks parses no block.
func ParseNoneBlocks(*input.Input) *sx.Pair { return nil }

// ---- Some plain text syntaxes

// ParsePlainBlocks parses the block as plain text with the given syntax.
func ParsePlainBlocks(inp *input.Input, syntax string) *sx.Pair {
	var sym *sx.Symbol
	if syntax == meta.ValueSyntaxHTML {
		sym = zsx.SymVerbatimHTML
	} else {
		sym = zsx.SymVerbatimCode
	}
	return sx.MakeList(
		sym,
		sx.MakeList(sx.Cons(sx.MakeString(""), sx.MakeString(syntax))),
		sx.MakeString(string(inp.ScanLineContent())),
	)
}

Added sz/parser_test.go.






















































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2024-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2024-present Detlef Stern
//-----------------------------------------------------------------------------

package sz_test

import (
	"testing"

	"t73f.de/r/zsc/sz"
	"t73f.de/r/zsx/input"
)

func TestParseNone(t *testing.T) {
	if got := sz.ParseNoneBlocks(nil); got != nil {
		t.Error("GOTB", got)
	}

	inp := input.NewInput([]byte("1234\n6789"))
	if got := sz.ParseNoneBlocks(inp); got != nil {
		t.Error("GOTI", got)
	}
}

func TestParsePlani(t *testing.T) {
	testcases := []struct {
		src       string
		syntax    string
		expBlocks string
	}{
		{"abc", "html", "(VERBATIM-HTML ((\"\" . \"html\")) \"abc\")"},
		{"abc\ndef", "html", "(VERBATIM-HTML ((\"\" . \"html\")) \"abc\\ndef\")"},
		{"abc", "text", "(VERBATIM-CODE ((\"\" . \"text\")) \"abc\")"},
		{"abc\nDEF", "text", "(VERBATIM-CODE ((\"\" . \"text\")) \"abc\\nDEF\")"},
	}
	for i, tc := range testcases {
		t.Run(tc.syntax+":"+tc.src, func(t *testing.T) {
			inp := input.NewInput([]byte(tc.src))
			if got := sz.ParsePlainBlocks(inp, tc.syntax).String(); tc.expBlocks != got {
				t.Errorf("%d: %q/%v\nexpected: %q\ngot     : %q", i, tc.src, tc.syntax, tc.expBlocks, got)
			}
		})
	}
}

Added sz/ref.go.






































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package sz

import (
	"net/url"
	"strings"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/domain/id"
	"t73f.de/r/zsx"
)

// MakeReference builds a reference node.
func MakeReference(sym *sx.Symbol, val string) *sx.Pair {
	return sx.Cons(sym, sx.Cons(sx.MakeString(val), sx.Nil()))
}

// GetReference returns the reference symbol and value.
func GetReference(ref *sx.Pair) (*sx.Symbol, string) {
	if ref != nil {
		if sym, isSymbol := sx.GetSymbol(ref.Car()); isSymbol {
			val, isString := sx.GetString(ref.Cdr())
			if !isString {
				val, isString = sx.GetString(ref.Tail().Car())
			}
			if isString {
				return sym, val.GetValue()
			}
		}
	}
	return nil, ""
}

// ScanReference scans a string and returns a reference.
//
// This function is very specific for Zettelstore.
func ScanReference(s string) *sx.Pair {
	if invalidReference(s) {
		return MakeReference(zsx.SymRefStateInvalid, s)
	}
	if strings.HasPrefix(s, api.QueryPrefix) {
		return MakeReference(SymRefStateQuery, s[len(api.QueryPrefix):])
	}
	if state, ok := localState(s); ok {
		if state.IsEqualSymbol(SymRefStateBased) {
			s = s[1:]
		}
		_, err := url.Parse(s)
		if err == nil {
			return MakeReference(state, s)
		}
	}
	u, err := url.Parse(s)
	if err != nil {
		return MakeReference(zsx.SymRefStateInvalid, s)
	}
	if !externalURL(u) {
		if _, err = id.Parse(u.Path); err == nil {
			return MakeReference(SymRefStateZettel, s)
		}
		if u.Path == "" && u.Fragment != "" {
			return MakeReference(zsx.SymRefStateSelf, s)
		}
	}
	return MakeReference(zsx.SymRefStateExternal, s)
}

func invalidReference(s string) bool { return s == "" || s == "00000000000000" }

func externalURL(u *url.URL) bool {
	return u.Scheme != "" || u.Opaque != "" || u.Host != "" || u.User != nil
}

func localState(path string) (*sx.Symbol, bool) {
	if len(path) > 0 && path[0] == '/' {
		if len(path) > 1 && path[1] == '/' {
			return SymRefStateBased, true
		}
		return zsx.SymRefStateHosted, true
	}
	if len(path) > 1 && path[0] == '.' {
		if len(path) > 2 && path[1] == '.' && path[2] == '/' {
			return zsx.SymRefStateHosted, true
		}
		return zsx.SymRefStateHosted, path[1] == '/'
	}
	return zsx.SymRefStateInvalid, false
}

Added sz/ref_test.go.








































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
// -----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
// -----------------------------------------------------------------------------

package sz_test

import (
	"testing"

	"t73f.de/r/zsc/sz"
	"t73f.de/r/zsx"
)

func TestParseReference(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		link string
		err  bool
		exp  string
	}{
		{"", true, ""},
		{"12345678901234", false, "(ZETTEL \"12345678901234\")"},
		{"123", false, "(EXTERNAL \"123\")"},
		{",://", true, ""},
	}

	for i, tc := range testcases {
		got := sz.ScanReference(tc.link)
		refSym, _ := sz.GetReference(got)
		gotIsValid := !refSym.IsEqual(zsx.SymRefStateInvalid)
		if gotIsValid == tc.err {
			t.Errorf(
				"TC=%d, expected parse error of %q: %v, but got %q", i, tc.link, tc.err, got)
		}
		if gotIsValid && got.String() != tc.exp {
			t.Errorf("TC=%d, Reference of %q is %q, but got %q", i, tc.link, tc.exp, got)
		}
	}
}

func TestReferenceIsZettelMaterial(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		link       string
		isZettel   bool
		isExternal bool
		isLocal    bool
	}{
		{"", false, false, false},
		{"00000000000000", false, false, false},
		{"http://zettelstore.de/z/ast", false, true, false},
		{"12345678901234", true, false, false},
		{"12345678901234#local", true, false, false},
		{"http://12345678901234", false, true, false},
		{"http://zettelstore.de/z/12345678901234", false, true, false},
		{"http://zettelstore.de/12345678901234", false, true, false},
		{"/12345678901234", false, false, true},
		{"//12345678901234", false, false, true},
		{"./12345678901234", false, false, true},
		{"../12345678901234", false, false, true},
		{".../12345678901234", false, true, false},
	}

	for i, tc := range testcases {
		ref := sz.ScanReference(tc.link)
		refSym, _ := sz.GetReference(ref)
		isZettel := refSym.IsEqual(sz.SymRefStateZettel)
		if isZettel != tc.isZettel {
			t.Errorf(
				"TC=%d, Reference %q isZettel=%v expected, but got %v",
				i,
				tc.link,
				tc.isZettel,
				isZettel)
		}
		isLocal := refSym.IsEqual(zsx.SymRefStateHosted) || refSym.IsEqual(sz.SymRefStateBased)
		if isLocal != tc.isLocal {
			t.Errorf(
				"TC=%d, Reference %q isLocal=%v expected, but got %v",
				i,
				tc.link,
				tc.isLocal, isLocal)
		}
		isExternal := refSym.IsEqual(zsx.SymRefStateExternal)
		if isExternal != tc.isExternal {
			t.Errorf(
				"TC=%d, Reference %q isExternal=%v expected, but got %v",
				i,
				tc.link,
				tc.isExternal,
				isExternal)
		}
	}
}

Added sz/sz.go.










































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package sz contains zettel data handling as sx expressions.
package sz

import (
	"t73f.de/r/sx"
	"t73f.de/r/zsx"
)

// GetMetaContent returns the metadata and the content of a sz encoded zettel.
func GetMetaContent(zettel sx.Object) (Meta, *sx.Pair) {
	if pair, isPair := sx.GetPair(zettel); isPair {
		m := pair.Car()
		if s := pair.Tail(); s != nil {
			if content, isContentPair := sx.GetPair(s.Car()); isContentPair {
				return MakeMeta(m), content
			}
		}
		return MakeMeta(m), nil
	}
	return nil, nil
}

// Meta map metadata keys to MetaValue.
type Meta map[string]MetaValue

// MetaValue is an extended metadata value:
//
//   - Type: the type assiciated with the metata key
//   - Key: the metadata key itself
//   - Value: the metadata value as an (sx-) object.
type MetaValue struct {
	Type  string
	Key   string
	Value sx.Object
}

// MakeMeta build a Meta based on a list of metadata objects.
func MakeMeta(obj sx.Object) Meta {
	if result := doMakeMeta(obj); len(result) > 0 {
		return result
	}
	return nil
}
func doMakeMeta(obj sx.Object) Meta {
	lst, isList := sx.GetPair(obj)
	if !isList || !lst.Car().IsEqual(SymMeta) {
		return nil
	}
	result := make(map[string]MetaValue)
	for node := range lst.Tail().Pairs() {
		if mv, found := makeMetaValue(node.Head()); found {
			result[mv.Key] = mv
		}
	}
	return result
}
func makeMetaValue(mnode *sx.Pair) (MetaValue, bool) {
	var result MetaValue
	typeSym, isSymbol := sx.GetSymbol(mnode.Car())
	if !isSymbol {
		return result, false
	}
	next := mnode.Tail()
	keySym, isSymbol := sx.GetSymbol(next.Car())
	if !isSymbol {
		return result, false
	}
	next = next.Tail()
	result.Type = typeSym.GetValue()
	result.Key = keySym.GetValue()
	result.Value = next.Car()
	return result, true
}

// GetString return the metadata string value associated with the given key.
func (m Meta) GetString(key string) string {
	if v, found := m[key]; found {
		return zsx.GoValue(v.Value)
	}
	return ""
}

// GetPair return the metadata value associated with the given key,
// as a list of objects.
func (m Meta) GetPair(key string) *sx.Pair {
	if mv, found := m[key]; found {
		if pair, isPair := sx.GetPair(mv.Value); isPair {
			return pair
		}
	}
	return nil
}

Added sz/zmk/block.go.



































































































































































































































































































































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package zmk

import (
	"fmt"

	"t73f.de/r/sx"
	"t73f.de/r/zsx"
	"t73f.de/r/zsx/input"
)

// parseBlock parses one block.
func (cp *Parser) parseBlock(lastPara *sx.Pair) (res *sx.Pair, cont bool) {
	inp := cp.inp
	pos := inp.Pos
	if cp.nestingLevel <= maxNestingLevel {
		cp.nestingLevel++
		defer func() { cp.nestingLevel-- }()

		var bn *sx.Pair
		success := false

		switch inp.Ch {
		case input.EOS:
			return nil, false
		case '\n', '\r':
			inp.EatEOL()
			cp.cleanupListsAfterEOL()
			return nil, false
		case ':':
			bn, success = cp.parseColon()
		case '@', '`', runeModGrave, '%', '~', '$':
			cp.clearStacked()
			bn, success = parseVerbatim(inp)
		case '"', '<':
			cp.clearStacked()
			bn, success = cp.parseRegion()
		case '=':
			cp.clearStacked()
			bn, success = cp.parseHeading()
		case '-':
			cp.clearStacked()
			bn, success = parseHRule(inp)
		case '*', '#', '>':
			cp.lastRow = nil
			cp.descrl = nil
			bn, success = cp.parseNestedList()
		case ';':
			cp.lists = nil
			cp.lastRow = nil
			bn, success = cp.parseDefTerm()
		case ' ':
			cp.lastRow = nil
			bn, success = nil, cp.parseIndent()
		case '|':
			cp.lists = nil
			cp.descrl = nil
			bn, success = cp.parseRow(), true
		case '{':
			cp.clearStacked()
			bn, success = cp.parseTransclusion()
		}

		if success {
			return bn, false
		}
	}
	inp.SetPos(pos)
	cp.clearStacked()
	ins := cp.parsePara()
	if startsWithSpaceSoftBreak(ins) {
		ins = ins.Tail().Tail()
	} else if lastPara != nil {
		lastPair := lastPara.LastPair()
		lastPair.ExtendBang(ins)
		return nil, true
	}
	return zsx.MakeParaList(ins), false
}

func startsWithSpaceSoftBreak(ins *sx.Pair) bool {
	if ins == nil {
		return false
	}
	pair0, isPair0 := sx.GetPair(ins.Car())
	if pair0 == nil || !isPair0 {
		return false
	}
	next := ins.Tail()
	if next == nil {
		return false
	}
	pair1, isPair1 := sx.GetPair(next.Car())
	if pair1 == nil || !isPair1 {
		return false
	}

	if pair0.Car().IsEqual(zsx.SymText) && isBreakSym(pair1.Car()) {
		if args := pair0.Tail(); args != nil {
			if val, isString := sx.GetString(args.Car()); isString {
				for _, ch := range val.GetValue() {
					if !input.IsSpace(ch) {
						return false
					}
				}
				return true
			}
		}
	}
	return false
}

var symSeparator = sx.MakeSymbol("sEpArAtOr")

func (cp *Parser) cleanupListsAfterEOL() {
	for _, l := range cp.lists {
		l.LastPair().Head().LastPair().AppendBang(sx.Cons(symSeparator, nil))
	}
	if descrl := cp.descrl; descrl != nil {
		if lastPair, pos := lastPairPos(descrl); pos > 2 && pos%2 != 0 {
			lastPair.Head().LastPair().AppendBang(sx.Cons(symSeparator, nil))
		}
	}
}

// parseColon determines which element should be parsed.
func (cp *Parser) parseColon() (*sx.Pair, bool) {
	inp := cp.inp
	if inp.PeekN(1) == ':' {
		cp.clearStacked()
		return cp.parseRegion()
	}
	return cp.parseDefDescr()
}

// parsePara parses paragraphed inline material as a sx List.
func (cp *Parser) parsePara() *sx.Pair {
	var lb sx.ListBuilder
	for {
		in := cp.parseInline()
		if in == nil {
			return lb.List()
		}
		lb.Add(in)
		if isBreakSym(in.Car()) {
			ch := cp.inp.Ch
			switch ch {
			// Must contain all cases from above switch in parseBlock.
			case input.EOS, '\n', '\r', '@', '`', runeModGrave, '%', '~', '$', '"', '<', '=', '-', '*', '#', '>', ';', ':', ' ', '|', '{':
				return lb.List()
			}
		}
	}
}

// countDelim read from input until a non-delimiter is found and returns number of delimiter chars.
func countDelim(inp *input.Input, delim rune) int {
	cnt := 0
	for inp.Ch == delim {
		cnt++
		inp.Next()
	}
	return cnt
}

// parseVerbatim parses a verbatim block.
func parseVerbatim(inp *input.Input) (*sx.Pair, bool) {
	fch := inp.Ch
	cnt := countDelim(inp, fch)
	if cnt < 3 {
		return nil, false
	}
	attrs := parseBlockAttributes(inp)
	inp.SkipToEOL()
	if inp.Ch == input.EOS {
		return nil, false
	}
	var sym *sx.Symbol
	switch fch {
	case '@':
		sym = zsx.SymVerbatimZettel
	case '`', runeModGrave:
		sym = zsx.SymVerbatimCode
	case '%':
		sym = zsx.SymVerbatimComment
	case '~':
		sym = zsx.SymVerbatimEval
	case '$':
		sym = zsx.SymVerbatimMath
	default:
		panic(fmt.Sprintf("%q is not a verbatim char", fch))
	}
	content := make([]byte, 0, 512)
	for {
		inp.EatEOL()
		posL := inp.Pos
		switch inp.Ch {
		case fch:
			if countDelim(inp, fch) >= cnt {
				inp.SkipToEOL()
				return zsx.MakeVerbatim(sym, attrs, string(content)), true
			}
			inp.SetPos(posL)
		case input.EOS:
			return nil, false
		}
		inp.SkipToEOL()
		if len(content) > 0 {
			content = append(content, '\n')
		}
		content = append(content, inp.Src[posL:inp.Pos]...)
	}
}

// parseRegion parses a block region.
func (cp *Parser) parseRegion() (*sx.Pair, bool) {
	inp := cp.inp
	fch := inp.Ch
	cnt := countDelim(inp, fch)
	if cnt < 3 {
		return nil, false
	}

	var sym *sx.Symbol
	switch fch {
	case ':':
		sym = zsx.SymRegionBlock
	case '<':
		sym = zsx.SymRegionQuote
	case '"':
		sym = zsx.SymRegionVerse
	default:
		panic(fmt.Sprintf("%q is not a region char", fch))
	}
	attrs := parseBlockAttributes(inp)
	inp.SkipToEOL()
	if inp.Ch == input.EOS {
		return nil, false
	}
	var blocksBuilder sx.ListBuilder
	var lastPara *sx.Pair
	inp.EatEOL()
	for {
		posL := inp.Pos
		switch inp.Ch {
		case fch:
			if countDelim(inp, fch) >= cnt {
				ins := cp.parseRegionLastLine()
				return zsx.MakeRegion(sym, attrs, blocksBuilder.List(), ins), true
			}
			inp.SetPos(posL)
		case input.EOS:
			return nil, false
		}
		bn, cont := cp.parseBlock(lastPara)
		if bn != nil {
			blocksBuilder.Add(bn)
		}
		if !cont {
			lastPara = bn
		}
	}
}

// parseRegionLastLine parses the last line of a region and returns its inline text.
func (cp *Parser) parseRegionLastLine() *sx.Pair {
	inp := cp.inp
	cp.clearStacked() // remove any lists defined in the region
	inp.SkipSpace()
	var region sx.ListBuilder
	for {
		switch inp.Ch {
		case input.EOS, '\n', '\r':
			return region.List()
		}
		in := cp.parseInline()
		if in == nil {
			return region.List()
		}
		region.Add(in)
	}
}

// parseHeading parses a head line.
func (cp *Parser) parseHeading() (*sx.Pair, bool) {
	inp := cp.inp
	delims := countDelim(inp, inp.Ch)
	if delims < 3 {
		return nil, false
	}
	if inp.Ch != ' ' {
		return nil, false
	}
	inp.Next()
	inp.SkipSpace()
	if delims > 7 {
		delims = 7
	}
	level := delims - 2
	var attrs *sx.Pair
	var text sx.ListBuilder
	for {
		if input.IsEOLEOS(inp.Ch) {
			return zsx.MakeHeading(level, attrs, text.List(), "", ""), true
		}
		in := cp.parseInline()
		if in == nil {
			return zsx.MakeHeading(level, attrs, text.List(), "", ""), true
		}
		text.Add(in)
		if inp.Ch == '{' && inp.Peek() != '{' {
			attrs = parseBlockAttributes(inp)
			inp.SkipToEOL()
			return zsx.MakeHeading(level, attrs, text.List(), "", ""), true
		}
	}
}

// parseHRule parses a horizontal rule.
func parseHRule(inp *input.Input) (*sx.Pair, bool) {
	if countDelim(inp, inp.Ch) < 3 {
		return nil, false
	}

	attrs := parseBlockAttributes(inp)
	inp.SkipToEOL()
	return zsx.MakeThematic(attrs), true
}

// parseNestedList parses a list.
func (cp *Parser) parseNestedList() (*sx.Pair, bool) {
	inp := cp.inp
	kinds := parseNestedListKinds(inp)
	if len(kinds) == 0 {
		return nil, false
	}
	inp.SkipSpace()
	if !kinds[len(kinds)-1].IsEqual(zsx.SymListQuote) && input.IsEOLEOS(inp.Ch) {
		return nil, false
	}

	if len(kinds) < len(cp.lists) {
		cp.lists = cp.lists[:len(kinds)]
	}
	ln, newLnCount := cp.buildNestedList(kinds)
	pv := cp.parseLinePara()
	bn := zsx.MakeBlock()
	if pv != nil {
		bn.AppendBang(zsx.MakeParaList(pv))
	}
	lastItemPair := ln.LastPair()
	lastItemPair.AppendBang(bn)
	return cp.cleanupParsedNestedList(newLnCount)
}

func parseNestedListKinds(inp *input.Input) []*sx.Symbol {
	result := make([]*sx.Symbol, 0, 8)
	for {
		var sym *sx.Symbol
		switch inp.Ch {
		case '*':
			sym = zsx.SymListUnordered
		case '#':
			sym = zsx.SymListOrdered
		case '>':
			sym = zsx.SymListQuote
		default:
			panic(fmt.Sprintf("%q is not a region char", inp.Ch))
		}
		result = append(result, sym)
		switch inp.Next() {
		case '*', '#', '>':
		case ' ', input.EOS, '\n', '\r':
			return result
		default:
			return nil
		}
	}
}

func (cp *Parser) buildNestedList(kinds []*sx.Symbol) (ln *sx.Pair, newLnCount int) {
	for i, kind := range kinds {
		if i < len(cp.lists) {
			if !cp.lists[i].Car().IsEqual(kind) {
				ln = sx.Cons(kind, sx.Cons(sx.Nil(), sx.Nil()))
				newLnCount++
				cp.lists[i] = ln
				cp.lists = cp.lists[:i+1]
			} else {
				ln = cp.lists[i]
			}
		} else {
			ln = sx.Cons(kind, sx.Cons(sx.Nil(), sx.Nil()))
			newLnCount++
			cp.lists = append(cp.lists, ln)
		}
	}
	return ln, newLnCount
}

func (cp *Parser) cleanupParsedNestedList(newLnCount int) (*sx.Pair, bool) {
	childPos := len(cp.lists) - 1
	parentPos := childPos - 1
	for range newLnCount {
		if parentPos < 0 {
			return cp.lists[0], true
		}
		parentLn := cp.lists[parentPos]
		childLn := cp.lists[childPos]
		if firstParent := parentLn.Tail().Tail(); firstParent != nil {
			// Add list to last item of the parent list
			lastParent := firstParent.LastPair()
			lastParent.Head().LastPair().AppendBang(childLn)
		} else {
			// Set list to first child of parent.
			parentLn.LastPair().AppendBang(zsx.MakeBlock(cp.lists[childPos]))
		}
		childPos--
		parentPos--
	}
	return nil, true
}

// parseDefTerm parses a term of a definition list.
func (cp *Parser) parseDefTerm() (res *sx.Pair, success bool) {
	inp := cp.inp
	if inp.Next() != ' ' {
		return nil, false
	}
	inp.Next()
	inp.SkipSpace()
	descrl := cp.descrl
	if descrl == nil {
		descrl = sx.Cons(zsx.SymDescription, sx.Cons(sx.Nil(), sx.Nil()))
		cp.descrl = descrl
		res = descrl
	}
	lastPair, pos := lastPairPos(descrl)
	for first := true; ; first = false {
		in := cp.parseInline()
		if in == nil {
			if pos%2 != 0 {
				// lastPair is either the empty description list or the last block of definitions
				return nil, false
			}
			// lastPair is the definition term
			return res, true
		}
		if pos%2 != 0 {
			// lastPair is either the empty description list or the last block of definitions
			lastPair = lastPair.AppendBang(sx.Cons(in, nil))
			pos++
		} else if first {
			// Previous term had no description
			lastPair = lastPair.
				AppendBang(zsx.MakeBlock()).
				AppendBang(sx.Cons(in, nil))
			pos += 2
		} else {
			// lastPair is the term part and we need to append the inline list just read
			lastPair.Head().LastPair().AppendBang(in)
		}
		if isBreakSym(in.Car()) {
			return res, true
		}
	}
}

// parseDefDescr parses a description of a definition list.
func (cp *Parser) parseDefDescr() (res *sx.Pair, success bool) {
	inp := cp.inp
	if inp.Next() != ' ' {
		return nil, false
	}
	inp.Next()
	inp.SkipSpace()
	descrl := cp.descrl
	lastPair, lpPos := lastPairPos(descrl)
	if descrl == nil || lpPos < 0 {
		// No term given
		return nil, false
	}

	pn := cp.parseLinePara()
	if pn == nil {
		return nil, false
	}

	newDef := zsx.MakeBlock(zsx.MakeParaList(pn))
	if lpPos%2 == 0 {
		// Just a term, but no definitions
		lastPair.AppendBang(zsx.MakeBlock(newDef))
	} else {
		// lastPara points a the last definition
		lastPair.Head().LastPair().AppendBang(newDef)
	}
	return nil, true
}

func lastPairPos(p *sx.Pair) (*sx.Pair, int) {
	cnt := 0
	for node := p; node != nil; {
		next := node.Tail()
		if next == nil {
			return node, cnt
		}
		node = next
		cnt++
	}
	return nil, -1
}

// parseIndent parses initial spaces to continue a list.
func (cp *Parser) parseIndent() bool {
	inp := cp.inp
	cnt := 0
	for {
		if inp.Next() != ' ' {
			break
		}
		cnt++
	}
	if cp.lists != nil {
		return cp.parseIndentForList(cnt)
	}
	if cp.descrl != nil {
		return cp.parseIndentForDescription(cnt)
	}
	return false
}

func (cp *Parser) parseIndentForList(cnt int) bool {
	if len(cp.lists) < cnt {
		cnt = len(cp.lists)
	}
	cp.lists = cp.lists[:cnt]
	if cnt == 0 {
		return false
	}
	pv := cp.parseLinePara()
	if pv == nil {
		return false
	}
	ln := cp.lists[cnt-1]
	lbn := ln.LastPair().Head()
	lpn := lbn.LastPair().Head()
	if lpn.Car().IsEqual(zsx.SymPara) {
		lpn.LastPair().SetCdr(pv)
	} else {
		lbn.LastPair().AppendBang(zsx.MakeParaList(pv))
	}
	return true
}

func (cp *Parser) parseIndentForDescription(cnt int) bool {
	descrl := cp.descrl
	lastPair, pos := lastPairPos(descrl)
	if cnt < 1 || pos < 2 {
		return false
	}
	if pos%2 == 0 {
		// Continuation of a definition term
		for {
			in := cp.parseInline()
			if in == nil {
				return true
			}
			lastPair.Head().LastPair().AppendBang(in)
			if isBreakSym(in.Car()) {
				return true
			}
		}
	}

	// Continuation of a definition description.
	// Either it is a continuation of a definition paragraph, or it is a new paragraph.
	pn := cp.parseLinePara()
	if pn == nil {
		return false
	}

	bn := lastPair.Head()

	// Check for new paragraph
	for curr := bn.Tail(); curr != nil; {
		obj := curr.Head()
		if obj == nil {
			break
		}
		next := curr.Tail()
		if next == nil {
			break
		}
		if symSeparator.IsEqual(next.Head().Car()) {
			// It is a new paragraph!
			obj.LastPair().AppendBang(zsx.MakeParaList(pn))
			return true
		}
		curr = next
	}

	// Continuation of existing paragraph
	para := bn.LastPair().Head().LastPair().Head()
	if para.Car().IsEqual(zsx.SymPara) {
		para.LastPair().SetCdr(pn)
	} else {
		bn.LastPair().AppendBang(zsx.MakeParaList(pn))
	}
	return true
}

// parseLinePara parses one paragraph of inline material.
func (cp *Parser) parseLinePara() *sx.Pair {
	var lb sx.ListBuilder
	for {
		in := cp.parseInline()
		if in == nil {
			return lb.List()
		}
		lb.Add(in)
		if isBreakSym(in.Car()) {
			return lb.List()
		}
	}
}

// parseRow parse one table row.
func (cp *Parser) parseRow() *sx.Pair {
	inp := cp.inp
	if inp.Peek() == '%' {
		inp.SkipToEOL()
		return nil
	}

	var row sx.ListBuilder
	for {
		inp.Next()
		cell := cp.parseCell()
		if cell != nil {
			row.Add(cell)
		}
		switch inp.Ch {
		case '\n', '\r':
			inp.EatEOL()
			fallthrough
		case input.EOS:
			// add to table
			if cp.lastRow == nil {
				if row.IsEmpty() {
					return nil
				}
				cp.lastRow = sx.Cons(row.List(), nil)
				return cp.lastRow.Cons(nil).Cons(zsx.SymTable)
			}
			cp.lastRow = cp.lastRow.AppendBang(row.List())
			return nil
		}
		// inp.Ch must be '|'
	}
}

// parseCell parses one single cell of a table row.
func (cp *Parser) parseCell() *sx.Pair {
	inp := cp.inp
	var cell sx.ListBuilder
	for {
		if input.IsEOLEOS(inp.Ch) {
			if cell.IsEmpty() {
				return nil
			}
			return zsx.MakeCell(nil, cell.List())
		}
		if inp.Ch == '|' {
			return zsx.MakeCell(nil, cell.List())
		}

		in := cp.parseInline()
		cell.Add(in)
	}
}

// parseTransclusion parses '{' '{' '{' ZID '}' '}' '}'
func (cp *Parser) parseTransclusion() (*sx.Pair, bool) {
	inp := cp.inp
	if countDelim(inp, '{') != 3 {
		return nil, false
	}
	posA, posE := inp.Pos, 0

loop:

	for {
		switch inp.Ch {
		case input.EOS:
			return nil, false
		case '\n', '\r', ' ', '\t':
			if !cp.isSpaceReference(inp.Src[posA:]) {
				return nil, false
			}
		case '\\':
			switch inp.Next() {
			case input.EOS, '\n', '\r':
				return nil, false
			}
		case '}':
			posE = inp.Pos
			if posA >= posE {
				return nil, false
			}
			if inp.Next() != '}' {
				continue
			}
			if inp.Next() != '}' {
				continue
			}
			break loop
		}
		inp.Next()
	}
	inp.Next() // consume last '}'
	attrs := parseBlockAttributes(inp)
	inp.SkipToEOL()
	refText := string(inp.Src[posA:posE])
	ref := cp.scanReference(refText)
	return zsx.MakeTransclusion(attrs, ref, sx.Nil()), true
}

Added sz/zmk/inline.go.
























































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package zmk

import (
	"fmt"
	"slices"
	"strings"

	"t73f.de/r/sx"
	"t73f.de/r/zsx"
	"t73f.de/r/zsx/input"
)

func (cp *Parser) parseInline() *sx.Pair {
	inp := cp.inp
	pos := inp.Pos
	if cp.nestingLevel <= maxNestingLevel {
		cp.nestingLevel++
		defer func() { cp.nestingLevel-- }()

		var in *sx.Pair
		success := false
		switch inp.Ch {
		case input.EOS:
			return nil
		case '\n', '\r':
			return parseSoftBreak(inp)
		case '[':
			switch inp.Next() {
			case '[':
				in, success = cp.parseLink('[', ']')
			case '@':
				in, success = cp.parseCite()
			case '^':
				in, success = cp.parseEndnote()
			case '!':
				in, success = cp.parseMark()
			}
		case '{':
			if inp.Next() == '{' {
				in, success = cp.parseEmbed('{', '}')
			}
		case '%':
			in, success = parseComment(inp)
		case '_', '*', '>', '~', '^', ',', '"', '#', ':':
			in, success = cp.parseFormat()
		case '\'', '`', '=', runeModGrave:
			in, success = parseLiteral(inp)
		case '$':
			in, success = parseLiteralMath(inp)
		case '\\':
			return parseBackslash(inp)
		case '-':
			in, success = parseNdash(inp)
		case '&':
			in, success = parseEntity(inp)
		}
		if success {
			return in
		}
	}
	inp.SetPos(pos)
	return parseText(inp)
}

func parseText(inp *input.Input) *sx.Pair { return zsx.MakeText(parseString(inp)) }

func parseString(inp *input.Input) string {
	pos := inp.Pos
	if inp.Ch == '\\' {
		inp.Next()
		return parseBackslashRest(inp)
	}
	for {
		switch inp.Next() {
		// The following case must contain all runes that occur in parseInline!
		// Plus the closing brackets ] and } and ) and the middle |
		case input.EOS, '\n', '\r', '[', ']', '{', '}', '(', ')', '|', '%', '_', '*', '>', '~', '^', ',', '"', '#', ':', '\'', '`', runeModGrave, '$', '=', '\\', '-', '&':
			return string(inp.Src[pos:inp.Pos])
		}
	}
}

func parseBackslash(inp *input.Input) *sx.Pair {
	switch inp.Next() {
	case '\n', '\r':
		inp.EatEOL()
		return zsx.MakeHard()
	default:
		return zsx.MakeText(parseBackslashRest(inp))
	}
}

func parseBackslashRest(inp *input.Input) string {
	if input.IsEOLEOS(inp.Ch) {
		return "\\"
	}
	if inp.Ch == ' ' {
		inp.Next()
		return "\u00a0"
	}
	pos := inp.Pos
	inp.Next()
	return string(inp.Src[pos:inp.Pos])
}

func parseSoftBreak(inp *input.Input) *sx.Pair {
	inp.EatEOL()
	return zsx.MakeSoft()
}

func (cp *Parser) parseLink(openCh, closeCh rune) (*sx.Pair, bool) {
	if refString, text, ok := cp.parseReference(openCh, closeCh); ok {
		attrs := parseInlineAttributes(cp.inp)
		if len(refString) > 0 {
			ref := cp.scanReference(refString)
			return zsx.MakeLink(attrs, ref, text), true
		}
	}
	return nil, false
}
func (cp *Parser) parseEmbed(openCh, closeCh rune) (*sx.Pair, bool) {
	if refString, text, ok := cp.parseReference(openCh, closeCh); ok {
		attrs := parseInlineAttributes(cp.inp)
		if len(refString) > 0 {
			return zsx.MakeEmbed(attrs, cp.scanReference(refString), "", text), true
		}
	}
	return nil, false
}

func (cp *Parser) parseReference(openCh, closeCh rune) (string, *sx.Pair, bool) {
	inp := cp.inp
	inp.Next()
	inp.SkipSpace()
	if inp.Ch == openCh {
		// Additional opening chars result in a fail
		return "", nil, false
	}
	var lb sx.ListBuilder
	pos := inp.Pos
	if !cp.isSpaceReference(inp.Src[pos:]) {
		hasSpace, ok := readReferenceToSep(inp, closeCh)
		if !ok {
			return "", nil, false
		}
		if inp.Ch == '|' { // First part must be inline text
			if pos == inp.Pos { // [[| or {{|
				return "", nil, false
			}
			cp.inp = input.NewInput(inp.Src[pos:inp.Pos])
			for {
				in := cp.parseInline()
				if in == nil {
					break
				}
				lb.Add(in)
			}
			cp.inp = inp
			inp.Next()
		} else {
			if hasSpace {
				return "", nil, false
			}
			inp.SetPos(pos)
		}
	}

	inp.SkipSpace()
	pos = inp.Pos
	if !cp.readReferenceToClose(closeCh) {
		return "", nil, false
	}
	ref := strings.TrimSpace(string(inp.Src[pos:inp.Pos]))
	if inp.Next() != closeCh {
		return "", nil, false
	}
	inp.Next()
	return ref, lb.List(), true
}

func readReferenceToSep(inp *input.Input, closeCh rune) (bool, bool) {
	hasSpace := false
	for {
		switch inp.Ch {
		case input.EOS:
			return false, false
		case '\n', '\r', ' ':
			hasSpace = true
		case '|':
			return hasSpace, true
		case '\\':
			switch inp.Next() {
			case input.EOS:
				return false, false
			case '\n', '\r':
				hasSpace = true
			}
		case '%':
			if inp.Next() == '%' {
				inp.SkipToEOL()
			}
			continue
		case closeCh:
			if inp.Next() == closeCh {
				return hasSpace, true
			}
			continue
		}
		inp.Next()
	}
}

func (cp *Parser) readReferenceToClose(closeCh rune) bool {
	inp := cp.inp
	pos := inp.Pos
	for {
		switch inp.Ch {
		case input.EOS:
			return false
		case '\t', '\r', '\n', ' ':
			if !cp.isSpaceReference(inp.Src[pos:]) {
				return false
			}
		case '\\':
			switch inp.Next() {
			case input.EOS, '\n', '\r':
				return false
			}
		case closeCh:
			return true
		}
		inp.Next()
	}
}

func (cp *Parser) parseCite() (*sx.Pair, bool) {
	inp := cp.inp
	switch inp.Next() {
	case ' ', ',', '|', ']', '\n', '\r':
		return nil, false
	}
	pos := inp.Pos
loop:
	for {
		switch inp.Ch {
		case input.EOS:
			return nil, false
		case ' ', ',', '|', ']', '\n', '\r':
			break loop
		}
		inp.Next()
	}
	posL := inp.Pos
	switch inp.Ch {
	case ' ', ',', '|':
		inp.Next()
	}
	ins, ok := cp.parseLinkLikeRest()
	if !ok {
		return nil, false
	}
	attrs := parseInlineAttributes(inp)
	return zsx.MakeCite(attrs, string(inp.Src[pos:posL]), ins), true
}

func (cp *Parser) parseEndnote() (*sx.Pair, bool) {
	cp.inp.Next()
	ins, ok := cp.parseLinkLikeRest()
	if !ok {
		return nil, false
	}
	attrs := parseInlineAttributes(cp.inp)
	return zsx.MakeEndnote(attrs, ins), true
}

func (cp *Parser) parseMark() (*sx.Pair, bool) {
	inp := cp.inp
	inp.Next()
	pos := inp.Pos
	for inp.Ch != '|' && inp.Ch != ']' {
		if !isNameRune(inp.Ch) {
			return nil, false
		}
		inp.Next()
	}
	mark := string(inp.Src[pos:inp.Pos])
	var ins *sx.Pair
	if inp.Ch == '|' {
		inp.Next()
		var ok bool
		ins, ok = cp.parseLinkLikeRest()
		if !ok {
			return nil, false
		}
	} else {
		inp.Next()
	}
	return zsx.MakeMark(mark, "", "", ins), true
	// Problematisch ist, dass hier noch nicht mn.Fragment und mn.Slug gesetzt werden.
	// Evtl. muss es ein PreMark-Symbol geben
}

func (cp *Parser) parseLinkLikeRest() (*sx.Pair, bool) {
	var ins sx.ListBuilder
	inp := cp.inp
	inp.SkipSpace()
	for inp.Ch != ']' {
		in := cp.parseInline()
		if in == nil {
			return nil, false
		}
		ins.Add(in)
		if input.IsEOLEOS(inp.Ch) && isBreakSym(in.Car()) {
			return nil, false
		}
	}
	inp.Next()
	return ins.List(), true
}

func parseComment(inp *input.Input) (*sx.Pair, bool) {
	if inp.Next() != '%' {
		return nil, false
	}
	for inp.Ch == '%' {
		inp.Next()
	}
	attrs := parseInlineAttributes(inp)
	inp.SkipSpace()
	pos := inp.Pos
	for {
		if input.IsEOLEOS(inp.Ch) {
			return zsx.MakeLiteral(zsx.SymLiteralComment, attrs, string(inp.Src[pos:inp.Pos])), true
		}
		inp.Next()
	}
}

var mapRuneFormat = map[rune]*sx.Symbol{
	'_': zsx.SymFormatEmph,
	'*': zsx.SymFormatStrong,
	'>': zsx.SymFormatInsert,
	'~': zsx.SymFormatDelete,
	'^': zsx.SymFormatSuper,
	',': zsx.SymFormatSub,
	'"': zsx.SymFormatQuote,
	'#': zsx.SymFormatMark,
	':': zsx.SymFormatSpan,
}

func (cp *Parser) parseFormat() (*sx.Pair, bool) {
	inp := cp.inp
	fch := inp.Ch
	symFormat, ok := mapRuneFormat[fch]
	if !ok {
		panic(fmt.Sprintf("%q is not a formatting char", fch))
	}
	// read 2nd formatting character
	if inp.Next() != fch {
		return nil, false
	}
	inp.Next()
	var inlines sx.ListBuilder
	for {
		if inp.Ch == input.EOS {
			return nil, false
		}
		if inp.Ch == fch {
			if inp.Next() == fch {
				inp.Next()
				attrs := parseInlineAttributes(inp)
				return zsx.MakeFormat(symFormat, attrs, inlines.List()), true
			}
			inlines.Add(zsx.MakeText(string(fch)))
		} else if in := cp.parseInline(); in != nil {
			if input.IsEOLEOS(inp.Ch) && isBreakSym(in.Car()) {
				return nil, false
			}
			inlines.Add(in)
		}
	}
}

var mapRuneLiteral = map[rune]*sx.Symbol{
	'`':          zsx.SymLiteralCode,
	runeModGrave: zsx.SymLiteralCode,
	'\'':         zsx.SymLiteralInput,
	'=':          zsx.SymLiteralOutput,
	// No '$': sz.SymLiteralMath, because pairing literal math is a little different
}

func parseLiteral(inp *input.Input) (*sx.Pair, bool) {
	fch := inp.Ch
	symLiteral, ok := mapRuneLiteral[fch]
	if !ok {
		panic(fmt.Sprintf("%q is not a formatting char", fch))
	}
	// read 2nd formatting character
	if inp.Next() != fch {
		return nil, false
	}
	inp.Next()
	var sb strings.Builder
	for {
		if inp.Ch == input.EOS {
			return nil, false
		}
		if inp.Ch == fch {
			if inp.Peek() == fch {
				inp.Next()
				inp.Next()
				return zsx.MakeLiteral(symLiteral, parseInlineAttributes(inp), sb.String()), true
			}
			sb.WriteRune(fch)
			inp.Next()
		} else {
			s := parseString(inp)
			sb.WriteString(s)
		}
	}
}

func parseLiteralMath(inp *input.Input) (res *sx.Pair, success bool) {
	// read 2nd formatting character
	if inp.Next() != '$' {
		return nil, false
	}
	inp.Next()
	pos := inp.Pos
	for {
		if inp.Ch == input.EOS {
			return nil, false
		}
		if inp.Ch == '$' && inp.Peek() == '$' {
			content := slices.Clone(inp.Src[pos:inp.Pos])
			inp.Next()
			inp.Next()
			return zsx.MakeLiteral(zsx.SymLiteralMath, parseInlineAttributes(inp), string(content)), true
		}
		inp.Next()
	}
}

func parseNdash(inp *input.Input) (*sx.Pair, bool) {
	if inp.Peek() != inp.Ch {
		return nil, false
	}
	inp.Next()
	inp.Next()
	return zsx.MakeText("\u2013"), true
}

func parseEntity(inp *input.Input) (*sx.Pair, bool) {
	if text, ok := zsx.ScanEntity(inp); ok {
		return zsx.MakeText(text), true
	}
	return nil, false
}

Added sz/zmk/post-processor.go.













































































































































































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

package zmk

import (
	"strings"

	"t73f.de/r/sx"
	"t73f.de/r/zsx"
)

var symInVerse = sx.MakeSymbol("in-verse")
var symNoBlock = sx.MakeSymbol("no-block")

type postProcessor struct{}

func (pp *postProcessor) VisitBefore(lst *sx.Pair, env *sx.Pair) (sx.Object, bool) {
	if lst == nil {
		return nil, true
	}
	sym, isSym := sx.GetSymbol(lst.Car())
	if !isSym {
		panic(lst)
	}
	if fn, found := symMap[sym]; found {
		return fn(pp, lst, env), true
	}
	return nil, false
}

func (pp *postProcessor) VisitAfter(lst *sx.Pair, _ *sx.Pair) sx.Object { return lst }

func (pp *postProcessor) visitPairList(lst *sx.Pair, env *sx.Pair) *sx.Pair {
	var pList sx.ListBuilder
	for node := range lst.Pairs() {
		if elem, isPair := sx.GetPair(zsx.Walk(pp, node.Head(), env)); isPair && elem != nil {
			pList.Add(elem)
		}
	}
	return pList.List()
}

var symMap map[*sx.Symbol]func(*postProcessor, *sx.Pair, *sx.Pair) *sx.Pair

func init() {
	symMap = map[*sx.Symbol]func(*postProcessor, *sx.Pair, *sx.Pair) *sx.Pair{
		zsx.SymBlock:           postProcessBlockList,
		zsx.SymPara:            postProcessInlineList,
		zsx.SymRegionBlock:     postProcessRegion,
		zsx.SymRegionQuote:     postProcessRegion,
		zsx.SymRegionVerse:     postProcessRegionVerse,
		zsx.SymVerbatimComment: postProcessVerbatim,
		zsx.SymVerbatimEval:    postProcessVerbatim,
		zsx.SymVerbatimMath:    postProcessVerbatim,
		zsx.SymVerbatimCode:    postProcessVerbatim,
		zsx.SymVerbatimZettel:  postProcessVerbatim,
		zsx.SymHeading:         postProcessHeading,
		zsx.SymListOrdered:     postProcessItemList,
		zsx.SymListUnordered:   postProcessItemList,
		zsx.SymListQuote:       postProcessQuoteList,
		zsx.SymDescription:     postProcessDescription,
		zsx.SymTable:           postProcessTable,

		zsx.SymInline:       postProcessInlineList,
		zsx.SymText:         postProcessText,
		zsx.SymSoft:         postProcessSoft,
		zsx.SymEndnote:      postProcessEndnote,
		zsx.SymMark:         postProcessMark,
		zsx.SymLink:         postProcessInlines4,
		zsx.SymEmbed:        postProcessEmbed,
		zsx.SymCite:         postProcessInlines4,
		zsx.SymFormatDelete: postProcessFormat,
		zsx.SymFormatEmph:   postProcessFormat,
		zsx.SymFormatInsert: postProcessFormat,
		zsx.SymFormatMark:   postProcessFormat,
		zsx.SymFormatQuote:  postProcessFormat,
		zsx.SymFormatStrong: postProcessFormat,
		zsx.SymFormatSpan:   postProcessFormat,
		zsx.SymFormatSub:    postProcessFormat,
		zsx.SymFormatSuper:  postProcessFormat,

		symSeparator: ignoreProcess,
	}
}

func ignoreProcess(*postProcessor, *sx.Pair, *sx.Pair) *sx.Pair { return nil }

func postProcessBlockList(pp *postProcessor, lst *sx.Pair, env *sx.Pair) *sx.Pair {
	result := pp.visitPairList(lst.Tail(), env)
	if result == nil {
		if noBlockPair := env.Assoc(symNoBlock); noBlockPair == nil || sx.IsTrue(noBlockPair.Cdr()) {
			return nil
		}
	}
	return result.Cons(lst.Car())
}

func postProcessInlineList(pp *postProcessor, lst *sx.Pair, env *sx.Pair) *sx.Pair {
	sym := lst.Car()
	if rest := pp.visitInlines(lst.Tail(), env); rest != nil {
		return rest.Cons(sym)
	}
	return nil
}

func postProcessRegion(pp *postProcessor, rn *sx.Pair, env *sx.Pair) *sx.Pair {
	return doPostProcessRegion(pp, rn, env, env)
}

func postProcessRegionVerse(pp *postProcessor, rn *sx.Pair, env *sx.Pair) *sx.Pair {
	return doPostProcessRegion(pp, rn, env.Cons(sx.Cons(symInVerse, nil)), env)
}

func doPostProcessRegion(pp *postProcessor, rn *sx.Pair, envBlock, envInline *sx.Pair) *sx.Pair {
	sym := rn.Car().(*sx.Symbol)
	next := rn.Tail()
	attrs := next.Car().(*sx.Pair)
	next = next.Tail()
	blocks := pp.visitPairList(next.Head(), envBlock)
	text := pp.visitInlines(next.Tail(), envInline)
	if blocks == nil && text == nil {
		return nil
	}
	return zsx.MakeRegion(sym, attrs, blocks, text)
}

func postProcessVerbatim(_ *postProcessor, verb *sx.Pair, _ *sx.Pair) *sx.Pair {
	if content, isString := sx.GetString(verb.Tail().Tail().Car()); isString && content.GetValue() != "" {
		return verb
	}
	return nil
}

func postProcessHeading(pp *postProcessor, hn *sx.Pair, env *sx.Pair) *sx.Pair {
	next := hn.Tail()
	level := next.Car().(sx.Int64)
	next = next.Tail()
	attrs := next.Car().(*sx.Pair)
	next = next.Tail()
	slug := next.Car().(sx.String)
	next = next.Tail()
	fragment := next.Car().(sx.String)
	if text := pp.visitInlines(next.Tail(), env); text != nil {
		return zsx.MakeHeading(int(level), attrs, text, slug.GetValue(), fragment.GetValue())
	}
	return nil
}

func postProcessItemList(pp *postProcessor, ln *sx.Pair, env *sx.Pair) *sx.Pair {
	attrs := ln.Tail().Head()
	elems := pp.visitListElems(ln.Tail(), env)
	if elems == nil {
		return nil
	}
	return zsx.MakeList(ln.Car().(*sx.Symbol), attrs, elems)
}

func postProcessQuoteList(pp *postProcessor, ln *sx.Pair, env *sx.Pair) *sx.Pair {
	attrs := ln.Tail().Head()
	elems := pp.visitListElems(ln.Tail(), env.Cons(sx.Cons(symNoBlock, nil)))

	// Collect multiple paragraph items into one item.

	var newElems sx.ListBuilder
	var newPara sx.ListBuilder

	addtoParagraph := func() {
		if !newPara.IsEmpty() {
			newElems.Add(sx.MakeList(zsx.SymBlock, newPara.List().Cons(zsx.SymPara)))
			newPara.Reset()
		}
	}
	for node := range elems.Pairs() {
		item := node.Head()
		if !item.Car().IsEqual(zsx.SymBlock) {
			continue
		}
		itemTail := item.Tail()
		if itemTail == nil || itemTail.Tail() != nil {
			addtoParagraph()
			newElems.Add(item)
			continue
		}
		if pn := itemTail.Head(); pn.Car().IsEqual(zsx.SymPara) {
			if !newPara.IsEmpty() {
				newPara.Add(sx.Cons(zsx.SymSoft, nil))
			}
			newPara.ExtendBang(pn.Tail())
			continue
		}
		addtoParagraph()
		newElems.Add(item)
	}
	addtoParagraph()
	return zsx.MakeList(ln.Car().(*sx.Symbol), attrs, newElems.List())
}

func (pp *postProcessor) visitListElems(ln *sx.Pair, env *sx.Pair) *sx.Pair {
	var pList sx.ListBuilder
	for node := range ln.Tail().Pairs() {
		if elem := zsx.Walk(pp, node.Head(), env); elem != nil {
			pList.Add(elem)
		}
	}
	return pList.List()
}

func postProcessDescription(pp *postProcessor, dl *sx.Pair, env *sx.Pair) *sx.Pair {
	attrs := dl.Tail().Head()
	var dList sx.ListBuilder
	isTerm := false
	for node := range dl.Tail().Tail().Pairs() {
		isTerm = !isTerm
		if isTerm {
			dList.Add(pp.visitInlines(node.Head(), env))
		} else {
			dList.Add(zsx.Walk(pp, node.Head(), env))
		}
	}
	return dList.List().Cons(attrs).Cons(dl.Car())
}

func postProcessTable(pp *postProcessor, tbl *sx.Pair, env *sx.Pair) *sx.Pair {
	sym := tbl.Car()
	next := tbl.Tail()
	header := next.Head()
	if header != nil {
		// Already post-processed
		return tbl
	}
	rows, width := pp.visitRows(next.Tail(), env)
	if rows == nil {
		// Header and row are nil -> no table
		return nil
	}
	header, rows, align := splitTableHeader(rows, width)
	alignRow(header, align)
	for node := range rows.Pairs() {
		alignRow(node.Head(), align)
	}
	return rows.Cons(header).Cons(sym)
}

func (pp *postProcessor) visitRows(rows *sx.Pair, env *sx.Pair) (*sx.Pair, int) {
	maxWidth := 0
	var pRows sx.ListBuilder
	for node := range rows.Pairs() {
		row := node.Head()
		row, width := pp.visitCells(row, env)
		if maxWidth < width {
			maxWidth = width
		}
		pRows.Add(row)
	}
	return pRows.List(), maxWidth
}

func (pp *postProcessor) visitCells(cells *sx.Pair, env *sx.Pair) (*sx.Pair, int) {
	width := 0
	var pCells sx.ListBuilder
	for node := range cells.Pairs() {
		cell := node.Head()
		rest := cell.Tail()
		attrs := rest.Head()
		ins := pp.visitInlines(rest.Tail(), env)
		pCells.Add(zsx.MakeCell(attrs, ins))
		width++
	}
	return pCells.List(), width
}

func splitTableHeader(rows *sx.Pair, width int) (header, realRows *sx.Pair, align []byte) {
	align = make([]byte, width)

	foundHeader := false
	cellCount := 0

	// assert: rows != nil (checked in postProcessTable)
	for node := range rows.Head().Pairs() {
		cell := node.Head()
		cellCount++
		rest := cell.Tail() // attrs := rest.Head()
		cellInlines := rest.Tail()
		if cellInlines == nil {
			continue
		}

		// elem is first cell inline element
		elem := cellInlines.Head()
		if elem.Car().IsEqual(zsx.SymText) {
			if s, isString := sx.GetString(elem.Tail().Car()); isString && s.GetValue() != "" {
				str := s.GetValue()
				if str[0] == '=' {
					foundHeader = true
					elem.SetCdr(sx.Cons(sx.MakeString(str[1:]), nil))
				}
			}
		}

		// move to the last cell inline element
		for {
			next := cellInlines.Tail()
			if next == nil {
				break
			}
			cellInlines = next
		}

		elem = cellInlines.Head()
		if elem.Car().IsEqual(zsx.SymText) {
			if s, isString := sx.GetString(elem.Tail().Car()); isString && s.GetValue() != "" {
				str := s.GetValue()
				lastByte := str[len(str)-1]
				if cellAlign, isValid := getCellAlignment(lastByte); isValid {
					elem.SetCdr(sx.Cons(sx.MakeString(str[0:len(str)-1]), nil))
					rest.SetCar(makeCellAttrs(cellAlign))
				}
				align[cellCount-1] = lastByte
			}
		}
	}

	if !foundHeader {
		return nil, rows, align
	}

	return rows.Head(), rows.Tail(), align
}

func alignRow(row *sx.Pair, defaultAlign []byte) {
	if row == nil {
		return
	}
	var lastCellNode *sx.Pair
	cellColumnNo := 0
	for node := range row.Pairs() {
		lastCellNode = node
		cell := node.Head()
		cellColumnNo++
		rest := cell.Tail() // attrs := rest.Head()
		if cellAlign, isValid := getCellAlignment(defaultAlign[cellColumnNo-1]); isValid {
			rest.SetCar(makeCellAttrs(cellAlign))
		}
		cellInlines := rest.Tail()
		if cellInlines == nil {
			continue
		}

		// elem is first cell inline element
		elem := cellInlines.Head()
		if elem.Car().IsEqual(zsx.SymText) {
			if s, isString := sx.GetString(elem.Tail().Car()); isString && s.GetValue() != "" {
				str := s.GetValue()
				cellAlign, isValid := getCellAlignment(str[0])
				if isValid {
					elem.SetCdr(sx.Cons(sx.MakeString(str[1:]), nil))
					rest.SetCar(makeCellAttrs(cellAlign))
				}
			}
		}
	}

	for cellColumnNo < len(defaultAlign) {
		var attrs *sx.Pair
		if cellAlign, isValid := getCellAlignment(defaultAlign[cellColumnNo]); isValid {
			attrs = makeCellAttrs(cellAlign)
		}
		lastCellNode = lastCellNode.AppendBang(zsx.MakeCell(attrs, nil))
		cellColumnNo++
	}
}

func makeCellAttrs(align sx.String) *sx.Pair {
	return sx.Cons(sx.Cons(zsx.SymAttrAlign, align), sx.Nil())
}

func getCellAlignment(ch byte) (sx.String, bool) {
	switch ch {
	case ':':
		return zsx.AttrAlignCenter, true
	case '<':
		return zsx.AttrAlignLeft, true
	case '>':
		return zsx.AttrAlignRight, true
	default:
		return sx.MakeString(""), false
	}
}

func (pp *postProcessor) visitInlines(lst *sx.Pair, env *sx.Pair) *sx.Pair {
	length := lst.Length()
	if length <= 0 {
		return nil
	}
	inVerse := env.Assoc(symInVerse) != nil
	vector := make([]*sx.Pair, 0, length)
	// 1st phase: process all childs, ignore ' ' / '\t' at start, and merge some elements
	for node := range lst.Pairs() {
		elem, isPair := sx.GetPair(zsx.Walk(pp, node.Head(), env))
		if !isPair || elem == nil {
			continue
		}
		elemSym := elem.Car()
		elemTail := elem.Tail()

		if inVerse && elemSym.IsEqual(zsx.SymText) {
			if s, isString := sx.GetString(elemTail.Car()); isString {
				verseText := s.GetValue()
				verseText = strings.ReplaceAll(verseText, " ", "\u00a0")
				elemTail.SetCar(sx.MakeString(verseText))
			}
		}

		if len(vector) == 0 {
			// If the 1st element is a TEXT, remove all ' ', '\t' at the beginning, if outside a verse block.
			if !elemSym.IsEqual(zsx.SymText) {
				vector = append(vector, elem)
				continue
			}

			elemText := elemTail.Car().(sx.String).GetValue()
			if elemText != "" && (elemText[0] == ' ' || elemText[0] == '\t') {
				for elemText != "" {
					if ch := elemText[0]; ch != ' ' && ch != '\t' {
						break
					}
					elemText = elemText[1:]
				}
				elemTail.SetCar(sx.MakeString(elemText))
			}
			if elemText != "" {
				vector = append(vector, elem)
			}
			continue
		}
		last := vector[len(vector)-1]
		lastSym := last.Car()

		if lastSym.IsEqual(zsx.SymText) && elemSym.IsEqual(zsx.SymText) {
			// Merge two TEXT elements into one
			lastText := last.Tail().Car().(sx.String).GetValue()
			elemText := elem.Tail().Car().(sx.String).GetValue()
			last.SetCdr(sx.Cons(sx.MakeString(lastText+elemText), sx.Nil()))
			continue
		}

		if lastSym.IsEqual(zsx.SymText) && elemSym.IsEqual(zsx.SymSoft) {
			// Merge (TEXT "... ") (SOFT) to (TEXT "...") (HARD)
			lastTail := last.Tail()
			if lastText := lastTail.Car().(sx.String).GetValue(); strings.HasSuffix(lastText, " ") {
				newText := removeTrailingSpaces(lastText)
				if newText == "" {
					vector[len(vector)-1] = sx.Cons(zsx.SymHard, sx.Nil())
					continue
				}
				lastTail.SetCar(sx.MakeString(newText))
				elemSym = zsx.SymHard
				elem.SetCar(elemSym)
			}
		}

		vector = append(vector, elem)
	}
	if len(vector) == 0 {
		return nil
	}

	// 2nd phase: remove (SOFT), (HARD) at the end, remove trailing spaces in (TEXT "...")
	lastPos := len(vector) - 1
	for lastPos >= 0 {
		elem := vector[lastPos]
		elemSym := elem.Car()
		if elemSym.IsEqual(zsx.SymText) {
			elemTail := elem.Tail()
			elemText := elemTail.Car().(sx.String).GetValue()
			newText := removeTrailingSpaces(elemText)
			if newText != "" {
				elemTail.SetCar(sx.MakeString(newText))
				break
			}
			lastPos--
		} else if isBreakSym(elemSym) {
			lastPos--
		} else {
			break
		}
	}
	if lastPos < 0 {
		return nil
	}

	result := sx.Cons(vector[0], nil)
	curr := result
	for i := 1; i <= lastPos; i++ {
		curr = curr.AppendBang(vector[i])
	}
	return result
}

func removeTrailingSpaces(s string) string {
	for len(s) > 0 {
		if ch := s[len(s)-1]; ch != ' ' && ch != '\t' {
			return s
		}
		s = s[0 : len(s)-1]
	}
	return ""
}

func postProcessText(_ *postProcessor, txt *sx.Pair, _ *sx.Pair) *sx.Pair {
	if tail := txt.Tail(); tail != nil {
		if content, isString := sx.GetString(tail.Car()); isString && content.GetValue() != "" {
			return txt
		}
	}
	return nil
}

func postProcessSoft(_ *postProcessor, sn *sx.Pair, env *sx.Pair) *sx.Pair {
	if env.Assoc(symInVerse) == nil {
		return sn
	}
	return sx.Cons(zsx.SymHard, nil)
}

func postProcessEndnote(pp *postProcessor, en *sx.Pair, env *sx.Pair) *sx.Pair {
	next := en.Tail()
	attrs := next.Car().(*sx.Pair)
	if text := pp.visitInlines(next.Tail(), env); text != nil {
		return zsx.MakeEndnote(attrs, text)
	}
	return zsx.MakeEndnote(attrs, sx.Nil())
}

func postProcessMark(pp *postProcessor, en *sx.Pair, env *sx.Pair) *sx.Pair {
	next := en.Tail()
	mark := next.Car().(sx.String)
	next = next.Tail()
	slug := next.Car().(sx.String)
	next = next.Tail()
	fragment := next.Car().(sx.String)
	text := pp.visitInlines(next.Tail(), env)
	return zsx.MakeMark(mark.GetValue(), slug.GetValue(), fragment.GetValue(), text)
}

func postProcessInlines4(pp *postProcessor, ln *sx.Pair, env *sx.Pair) *sx.Pair {
	sym := ln.Car()
	next := ln.Tail()
	attrs := next.Car()
	next = next.Tail()
	val3 := next.Car()
	text := pp.visitInlines(next.Tail(), env)
	return text.Cons(val3).Cons(attrs).Cons(sym)
}

func postProcessEmbed(pp *postProcessor, ln *sx.Pair, env *sx.Pair) *sx.Pair {
	next := ln.Tail()
	attrs := next.Car().(*sx.Pair)
	next = next.Tail()
	ref := next.Car()
	next = next.Tail()
	syntax := next.Car().(sx.String)
	text := pp.visitInlines(next.Tail(), env)
	return zsx.MakeEmbed(attrs, ref, syntax.GetValue(), text)
}

func postProcessFormat(pp *postProcessor, fn *sx.Pair, env *sx.Pair) *sx.Pair {
	symFormat := fn.Car().(*sx.Symbol)
	next := fn.Tail() // Attrs
	attrs := next.Car().(*sx.Pair)
	next = next.Tail() // Possible inlines
	if next == nil {
		return fn
	}
	inlines := pp.visitInlines(next, env)
	return zsx.MakeFormat(symFormat, attrs, inlines)
}

Added sz/zmk/zmk.go.



















































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package zmk provides a parser for zettelmarkup.
package zmk

import (
	"maps"
	"slices"
	"strings"
	"unicode"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/api"
	"t73f.de/r/zsc/sz"
	"t73f.de/r/zsx"
	"t73f.de/r/zsx/input"
)

// Parser allows to parse its plain text input into Zettelmarkup.
type Parser struct {
	inp          *input.Input // Input stream
	lists        []*sx.Pair   // Stack of lists
	lastRow      *sx.Pair     // Last row of table, or nil if not in table.
	descrl       *sx.Pair     // Current description list
	nestingLevel int          // Count nesting of block and inline elements

	scanReference    func(string) *sx.Pair // Builds a reference node from a given string reference
	isSpaceReference func([]byte) bool     // Returns true, if src starts with a reference that allows white space
}

// Initialize the parser with the input stream and a reference scanner.
func (cp *Parser) Initialize(inp *input.Input) {
	var zeroParser Parser
	*cp = zeroParser
	cp.inp = inp
	cp.scanReference = sz.ScanReference
	cp.isSpaceReference = withQueryPrefix
}

// Parse tries to parse the input as a block element.
func (cp *Parser) Parse() *sx.Pair {

	var lastPara *sx.Pair
	var blkBuild sx.ListBuilder
	for cp.inp.Ch != input.EOS {
		bn, cont := cp.parseBlock(lastPara)
		if bn != nil {
			blkBuild.Add(bn)
		}
		if !cont {
			if bn.Car().IsEqual(zsx.SymPara) {
				lastPara = bn
			} else {
				lastPara = nil
			}
		}
	}
	if cp.nestingLevel != 0 {
		panic("Nesting level was not decremented")
	}

	bnl := blkBuild.List()
	var pp postProcessor
	if bs := pp.visitPairList(bnl, nil); bs != nil {
		return bs.Cons(zsx.SymBlock)
	}
	return nil
}

func withQueryPrefix(src []byte) bool {
	return len(src) > len(api.QueryPrefix) && string(src[:len(api.QueryPrefix)]) == api.QueryPrefix
}

// runeModGrave is Unicode code point U+02CB (715) called "MODIFIER LETTER
// GRAVE ACCENT". On the iPad it is much more easier to type in this code point
// than U+0060 (96) "Grave accent" (aka backtick). Therefore, U+02CB will be
// considered equivalent to U+0060.
const runeModGrave = 'Ë‹' // This is NOT '`'!

const maxNestingLevel = 50

// clearStacked removes all multi-line nodes from parser.
func (cp *Parser) clearStacked() {
	cp.lists = nil
	cp.lastRow = nil
	cp.descrl = nil
}

type attrMap map[string]string

func (attrs attrMap) updateAttrs(key, val string) {
	if prevVal := attrs[key]; len(prevVal) > 0 {
		attrs[key] = prevVal + " " + val
	} else {
		attrs[key] = val
	}
}

func (attrs attrMap) asPairAssoc() *sx.Pair {
	var lb sx.ListBuilder
	for _, key := range slices.Sorted(maps.Keys(attrs)) {
		lb.Add(sx.Cons(sx.MakeString(key), sx.MakeString(attrs[key])))
	}
	return lb.List()
}

func parseNormalAttribute(inp *input.Input, attrs attrMap) bool {
	posK := inp.Pos
	for isNameRune(inp.Ch) {
		inp.Next()
	}
	if posK == inp.Pos {
		return false
	}
	key := string(inp.Src[posK:inp.Pos])
	if inp.Ch != '=' {
		attrs[key] = ""
		return true
	}
	return parseAttributeValue(inp, key, attrs)
}

func parseAttributeValue(inp *input.Input, key string, attrs attrMap) bool {
	if inp.Next() == '"' {
		return parseQuotedAttributeValue(inp, key, attrs)
	}
	posV := inp.Pos
	for {
		switch inp.Ch {
		case input.EOS:
			return false
		case '\n', '\r', ' ', ',', '}':
			attrs.updateAttrs(key, string(inp.Src[posV:inp.Pos]))
			return true
		}
		inp.Next()
	}
}

func parseQuotedAttributeValue(inp *input.Input, key string, attrs attrMap) bool {
	inp.Next()
	var sb strings.Builder
	for {
		switch inp.Ch {
		case input.EOS:
			return false
		case '"':
			attrs.updateAttrs(key, sb.String())
			inp.Next()
			return true
		case '\\':
			switch inp.Next() {
			case input.EOS, '\n', '\r':
				return false
			}
			fallthrough
		default:
			sb.WriteRune(inp.Ch)
			inp.Next()
		}
	}

}

func parseBlockAttributes(inp *input.Input) *sx.Pair {
	pos := inp.Pos
	for isNameRune(inp.Ch) {
		inp.Next()
	}
	if pos < inp.Pos {
		return attrMap{"": string(inp.Src[pos:inp.Pos])}.asPairAssoc()
	}

	// No immediate name: skip spaces
	inp.SkipSpace()
	return parseInlineAttributes(inp)
}

func parseInlineAttributes(inp *input.Input) *sx.Pair {
	pos := inp.Pos
	if attrs, success := doParseAttributes(inp); success {
		return attrs
	}
	inp.SetPos(pos)
	return nil
}

// doParseAttributes reads attributes.
func doParseAttributes(inp *input.Input) (*sx.Pair, bool) {
	if inp.Ch != '{' {
		return nil, false
	}
	inp.Next()
	a := attrMap{}
	if !parseAttributeValues(inp, a) {
		return nil, false
	}
	inp.Next()
	return a.asPairAssoc(), true
}

func parseAttributeValues(inp *input.Input, a attrMap) bool {
	for {
		skipSpaceLine(inp)
		switch inp.Ch {
		case input.EOS:
			return false
		case '}':
			return true
		case '.':
			inp.Next()
			posC := inp.Pos
			for isNameRune(inp.Ch) {
				inp.Next()
			}
			if posC == inp.Pos {
				return false
			}
			a.updateAttrs("class", string(inp.Src[posC:inp.Pos]))
		case '=':
			delete(a, "")
			if !parseAttributeValue(inp, "", a) {
				return false
			}
		default:
			if !parseNormalAttribute(inp, a) {
				return false
			}
		}

		switch inp.Ch {
		case '}':
			return true
		case '\n', '\r':
		case ' ', ',':
			inp.Next()
		default:
			return false
		}
	}
}

func skipSpaceLine(inp *input.Input) {
	for {
		switch inp.Ch {
		case ' ':
			inp.Next()
		case '\n', '\r':
			inp.EatEOL()
		default:
			return
		}
	}
}

func isNameRune(ch rune) bool {
	return unicode.IsLetter(ch) || unicode.IsDigit(ch) || ch == '-' || ch == '_'
}

// isBreakSym return true if the object is either a soft or a hard break symbol.
func isBreakSym(obj sx.Object) bool {
	return zsx.SymSoft.IsEqual(obj) || zsx.SymHard.IsEqual(obj)
}

Added sz/zmk/zmk_fuzz_test.go.
































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

package zmk_test

import (
	"testing"

	"t73f.de/r/zsc/sz/zmk"
	"t73f.de/r/zsx/input"
)

func FuzzParseBlocks(f *testing.F) {
	var parser zmk.Parser
	f.Fuzz(func(t *testing.T, src []byte) {
		t.Parallel()
		inp := input.NewInput(src)
		parser.Initialize(inp)
		parser.Parse()
	})
}

Added sz/zmk/zmk_test.go.























































































































































































































































































































































































































































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2020-present Detlef Stern
//-----------------------------------------------------------------------------

// Package zmk_test provides some tests for the zettelmarkup parser.
package zmk_test

import (
	"fmt"
	"strings"
	"testing"

	"t73f.de/r/sx"
	"t73f.de/r/zsc/sz/zmk"
	"t73f.de/r/zsx"
	"t73f.de/r/zsx/input"
)

type TestCase struct{ source, want string }
type TestCases []TestCase
type symbolMap map[string]*sx.Symbol

func replace(s string, sm symbolMap, tcs TestCases) TestCases {
	var sym string
	if len(sm) > 0 {
		sym = sm[s].GetValue()
	}
	var testCases TestCases
	for _, tc := range tcs {
		source := strings.ReplaceAll(tc.source, "$", s)
		want := tc.want
		if sym != "" {
			want = strings.ReplaceAll(want, "$%", sym)
		}
		want = strings.ReplaceAll(want, "$", s)
		testCases = append(testCases, TestCase{source, want})
	}
	return testCases
}

func checkTcs(t *testing.T, tcs TestCases) {
	t.Helper()

	var parser zmk.Parser
	for tcn, tc := range tcs {
		t.Run(fmt.Sprintf("TC=%02d,src=%q", tcn, tc.source), func(st *testing.T) {
			st.Helper()
			inp := input.NewInput([]byte(tc.source))
			parser.Initialize(inp)
			ast := parser.Parse()
			zsx.Walk(astWalker{}, ast, nil)
			got := ast.String()
			if tc.want != got {
				st.Errorf("\nwant=%q\n got=%q", tc.want, got)
			}
		})
	}
}

type astWalker struct{}

func (astWalker) VisitBefore(node *sx.Pair, env *sx.Pair) (sx.Object, bool) { return sx.Nil(), false }
func (astWalker) VisitAfter(node *sx.Pair, env *sx.Pair) sx.Object          { return node }

func TestEOL(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"", "()"},
		{"\n", "()"},
		{"\r", "()"},
		{"\r\n", "()"},
		{"\n\n", "()"},
	})
}

func TestText(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"abcd", "(BLOCK (PARA (TEXT \"abcd\")))"},
		{"ab cd", "(BLOCK (PARA (TEXT \"ab cd\")))"},
		{"abcd ", "(BLOCK (PARA (TEXT \"abcd\")))"},
		{" abcd", "(BLOCK (PARA (TEXT \"abcd\")))"},
		{"\\", "(BLOCK (PARA (TEXT \"\\\\\")))"},
		{"\\\n", "()"},
		{"\\\ndef", "(BLOCK (PARA (HARD) (TEXT \"def\")))"},
		{"\\\r", "()"},
		{"\\\rdef", "(BLOCK (PARA (HARD) (TEXT \"def\")))"},
		{"\\\r\n", "()"},
		{"\\\r\ndef", "(BLOCK (PARA (HARD) (TEXT \"def\")))"},
		{"\\a", "(BLOCK (PARA (TEXT \"a\")))"},
		{"\\aa", "(BLOCK (PARA (TEXT \"aa\")))"},
		{"a\\a", "(BLOCK (PARA (TEXT \"aa\")))"},
		{"\\+", "(BLOCK (PARA (TEXT \"+\")))"},
		{"\\ ", "(BLOCK (PARA (TEXT \"\u00a0\")))"},
		{"http://a, http://b", "(BLOCK (PARA (TEXT \"http://a, http://b\")))"},
	})
}

func TestSpace(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{" ", "()"},
		{"\t", "()"},
		{"  ", "()"},
	})
}

func TestSoftBreak(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"x\ny", "(BLOCK (PARA (TEXT \"x\") (SOFT) (TEXT \"y\")))"},
		{"z\n", "(BLOCK (PARA (TEXT \"z\")))"},
		{" \n ", "()"},
		{" \n", "()"},
	})
}

func TestHardBreak(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"x  \ny", "(BLOCK (PARA (TEXT \"x\") (HARD) (TEXT \"y\")))"},
		{"z  \n", "(BLOCK (PARA (TEXT \"z\")))"},
		{"   \n ", "()"},
		{"   \n", "()"},
	})
}

func TestLink(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[", "(BLOCK (PARA (TEXT \"[\")))"},
		{"[[", "(BLOCK (PARA (TEXT \"[[\")))"},
		{"[[|", "(BLOCK (PARA (TEXT \"[[|\")))"},
		{"[[]", "(BLOCK (PARA (TEXT \"[[]\")))"},
		{"[[|]", "(BLOCK (PARA (TEXT \"[[|]\")))"},
		{"[[]]", "(BLOCK (PARA (TEXT \"[[]]\")))"},
		{"[[|]]", "(BLOCK (PARA (TEXT \"[[|]]\")))"},
		{"[[ ]]", "(BLOCK (PARA (TEXT \"[[ ]]\")))"},
		{"[[\n]]", "(BLOCK (PARA (TEXT \"[[\") (SOFT) (TEXT \"]]\")))"},
		{"[[ a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\"))))"},
		{"[[a ]]", "(BLOCK (PARA (TEXT \"[[a ]]\")))"},
		{"[[a\n]]", "(BLOCK (PARA (TEXT \"[[a\") (SOFT) (TEXT \"]]\")))"},
		{"[[a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\"))))"},
		{"[[12345678901234]]", "(BLOCK (PARA (LINK () (ZETTEL \"12345678901234\"))))"},
		{"[[a]", "(BLOCK (PARA (TEXT \"[[a]\")))"},
		{"[[|a]]", "(BLOCK (PARA (TEXT \"[[|a]]\")))"},
		{"[[b|]]", "(BLOCK (PARA (TEXT \"[[b|]]\")))"},
		{"[[b|a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"b\"))))"},
		{"[[b| a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"b\"))))"},
		{"[[b%c|a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"b%c\"))))"},
		{"[[b%%c|a]]", "(BLOCK (PARA (TEXT \"[[b\") (LITERAL-COMMENT () \"c|a]]\")))"},
		{"[[b|a]", "(BLOCK (PARA (TEXT \"[[b|a]\")))"},
		{"[[b\nc|a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"b\") (SOFT) (TEXT \"c\"))))"},
		{"[[b c|a#n]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a#n\") (TEXT \"b c\"))))"},
		{"[[a]]go", "(BLOCK (PARA (LINK () (EXTERNAL \"a\")) (TEXT \"go\")))"},
		{"[[b|a]]{go}", "(BLOCK (PARA (LINK ((\"go\" . \"\")) (EXTERNAL \"a\") (TEXT \"b\"))))"},
		{"[[[[a]]|b]]", "(BLOCK (PARA (TEXT \"[[\") (LINK () (EXTERNAL \"a\")) (TEXT \"|b]]\")))"},
		{"[[a[b]c|d]]", "(BLOCK (PARA (LINK () (EXTERNAL \"d\") (TEXT \"a[b]c\"))))"},
		{"[[[b]c|d]]", "(BLOCK (PARA (TEXT \"[\") (LINK () (EXTERNAL \"d\") (TEXT \"b]c\"))))"},
		{"[[a[]c|d]]", "(BLOCK (PARA (LINK () (EXTERNAL \"d\") (TEXT \"a[]c\"))))"},
		{"[[a[b]|d]]", "(BLOCK (PARA (LINK () (EXTERNAL \"d\") (TEXT \"a[b]\"))))"},
		{"[[\\|]]", "(BLOCK (PARA (LINK () (EXTERNAL \"\\\\|\"))))"},
		{"[[\\||a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"|\"))))"},
		{"[[b\\||a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"b|\"))))"},
		{"[[b\\|c|a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"b|c\"))))"},
		{"[[\\]]]", "(BLOCK (PARA (LINK () (EXTERNAL \"\\\\]\"))))"},
		{"[[\\]|a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"]\"))))"},
		{"[[b\\]|a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"b]\"))))"},
		{"[[\\]\\||a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"a\") (TEXT \"]|\"))))"},
		{"[[http://a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"http://a\"))))"},
		{"[[http://a|http://a]]", "(BLOCK (PARA (LINK () (EXTERNAL \"http://a\") (TEXT \"http://a\"))))"},
		{"[[[[a]]]]", "(BLOCK (PARA (TEXT \"[[\") (LINK () (EXTERNAL \"a\")) (TEXT \"]]\")))"},
		{"[[query:title]]", "(BLOCK (PARA (LINK () (QUERY \"title\"))))"},
		{"[[query:title syntax]]", "(BLOCK (PARA (LINK () (QUERY \"title syntax\"))))"},
		{"[[query:title | action]]", "(BLOCK (PARA (LINK () (QUERY \"title | action\"))))"},
		{"[[Text|query:title]]", "(BLOCK (PARA (LINK () (QUERY \"title\") (TEXT \"Text\"))))"},
		{"[[Text|query:title syntax]]", "(BLOCK (PARA (LINK () (QUERY \"title syntax\") (TEXT \"Text\"))))"},
		{"[[Text|query:title | action]]", "(BLOCK (PARA (LINK () (QUERY \"title | action\") (TEXT \"Text\"))))"},
	})
}

func TestEmbed(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"{", "(BLOCK (PARA (TEXT \"{\")))"},
		{"{{", "(BLOCK (PARA (TEXT \"{{\")))"},
		{"{{|", "(BLOCK (PARA (TEXT \"{{|\")))"},
		{"{{}", "(BLOCK (PARA (TEXT \"{{}\")))"},
		{"{{|}", "(BLOCK (PARA (TEXT \"{{|}\")))"},
		{"{{}}", "(BLOCK (PARA (TEXT \"{{}}\")))"},
		{"{{|}}", "(BLOCK (PARA (TEXT \"{{|}}\")))"},
		{"{{ }}", "(BLOCK (PARA (TEXT \"{{ }}\")))"},
		{"{{\n}}", "(BLOCK (PARA (TEXT \"{{\") (SOFT) (TEXT \"}}\")))"},
		{"{{a }}", "(BLOCK (PARA (TEXT \"{{a }}\")))"},
		{"{{a\n}}", "(BLOCK (PARA (TEXT \"{{a\") (SOFT) (TEXT \"}}\")))"},
		{"{{a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\")))"},
		{"{{12345678901234}}", "(BLOCK (PARA (EMBED () (ZETTEL \"12345678901234\") \"\")))"},
		{"{{ a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\")))"},
		{"{{a}", "(BLOCK (PARA (TEXT \"{{a}\")))"},
		{"{{|a}}", "(BLOCK (PARA (TEXT \"{{|a}}\")))"},
		{"{{b|}}", "(BLOCK (PARA (TEXT \"{{b|}}\")))"},
		{"{{b|a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"b\"))))"},
		{"{{b| a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"b\"))))"},
		{"{{b|a}", "(BLOCK (PARA (TEXT \"{{b|a}\")))"},
		{"{{b\nc|a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"b\") (SOFT) (TEXT \"c\"))))"},
		{"{{b c|a#n}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a#n\") \"\" (TEXT \"b c\"))))"},
		{"{{a}}{go}", "(BLOCK (PARA (EMBED ((\"go\" . \"\")) (EXTERNAL \"a\") \"\")))"},
		{"{{{{a}}|b}}", "(BLOCK (PARA (TEXT \"{{\") (EMBED () (EXTERNAL \"a\") \"\") (TEXT \"|b}}\")))"},
		{"{{\\|}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"\\\\|\") \"\")))"},
		{"{{\\||a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"|\"))))"},
		{"{{b\\||a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"b|\"))))"},
		{"{{b\\|c|a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"b|c\"))))"},
		{"{{\\}}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"\\\\}\") \"\")))"},
		{"{{\\}|a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"}\"))))"},
		{"{{b\\}|a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"b}\"))))"},
		{"{{\\}\\||a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"a\") \"\" (TEXT \"}|\"))))"},
		{"{{http://a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"http://a\") \"\")))"},
		{"{{http://a|http://a}}", "(BLOCK (PARA (EMBED () (EXTERNAL \"http://a\") \"\" (TEXT \"http://a\"))))"},
		{"{{{{a}}}}", "(BLOCK (PARA (TEXT \"{{\") (EMBED () (EXTERNAL \"a\") \"\") (TEXT \"}}\")))"},
	})
}

func TestCite(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[@", "(BLOCK (PARA (TEXT \"[@\")))"},
		{"[@]", "(BLOCK (PARA (TEXT \"[@]\")))"},
		{"[@a]", "(BLOCK (PARA (CITE () \"a\")))"},
		{"[@ a]", "(BLOCK (PARA (TEXT \"[@ a]\")))"},
		{"[@a ]", "(BLOCK (PARA (CITE () \"a\")))"},
		{"[@a\n]", "(BLOCK (PARA (CITE () \"a\")))"},
		{"[@a\nx]", "(BLOCK (PARA (CITE () \"a\" (SOFT) (TEXT \"x\"))))"},
		{"[@a\n\n]", "(BLOCK (PARA (TEXT \"[@a\")) (PARA (TEXT \"]\")))"},
		{"[@a,\n]", "(BLOCK (PARA (CITE () \"a\")))"},
		{"[@a,n]", "(BLOCK (PARA (CITE () \"a\" (TEXT \"n\"))))"},
		{"[@a| n]", "(BLOCK (PARA (CITE () \"a\" (TEXT \"n\"))))"},
		{"[@a|n ]", "(BLOCK (PARA (CITE () \"a\" (TEXT \"n\"))))"},
		{"[@a,[@b]]", "(BLOCK (PARA (CITE () \"a\" (CITE () \"b\"))))"},
		{"[@a]{color=green}", "(BLOCK (PARA (CITE ((\"color\" . \"green\")) \"a\")))"},
	})
	checkTcs(t, TestCases{
		{"[@a\n\n]", "(BLOCK (PARA (TEXT \"[@a\")) (PARA (TEXT \"]\")))"},
	})
}

func TestEndnote(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[^", "(BLOCK (PARA (TEXT \"[^\")))"},
		{"[^]", "(BLOCK (PARA (ENDNOTE ())))"},
		{"[^abc]", "(BLOCK (PARA (ENDNOTE () (TEXT \"abc\"))))"},
		{"[^abc ]", "(BLOCK (PARA (ENDNOTE () (TEXT \"abc\"))))"},
		{"[^abc\ndef]", "(BLOCK (PARA (ENDNOTE () (TEXT \"abc\") (SOFT) (TEXT \"def\"))))"},
		{"[^abc\n\ndef]", "(BLOCK (PARA (TEXT \"[^abc\")) (PARA (TEXT \"def]\")))"},
		{"[^abc[^def]]", "(BLOCK (PARA (ENDNOTE () (TEXT \"abc\") (ENDNOTE () (TEXT \"def\")))))"},
		{"[^abc]{-}", "(BLOCK (PARA (ENDNOTE ((\"-\" . \"\")) (TEXT \"abc\"))))"},
	})
	checkTcs(t, TestCases{
		{"[^abc\n\ndef]", "(BLOCK (PARA (TEXT \"[^abc\")) (PARA (TEXT \"def]\")))"},
	})
}

func TestMark(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[!", "(BLOCK (PARA (TEXT \"[!\")))"},
		{"[!\n", "(BLOCK (PARA (TEXT \"[!\")))"},
		{"[!]", "(BLOCK (PARA (MARK \"\" \"\" \"\")))"},
		{"[!][!]", "(BLOCK (PARA (MARK \"\" \"\" \"\") (MARK \"\" \"\" \"\")))"},
		{"[! ]", "(BLOCK (PARA (TEXT \"[! ]\")))"},
		{"[!a]", "(BLOCK (PARA (MARK \"a\" \"\" \"\")))"},
		{"[!a][!a]", "(BLOCK (PARA (MARK \"a\" \"\" \"\") (MARK \"a\" \"\" \"\")))"},
		{"[!a ]", "(BLOCK (PARA (TEXT \"[!a ]\")))"},
		{"[!a_]", "(BLOCK (PARA (MARK \"a_\" \"\" \"\")))"},
		{"[!a_][!a]", "(BLOCK (PARA (MARK \"a_\" \"\" \"\") (MARK \"a\" \"\" \"\")))"},
		{"[!a-b]", "(BLOCK (PARA (MARK \"a-b\" \"\" \"\")))"},
		{"[!a|b]", "(BLOCK (PARA (MARK \"a\" \"\" \"\" (TEXT \"b\"))))"},
		{"[!a|]", "(BLOCK (PARA (MARK \"a\" \"\" \"\")))"},
		{"[!|b]", "(BLOCK (PARA (MARK \"\" \"\" \"\" (TEXT \"b\"))))"},
		{"[!|b ]", "(BLOCK (PARA (MARK \"\" \"\" \"\" (TEXT \"b\"))))"},
		{"[!|b c]", "(BLOCK (PARA (MARK \"\" \"\" \"\" (TEXT \"b c\"))))"},
	})
}

func TestComment(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"%", "(BLOCK (PARA (TEXT \"%\")))"},
		{"%%", "(BLOCK (PARA (LITERAL-COMMENT () \"\")))"},
		{"%\n", "(BLOCK (PARA (TEXT \"%\")))"},
		{"%%\n", "(BLOCK (PARA (LITERAL-COMMENT () \"\")))"},
		{"%%a", "(BLOCK (PARA (LITERAL-COMMENT () \"a\")))"},
		{"%%%a", "(BLOCK (PARA (LITERAL-COMMENT () \"a\")))"},
		{"%% a", "(BLOCK (PARA (LITERAL-COMMENT () \"a\")))"},
		{"%%%  a", "(BLOCK (PARA (LITERAL-COMMENT () \"a\")))"},
		{"%% % a", "(BLOCK (PARA (LITERAL-COMMENT () \"% a\")))"},
		{"%%a", "(BLOCK (PARA (LITERAL-COMMENT () \"a\")))"},
		{"a%%b", "(BLOCK (PARA (TEXT \"a\") (LITERAL-COMMENT () \"b\")))"},
		{"a %%b", "(BLOCK (PARA (TEXT \"a \") (LITERAL-COMMENT () \"b\")))"},
		{" %%b", "(BLOCK (PARA (LITERAL-COMMENT () \"b\")))"},
		{"%%b ", "(BLOCK (PARA (LITERAL-COMMENT () \"b \")))"},
		{"100%", "(BLOCK (PARA (TEXT \"100%\")))"},
		{"%%{=}a", "(BLOCK (PARA (LITERAL-COMMENT ((\"\" . \"\")) \"a\")))"},
	})
}

func TestFormat(t *testing.T) {
	symMap := symbolMap{
		"_": zsx.SymFormatEmph,
		"*": zsx.SymFormatStrong,
		">": zsx.SymFormatInsert,
		"~": zsx.SymFormatDelete,
		"^": zsx.SymFormatSuper,
		",": zsx.SymFormatSub,
		"#": zsx.SymFormatMark,
		":": zsx.SymFormatSpan,
	}
	t.Parallel()
	// Not for Insert / '>', because collision with quoted list
	// Not for Quote / '"', because escaped representation.
	for _, ch := range []string{"_", "*", "~", "^", ",", "#", ":"} {
		checkTcs(t, replace(ch, symMap, TestCases{
			{"$", "(BLOCK (PARA (TEXT \"$\")))"},
			{"$$", "(BLOCK (PARA (TEXT \"$$\")))"},
			{"$$$", "(BLOCK (PARA (TEXT \"$$$\")))"},
			{"$$$$", "(BLOCK (PARA ($% ())))"},
		}))
	}
	// Not for Quote / '"', because escaped representation.
	for _, ch := range []string{"_", "*", ">", "~", "^", ",", "#", ":"} {
		checkTcs(t, replace(ch, symMap, TestCases{
			{"$$a$$", "(BLOCK (PARA ($% () (TEXT \"a\"))))"},
			{"$$a$$$", "(BLOCK (PARA ($% () (TEXT \"a\")) (TEXT \"$\")))"},
			{"$$$a$$", "(BLOCK (PARA ($% () (TEXT \"$a\"))))"},
			{"$$$a$$$", "(BLOCK (PARA ($% () (TEXT \"$a\")) (TEXT \"$\")))"},
			{"$\\$", "(BLOCK (PARA (TEXT \"$$\")))"},
			{"$\\$$", "(BLOCK (PARA (TEXT \"$$$\")))"},
			{"$$\\$", "(BLOCK (PARA (TEXT \"$$$\")))"},
			{"$$a\\$$", "(BLOCK (PARA (TEXT \"$$a$$\")))"},
			{"$$a$\\$", "(BLOCK (PARA (TEXT \"$$a$$\")))"},
			{"$$a\\$$$", "(BLOCK (PARA ($% () (TEXT \"a$\"))))"},
			{"$$a\na$$", "(BLOCK (PARA ($% () (TEXT \"a\") (SOFT) (TEXT \"a\"))))"},
			{"$$a\n\na$$", "(BLOCK (PARA (TEXT \"$$a\")) (PARA (TEXT \"a$$\")))"},
			{"$$a$${go}", "(BLOCK (PARA ($% ((\"go\" . \"\")) (TEXT \"a\"))))"},
		}))
		checkTcs(t, replace(ch, symMap, TestCases{
			{"$$a\n\na$$", "(BLOCK (PARA (TEXT \"$$a\")) (PARA (TEXT \"a$$\")))"},
		}))
	}
	checkTcs(t, replace(`"`, symbolMap{`"`: zsx.SymFormatQuote}, TestCases{
		{"$", "(BLOCK (PARA (TEXT \"\\\"\")))"},
		{"$$", "(BLOCK (PARA (TEXT \"\\\"\\\"\")))"},
		{"$$$", "(BLOCK (PARA (TEXT \"\\\"\\\"\\\"\")))"},
		{"$$$$", "(BLOCK (PARA ($% ())))"},

		{"$$a$$", "(BLOCK (PARA ($% () (TEXT \"a\"))))"},
		{"$$a$$$", "(BLOCK (PARA ($% () (TEXT \"a\")) (TEXT \"\\\"\")))"},
		{"$$$a$$", "(BLOCK (PARA ($% () (TEXT \"\\\"a\"))))"},
		{"$$$a$$$", "(BLOCK (PARA ($% () (TEXT \"\\\"a\")) (TEXT \"\\\"\")))"},
		{"$\\$", "(BLOCK (PARA (TEXT \"\\\"\\\"\")))"},
		{"$\\$$", "(BLOCK (PARA (TEXT \"\\\"\\\"\\\"\")))"},
		{"$$\\$", "(BLOCK (PARA (TEXT \"\\\"\\\"\\\"\")))"},
		{"$$a\\$$", "(BLOCK (PARA (TEXT \"\\\"\\\"a\\\"\\\"\")))"},
		{"$$a$\\$", "(BLOCK (PARA (TEXT \"\\\"\\\"a\\\"\\\"\")))"},
		{"$$a\\$$$", "(BLOCK (PARA ($% () (TEXT \"a\\\"\"))))"},
		{"$$a\na$$", "(BLOCK (PARA ($% () (TEXT \"a\") (SOFT) (TEXT \"a\"))))"},
		{"$$a\n\na$$", "(BLOCK (PARA (TEXT \"\\\"\\\"a\")) (PARA (TEXT \"a\\\"\\\"\")))"},
		{"$$a$${go}", "(BLOCK (PARA ($% ((\"go\" . \"\")) (TEXT \"a\"))))"},
	}))
	checkTcs(t, TestCases{
		{"__****__", "(BLOCK (PARA (FORMAT-EMPH () (FORMAT-STRONG ()))))"},
		{"__**a**__", "(BLOCK (PARA (FORMAT-EMPH () (FORMAT-STRONG () (TEXT \"a\")))))"},
		{"__**__**", "(BLOCK (PARA (TEXT \"__\") (FORMAT-STRONG () (TEXT \"__\"))))"},
	})
}

func TestLiteral(t *testing.T) {
	symMap := symbolMap{
		"`": zsx.SymLiteralCode,
		"'": zsx.SymLiteralInput,
		"=": zsx.SymLiteralOutput,
	}
	t.Parallel()
	for _, ch := range []string{"`", "'", "="} {
		checkTcs(t, replace(ch, symMap, TestCases{
			{"$", "(BLOCK (PARA (TEXT \"$\")))"},
			{"$$", "(BLOCK (PARA (TEXT \"$$\")))"},
			{"$$$", "(BLOCK (PARA (TEXT \"$$$\")))"},
			{"$$$$", "(BLOCK (PARA ($% () \"\")))"},
			{"$$a$$", "(BLOCK (PARA ($% () \"a\")))"},
			{"$$a$$$", "(BLOCK (PARA ($% () \"a\") (TEXT \"$\")))"},
			{"$$$a$$", "(BLOCK (PARA ($% () \"$a\")))"},
			{"$$$a$$$", "(BLOCK (PARA ($% () \"$a\") (TEXT \"$\")))"},
			{"$\\$", "(BLOCK (PARA (TEXT \"$$\")))"},
			{"$\\$$", "(BLOCK (PARA (TEXT \"$$$\")))"},
			{"$$\\$", "(BLOCK (PARA (TEXT \"$$$\")))"},
			{"$$a\\$$", "(BLOCK (PARA (TEXT \"$$a$$\")))"},
			{"$$a$\\$", "(BLOCK (PARA (TEXT \"$$a$$\")))"},
			{"$$a\\$$$", "(BLOCK (PARA ($% () \"a$\")))"},
			{"$$a$${go}", "(BLOCK (PARA ($% ((\"go\" . \"\")) \"a\")))"},
		}))
	}
	checkTcs(t, TestCases{
		{"``<script `` abc", "(BLOCK (PARA (LITERAL-CODE () \"<script \") (TEXT \" abc\")))"},
		{"''````''", "(BLOCK (PARA (LITERAL-INPUT () \"````\")))"},
		{"''``a``''", "(BLOCK (PARA (LITERAL-INPUT () \"``a``\")))"},
		{"''``''``", "(BLOCK (PARA (LITERAL-INPUT () \"``\") (TEXT \"``\")))"},
		{"''\\'''", "(BLOCK (PARA (LITERAL-INPUT () \"'\")))"},
	})
}

func TestLiteralMath(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"$", "(BLOCK (PARA (TEXT \"$\")))"},
		{"$$", "(BLOCK (PARA (TEXT \"$$\")))"},
		{"$$$", "(BLOCK (PARA (TEXT \"$$$\")))"},
		{"$$$$", "(BLOCK (PARA (LITERAL-MATH () \"\")))"},
		{"$$a$$", "(BLOCK (PARA (LITERAL-MATH () \"a\")))"},
		{"$$a$$$", "(BLOCK (PARA (LITERAL-MATH () \"a\") (TEXT \"$\")))"},
		{"$$$a$$", "(BLOCK (PARA (LITERAL-MATH () \"$a\")))"},
		{"$$$a$$$", "(BLOCK (PARA (LITERAL-MATH () \"$a\") (TEXT \"$\")))"},
		{`$\$`, "(BLOCK (PARA (TEXT \"$$\")))"},
		{`$\$$`, "(BLOCK (PARA (TEXT \"$$$\")))"},
		{`$$\$`, "(BLOCK (PARA (TEXT \"$$$\")))"},
		{`$$a\$$`, "(BLOCK (PARA (LITERAL-MATH () \"a\\\\\")))"},
		{`$$a$\$`, "(BLOCK (PARA (TEXT \"$$a$$\")))"},
		{`$$a\$$$`, "(BLOCK (PARA (LITERAL-MATH () \"a\\\\\") (TEXT \"$\")))"},
		{"$$a$${go}", "(BLOCK (PARA (LITERAL-MATH ((\"go\" . \"\")) \"a\")))"},
	})
}

func TestMixFormatCode(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"__abc__\n**def**", "(BLOCK (PARA (FORMAT-EMPH () (TEXT \"abc\")) (SOFT) (FORMAT-STRONG () (TEXT \"def\"))))"},
		{"''abc''\n==def==", "(BLOCK (PARA (LITERAL-INPUT () \"abc\") (SOFT) (LITERAL-OUTPUT () \"def\")))"},
		{"__abc__\n==def==", "(BLOCK (PARA (FORMAT-EMPH () (TEXT \"abc\")) (SOFT) (LITERAL-OUTPUT () \"def\")))"},
		{"__abc__\n``def``", "(BLOCK (PARA (FORMAT-EMPH () (TEXT \"abc\")) (SOFT) (LITERAL-CODE () \"def\")))"},
		{
			"\"\"ghi\"\"\n::abc::\n``def``\n",
			"(BLOCK (PARA (FORMAT-QUOTE () (TEXT \"ghi\")) (SOFT) (FORMAT-SPAN () (TEXT \"abc\")) (SOFT) (LITERAL-CODE () \"def\")))",
		},
	})
}

func TestNDash(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"--", "(BLOCK (PARA (TEXT \"\u2013\")))"},
		{"a--b", "(BLOCK (PARA (TEXT \"a\u2013b\")))"},
	})
}

func TestEntity(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"&", "(BLOCK (PARA (TEXT \"&\")))"},
		{"&;", "(BLOCK (PARA (TEXT \"&;\")))"},
		{"&#;", "(BLOCK (PARA (TEXT \"&#;\")))"},
		{"&#1a;", "(BLOCK (PARA (TEXT \"&#1a;\")))"},
		{"&#x;", "(BLOCK (PARA (TEXT \"&#x;\")))"},
		{"&#x0z;", "(BLOCK (PARA (TEXT \"&#x0z;\")))"},
		{"&1;", "(BLOCK (PARA (TEXT \"&1;\")))"},
		{"&#9;", "(BLOCK (PARA (TEXT \"&#9;\")))"}, // Numeric entities below space are not allowed.
		{"&#x1f;", "(BLOCK (PARA (TEXT \"&#x1f;\")))"},

		// Good cases
		{"&lt;", "(BLOCK (PARA (TEXT \"<\")))"},
		{"&#48;", "(BLOCK (PARA (TEXT \"0\")))"},
		{"&#x4A;", "(BLOCK (PARA (TEXT \"J\")))"},
		{"&#X4a;", "(BLOCK (PARA (TEXT \"J\")))"},
		{"&hellip;", "(BLOCK (PARA (TEXT \"\u2026\")))"},
		{"&nbsp;", "(BLOCK (PARA (TEXT \"\u00a0\")))"},
		{"E: &amp;,&#63;;&#x63;.", "(BLOCK (PARA (TEXT \"E: &,?;c.\")))"},
	})
}

func TestVerbatimZettel(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"@@@\n@@@", "()"},
		{"@@@\nabc\n@@@", "(BLOCK (VERBATIM-ZETTEL () \"abc\"))"},
		{"@@@@def\nabc\n@@@@", "(BLOCK (VERBATIM-ZETTEL ((\"\" . \"def\")) \"abc\"))"},
	})
}

func TestVerbatimCode(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"```\n```", "()"},
		{"```\nabc\n```", "(BLOCK (VERBATIM-CODE () \"abc\"))"},
		{"```\nabc\n````", "(BLOCK (VERBATIM-CODE () \"abc\"))"},
		{"````\nabc\n````", "(BLOCK (VERBATIM-CODE () \"abc\"))"},
		{"````\nabc\n```\n````", "(BLOCK (VERBATIM-CODE () \"abc\\n```\"))"},
		{"````go\nabc\n````", "(BLOCK (VERBATIM-CODE ((\"\" . \"go\")) \"abc\"))"},
	})
}

func TestVerbatimEval(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"~~~\n~~~", "()"},
		{"~~~\nabc\n~~~", "(BLOCK (VERBATIM-EVAL () \"abc\"))"},
		{"~~~\nabc\n~~~~", "(BLOCK (VERBATIM-EVAL () \"abc\"))"},
		{"~~~~\nabc\n~~~~", "(BLOCK (VERBATIM-EVAL () \"abc\"))"},
		{"~~~~\nabc\n~~~\n~~~~", "(BLOCK (VERBATIM-EVAL () \"abc\\n~~~\"))"},
		{"~~~~go\nabc\n~~~~", "(BLOCK (VERBATIM-EVAL ((\"\" . \"go\")) \"abc\"))"},
	})
}

func TestVerbatimMath(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"$$$\n$$$", "()"},
		{"$$$\nabc\n$$$", "(BLOCK (VERBATIM-MATH () \"abc\"))"},
		{"$$$\nabc\n$$$$", "(BLOCK (VERBATIM-MATH () \"abc\"))"},
		{"$$$$\nabc\n$$$$", "(BLOCK (VERBATIM-MATH () \"abc\"))"},
		{"$$$$\nabc\n$$$\n$$$$", "(BLOCK (VERBATIM-MATH () \"abc\\n$$$\"))"},
		{"$$$$go\nabc\n$$$$", "(BLOCK (VERBATIM-MATH ((\"\" . \"go\")) \"abc\"))"},
	})
}

func TestVerbatimComment(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"%%%\n%%%", "()"},
		{"%%%\nabc\n%%%", "(BLOCK (VERBATIM-COMMENT () \"abc\"))"},
		{"%%%%go\nabc\n%%%%", "(BLOCK (VERBATIM-COMMENT ((\"\" . \"go\")) \"abc\"))"},
	})
}

func TestPara(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"a\n\nb", "(BLOCK (PARA (TEXT \"a\")) (PARA (TEXT \"b\")))"},
		{"a\n \nb", "(BLOCK (PARA (TEXT \"a\")) (PARA (TEXT \"b\")))"},
	})
}

func TestSpanRegion(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{":::\n:::", "()"},
		{":::\nabc\n:::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"abc\")))))"},
		{":::\nabc\n::::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"abc\")))))"},
		{"::::\nabc\n::::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"abc\")))))"},
		{"::::\nabc\n:::\ndef\n:::\n::::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"abc\")) (REGION-BLOCK () ((PARA (TEXT \"def\")))))))"},
		{":::{go}\n:::a", "(BLOCK (REGION-BLOCK ((\"go\" . \"\")) () (TEXT \"a\")))"},
		{":::\nabc\n::: def ", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"abc\"))) (TEXT \"def\")))"},
	})
}

func TestQuoteRegion(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"<<<\n<<<", "()"},
		{"<<<\nabc\n<<<", "(BLOCK (REGION-QUOTE () ((PARA (TEXT \"abc\")))))"},
		{"<<<\nabc\n<<<<", "(BLOCK (REGION-QUOTE () ((PARA (TEXT \"abc\")))))"},
		{"<<<<\nabc\n<<<<", "(BLOCK (REGION-QUOTE () ((PARA (TEXT \"abc\")))))"},
		{"<<<<\nabc\n<<<\ndef\n<<<\n<<<<", "(BLOCK (REGION-QUOTE () ((PARA (TEXT \"abc\")) (REGION-QUOTE () ((PARA (TEXT \"def\")))))))"},
		{"<<<go\n<<< a", "(BLOCK (REGION-QUOTE ((\"\" . \"go\")) () (TEXT \"a\")))"},
		{"<<<\nabc\n<<< def ", "(BLOCK (REGION-QUOTE () ((PARA (TEXT \"abc\"))) (TEXT \"def\")))"},
	})
}

func TestVerseRegion(t *testing.T) {
	t.Parallel()
	checkTcs(t, replace("\"", nil, TestCases{
		{"$$$\n$$$", "()"},
		{"$$$\nabc\n$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"abc\")))))"},
		{"$$$\nabc\n$$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"abc\")))))"},
		{"$$$$\nabc\n$$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"abc\")))))"},
		{"$$$\nabc\ndef\n$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"abc\") (HARD) (TEXT \"def\")))))"},
		{"$$$$\nabc\n$$$\ndef\n$$$\n$$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"abc\")) (REGION-VERSE () ((PARA (TEXT \"def\")))))))"},
		{"$$$go\n$$$x", "(BLOCK (REGION-VERSE ((\"\" . \"go\")) () (TEXT \"x\")))"},
		{"$$$\nabc\n$$$ def ", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"abc\"))) (TEXT \"def\")))"},
		{"$$$\n space \n$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"\u00a0space\u00a0\")))))"},
		{"$$$\n  spaces  \n$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"\u00a0\u00a0spaces\u00a0\u00a0\")))))"},
		{"$$$\n  spaces  \n space  \n$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"\u00a0\u00a0spaces\u00a0\u00a0\") (HARD) (TEXT \"\u00a0space\u00a0\u00a0\")))))"},
		{"$$$\n space space \n$$$", "(BLOCK (REGION-VERSE () ((PARA (TEXT \"\u00a0space\u00a0space\u00a0\")))))"},
	}))
}

func TestHeading(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"=h", "(BLOCK (PARA (TEXT \"=h\")))"},
		{"= h", "(BLOCK (PARA (TEXT \"= h\")))"},
		{"==h", "(BLOCK (PARA (TEXT \"==h\")))"},
		{"== h", "(BLOCK (PARA (TEXT \"== h\")))"},
		{"===h", "(BLOCK (PARA (TEXT \"===h\")))"},
		{"===", "(BLOCK (PARA (TEXT \"===\")))"},
		{"=== h", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h\")))"},
		{"===  h", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h\")))"},
		{"==== h", "(BLOCK (HEADING 2 () \"\" \"\" (TEXT \"h\")))"},
		{"===== h", "(BLOCK (HEADING 3 () \"\" \"\" (TEXT \"h\")))"},
		{"====== h", "(BLOCK (HEADING 4 () \"\" \"\" (TEXT \"h\")))"},
		{"======= h", "(BLOCK (HEADING 5 () \"\" \"\" (TEXT \"h\")))"},
		{"======== h", "(BLOCK (HEADING 5 () \"\" \"\" (TEXT \"h\")))"},
		{"=", "(BLOCK (PARA (TEXT \"=\")))"},
		{"=== h=__=a__", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h=\") (FORMAT-EMPH () (TEXT \"=a\"))))"},
		{"=\n", "(BLOCK (PARA (TEXT \"=\")))"},
		{"a=", "(BLOCK (PARA (TEXT \"a=\")))"},
		{" =", "(BLOCK (PARA (TEXT \"=\")))"},
		{"=== h\na", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h\")) (PARA (TEXT \"a\")))"},
		{"=== h i {-}", "(BLOCK (HEADING 1 ((\"-\" . \"\")) \"\" \"\" (TEXT \"h i\")))"},
		{"=== h {{a}}", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h \") (EMBED () (EXTERNAL \"a\") \"\")))"},
		{"=== h{{a}}", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h\") (EMBED () (EXTERNAL \"a\") \"\")))"},
		{"=== {{a}}", "(BLOCK (HEADING 1 () \"\" \"\" (EMBED () (EXTERNAL \"a\") \"\")))"},
		{"=== h {{a}}{-}", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h \") (EMBED ((\"-\" . \"\")) (EXTERNAL \"a\") \"\")))"},
		{"=== h {{a}} {-}", "(BLOCK (HEADING 1 ((\"-\" . \"\")) \"\" \"\" (TEXT \"h \") (EMBED () (EXTERNAL \"a\") \"\")))"},
		{"=== h {-}{{a}}", "(BLOCK (HEADING 1 ((\"-\" . \"\")) \"\" \"\" (TEXT \"h\")))"},
		{"=== h{id=abc}", "(BLOCK (HEADING 1 ((\"id\" . \"abc\")) \"\" \"\" (TEXT \"h\")))"},
		{"=== h\n=== h", "(BLOCK (HEADING 1 () \"\" \"\" (TEXT \"h\")) (HEADING 1 () \"\" \"\" (TEXT \"h\")))"},
	})
}

func TestHRule(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"-", "(BLOCK (PARA (TEXT \"-\")))"},
		{"---", "(BLOCK (THEMATIC ()))"},
		{"----", "(BLOCK (THEMATIC ()))"},
		{"---A", "(BLOCK (THEMATIC ((\"\" . \"A\"))))"},
		{"---A-", "(BLOCK (THEMATIC ((\"\" . \"A-\"))))"},
		{"-1", "(BLOCK (PARA (TEXT \"-1\")))"},
		{"2-1", "(BLOCK (PARA (TEXT \"2-1\")))"},
		{"---  {  go  }  ", "(BLOCK (THEMATIC ((\"go\" . \"\"))))"},
		{"---  {  .go  }  ", "(BLOCK (THEMATIC ((\"class\" . \"go\"))))"},
		{`---{lang=zmk}`, "(BLOCK (THEMATIC ((\"lang\" . \"zmk\"))))"},
		{`---{lang="zmk"}`, "(BLOCK (THEMATIC ((\"lang\" . \"zmk\"))))"},
	})
}

func TestList(t *testing.T) {
	t.Parallel()
	// No ">" in the following, because quotation lists may have empty items.
	for _, ch := range []string{"*", "#"} {
		checkTcs(t, replace(ch, nil, TestCases{
			{"$", "(BLOCK (PARA (TEXT \"$\")))"},
			{"$$", "(BLOCK (PARA (TEXT \"$$\")))"},
			{"$$$", "(BLOCK (PARA (TEXT \"$$$\")))"},
			{"$ ", "(BLOCK (PARA (TEXT \"$\")))"},
			{"$$ ", "(BLOCK (PARA (TEXT \"$$\")))"},
			{"$$$ ", "(BLOCK (PARA (TEXT \"$$$\")))"},
		}))
	}
	checkTcs(t, TestCases{
		{"* abc", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))))"},
		{"** abc", "(BLOCK (UNORDERED () (BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))))))"},
		{"*** abc", "(BLOCK (UNORDERED () (BLOCK (UNORDERED () (BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))))))))"},
		{"**** abc", "(BLOCK (UNORDERED () (BLOCK (UNORDERED () (BLOCK (UNORDERED () (BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))))))))))"},
		{"** abc\n**** def", "(BLOCK (UNORDERED () (BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")) (UNORDERED () (BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"def\")))))))))))"},
		{"* abc\ndef", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))) (PARA (TEXT \"def\")))"},
		{"* abc\n def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))) (PARA (TEXT \"def\")))"},
		{"* abc\n* def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\"))) (BLOCK (PARA (TEXT \"def\")))))"},
		{"* abc\n  def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\") (SOFT) (TEXT \"def\")))))"},
		{"* abc\n   def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\") (SOFT) (TEXT \"def\")))))"},
		{"* abc\n\ndef", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))) (PARA (TEXT \"def\")))"},
		{"* abc\n\n def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))) (PARA (TEXT \"def\")))"},
		{"* abc\n\n  def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")) (PARA (TEXT \"def\")))))"},
		{"* abc\n\n   def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")) (PARA (TEXT \"def\")))))"},
		{"* abc\n** def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")) (UNORDERED () (BLOCK (PARA (TEXT \"def\")))))))"},
		{"* abc\n** def\n* ghi", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")) (UNORDERED () (BLOCK (PARA (TEXT \"def\"))))) (BLOCK (PARA (TEXT \"ghi\")))))"},
		{"* abc\n\n  def\n* ghi", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")) (PARA (TEXT \"def\"))) (BLOCK (PARA (TEXT \"ghi\")))))"},
		{"* abc\n** def\n   ghi\n  jkl", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")) (UNORDERED () (BLOCK (PARA (TEXT \"def\") (SOFT) (TEXT \"ghi\")))) (PARA (TEXT \"jkl\")))))"},

		// A list does not last beyond a region
		{":::\n# abc\n:::\n# def", "(BLOCK (REGION-BLOCK () ((ORDERED () (BLOCK (PARA (TEXT \"abc\")))))) (ORDERED () (BLOCK (PARA (TEXT \"def\")))))"},

		// A HRule creates a new list
		{"* abc\n---\n* def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))) (THEMATIC ()) (UNORDERED () (BLOCK (PARA (TEXT \"def\")))))"},

		// Changing list type adds a new list
		{"* abc\n# def", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))) (ORDERED () (BLOCK (PARA (TEXT \"def\")))))"},

		// Quotation lists may have empty items
		{">", "(BLOCK (QUOTATION () (BLOCK)))"},

		// Empty continuation
		{"* abc\n  ", "(BLOCK (UNORDERED () (BLOCK (PARA (TEXT \"abc\")))))"},
	})
}

func TestQuoteList(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"> w1 w2", "(BLOCK (QUOTATION () (BLOCK (PARA (TEXT \"w1 w2\")))))"},
		{"> w1\n> w2", "(BLOCK (QUOTATION () (BLOCK (PARA (TEXT \"w1\") (SOFT) (TEXT \"w2\")))))"},
		{"> w1\n>w2", "(BLOCK (QUOTATION () (BLOCK (PARA (TEXT \"w1\")))) (PARA (TEXT \">w2\")))"},
		{"> w1\n>\n>w2", "(BLOCK (QUOTATION () (BLOCK (PARA (TEXT \"w1\"))) (BLOCK)) (PARA (TEXT \">w2\")))"},
		{"> w1\n> \n> w2", "(BLOCK (QUOTATION () (BLOCK (PARA (TEXT \"w1\"))) (BLOCK) (BLOCK (PARA (TEXT \"w2\")))))"},
	})
}

func TestEnumAfterPara(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"abc\n* def", "(BLOCK (PARA (TEXT \"abc\")) (UNORDERED () (BLOCK (PARA (TEXT \"def\")))))"},
		{"abc\n*def", "(BLOCK (PARA (TEXT \"abc\") (SOFT) (TEXT \"*def\")))"},
	})
}

func TestDefinition(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{";", "(BLOCK (PARA (TEXT \";\")))"},
		{"; ", "(BLOCK (PARA (TEXT \";\")))"},
		{"; abc", "(BLOCK (DESCRIPTION () ((TEXT \"abc\"))))"},
		{"; abc\ndef", "(BLOCK (DESCRIPTION () ((TEXT \"abc\"))) (PARA (TEXT \"def\")))"},
		{"; abc\n def", "(BLOCK (DESCRIPTION () ((TEXT \"abc\"))) (PARA (TEXT \"def\")))"},
		{"; abc\n  def", "(BLOCK (DESCRIPTION () ((TEXT \"abc\") (SOFT) (TEXT \"def\"))))"},
		{"; abc\n  def\n  ghi", "(BLOCK (DESCRIPTION () ((TEXT \"abc\") (SOFT) (TEXT \"def\") (SOFT) (TEXT \"ghi\"))))"},
		{":", "(BLOCK (PARA (TEXT \":\")))"},
		{": ", "(BLOCK (PARA (TEXT \":\")))"},
		{": abc", "(BLOCK (PARA (TEXT \": abc\")))"},
		{"; abc\n: def", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\"))))))"},
		{"; abc\n: def\nghi", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\"))))) (PARA (TEXT \"ghi\")))"},
		{"; abc\n: def\n ghi", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\"))))) (PARA (TEXT \"ghi\")))"},
		{"; abc\n: def\n  ghi", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\") (SOFT) (TEXT \"ghi\"))))))"},
		{"; abc\n: def\n\n  ghi", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\")) (PARA (TEXT \"ghi\"))))))"},
		{"; abc\n: def\n\n  ghi\n\n  jkl", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\")) (PARA (TEXT \"ghi\")) (PARA (TEXT \"jkl\"))))))"},
		{"; abc\n:", "(BLOCK (DESCRIPTION () ((TEXT \"abc\"))) (PARA (TEXT \":\")))"},
		{"; abc\n: def\n: ghi", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\"))) (BLOCK (PARA (TEXT \"ghi\"))))))"},
		{"; abc\n: def\n; ghi\n: jkl", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\")))) ((TEXT \"ghi\")) (BLOCK (BLOCK (PARA (TEXT \"jkl\"))))))"},

		// Empty description
		{"; abc\n: ", "(BLOCK (DESCRIPTION () ((TEXT \"abc\"))) (PARA (TEXT \":\")))"},
		// Empty continuation of definition
		{"; abc\n: def\n  ", "(BLOCK (DESCRIPTION () ((TEXT \"abc\")) (BLOCK (BLOCK (PARA (TEXT \"def\"))))))"},
	})
}

func TestTable(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"|", "()"},
		{"||", "(BLOCK (TABLE () ((CELL ()))))"},
		{"| |", "(BLOCK (TABLE () ((CELL ()))))"},
		{"|a", "(BLOCK (TABLE () ((CELL () (TEXT \"a\")))))"},
		{"|a|", "(BLOCK (TABLE () ((CELL () (TEXT \"a\")))))"},
		{"|a| ", "(BLOCK (TABLE () ((CELL () (TEXT \"a\")) (CELL ()))))"},
		{"|a|b", "(BLOCK (TABLE () ((CELL () (TEXT \"a\")) (CELL () (TEXT \"b\")))))"},
		{"|a\n|b", "(BLOCK (TABLE () ((CELL () (TEXT \"a\"))) ((CELL () (TEXT \"b\")))))"},
		{"|a|b\n|c|d", "(BLOCK (TABLE () ((CELL () (TEXT \"a\")) (CELL () (TEXT \"b\"))) ((CELL () (TEXT \"c\")) (CELL () (TEXT \"d\")))))"},
		{"|%", "()"},
		{"|=a", "(BLOCK (TABLE ((CELL () (TEXT \"a\")))))"},
		{"|=a\n|b", "(BLOCK (TABLE ((CELL () (TEXT \"a\"))) ((CELL () (TEXT \"b\")))))"},
		{"|a|b\n|%---\n|c|d", "(BLOCK (TABLE () ((CELL () (TEXT \"a\")) (CELL () (TEXT \"b\"))) ((CELL () (TEXT \"c\")) (CELL () (TEXT \"d\")))))"},
		{"|a|b\n|c", "(BLOCK (TABLE () ((CELL () (TEXT \"a\")) (CELL () (TEXT \"b\"))) ((CELL () (TEXT \"c\")) (CELL ()))))"},
		{"|=<a>\n|b|c", "(BLOCK (TABLE ((CELL ((align . \"left\")) (TEXT \"a\")) (CELL ())) ((CELL ((align . \"right\")) (TEXT \"b\")) (CELL () (TEXT \"c\")))))"},
		{"|=<a|=b>\n||", "(BLOCK (TABLE ((CELL ((align . \"left\")) (TEXT \"a\")) (CELL ((align . \"right\")) (TEXT \"b\"))) ((CELL ()) (CELL ((align . \"right\"))))))"},
	})
}

func TestTransclude(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"{{{a}}}", "(BLOCK (TRANSCLUDE () (EXTERNAL \"a\")))"},
		{"{{{a}}}b", "(BLOCK (TRANSCLUDE ((\"\" . \"b\")) (EXTERNAL \"a\")))"},
		{"{{{a}}}}", "(BLOCK (TRANSCLUDE () (EXTERNAL \"a\")))"},
		{"{{{a\\}}}}", "(BLOCK (TRANSCLUDE () (EXTERNAL \"a\\\\}\")))"},
		{"{{{a\\}}}}b", "(BLOCK (TRANSCLUDE ((\"\" . \"b\")) (EXTERNAL \"a\\\\}\")))"},
		{"{{{a}}", "(BLOCK (PARA (TEXT \"{\") (EMBED () (EXTERNAL \"a\") \"\")))"},
		{"{{{a}}}{go=b}", "(BLOCK (TRANSCLUDE ((\"go\" . \"b\")) (EXTERNAL \"a\")))"},
	})
}

func TestBlockAttr(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{":::go\na\n:::", "(BLOCK (REGION-BLOCK ((\"\" . \"go\")) ((PARA (TEXT \"a\")))))"},
		{":::go=\na\n:::", "(BLOCK (REGION-BLOCK ((\"\" . \"go\")) ((PARA (TEXT \"a\")))))"},
		{":::{}\na\n:::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"a\")))))"},
		{":::{ }\na\n:::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"a\")))))"},
		{":::{.go}\na\n:::", "(BLOCK (REGION-BLOCK ((\"class\" . \"go\")) ((PARA (TEXT \"a\")))))"},
		{":::{=go}\na\n:::", "(BLOCK (REGION-BLOCK ((\"\" . \"go\")) ((PARA (TEXT \"a\")))))"},
		{":::{go}\na\n:::", "(BLOCK (REGION-BLOCK ((\"go\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::{go=py}\na\n:::", "(BLOCK (REGION-BLOCK ((\"go\" . \"py\")) ((PARA (TEXT \"a\")))))"},
		{":::{.go=py}\na\n:::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"a\")))))"},
		{":::{go=}\na\n:::", "(BLOCK (REGION-BLOCK ((\"go\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::{.go=}\na\n:::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"a\")))))"},
		{":::{go py}\na\n:::", "(BLOCK (REGION-BLOCK ((\"go\" . \"\") (\"py\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::{go\npy}\na\n:::", "(BLOCK (REGION-BLOCK ((\"go\" . \"\") (\"py\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::{.go py}\na\n:::", "(BLOCK (REGION-BLOCK ((\"class\" . \"go\") (\"py\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::{go .py}\na\n:::", "(BLOCK (REGION-BLOCK ((\"class\" . \"py\") (\"go\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::{.go py=3}\na\n:::", "(BLOCK (REGION-BLOCK ((\"class\" . \"go\") (\"py\" . \"3\")) ((PARA (TEXT \"a\")))))"},
		{":::  {  go  }  \na\n:::", "(BLOCK (REGION-BLOCK ((\"go\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::  {  .go  }  \na\n:::", "(BLOCK (REGION-BLOCK ((\"class\" . \"go\")) ((PARA (TEXT \"a\")))))"},
	})
	checkTcs(t, replace("\"", nil, TestCases{
		{":::{py=3}\na\n:::", "(BLOCK (REGION-BLOCK ((\"py\" . \"3\")) ((PARA (TEXT \"a\")))))"},
		{":::{py=$2 3$}\na\n:::", "(BLOCK (REGION-BLOCK ((\"py\" . \"2 3\")) ((PARA (TEXT \"a\")))))"},
		{":::{py=$2\\$3$}\na\n:::", "(BLOCK (REGION-BLOCK ((\"py\" . \"2\\\"3\")) ((PARA (TEXT \"a\")))))"},
		{":::{py=2$3}\na\n:::", "(BLOCK (REGION-BLOCK ((\"py\" . \"2\\\"3\")) ((PARA (TEXT \"a\")))))"},
		{":::{py=$2\n3$}\na\n:::", "(BLOCK (REGION-BLOCK ((\"py\" . \"2\\n3\")) ((PARA (TEXT \"a\")))))"},
		{":::{py=$2 3}\na\n:::", "(BLOCK (REGION-BLOCK () ((PARA (TEXT \"a\")))))"},
		{":::{py=2 py=3}\na\n:::", "(BLOCK (REGION-BLOCK ((\"py\" . \"2 3\")) ((PARA (TEXT \"a\")))))"},
		{":::{.go .py}\na\n:::", "(BLOCK (REGION-BLOCK ((\"class\" . \"go py\")) ((PARA (TEXT \"a\")))))"},
		{":::{go go}\na\n:::", "(BLOCK (REGION-BLOCK ((\"go\" . \"\")) ((PARA (TEXT \"a\")))))"},
		{":::{=py =go}\na\n:::", "(BLOCK (REGION-BLOCK ((\"\" . \"go\")) ((PARA (TEXT \"a\")))))"},
	}))
}

func TestInlineAttr(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"::a::{}", "(BLOCK (PARA (FORMAT-SPAN () (TEXT \"a\"))))"},
		{"::a::{ }", "(BLOCK (PARA (FORMAT-SPAN () (TEXT \"a\"))))"},
		{"::a::{.go}", "(BLOCK (PARA (FORMAT-SPAN ((\"class\" . \"go\")) (TEXT \"a\"))))"},
		{"::a::{=go}", "(BLOCK (PARA (FORMAT-SPAN ((\"\" . \"go\")) (TEXT \"a\"))))"},
		{"::a::{go}", "(BLOCK (PARA (FORMAT-SPAN ((\"go\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{go=py}", "(BLOCK (PARA (FORMAT-SPAN ((\"go\" . \"py\")) (TEXT \"a\"))))"},
		{"::a::{.go=py}", "(BLOCK (PARA (FORMAT-SPAN () (TEXT \"a\")) (TEXT \"{.go=py}\")))"},
		{"::a::{go=}", "(BLOCK (PARA (FORMAT-SPAN ((\"go\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{.go=}", "(BLOCK (PARA (FORMAT-SPAN () (TEXT \"a\")) (TEXT \"{.go=}\")))"},
		{"::a::{go py}", "(BLOCK (PARA (FORMAT-SPAN ((\"go\" . \"\") (\"py\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{go\npy}", "(BLOCK (PARA (FORMAT-SPAN ((\"go\" . \"\") (\"py\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{.go py}", "(BLOCK (PARA (FORMAT-SPAN ((\"class\" . \"go\") (\"py\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{go .py}", "(BLOCK (PARA (FORMAT-SPAN ((\"class\" . \"py\") (\"go\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{  \n go \n .py\n  \n}", "(BLOCK (PARA (FORMAT-SPAN ((\"class\" . \"py\") (\"go\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{  \n go \n .py\n\n}", "(BLOCK (PARA (FORMAT-SPAN ((\"class\" . \"py\") (\"go\" . \"\")) (TEXT \"a\"))))"},
		{"::a::{\ngo\n}", "(BLOCK (PARA (FORMAT-SPAN ((\"go\" . \"\")) (TEXT \"a\"))))"},
	})
	checkTcs(t, TestCases{
		{"::a::{py=3}", "(BLOCK (PARA (FORMAT-SPAN ((\"py\" . \"3\")) (TEXT \"a\"))))"},
		{"::a::{py=\"2 3\"}", "(BLOCK (PARA (FORMAT-SPAN ((\"py\" . \"2 3\")) (TEXT \"a\"))))"},
		{"::a::{py=\"2\\\"3\"}", "(BLOCK (PARA (FORMAT-SPAN ((\"py\" . \"2\\\"3\")) (TEXT \"a\"))))"},
		{"::a::{py=2\"3}", "(BLOCK (PARA (FORMAT-SPAN ((\"py\" . \"2\\\"3\")) (TEXT \"a\"))))"},
		{"::a::{py=\"2\n3\"}", "(BLOCK (PARA (FORMAT-SPAN ((\"py\" . \"2\\n3\")) (TEXT \"a\"))))"},
		{"::a::{py=\"2 3}", "(BLOCK (PARA (FORMAT-SPAN () (TEXT \"a\")) (TEXT \"{py=\\\"2 3}\")))"},
	})
	checkTcs(t, TestCases{
		{"::a::{py=2 py=3}", "(BLOCK (PARA (FORMAT-SPAN ((\"py\" . \"2 3\")) (TEXT \"a\"))))"},
		{"::a::{.go .py}", "(BLOCK (PARA (FORMAT-SPAN ((\"class\" . \"go py\")) (TEXT \"a\"))))"},
	})
}

func TestTemp(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"", "()"},
	})
}

Changes to text/text.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18



19
20
21
22
23
24
25

26
27
28
29
30
31
32
33



34
35
36
37
38
39

40
41
42
43
44
45
46
47
48

49

50
51
52
53
54
55
56
57

58
59
60


61
62

63
64
65
66



67
68
69
70
71



72
73
74
75

76
77
78

79
80
81
82
83
84
85
86















87
88

89
90

91
92
93
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19


20
21
22
23
24
25
26

27

28








29
30
31



32


33





34
35
36
37
38

39
40
41
42
43
44
45
46

47



48
49


50




51
52
53
54
55



56
57
58
59
60
61

62
63
64

65
66
67
68
69




70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

86
87

88
89
90
91








+
+
+








-
-
+
+
+




-

-
+
-
-
-
-
-
-
-
-
+
+
+
-
-
-

-
-
+
-
-
-
-
-




+
-
+







-
+
-
-
-
+
+
-
-
+
-
-
-
-
+
+
+


-
-
-
+
+
+



-
+


-
+




-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
+

-
+



//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

// Package text provides types, constants and function to work with text output.
package text

import (
	"strings"

	"codeberg.org/t73fde/sxpf"
	"zettelstore.de/c/sexpr"
	"t73f.de/r/sx"
	"t73f.de/r/zsx"
	"t73f.de/r/zsx/input"
)

// Encoder is the structure to hold relevant data to execute the encoding.
type Encoder struct {
	sf sxpf.SymbolFactory
	sb strings.Builder

}
	symText  *sxpf.Symbol
	symSpace *sxpf.Symbol
	symSoft  *sxpf.Symbol
	symHard  *sxpf.Symbol
	symQuote *sxpf.Symbol
}

func NewEncoder(sf sxpf.SymbolFactory) *Encoder {

// NewEncoder returns a new text encoder.
func NewEncoder() *Encoder {
	if sf == nil {
		return nil
	}
	enc := &Encoder{
		sf:       sf,
		sb:       strings.Builder{},
		sb: strings.Builder{},
		symText:  sf.MustMake(sexpr.NameSymText),
		symSpace: sf.MustMake(sexpr.NameSymSpace),
		symSoft:  sf.MustMake(sexpr.NameSymSoft),
		symHard:  sf.MustMake(sexpr.NameSymHard),
		symQuote: sf.MustMake(sexpr.NameSymQuote),
	}
	return enc
}

// Encode the object list as a string.
func (enc *Encoder) Encode(lst *sxpf.List) string {
func (enc *Encoder) Encode(lst *sx.Pair) string {
	enc.executeList(lst)
	result := enc.sb.String()
	enc.sb.Reset()
	return result
}

// EvaluateInlineString returns the text content of the given inline list as a string.
func EvaluateInlineString(lst *sxpf.List) string {
func EvaluateInlineString(lst *sx.Pair) string {
	if sf := sxpf.FindSymbolFactory(lst); sf != nil {
		return NewEncoder(sf).Encode(lst)
	}
	return NewEncoder().Encode(lst)
}
	return ""
}


func (enc *Encoder) executeList(lst *sxpf.List) {
	for elem := lst; elem != nil; elem = elem.Tail() {
		enc.execute(elem.Car())
func (enc *Encoder) executeList(lst *sx.Pair) {
	for obj := range lst.Values() {
		enc.execute(obj)
	}
}
func (enc *Encoder) execute(obj sxpf.Object) {
	cmd, ok := obj.(*sxpf.List)
	if !ok {
func (enc *Encoder) execute(obj sx.Object) {
	cmd, isPair := sx.GetPair(obj)
	if !isPair {
		return
	}
	sym := cmd.Car()
	if sxpf.IsNil(sym) {
	if sx.IsNil(sym) {
		return
	}
	if sym.IsEqual(enc.symText) {
	if sym.IsEqual(zsx.SymText) {
		args := cmd.Tail()
		if args == nil {
			return
		}
		if val, ok2 := args.Car().(sxpf.String); ok2 {
			enc.sb.WriteString(val.String())
		}
	} else if sym.IsEqual(enc.symSpace) || sym.IsEqual(enc.symSoft) {
		if val, isString := sx.GetString(args.Car()); isString {
			hadSpace := false
			for _, ch := range val.GetValue() {
				if input.IsSpace(ch) {
					if !hadSpace {
						enc.sb.WriteByte(' ')
						hadSpace = true
					}
				} else {
					enc.sb.WriteRune(ch)
					hadSpace = false
				}
			}
		}
	} else if sym.IsEqual(zsx.SymSoft) {
		enc.sb.WriteByte(' ')
	} else if sym.IsEqual(enc.symHard) {
	} else if sym.IsEqual(zsx.SymHard) {
		enc.sb.WriteByte('\n')
	} else if !sym.IsEqual(enc.symQuote) {
	} else if !sym.IsEqual(sx.SymbolQuote) {
		enc.executeList(cmd.Tail())
	}
}

Changes to text/text_test.go.

1
2
3
4
5
6
7
8



9
10
11
12
13
14
15
16
17
18
19



20
21
22

23
24
25
26
27
28
29


30
31
32

33
34
35
36
37
38


39
40
41
42
43
44
45
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19



20
21
22
23
24

25
26
27
28
29
30
31

32
33
34
35

36
37
38
39
40


41
42
43
44
45
46
47
48
49








+
+
+








-
-
-
+
+
+


-
+






-
+
+


-
+




-
-
+
+







//-----------------------------------------------------------------------------
// Copyright (c) 2022-present Detlef Stern
//
// This file is part of zettelstore-client.
//
// Zettelstore client is licensed under the latest version of the EUPL
// (European Union Public License). Please see file LICENSE.txt for your rights
// and obligations under this license.
//
// SPDX-License-Identifier: EUPL-1.2
// SPDX-FileCopyrightText: 2022-present Detlef Stern
//-----------------------------------------------------------------------------

package text_test

import (
	"strings"
	"testing"

	"codeberg.org/t73fde/sxpf"
	"codeberg.org/t73fde/sxpf/reader"
	"zettelstore.de/c/text"
	"t73f.de/r/sx"
	"t73f.de/r/sx/sxreader"
	"t73f.de/r/zsc/text"
)

func TestSexprText(t *testing.T) {
func TestSzText(t *testing.T) {
	testcases := []struct {
		src string
		exp string
	}{
		{"()", ""},
		{`(INLINE (TEXT "a"))`, "a"},
		{`(INLINE (SPACE "a"))`, " "},
		{`(INLINE (TEXT " "))`, " "},
		{`(INLINE (TEXT "  "))`, " "},
	}
	for i, tc := range testcases {
		sval, err := reader.MakeReader(strings.NewReader(tc.src)).Read()
		sval, err := sxreader.MakeReader(strings.NewReader(tc.src)).Read()
		if err != nil {
			t.Error(err)
			continue
		}
		seq, ok := sval.(*sxpf.List)
		if !ok {
		seq, isPair := sx.GetPair(sval)
		if !isPair {
			t.Errorf("%d: not a list: %v", i, sval)
		}
		got := text.EvaluateInlineString(seq)
		if got != tc.exp {
			t.Errorf("%d: EncodeBlock(%q) == %q, but got %q", i, tc.src, tc.exp, got)
		}
	}

Changes to www/changes.wiki.

1
2


























































3
4


5
6
7
8
9
10
11
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

62
63
64
65
66
67
68
69
70


+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
+
+







<title>Change Log</title>

<a name="0_21"></a>
<h2>Changes for Version 0.21.0 (pending)</h2>
  *  Sz encoding changed (LINK-* -> LINK, lists, descriptions, block BLOBs,
     tables and its cells got attributes; cell attributes now defines cell
     alignment).

<a name="0_20"></a>
<h2>Changes for Version 0.20.0 (2025-03-07)</h2>
  *  Add Zettelmarkup-Parser that translates to sz expressions
  *  Add domain specific data structure from main zettelstore
  *  <code>client.QueryZettelData</code> will support the new encoding of
     result lists. This make the client incompatible to version 0.19.

<a name="0_19"></a>
<h2>Changes for Version 0.19.0 (2024-12-13)</h2>
  *  Remove support for rename operation; removed all associated constants
  *  Make quote handling in shtml public, to be used by other encoders
  *  shtml generates external links with rel attribute
  *  Add some input handling methods
  *  Enhance docs for api/client

<a name="0_18"></a>
<h2>Changes for Version 0.18.0 (2024-07-11)</h2>
  *  Add client method <code>GetApplicationZid</code> to retrieve the zettel
     identifier of an configuration zettel for a specific application.
  *  Rename to be package <code>t73f.de/r/zsc</code>
  *  Reserve some zettel identifier for future use
  *  Mark <code>client.Client.RenameZettel</code> as deprecated
  *  Remove space node from (Sx-) AST

<a name="0_17"></a>
<h2>Changes for Version 0.17.0 (2024-03-04)</h2>
  *  Generic GET method for HTTP client.
  *  Adapt to sz changes; see manual for current syntax.
  *  Got some packages from Zettelstore, for general use.

<a name="0_16"></a>
<h2>Changes for Version 0.16.0 (2023-11-30)</h2>
  *  Refactor shtml transformator to support evaluating the language tree of a
     zettel AST. Its API has been changes, since evaluation is now top-down,
     where previous transformation was bottom.up.
  *  Add API call to retrieve role zettel.
  *  Added constants for role zettel and to mark text within zettelmarkup.

<a name="0_15"></a>
<h2>Changes for Version 0.15.0 (2023-10-26)</h2>
  *  Tag zettel: API, constant values
  *  Refactorings b/c Sx

<a name="0_14"></a>
<h2>Changes for Version 0.14.0 (2023-09-23)</h2>
  *  Remove support for JSON encoding

<a name="0_13"></a>
<h2>Changes for Version 0.13.0 (2023-08-07)</h2>
  *  API uses plain data or sx data, but no JSON encoded data.
  *  Dependency sx is now hosted on Fossil repository, same for this library.

<a name="0_12"></a>
<h2>Changes for Version 0.12.0 (pending)</h2>
<h2>Changes for Version 0.12.0 (2023-06-05)</h2>
  *  Rename "sexpr" to "sz".

<a name="0_11"></a>
<h2>Changes for Version 0.11.0 (2023-03-27)</h2>
  *  Remove all zjson related declarations.
  *  Generate HTML via SxHTML, not manually and direct.

<a name="0_10"></a>

Changes to www/index.wiki.

1
2
3
4
5
6
7
8
9
10
11






12























1
2
3
4
5






6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35





-
-
-
-
-
-
+
+
+
+
+
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
<title>Home</title>

This repository contains Go client software to
access [https://zettelstore.de|Zettelstore] via its API.

<h3>Latest Release: 0.11.0 (2023-03-27)</h3>
  *  [./changes.wiki#0_11|Change summary]
  *  [/timeline?p=v0.11.0&bt=v0.10.0&y=ci|Check-ins for version 0.11.0],
     [/vdiff?to=v0.11.0&from=v0.10.0|content diff]
  *  [/timeline?df=v0.11.0&y=ci|Check-ins derived from the 0.11.0 release],
     [/vdiff?from=v0.11.0&to=trunk|content diff]
<h3>Latest Release: 0.20.0 (2025-03-07)</h3>
  *  [./changes.wiki#0_20|Change summary]
  *  [/timeline?p=v0.20.0&bt=v0.19.0&y=ci|Check-ins for version 0.20],
     [/vdiff?to=v0.20.0&from=v0.19.0|content diff]
  *  [/timeline?df=v0.20.0&y=ci|Check-ins derived from the 0.20 release],
     [/vdiff?from=v0.20.0&to=trunk|content diff]
  *  [/timeline?t=release|Timeline of all past releases]

<h2>Usage instructions</h2>

To import this library into your own [https://go.dev/|Go] software, you need to
run the <code>go get</code> command. Since Go does not handle non-standard
software and platforms well, some additional steps are required.

First, install the version control system [https://fossil-scm.org|Fossil],
which is a superior alternative to Git in many use cases. Fossil is just a
single executable, nothing more. Make sure it is included in your system's
command search path.

Then, run the following Go command to retrieve a specific version of
this library:

<code>GOVCS=t73f.de:fossil go get t73f.de/r/zsc@HASH</code>

Here, <code>HASH</code> represents the commit hash of the version you want to
use.

Go currently does not seem to support software versioning for projects managed
by Fossil. This is why the hash value is required. However, this method works
reliably.