Zettelstore

Check-in Differences
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Difference From v0.5.0 To trunk

2022-08-13
16:24
Refactor search: remove negate bool in expValue ... (Leaf check-in: 184aed8a5a user: stern tags: trunk)
2022-08-12
17:22
Remove internal K/V handling of searches and use search expression parser instead ... (check-in: 0fac31e9d1 user: stern tags: trunk)
2022-08-02
08:33
Create new branch named "release-0.5" ... (check-in: 2ccbcee00f user: stern tags: release-0.5)
2022-08-01
11:47
Increase version to 0.6.0-dev to begin next development cycle ... (check-in: e61bb9ce88 user: stern tags: trunk)
2022-07-29
14:21
Version 0.5.0 ... (check-in: 7138adfeb5 user: stern tags: trunk, release, v0.5.0)
14:05
Adapt to new client version ... (check-in: 26b3fdba14 user: stern tags: trunk)

Changes to README.md.

19
20
21
22
23
24
25
26
often connects to Zettelstore via its API. Some of the software packages may be
experimental.

The software, including the manual, is licensed
under the [European Union Public License 1.2 (or
later)](https://zettelstore.de/home/file?name=LICENSE.txt&ci=trunk).

[Stay tuned](https://twitter.com/zettelstore)…







|
19
20
21
22
23
24
25
26
often connects to Zettelstore via its API. Some of the software packages may be
experimental.

The software, including the manual, is licensed
under the [European Union Public License 1.2 (or
later)](https://zettelstore.de/home/file?name=LICENSE.txt&ci=trunk).

[Stay tuned](https://twitter.com/zettelstore) …

Changes to VERSION.

1
0.5.0
|
1
0.7.0-dev

Changes to ast/ast.go.

80
81
82
83
84
85
86

87
88
	RefStateInvalid  RefState = iota // Invalid Reference
	RefStateZettel                   // Reference to an internal zettel
	RefStateSelf                     // Reference to same zettel with a fragment
	RefStateFound                    // Reference to an existing internal zettel, URL is ajusted
	RefStateBroken                   // Reference to a non-existing internal zettel
	RefStateHosted                   // Reference to local hosted non-Zettel, without URL change
	RefStateBased                    // Reference to local non-Zettel, to be prefixed

	RefStateExternal                 // Reference to external material
)







>


80
81
82
83
84
85
86
87
88
89
	RefStateInvalid  RefState = iota // Invalid Reference
	RefStateZettel                   // Reference to an internal zettel
	RefStateSelf                     // Reference to same zettel with a fragment
	RefStateFound                    // Reference to an existing internal zettel, URL is ajusted
	RefStateBroken                   // Reference to a non-existing internal zettel
	RefStateHosted                   // Reference to local hosted non-Zettel, without URL change
	RefStateBased                    // Reference to local non-Zettel, to be prefixed
	RefStateSearch                   // Reference to a zettel search
	RefStateExternal                 // Reference to external material
)

Changes to ast/ref.go.

8
9
10
11
12
13
14

15
16
17



18
19
20
21
22
23
24



25
26
27
28
29
30
31
// under this license.
//-----------------------------------------------------------------------------

package ast

import (
	"net/url"


	"zettelstore.de/z/domain/id"
)




// ParseReference parses a string and returns a reference.
func ParseReference(s string) *Reference {
	switch s {
	case "", "00000000000000":
		return &Reference{URL: nil, Value: s, State: RefStateInvalid}
	}



	if state, ok := localState(s); ok {
		if state == RefStateBased {
			s = s[1:]
		}
		u, err := url.Parse(s)
		if err == nil {
			return &Reference{URL: u, Value: s, State: state}







>



>
>
>



<
|


>
>
>







8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32
33
34
35
36
37
// under this license.
//-----------------------------------------------------------------------------

package ast

import (
	"net/url"
	"strings"

	"zettelstore.de/z/domain/id"
)

// SearchPrefix is the prefix that denotes a search expression.
const SearchPrefix = "search:"

// ParseReference parses a string and returns a reference.
func ParseReference(s string) *Reference {

	if s == "" || s == "00000000000000" {
		return &Reference{URL: nil, Value: s, State: RefStateInvalid}
	}
	if strings.HasPrefix(s, SearchPrefix) {
		return &Reference{URL: nil, Value: s[len(SearchPrefix):], State: RefStateSearch}
	}
	if state, ok := localState(s); ok {
		if state == RefStateBased {
			s = s[1:]
		}
		u, err := url.Parse(s)
		if err == nil {
			return &Reference{URL: u, Value: s, State: state}
62
63
64
65
66
67
68



69
70
71
72
73
74
75
	return RefStateInvalid, false
}

// String returns the string representation of a reference.
func (r Reference) String() string {
	if r.URL != nil {
		return r.URL.String()



	}
	return r.Value
}

// IsValid returns true if reference is valid
func (r *Reference) IsValid() bool { return r.State != RefStateInvalid }








>
>
>







68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
	return RefStateInvalid, false
}

// String returns the string representation of a reference.
func (r Reference) String() string {
	if r.URL != nil {
		return r.URL.String()
	}
	if r.State == RefStateSearch {
		return SearchPrefix + r.Value
	}
	return r.Value
}

// IsValid returns true if reference is valid
func (r *Reference) IsValid() bool { return r.State != RefStateInvalid }

Changes to box/box.go.

246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
}

func (err *ErrNotAllowed) Error() string {
	if err.User == nil {
		if err.Zid.IsValid() {
			return fmt.Sprintf(
				"operation %q on zettel %v not allowed for not authorized user",
				err.Op,
				err.Zid.String())
		}
		return fmt.Sprintf("operation %q not allowed for not authorized user", err.Op)
	}
	if err.Zid.IsValid() {
		return fmt.Sprintf(
			"operation %q on zettel %v not allowed for user %v/%v",
			err.Op,
			err.Zid.String(),
			err.User.GetDefault(api.KeyUserID, "?"),
			err.User.Zid.String())
	}
	return fmt.Sprintf(
		"operation %q not allowed for user %v/%v",
		err.Op,
		err.User.GetDefault(api.KeyUserID, "?"),
		err.User.Zid.String())
}

// Is return true, if the error is of type ErrNotAllowed.
func (*ErrNotAllowed) Is(error) bool { return true }

// ErrStarted is returned when trying to start an already started box.
var ErrStarted = errors.New("box is already started")







|
<






<
<
|
<



<
|
<







246
247
248
249
250
251
252
253

254
255
256
257
258
259


260

261
262
263

264

265
266
267
268
269
270
271
}

func (err *ErrNotAllowed) Error() string {
	if err.User == nil {
		if err.Zid.IsValid() {
			return fmt.Sprintf(
				"operation %q on zettel %v not allowed for not authorized user",
				err.Op, err.Zid)

		}
		return fmt.Sprintf("operation %q not allowed for not authorized user", err.Op)
	}
	if err.Zid.IsValid() {
		return fmt.Sprintf(
			"operation %q on zettel %v not allowed for user %v/%v",


			err.Op, err.Zid, err.User.GetDefault(api.KeyUserID, "?"), err.User.Zid)

	}
	return fmt.Sprintf(
		"operation %q not allowed for user %v/%v",

		err.Op, err.User.GetDefault(api.KeyUserID, "?"), err.User.Zid)

}

// Is return true, if the error is of type ErrNotAllowed.
func (*ErrNotAllowed) Is(error) bool { return true }

// ErrStarted is returned when trying to start an already started box.
var ErrStarted = errors.New("box is already started")

Changes to box/constbox/base.css.

99
100
101
102
103
104
105


106
107
108

109
110
111
112
113
114
115
116
117
118
119
  blockquote p { margin-bottom: .5rem }
  blockquote cite { font-style: normal }
  table {
    border-collapse: collapse;
    border-spacing: 0;
    max-width: 100%;
  }


  th,td {
    text-align: left;
    padding: .25rem .5rem;

  }
  td { border-bottom: 1px solid hsl(0, 0%, 85%) }
  thead th { border-bottom: 2px solid hsl(0, 0%, 70%) }
  tfoot th { border-top: 2px solid hsl(0, 0%, 70%) }
  main form {
    padding: 0 .5em;
    margin: .5em 0 0 0;
  }
  main form:after {
    content: ".";
    display: block;







>
>
|


>

<
<
<







99
100
101
102
103
104
105
106
107
108
109
110
111
112



113
114
115
116
117
118
119
  blockquote p { margin-bottom: .5rem }
  blockquote cite { font-style: normal }
  table {
    border-collapse: collapse;
    border-spacing: 0;
    max-width: 100%;
  }
  thead>tr>td { border-bottom: 2px solid hsl(0, 0%, 70%); font-weight: bold }
  tfoot>tr>td { border-top: 2px solid hsl(0, 0%, 70%); font-weight: bold }
  td {
    text-align: left;
    padding: .25rem .5rem;
    border-bottom: 1px solid hsl(0, 0%, 85%)
  }



  main form {
    padding: 0 .5em;
    margin: .5em 0 0 0;
  }
  main form:after {
    content: ".";
    display: block;
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
    padding: .5rem 1rem;
  }
  .zs-error {
    background-color: lightpink;
    border-style: none !important;
    font-weight: bold;
  }
  td.left,th.left { text-align:left }
  td.center,th.center { text-align:center }
  td.right,th.right { text-align:right }
  .zs-font-size-0 { font-size:75% }
  .zs-font-size-1 { font-size:83% }
  .zs-font-size-2 { font-size:100% }
  .zs-font-size-3 { font-size:117% }
  .zs-font-size-4 { font-size:150% }
  .zs-font-size-5 { font-size:200% }
  .zs-deprecated { border-style: dashed; padding: .2rem }







|
|
|







195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
    padding: .5rem 1rem;
  }
  .zs-error {
    background-color: lightpink;
    border-style: none !important;
    font-weight: bold;
  }
  td.left { text-align:left }
  td.center { text-align:center }
  td.right { text-align:right }
  .zs-font-size-0 { font-size:75% }
  .zs-font-size-1 { font-size:83% }
  .zs-font-size-2 { font-size:100% }
  .zs-font-size-3 { font-size:117% }
  .zs-font-size-4 { font-size:150% }
  .zs-font-size-5 { font-size:200% }
  .zs-deprecated { border-style: dashed; padding: .2rem }

Changes to box/constbox/info.mustache.

17
18
19
20
21
22
23








24
25
26
27
28
29
30
<ul>
{{#LocLinks}}
{{#Valid}}<li><a href="{{{Zid}}}">{{Zid}}</a></li>{{/Valid}}
{{^Valid}}<li>{{Zid}}</li>{{/Valid}}
{{/LocLinks}}
</ul>
{{/HasLocLinks}}








{{#HasExtLinks}}
<h3>External</h3>
<ul>
{{#ExtLinks}}
<li><a href="{{{.}}}"{{{ExtNewWindow}}}>{{.}}</a></li>
{{/ExtLinks}}
</ul>







>
>
>
>
>
>
>
>







17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
<ul>
{{#LocLinks}}
{{#Valid}}<li><a href="{{{Zid}}}">{{Zid}}</a></li>{{/Valid}}
{{^Valid}}<li>{{Zid}}</li>{{/Valid}}
{{/LocLinks}}
</ul>
{{/HasLocLinks}}
{{#HasSearchLinks}}
<h3>Searches</h3>
<ul>
{{#SearchLinks}}
<li><a href="{{{URL}}}">{{Text}}</a></li>
{{/SearchLinks}}
</ul>
{{/HasSearchLinks}}
{{#HasExtLinks}}
<h3>External</h3>
<ul>
{{#ExtLinks}}
<li><a href="{{{.}}}"{{{ExtNewWindow}}}>{{.}}</a></li>
{{/ExtLinks}}
</ul>

Changes to box/constbox/listzettel.mustache.

1
2
3



4
5
6
<header>
<h1>{{Title}}</h1>
</header>



<ul>
{{#Metas}}<li><a href="{{{URL}}}">{{{Text}}}</a></li>
{{/Metas}}</ul>



>
>
>



1
2
3
4
5
6
7
8
9
<header>
<h1>{{Title}}</h1>
</header>
<form action="{{{SearchURL}}}">
<input class="zs-input" type="text" placeholder="Search.." name="{{QueryKeySearch}}" value="{{SearchValue}}">
</form>
<ul>
{{#Metas}}<li><a href="{{{URL}}}">{{{Text}}}</a></li>
{{/Metas}}</ul>

Changes to cmd/cmd_run.go.

61
62
63
64
65
66
67
68
69

70
71
72
73
74
75
76
	ucAuthenticate := usecase.NewAuthenticate(authLog, authManager, authManager, boxManager)
	ucIsAuth := usecase.NewIsAuthenticated(ucLog, webSrv, authManager)
	ucCreateZettel := usecase.NewCreateZettel(ucLog, rtConfig, protectedBoxManager)
	ucGetMeta := usecase.NewGetMeta(protectedBoxManager)
	ucGetAllMeta := usecase.NewGetAllMeta(protectedBoxManager)
	ucGetZettel := usecase.NewGetZettel(protectedBoxManager)
	ucParseZettel := usecase.NewParseZettel(rtConfig, ucGetZettel)
	ucEvaluate := usecase.NewEvaluate(rtConfig, ucGetZettel, ucGetMeta)
	ucListMeta := usecase.NewListMeta(protectedBoxManager)

	ucListSyntax := usecase.NewListSyntax(protectedBoxManager)
	ucListRoles := usecase.NewListRoles(protectedBoxManager)
	ucListTags := usecase.NewListTags(protectedBoxManager)
	ucZettelContext := usecase.NewZettelContext(protectedBoxManager, rtConfig)
	ucDelete := usecase.NewDeleteZettel(ucLog, protectedBoxManager)
	ucUpdate := usecase.NewUpdateZettel(ucLog, protectedBoxManager)
	ucRename := usecase.NewRenameZettel(ucLog, protectedBoxManager)







<

>







61
62
63
64
65
66
67

68
69
70
71
72
73
74
75
76
	ucAuthenticate := usecase.NewAuthenticate(authLog, authManager, authManager, boxManager)
	ucIsAuth := usecase.NewIsAuthenticated(ucLog, webSrv, authManager)
	ucCreateZettel := usecase.NewCreateZettel(ucLog, rtConfig, protectedBoxManager)
	ucGetMeta := usecase.NewGetMeta(protectedBoxManager)
	ucGetAllMeta := usecase.NewGetAllMeta(protectedBoxManager)
	ucGetZettel := usecase.NewGetZettel(protectedBoxManager)
	ucParseZettel := usecase.NewParseZettel(rtConfig, ucGetZettel)

	ucListMeta := usecase.NewListMeta(protectedBoxManager)
	ucEvaluate := usecase.NewEvaluate(rtConfig, ucGetZettel, ucGetMeta, ucListMeta)
	ucListSyntax := usecase.NewListSyntax(protectedBoxManager)
	ucListRoles := usecase.NewListRoles(protectedBoxManager)
	ucListTags := usecase.NewListTags(protectedBoxManager)
	ucZettelContext := usecase.NewZettelContext(protectedBoxManager, rtConfig)
	ucDelete := usecase.NewDeleteZettel(ucLog, protectedBoxManager)
	ucUpdate := usecase.NewUpdateZettel(ucLog, protectedBoxManager)
	ucRename := usecase.NewRenameZettel(ucLog, protectedBoxManager)

Deleted cmd/fd_limit.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//go:build !darwin
// +build !darwin

package cmd

func raiseFdLimit() error { return nil }
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
































Deleted cmd/fd_limit_raise.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

//go:build darwin
// +build darwin

package cmd

import (
	"fmt"
	"syscall"

	"zettelstore.de/z/kernel"
)

const minFiles = 1048576

func raiseFdLimit() error {
	var rLimit syscall.Rlimit
	err := syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rLimit)
	if err != nil {
		return err
	}
	if rLimit.Cur >= minFiles {
		return nil
	}
	rLimit.Cur = minFiles
	if rLimit.Cur > rLimit.Max {
		rLimit.Cur = rLimit.Max
	}
	err = syscall.Setrlimit(syscall.RLIMIT_NOFILE, &rLimit)
	if err != nil {
		return err
	}
	err = syscall.Getrlimit(syscall.RLIMIT_NOFILE, &rLimit)
	if err != nil {
		return err
	}
	if rLimit.Cur < minFiles {
		msg := fmt.Sprintf("Make sure you have no more than %d files in all your boxes if you enabled notification\n", rLimit.Cur)
		kernel.Main.GetKernelLogger().Mandatory().Msg(msg)
	}
	return nil
}
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<






































































































Changes to cmd/main.go.

204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
	ok = setConfigValue(ok, kernel.AuthService, kernel.AuthOwner, cfg.GetDefault(keyOwner, ""))
	ok = setConfigValue(ok, kernel.AuthService, kernel.AuthReadonly, cfg.GetBool(keyReadOnly))

	ok = setConfigValue(
		ok, kernel.BoxService, kernel.BoxDefaultDirType,
		cfg.GetDefault(keyDefaultDirBoxType, kernel.BoxDirTypeNotify))
	ok = setConfigValue(ok, kernel.BoxService, kernel.BoxURIs+"1", "dir:./zettel")
	format := kernel.BoxURIs + "%v"
	for i := 1; ; i++ {
		key := fmt.Sprintf(format, i)
		val, found := cfg.Get(key)
		if !found {
			break
		}
		ok = setConfigValue(ok, kernel.BoxService, key, val)
	}








<

|







204
205
206
207
208
209
210

211
212
213
214
215
216
217
218
219
	ok = setConfigValue(ok, kernel.AuthService, kernel.AuthOwner, cfg.GetDefault(keyOwner, ""))
	ok = setConfigValue(ok, kernel.AuthService, kernel.AuthReadonly, cfg.GetBool(keyReadOnly))

	ok = setConfigValue(
		ok, kernel.BoxService, kernel.BoxDefaultDirType,
		cfg.GetDefault(keyDefaultDirBoxType, kernel.BoxDirTypeNotify))
	ok = setConfigValue(ok, kernel.BoxService, kernel.BoxURIs+"1", "dir:./zettel")

	for i := 1; ; i++ {
		key := kernel.BoxURIs + strconv.Itoa(i)
		val, found := cfg.Get(key)
		if !found {
			break
		}
		ok = setConfigValue(ok, kernel.BoxService, key, val)
	}

262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
		fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
		return 2
	}

	kern := kernel.Main
	var createManager kernel.CreateBoxManagerFunc
	if command.Boxes {
		err := raiseFdLimit()
		if err != nil {
			logger := kern.GetKernelLogger()
			logger.IfErr(err).Msg("Raising some limitions did not work")
			logger.Error().Msg("Prepare to encounter errors. Most of them can be mitigated. See the manual for details")
			kern.SetConfig(kernel.BoxService, kernel.BoxDefaultDirType, kernel.BoxDirTypeSimple)
		}
		createManager = func(boxURIs []*url.URL, authManager auth.Manager, rtConfig config.Config) (box.Manager, error) {
			compbox.Setup(cfg)
			return manager.New(boxURIs, authManager, rtConfig)
		}
	} else {
		createManager = func([]*url.URL, auth.Manager, config.Config) (box.Manager, error) { return nil, nil }
	}







<
<
<
<
<
<
<







261
262
263
264
265
266
267







268
269
270
271
272
273
274
		fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
		return 2
	}

	kern := kernel.Main
	var createManager kernel.CreateBoxManagerFunc
	if command.Boxes {







		createManager = func(boxURIs []*url.URL, authManager auth.Manager, rtConfig config.Config) (box.Manager, error) {
			compbox.Setup(cfg)
			return manager.New(boxURIs, authManager, rtConfig)
		}
	} else {
		createManager = func([]*url.URL, auth.Manager, config.Config) (box.Manager, error) { return nil, nil }
	}
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
		fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
	}
	kern.Shutdown(true)
	return exitCode
}

// runSimple is called, when the user just starts the software via a double click
// or via a simple call ``./zettelstore`` on the command line.
func runSimple() int {
	if _, err := searchAndReadConfiguration(); err == nil {
		return executeCommand(strRunSimple)
	}
	dir := "./zettel"
	if err := os.MkdirAll(dir, 0750); err != nil {
		fmt.Fprintf(os.Stderr, "Unable to create zettel directory %q (%s)\n", dir, err)







|







299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
		fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
	}
	kern.Shutdown(true)
	return exitCode
}

// runSimple is called, when the user just starts the software via a double click
// or via a simple call ./zettelstore on the command line.
func runSimple() int {
	if _, err := searchAndReadConfiguration(); err == nil {
		return executeCommand(strRunSimple)
	}
	dir := "./zettel"
	if err := os.MkdirAll(dir, 0750); err != nil {
		fmt.Fprintf(os.Stderr, "Unable to create zettel directory %q (%s)\n", dir, err)

Changes to docs/development/20210916194900.zettel.

1
2
3
4
5
6
7
8


9
10
11
12
13
14
15
id: 20210916194900
title: Checklist for Release
role: zettel
syntax: zmk
modified: 20220309105459

# Sync with the official repository
#* ``fossil sync -u``


# Make sure that all dependencies are up-to-date.
#* ``cat go.mod``
# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# All internal tests must succeed:
#* ``go run tools/build.go relcheck`` (alternatively: ``make relcheck``).
# The API tests must succeed on every development platform:








>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
id: 20210916194900
title: Checklist for Release
role: zettel
syntax: zmk
modified: 20220309105459

# Sync with the official repository
#* ``fossil sync -u``
# Make sure that there is no workspace defined.
#* ``ls ..`` must not have a file ''go.work'', in no parent folder.
# Make sure that all dependencies are up-to-date.
#* ``cat go.mod``
# Clean up your Go workspace:
#* ``go run tools/build.go clean`` (alternatively: ``make clean``).
# All internal tests must succeed:
#* ``go run tools/build.go relcheck`` (alternatively: ``make relcheck``).
# The API tests must succeed on every development platform:
28
29
30
31
32
33
34
35

36
37
38
39
40
41
42
# On every platform (esp. macOS), the box with 10.000 zettel must run properly:
#* ``./zettelstore -d DIR``
# Update files in directory ''www''
#* index.wiki
#* download.wiki
#* changes.wiki
#* plan.wiki
# Set file ''VERSION'' to the new release version

# Disable Fossil autosync mode:
#* ``fossil setting autosync off``
# Commit the new release version:
#* ``fossil commit --tag release --tag vVERSION -m "Version VERSION"``
#* **Important:** the tag must follow the given pattern, e.g. ''v0.0.15''.
   Otherwise client will not be able to import ''zettelkasten.de/z''.
# Clean up your Go workspace:







|
>







30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# On every platform (esp. macOS), the box with 10.000 zettel must run properly:
#* ``./zettelstore -d DIR``
# Update files in directory ''www''
#* index.wiki
#* download.wiki
#* changes.wiki
#* plan.wiki
# Set file ''VERSION'' to the new release version.
  It _must_ consist of three digits: MAJOR.MINOR.PATCH, even if PATCH is zero
# Disable Fossil autosync mode:
#* ``fossil setting autosync off``
# Commit the new release version:
#* ``fossil commit --tag release --tag vVERSION -m "Version VERSION"``
#* **Important:** the tag must follow the given pattern, e.g. ''v0.0.15''.
   Otherwise client will not be able to import ''zettelkasten.de/z''.
# Clean up your Go workspace:

Changes to docs/manual/00001000000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

19
20
21
22
id: 00001000000000
title: Zettelstore Manual
role: manual
tags: #manual #zettelstore
syntax: zmk
modified: 20211027121716

* [[Introduction|00001001000000]]
* [[Design goals|00001002000000]]
* [[Installation|00001003000000]]
* [[Configuration|00001004000000]]
* [[Structure of Zettelstore|00001005000000]]
* [[Layout of a zettel|00001006000000]]
* [[Zettelmarkup|00001007000000]]
* [[Other markup languages|00001008000000]]
* [[Security|00001010000000]]
* [[API|00001012000000]]
* [[Web user interface|00001014000000]]

* [[Troubleshooting|00001018000000]]
* Frequently asked questions

Licensed under the EUPL-1.2-or-later.





|












>




1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
id: 00001000000000
title: Zettelstore Manual
role: manual
tags: #manual #zettelstore
syntax: zmk
modified: 20220803183647

* [[Introduction|00001001000000]]
* [[Design goals|00001002000000]]
* [[Installation|00001003000000]]
* [[Configuration|00001004000000]]
* [[Structure of Zettelstore|00001005000000]]
* [[Layout of a zettel|00001006000000]]
* [[Zettelmarkup|00001007000000]]
* [[Other markup languages|00001008000000]]
* [[Security|00001010000000]]
* [[API|00001012000000]]
* [[Web user interface|00001014000000]]
* [[Tips and Tricks|00001017000000]]
* [[Troubleshooting|00001018000000]]
* Frequently asked questions

Licensed under the EUPL-1.2-or-later.

Changes to docs/manual/00001004050000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004050000
title: Command line parameters
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20211124141554

Zettelstore is not just a service that provides services of a zettelkasten.
It allows to some tasks to be executed at the command line.
Typically, the task (""sub-command"") will be given at the command line as the first parameter.

If no parameter is given, the Zettelstore is called as
```





|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001004050000
title: Command line parameters
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20220805174626

Zettelstore is not just a service that provides services of a zettelkasten.
It allows to some tasks to be executed at the command line.
Typically, the task (""sub-command"") will be given at the command line as the first parameter.

If no parameter is given, the Zettelstore is called as
```
22
23
24
25
26
27
28


=== Sub-commands
* [[``zettelstore help``|00001004050200]] lists all available sub-commands.
* [[``zettelstore version``|00001004050400]] to display version information of Zettelstore.
* [[``zettelstore run``|00001004051000]] to start the Zettelstore service.
* [[``zettelstore run-simple``|00001004051100]] is typically called, when you start Zettelstore by a double.click in your GUI.
* [[``zettelstore file``|00001004051200]] to render files manually without activated/running Zettelstore services.
* [[``zettelstore password``|00001004051400]] to calculate data for [[user authentication|00001010040200]].









>
>
22
23
24
25
26
27
28
29
30
=== Sub-commands
* [[``zettelstore help``|00001004050200]] lists all available sub-commands.
* [[``zettelstore version``|00001004050400]] to display version information of Zettelstore.
* [[``zettelstore run``|00001004051000]] to start the Zettelstore service.
* [[``zettelstore run-simple``|00001004051100]] is typically called, when you start Zettelstore by a double.click in your GUI.
* [[``zettelstore file``|00001004051200]] to render files manually without activated/running Zettelstore services.
* [[``zettelstore password``|00001004051400]] to calculate data for [[user authentication|00001010040200]].

To measure potential bottlenecks within the software Zettelstore, there are some [[command line flags for profiling the application|00001004059900]].

Changes to docs/manual/00001006020000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


24
25
26
27
28
29
30
id: 00001006020000
title: Supported Metadata Keys
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20220628111132

Although you are free to define your own metadata, by using any key (according to the [[syntax|00001006010000]]), some keys have a special meaning that is enforced by Zettelstore.
See the [[computed list of supported metadata keys|00000000000090]] for details.

Most keys conform to a [[type|00001006030000]].

; [!all-tags|''all-tags'']
: A property (a computed values that is not stored) that contains both the value of [[''tags''|#tags]] together with all [[tags|00001007040000#tag]] that are specified within the content.
; [!back|''back'']
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata, that are not referenced by this zettel.
  Basically, it is the value of [[''backward''|#backward]], but without any zettel identifier that is contained in [[''forward''|#forward]].
; [!backward|''backward'']
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata.
  References within invertible values are not included here, e.g. [[''precursor''|#precursor]].
; [!box-number|''box-number'']
: Is a computed value and contains the number of the box where the zettel was found.
  For all but the [[predefined zettel|00001005090000]], this number is equal to the number __X__ specified in startup configuration key [[''box-uri-__X__''|00001004010000#box-uri-x]].


; [!copyright|''copyright'']
: Defines a copyright string that will be encoded.
  If not given, the value ''default-copyright'' from the  [[configuration zettel|00001004020000#default-copyright]] will be used.
; [!credential|''credential'']
: Contains the hashed password, as it was emitted by [[``zettelstore password``|00001004051400]].
  It is internally created by hashing the password, the [[zettel identifier|00001006050000]], and the value of the ''ident'' key.






|







|









>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
id: 00001006020000
title: Supported Metadata Keys
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20220812145821

Although you are free to define your own metadata, by using any key (according to the [[syntax|00001006010000]]), some keys have a special meaning that is enforced by Zettelstore.
See the [[computed list of supported metadata keys|00000000000090]] for details.

Most keys conform to a [[type|00001006030000]].

; [!all-tags|''all-tags'']
: A property (a computed values that is not stored) that contains both the value of [[''tags''|#tags]] and the value of [[''content-tags''|#content-tags]].
; [!back|''back'']
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata, that are not referenced by this zettel.
  Basically, it is the value of [[''backward''|#backward]], but without any zettel identifier that is contained in [[''forward''|#forward]].
; [!backward|''backward'']
: Is a property that contains the identifier of all zettel that reference the zettel of this metadata.
  References within invertible values are not included here, e.g. [[''precursor''|#precursor]].
; [!box-number|''box-number'']
: Is a computed value and contains the number of the box where the zettel was found.
  For all but the [[predefined zettel|00001005090000]], this number is equal to the number __X__ specified in startup configuration key [[''box-uri-__X__''|00001004010000#box-uri-x]].
; [!content-tags|''content-tags'']
: A property that contains all [[inline tags|00001007040000#tag]] defined within the content.
; [!copyright|''copyright'']
: Defines a copyright string that will be encoded.
  If not given, the value ''default-copyright'' from the  [[configuration zettel|00001004020000#default-copyright]] will be used.
; [!credential|''credential'']
: Contains the hashed password, as it was emitted by [[``zettelstore password``|00001004051400]].
  It is internally created by hashing the password, the [[zettel identifier|00001006050000]], and the value of the ''ident'' key.

57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
  Basically the inverse of key [[''folge''|#folge]].
; [!published|''published'']
: This property contains the timestamp of the mast modification / creation of the zettel.
  If [[''modified''|#modified]] is set, it contains the same value.
  Otherwise, if the zettel identifier contains a valid timestamp, the identifier is used.
  In all other cases, this property is not set.

  It can be used for [[sorting|00001012052000]] zettel based on their publication date.

  It is a computed value.
  There is no need to set it via Zettelstore.
; [!read-only|''read-only'']
: Marks a zettel as read-only.
  The interpretation of [[supported values|00001006020400]] for this key depends, whether authentication is [[enabled|00001010040100]] or not.
; [!role|''role'']







|







59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
  Basically the inverse of key [[''folge''|#folge]].
; [!published|''published'']
: This property contains the timestamp of the mast modification / creation of the zettel.
  If [[''modified''|#modified]] is set, it contains the same value.
  Otherwise, if the zettel identifier contains a valid timestamp, the identifier is used.
  In all other cases, this property is not set.

  It can be used for [[sorting|00001007700000]] zettel based on their publication date.

  It is a computed value.
  There is no need to set it via Zettelstore.
; [!read-only|''read-only'']
: Marks a zettel as read-only.
  The interpretation of [[supported values|00001006020400]] for this key depends, whether authentication is [[enabled|00001010040100]] or not.
; [!role|''role'']

Changes to docs/manual/00001007000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007000000
title: Zettelmarkup
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220113185501

Zettelmarkup is a rich plain-text based markup language for writing zettel content.
Besides the zettel content, Zettelmarkup is also used for specifying the title of a zettel, regardless of the syntax of a zettel.

Zettelmarkup supports the longevity of stored notes by providing a syntax that any person can easily read, as well as a computer.
Zettelmarkup can be much easier parsed / consumed by a software compared to other markup languages.
Writing a parser for [[Markdown|https://daringfireball.net/projects/markdown/syntax]] is quite challenging.





|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007000000
title: Zettelmarkup
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220810194655

Zettelmarkup is a rich plain-text based markup language for writing zettel content.
Besides the zettel content, Zettelmarkup is also used for specifying the title of a zettel, regardless of the syntax of a zettel.

Zettelmarkup supports the longevity of stored notes by providing a syntax that any person can easily read, as well as a computer.
Zettelmarkup can be much easier parsed / consumed by a software compared to other markup languages.
Writing a parser for [[Markdown|https://daringfireball.net/projects/markdown/syntax]] is quite challenging.
30
31
32
33
34
35
36

37

However, the Zettelstore supports CommonMark as a zettel syntax, so you can mix both Zettelmarkup zettel and CommonMark zettel in one store to get the best of both worlds.

* [[General principles|00001007010000]]
* [[Basic definitions|00001007020000]]
* [[Block-structured elements|00001007030000]]
* [[Inline-structured element|00001007040000]]
* [[Attributes|00001007050000]]

* [[Summary of formatting characters|00001007060000]]








>
|
>
30
31
32
33
34
35
36
37
38
39
However, the Zettelstore supports CommonMark as a zettel syntax, so you can mix both Zettelmarkup zettel and CommonMark zettel in one store to get the best of both worlds.

* [[General principles|00001007010000]]
* [[Basic definitions|00001007020000]]
* [[Block-structured elements|00001007030000]]
* [[Inline-structured element|00001007040000]]
* [[Attributes|00001007050000]]
* [[Search expressions|00001007700000]]
* [[Summary of formatting characters|00001007800000]]
* [[Tutorial|00001007900000]]

Changes to docs/manual/00001007031100.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
id: 00001007031100
title: Zettelmarkup: Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133058

A transclusion allows to include the content of another zettel into the current zettel just by referencing the other zettel.

The transclusion specification begins with three consecutive left curly bracket characters (""''{''"", U+007B) at the first position of a line and ends with three consecutive right curly bracket characters (""''}''"", U+007D).
The curly brackets delimit the [[zettel identifier|00001006050000]] to be included.

First, the referenced zettel is read.
If it contains some transclusions itself, these will be expanded, recursively.
When a recursion is detected, expansion does not take place.
Instead an error message replaces the transclude specification.

An error message is also given, if the zettel cannot be read or if too many transclusions are made.
The maximum number of transclusion can be controlled by setting the value [[''max-transclusions''|00001004020000#max-transclusions]] of the runtime configuration zettel.

If everything went well, the referenced, expanded zettel will replace the transclusion element.

For example, to include the text of the Zettel titled ""Zettel identifier"", just specify its identifier [[''00001006050000''|00001006050000]] in the transclude element:
```zmk
{{{00001006050000}}}
```
This will result in:
:::example
{{{00001006050000}}}
:::

Please note: if the referenced zettel is changed, all transclusions will also change.

This allows, for example, to create a bigger document just by transcluding smaller zettel.

=== See also
[[Inline-mode transclusion|00001007040324]] does not work at the paragraph / block level, but is used for [[inline-structured elements|00001007040000]].





|

|


|

<
<
<
<
|
|
<
|
<
|
<
<
<
<
|
<
<
<

<
|
<
|
<
|
1
2
3
4
5
6
7
8
9
10
11
12




13
14

15

16




17



18

19

20

21
id: 00001007031100
title: Zettelmarkup: Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220809144920

A transclusion allows to include the content of other zettel into the current zettel.

The transclusion specification begins with three consecutive left curly bracket characters (""''{''"", U+007B) at the first position of a line and ends with three consecutive right curly bracket characters (""''}''"", U+007D).
The curly brackets delimit either a [[zettel identifier|00001006050000]] or a searched zettel list.





This leads to two variants of transclusion:
# Transclusion of the content of another zettel into the current zettel.

  This is done if you specify a zettel identifier, and is called ""zettel transclusion"".

# Transclusion of the list of zettel references that satisfy a [[search expression|00001007700000]].




  This is called ""search transclusion"".





The variants are described on separate zettel:

* [[Zettel transclusion|00001007031110]]

* [[Search transclusion|00001007031140]]

Added docs/manual/00001007031110.zettel.











































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
id: 00001007031110
title: Zettelmarkup: Zettel Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk

A zettel transclusion is specified by the following sequence, starting at the first position in a line: ''{{{zettel-identifier}}}''.

When evaluated, the referenced zettel is read.
If it contains some transclusions itself, these will be expanded, recursively.
When a recursion is detected, expansion does not take place.
Instead an error message replaces the transclude specification.

An error message is also given, if the zettel cannot be read or if too many transclusions are made.
The maximum number of transclusion can be controlled by setting the value [[''max-transclusions''|00001004020000#max-transclusions]] of the runtime configuration zettel.

If everything went well, the referenced, expanded zettel will replace the transclusion element.

For example, to include the text of the Zettel titled ""Zettel identifier"", just specify its identifier [[''00001006050000''|00001006050000]] in the transclude element:
```zmk
{{{00001006050000}}}
```
This will result in:
:::zs-example
{{{00001006050000}}}
:::

Please note: if the referenced zettel is changed, all transclusions will also change.

This allows, for example, to create a bigger document just by transcluding smaller zettel.

In addition, if a zettel __z__ transcludes a zettel __t__, but the current user is not allowed to view zettel __t__ (but zettel __z__), then the transclusion will not take place.
To the current user, it seems that there was no transclusion in zettel __z__.
This allows to create a zettel with content that seems to be changed, depending on the authorization of the current user.

=== See also
[[Inline-mode transclusion|00001007040324]] does not work at the paragraph / block level, but is used for [[inline-structured elements|00001007040000]].

Added docs/manual/00001007031140.zettel.





























































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
id: 00001007031140
title: Zettelmarkup: Search Transclusion
role: manual
tags: #manual #search #zettelmarkup #zettelstore
syntax: zmk
modified: 20220811141604

A search transclusion is specified by the following sequence, starting at the first position in a line: ''{{{search:search-expression}}}''.
The line must literally start with the sequence ''{{{search:''.
Everything after this prefix is interpreted as a [[search expression|00001007700000]].

When evaluated, the search expression is evaluated, leading to a list of [[links|00001007040310]] to zettel, matching the search expression.
Every link references the found zettel, with its title as link text.

This list replaces the search transclusion element.

For example, to include the list of all zettel with the [[all-tags|00001006020000#all-tags]] ""#search"", ordered by title specify the following search transclude element:
```zmk
{{{search:all-tags:#search ORDER title}}}
```
This will result in:
:::zs-example
{{{search:all-tags:#search ORDER title}}}
:::

Please note: if the referenced zettel is changed, all transclusions will also change.

For example, this allows to create a dynamic list of zettel inside a zettel, maybe to provide some introductory text followed by a list of child zettel.

The search will deliver only those zettel, which the current user is allowed to read.

Changes to docs/manual/00001007040000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007040000
title: Zettelmarkup: Inline-Structured Elements
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131736

Most characters you type is concerned with inline-structured elements.
The content of a zettel contains is many cases just ordinary text, lightly formatted.
Inline-structured elements allow to format your text and add some helpful links or images.
Sometimes, you want to enter characters that have no representation on your keyboard.

; Text formatting





|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001007040000
title: Zettelmarkup: Inline-Structured Elements
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220809171453

Most characters you type is concerned with inline-structured elements.
The content of a zettel contains is many cases just ordinary text, lightly formatted.
Inline-structured elements allow to format your text and add some helpful links or images.
Sometimes, you want to enter characters that have no representation on your keyboard.

; Text formatting
59
60
61
62
63
64
65
66
67
68

Since some Unicode character are used quite often, a special notation is introduced for them:

* Two consecutive hyphen-minus characters result in an __en-dash__ character.
  It is typically used in numeric ranges.
  ``pages 4--7`` will be rendered in HTML as: ::pages 4--7::{=example}.
  Alternative specifications are: ``&ndash;``, ``&x8211``, and ``&#x2013``.
* Three consecutive full stop characters (""''.''"", U+002E) after a space result in an horizontal ellipsis character.
  ``to be continued ... later`` will be rendered in HTML as: ::to be continued, ... later::{=example}.
  Alternative specifications are: ``&hellip;``, ``&x8230``, and ``&#x2026``.







<
<
<
59
60
61
62
63
64
65




Since some Unicode character are used quite often, a special notation is introduced for them:

* Two consecutive hyphen-minus characters result in an __en-dash__ character.
  It is typically used in numeric ranges.
  ``pages 4--7`` will be rendered in HTML as: ::pages 4--7::{=example}.
  Alternative specifications are: ``&ndash;``, ``&x8211``, and ``&#x2013``.



Changes to docs/manual/00001007040310.zettel.

1
2
3
4
5
6
7
8
9
10
11


12
13
14
15

16
17
18
19



20



21
22
23

24


id: 00001007040310
title: Zettelmarkup: Links
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218131639

There are two kinds of links, regardless of links to (internal) other zettel or to (external) material.
Both kinds begin with two consecutive left square bracket characters (""''[''"", U+005B) and ends with two consecutive right square bracket characters (""'']''"", U+005D).

The first form provides some text plus the link specification, delimited by a vertical bar character (""''|''"", U+007C): ``[[text|linkspecification]]``.



The second form just provides a link specification between the square brackets.
Its text is derived from the link specification, e.g. by interpreting the link specification as text: ``[[linkspecification]]``.


The link specification for another zettel within the same Zettelstore is just the [[zettel identifier|00001006050000]].
To reference some content within a zettel, you can append a number sign character (""''#''"", U+0023) and the name of the mark to the zettel identifier.
The resulting reference is called ""zettel reference"".




To specify some material outside the Zettelstore, just use an normal Uniform Resource Identifier (URI) as defined by [[RFC\ 3986|https://tools.ietf.org/html/rfc3986]].



If the URL begins with the slash character (""/"", U+002F), or if it begins with ""./"" or with ""../"", i.e. without scheme, user info, and host name, the reference will be treated as a ""local reference"", otherwise as an ""external reference"".
If the URL begins with two slash characters, it will be interpreted relative to the value of [[''url-prefix''|00001004010000#url-prefix]].


The text in the second form is just a sequence of [[inline elements|00001007040000]].







|





>
>




>




>
>
>
|
>
>
>
|
|

>
|
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
id: 00001007040310
title: Zettelmarkup: Links
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220808161918

There are two kinds of links, regardless of links to (internal) other zettel or to (external) material.
Both kinds begin with two consecutive left square bracket characters (""''[''"", U+005B) and ends with two consecutive right square bracket characters (""'']''"", U+005D).

The first form provides some text plus the link specification, delimited by a vertical bar character (""''|''"", U+007C): ``[[text|linkspecification]]``.
The text is a sequence of [[inline elements|00001007040000]].
However, it should not contain links itself.

The second form just provides a link specification between the square brackets.
Its text is derived from the link specification, e.g. by interpreting the link specification as text: ``[[linkspecification]]``.

=== Link specifications
The link specification for another zettel within the same Zettelstore is just the [[zettel identifier|00001006050000]].
To reference some content within a zettel, you can append a number sign character (""''#''"", U+0023) and the name of the mark to the zettel identifier.
The resulting reference is called ""zettel reference"".

If the link specification begins with the string ''search:'', the text following this string will be interpreted as a [[search expression|00001007700000]].
The resulting reference is called ""search reference"".
When this type of references is rendered, it will reference a list of all zettel that fulfills the search expression.

A link specification starting with one slash character (""''/''"", U+002F), or one or two full stop characters (""''.''"", U+002E) followed by a slash character,
will be interpreted as a local reference, called ""hosted reference"".
Such references will be interpreted relative to the web server hosting the Zettelstore.

If a link specification begins with two slash characters, it will be interpreted relative to the value of [[''url-prefix''|00001004010000#url-prefix]].

To specify some material outside the Zettelstore, just use an normal Uniform Resource Identifier (URI) as defined by [[RFC\ 3986|https://tools.ietf.org/html/rfc3986]].

=== Other topics
If the link references another zettel, and this zettel is not readable for the current user, because of a missing access rights, then only the associated text is presented.

Changes to docs/manual/00001007040320.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
id: 00001007040320
title: Zettelmarkup: Inline Embedding / Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220218133039

To some degree, an specification for embedded material is conceptually not too far away from a specification for [[linked material|00001007040310]].
Both contain a reference specification and optionally some text.
In contrast to a link, the specification of embedded material must currently resolve to some kind of real content.
This content replaces the embed specification.

An embed specification begins with two consecutive left curly bracket characters (""''{''"", U+007B) and ends with two consecutive right curly bracket characters (""''}''"", U+007D).
The curly brackets delimits either a reference specification or some text, a vertical bar character and the link specification, similar to a link.

One difference to a link: if the text was not given, an empty string is assumed.

The reference must point to some content, either zettel content or URL-referenced content.

If the referenced zettel does not exist, or is not readable, a [[spinning emoji|00000000040001]] is presented as a visual hint:
 
Example: ``{{00000000000000}}`` will be rendered as ::{{00000000000000}}::{=example}.

There are two kind of content:
# [[image content|00001007040322]],
# [[textual content|00001007040324]].





|












>
|






1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
id: 00001007040320
title: Zettelmarkup: Inline Embedding / Transclusion
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
modified: 20220803183936

To some degree, an specification for embedded material is conceptually not too far away from a specification for [[linked material|00001007040310]].
Both contain a reference specification and optionally some text.
In contrast to a link, the specification of embedded material must currently resolve to some kind of real content.
This content replaces the embed specification.

An embed specification begins with two consecutive left curly bracket characters (""''{''"", U+007B) and ends with two consecutive right curly bracket characters (""''}''"", U+007D).
The curly brackets delimits either a reference specification or some text, a vertical bar character and the link specification, similar to a link.

One difference to a link: if the text was not given, an empty string is assumed.

The reference must point to some content, either zettel content or URL-referenced content.
If the current user is not allowed to read the referenced zettel, the inline transclusion / embedding is ignored.
If the referenced zettel does not exist, or is not readable because of other reasons, a [[spinning emoji|00000000040001]] is presented as a visual hint:
 
Example: ``{{00000000000000}}`` will be rendered as ::{{00000000000000}}::{=example}.

There are two kind of content:
# [[image content|00001007040322]],
# [[textual content|00001007040324]].

Deleted docs/manual/00001007060000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
id: 00001007060000
title: Zettelmarkup: Summary of Formatting Characters
role: manual
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
modified: 20220311120759

The following table gives an overview about the use of all characters that begin a markup element.

|= Character :|= [[Blocks|00001007030000]] <|= [[Inlines|00001007040000]] <
| ''!''  | (free) | (free)
| ''"''  | [[Verse block|00001007030700]] | [[Short inline quote|00001007040100]]
| ''#''  | [[Ordered list|00001007030200]] | [[Tag|00001007040000]]
| ''$''  | (reserved) | (reserved)
| ''%''  | [[Comment block|00001007030900]] | [[Comment|00001007040000]]
| ''&''  | (free) | [[Entity|00001007040000]]
| ''\'''  | (free)  | [[Computer input|00001007040200]]
| ''(''  | (free) | (free)
| '')''  | (free) | (free)
| ''*''  | [[Unordered list|00001007030200]] | [[strongly emphasized text|00001007040100]]
| ''+''  | (free) | (free)
| '',''  | (free) | [[Sub-scripted text|00001007040100]]
| ''-''  | [[Horizontal rule|00001007030400]] | ""[[en-dash|00001007040000]]""
| ''.''  | (free) | [[Horizontal ellipsis|00001007040000]]
| ''/''  | (free) | (free)
| '':''  | [[Region block|00001007030800]] / [[description text|00001007030100]] | [[Inline region|00001007040100]]
| '';''  | [[Description term|00001007030100]] | [[Small text|00001007040100]]
| ''<''  | [[Quotation block|00001007030600]] | (free)
| ''=''  | [[Headings|00001007030300]] | [[Computer output|00001007040200]]
| ''>''  | [[Quotation lists|00001007030200]] | [[Inserted text|00001007040100]]
| ''?''  | (free) | (free)
| ''@''  | [[Inline-Zettel block|00001007031200]] | [[Inline-zettel snippet|00001007040200#inline-zettel-snippet]]
| ''[''  | (reserved)  | [[Linked material|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]
| ''\\'' | (blocked by inline meaning) | [[Escape character|00001007040000]]
| '']''  | (reserved) | End of [[link|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]
| ''^''  | (free) | [[Super-scripted text|00001007040100]]
| ''_''  | (free) | [[Emphasized text|00001007040100]]
| ''`''  | [[Verbatim block|00001007030500]] | [[Literal text|00001007040200]]
| ''{''  | [[Transclusion|00001007031100]] | [[Embedded material|00001007040300]], [[Attribute|00001007050000]]
| ''|''  | [[Table row / table cell|00001007031000]] | Separator within link and [[embed|00001007040320]] formatting
| ''}''  | End of [[Transclusion|00001007031100]] | End of embedded material, End of Attribute
| ''~''  | [[Evaluation block|00001007031300]] | [[Deleted text|00001007040100]]
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<




















































































Added docs/manual/00001007700000.zettel.





























































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
id: 00001007700000
title: Search expression
role: manual
tags: #manual #search #zettelstore
syntax: zmk
modified: 20220812144558

A search expression allows you to search for specific zettel.
You may select zettel based on a full-text search, based on specifc metadata values, or both.

In its simplest form, a search expression just contains a string to be search for with the help of a full-text search.
For example, the string ''syntax'' will search for all zettel containing the word ""syntax"".

If you want to search for all zettel with a title containing the word ""syntax"", you must specify ''title:syntax''.
""title"" names the [[metadata key|00001006010000]], in this case the [[supported metadata key ""title""|00001006020000#title]].
The colon character (""'':''"") is a [[search operator|00001007705000]], in this example to specify a match.
""syntax"" is the [[search value|00001007706000]] that must match to the value of the given metadata key, here ""title"".

A search expression may contain more than one search term, such as ''title:syntax''.
Search terms must be separated by one or more space characters, for example ''title:syntax title:search''.
All terms of a select expression must be true so that a zettel is selected.

* [[Search terms|00001007702000]]
* [[Search operator|00001007705000]]
* [[Search value|00001007706000]]

A search expression follows a [[formal syntax|00001007780000]].

Here are some examples of search expressions, which can be used to manage a Zettelstore:
{{{00001007790000}}}

Added docs/manual/00001007702000.zettel.











































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
id: 00001007702000
title: Search term
role: manual
tags: #manual #search #zettelstore
syntax: zmk
modified: 20220808130055

A search term allows you to specify one search restriction.
The result [[search expression|00001007700000]], which contains more than one search term, will be the applications of all restrictions.

A search term can be one of the following:
* A metadata-based search, by specifying the name of a [[metadata key|00001006010000]], followed by a [[search operator|00001007705000]], followed by an optional [[search value|00001007706000]].

  All zettel containing the given metadata key with a allowed value (depending on the search operator) are selected.

  If no search value is given, then all zettel containing the given metadata key are selected (or ignored, for a negated search operator).
* An optional [[search operator|00001007705000]], followed by a [[search value|00001007706000]].

  This specifies a full-text search for the given search value.

  **Note:** the search value will be normalized according to Unicode NKFD, ignoring everything except letters and numbers.
  Therefore, the following search expression are essentially the same: ''"search syntax"'' and ''search syntax''.
  The first is a search expression with one search value, which is normalized to two strings to be searched for.
  The second is a search expression containing two search values, giving two string to be searched for.
* The string ''NEGATE'' will negate (sic!) the behavior of the whole search expression.
  If it occurs multiple times, the negation will be negated.
* The string ''ORDER'', followed by a non-empty sequence of spaces and the name of a metadata key, will specify an ordering of the result list.
  If you include the string ''REVERSE'' after ''ORDER'' but before the metadata key, the ordering will be reversed.

  Example: ''ORDER published'' will order the resulting list based on the publishing data, while ''ORDER REVERSED published'' will return a reversed result order.

  Currently, only the first term specifying the order of the resulting list will be used.
  Other ordering terms will be ignored.

  An explicit order field will take precedence over the random order described below.

* The string ''RANDOM'' will provide a random order of the resulting list.

  Currently, only the first term specifying the order of the resulting list will be used.
  Other ordering terms will be ignored.

  A random order specification will be ignored, if there is an explicit ordering given.

  Example: ''RANDOM ORDER published'' will be interpreted as ''ORDER published''.

* The string ''OFFSET'', followed by a non-empty sequence of spaces and a number greater zero (called ""N"").

  This will ignore the first N elements of the result list, based on the specified sort order.
  A zero value of N will produce the same result as if nothing was specified.
  If specified multiple times, the higher value takes precedence.

  Example: ''OFFSET 4 OFFSET 8'' will be interpreted as ''OFFSET 8''.

* The string ''LIMIT'', followed by a non-empty sequence of spaces and a number greater zero (called ""N"").

  This will limit the result list to the first N elements, based on the specified sort order.
  A zero value of N will produce the same result as if nothing was specified.
  If specified multiple times, the lower value takes precedence.

  Example: ''LIMIT 4 LIMIT 8'' will be interpreted as ''LIMIT 4''.

You may have noted that the specifications of first two items overlap somehow.
This is resolved by the following rule:
* A search term containing no [[search operator character|00001007705000]] is treated as a full-text search.
* The first search operator character found in a search term divides the term into two pieces.
  If the first piece, from the beginning of the search term to the search operator character, is syntactically a metadata key, the search term is treated as a metadata-based search.
* Otherwise, the search term is treated as a full-text search.

If a term like ''ORDER'', ''ORDER REVERSE'', ''OFFSET'', or ''LIMIT'' is not followed by an appropriate value, it is interpreted as a search value for a full-text search. For example, ''ORDER 123'' will search for a zettel conatining the strings ""ORDER"" (case-insensitive) and ""123"".

Added docs/manual/00001007705000.zettel.



























































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
id: 00001007705000
title: Search operator
role: manual
tags: #manual #search #zettelstore
syntax: zmk
modified: 20220812184530

A search operator specifies how the comparison of a search value and a zettel should be executed.
Every comparison is done case-insensitive, treating all uppercase letters the same as lowercase letters.

The following are allowed search operator characters:
* The exclamation mark character (""!"", U+0021) negates the meaning
* The tilde character (""''~''"", U+007E) compares on containment (""contains operator"")
* The greater-than sign character (""''>''"", U+003E) matches if there is some prefix (""prefix operator"")
* The less-than sign character (""''<''"", U+003C) compares a suffix relationship (""suffix operator"")
* The colon character (""'':''"", U+003A) compares on equal words (""equal operator"")

Since the exclamation mark character can be combined with the other, there are 10 possible combinations:
# ""''!''"": is an abbreviation of the ""''!~''"" operator.
# ""''~''"": is successful if the search value is contained in the value to be compared.
# ""''!~''"": is successful if the search value is not contained in the value to be compared.
# ""'':''"": is successful if the search value is equal to one word of the value to be compared.
# ""''!:''"": is successful if the search value is not equal to any word of the value to be compared.
# ""''>''"": is successful if the search value is a prefix of the value to be compared.
# ""''!>''"": is successful if the search value is not a prefix of the value to be compared.
# ""''<''"": is successful if the search value is a suffix of the value to be compared.
# ""''!<''"": is successful if the search value is not a suffix of the value to be compared.
# ""''''"": a missing search operator can only occur for a full-text search.
  It is equal to the ""''~''"" operator.

Added docs/manual/00001007706000.zettel.





















>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
id: 00001007706000
title: Search value
role: manual
tags: #manual #search #zettelstore
syntax: zmk
modified: 20220807162031

A search value specifies a value to be searched for, depending on the [[search operator|00001007705000]].

A search value should be lower case, because all comparisons are done in a case-insensitive way and there are some upper case keywords planned.

Added docs/manual/00001007780000.zettel.

















































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
id: 00001007780000
title: Forma syntax of search expressions
role: manual
tags: #manual #reference #search #zettelstore
syntax: zmk
modified: 20220811141423

```
SearchExpression := SearchTerm (SPACE+ SearchTerm)*.
SearchTerm       := "NEGATE"
                  | SearchOperator? SearchValue
                  | SearchKey SearchOperator SearchValue?
                  | "RANDOM"
                  | "ORDER" SPACE+ ("REVERSE" SPACE+)? SearchKey
                  | "OFFSET" SPACE+ PosInt
                  | "LIMIT" SPACE+ PosInt.
SearchValue      := NO-SPACE (NO-SPACE)*.
SearchKey        := MetadataKey.
SearchOperator   := '!'
                 | ('!')? '='                ← removed in version 0.7.0
                 | ('!')? (':' | '<' | '>').
PosInt           := '0'
                  | ('1' .. '9') DIGIT*.
```

Added docs/manual/00001007790000.zettel.



































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
id: 00001007790000
title: Useful search expressions
role: manual
tags: #example #manual #search #zettelstore
syntax: zmk
modified: 20220811141224

|= Search Expression |= Meaning
| [[search:role:configuration]] | Zettel that contains some configuration data for the Zettelstore
| [[search:ORDER REVERSE id LIMIT 40]] | 40 recently created zettel
| [[search:ORDER REVERSE published LIMIT 40]] | 40 recently updated zettel
| [[search:RANDOM LIMIT 40]] | 40 random zettel
| [[search:dead:]] | Zettel with invalid / dead links
| [[search:backward!: precursor!:]] | Zettel that are not referenced by other zettel
| [[search:all-tags!:]] | Zettel without any tags
| [[search:tags!:]] | Zettel without tags that are defined within metadata
| [[search:content-tags:]] | Zettel with tags within content

Added docs/manual/00001007800000.zettel.





















































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
id: 00001007800000
title: Zettelmarkup: Summary of Formatting Characters
role: manual
tags: #manual #reference #zettelmarkup #zettelstore
syntax: zmk
modified: 20220810095559

The following table gives an overview about the use of all characters that begin a markup element.

|= Character :|= [[Blocks|00001007030000]] <|= [[Inlines|00001007040000]] <
| ''!''  | (free) | (free)
| ''"''  | [[Verse block|00001007030700]] | [[Short inline quote|00001007040100]]
| ''#''  | [[Ordered list|00001007030200]] | [[Tag|00001007040000]]
| ''$''  | (reserved) | (reserved)
| ''%''  | [[Comment block|00001007030900]] | [[Comment|00001007040000]]
| ''&''  | (free) | [[Entity|00001007040000]]
| ''\'''  | (free)  | [[Computer input|00001007040200]]
| ''(''  | (free) | (free)
| '')''  | (free) | (free)
| ''*''  | [[Unordered list|00001007030200]] | [[strongly emphasized text|00001007040100]]
| ''+''  | (free) | (free)
| '',''  | (free) | [[Sub-scripted text|00001007040100]]
| ''-''  | [[Horizontal rule|00001007030400]] | ""[[en-dash|00001007040000]]""
| ''.''  | (free) | (free)
| ''/''  | (free) | (free)
| '':''  | [[Region block|00001007030800]] / [[description text|00001007030100]] | [[Inline region|00001007040100]]
| '';''  | [[Description term|00001007030100]] | (free)
| ''<''  | [[Quotation block|00001007030600]] | (free)
| ''=''  | [[Headings|00001007030300]] | [[Computer output|00001007040200]]
| ''>''  | [[Quotation lists|00001007030200]] | [[Inserted text|00001007040100]]
| ''?''  | (free) | (free)
| ''@''  | [[Inline-Zettel block|00001007031200]] | [[Inline-zettel snippet|00001007040200#inline-zettel-snippet]]
| ''[''  | (reserved)  | [[Linked material|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]
| ''\\'' | (blocked by inline meaning) | [[Escape character|00001007040000]]
| '']''  | (reserved) | End of [[link|00001007040300]], [[citation key|00001007040300]], [[footnote|00001007040300]], [[mark|00001007040300]]
| ''^''  | (free) | [[Super-scripted text|00001007040100]]
| ''_''  | (free) | [[Emphasized text|00001007040100]]
| ''`''  | [[Verbatim block|00001007030500]] | [[Literal text|00001007040200]]
| ''{''  | [[Transclusion|00001007031100]] | [[Embedded material|00001007040300]], [[Attribute|00001007050000]]
| ''|''  | [[Table row / table cell|00001007031000]] | Separator within link and [[embed|00001007040320]] formatting
| ''}''  | End of [[Transclusion|00001007031100]] | End of embedded material, End of Attribute
| ''~''  | [[Evaluation block|00001007031300]] | [[Deleted text|00001007040100]]

Added docs/manual/00001007900000.zettel.



















>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
id: 00001007900000
title: Zettelmarkup: Tutorial
role: manual
tags: #manual #tutorial #zettelmarkup #zettelstore
syntax: zmk
modified: 20220811135314

* [[First steps|00001007903000]]: learn something about paragraphs, emphasized text, and lists.
* [[Second steps|00001007906000]]: know about links, thematic breaks, and headings.

Added docs/manual/00001007903000.zettel.



























































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
id: 00001007903000
title: Zettelmarkup: First Steps
role: manual
tags: #manual #tutorial #zettelmarkup #zettelstore
syntax: zmk
modified: 20220811122618

[[Zettelmarkup|00001007000000]] allows you to leave your text as it is, at least in many situations.
Some characters have a special meaning, but you have to enter them is a defined way to see a visible change.
Zettelmarkup is designed to be used for zettel, which are relatively short.
It allows to produce longer texts, but you should probably use a different tool, if you want to produce an scientific paper, to name an example.

=== Paragraphs
The most important concept of Zettelmarkup is the __paragraph__.
Ordinary text is interpreted as part of a paragraph.
Paragraphs are typically separated by one or more blank lines.

Therefore, line endings are more or less ignored within one paragraph.
Zettelmarkup will recognize the end of a line, and sore it as a ""soft break".
A soft break is rendered in most cases as a space character.

Within a paragraph you can style your text with [[special markup|00001007040000]].
Some examples:

|= Zettelmarkup | Rendered output | Instruction
| ''An __emphasized__ word'' | An __emphasized__ word | Put two underscore characters before and after the text you want to emphasize
| ''Someone uses **bold** text'' | Someone uses **bold** text | Put two asterisks before and after the text you want to see bold
| ''He says: ""I love you!""'' | Her says: ""I love you!"" | Put two quotation mark characters before and after the text you want to quote.

You probably see a principle.

One nice thing about the quotation mark characters: they are rendered according to the current language.
Examples: ""english""{lang=en}, ""french""{lang=fr}, ""german""{lang=de}, ""finnish""{lang=fi}.
You will see later, how to change the current language.

=== Lists
Quite often, text consists of lists.
Zettelmarkup supports different types of lists.
The most important lists are:
* Unnumbered lists,
* Numbered lists.

You produce an unnumbered list element by writing an asterisk character followed by a space character at the beginning of a line.
Since a list typically consists of more than one element, the following elements will also start at their own line:

```zmk
* First item
* Second item
* Third item
```

This is rendered as:

:::zs-example
* First item
* Second item
* Third item
:::

Similar, an numbered list element begins a line with the number sign (sic!) followed by a space character:

```zmk
# First item
# Second item
# Third item
```

This is rendered as:

:::zs-example
# First item
# Second item
# Third item
:::

---
After trying out these markup elements, you might want to continue with the [[second steps|00001007906000]].

Added docs/manual/00001007906000.zettel.







































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
id: 00001007906000
title: Zettelmarkup: Second Steps
role: manual
tags: #manual #tutorial #zettelmarkup #zettelstore
syntax: zmk
modified: 20220811135024

After you have [[learned|00001007903000]] the basic concepts and markup of Zettelmarkup (paragraphs, emphasized text, and lists), this zettel introduces you into the concepts of links, thematic breaks, and headings.

=== Links
A Zettelstore is much more useful, if you connect related zettel.
If you read a zettel later, this allows you to know about the context of a zettel.
[[Zettelmarkup|00001007000000]] allows you to specify such a connection.
A connection can be specified within a paragraph via [[Links|00001007040310]].

* A link always starts with two left square bracket characters and ends with two right square bracket characters: ''[[...]]''.
* Within these character sequences you specify the [[zettel identifier|00001006050000]] of the zettel you want to reference: ''[[00001007903000]]'' will connect to zettel containing the first steps into Zettelmarkup.
* In addition, you should give the link a more readable description.
  This is done by prepending the description before the reference and use the vertical bar character to separate both: ''[[First Steps|00001007903000]]''.

You are not restricted to reference your zettel.
Alternatively, you might specify an URL of an external website: ''[[Zettelstore|https://zettelstore.de]]''.
Of course, if you just want to specify the URL, you are allowed to omit the description: ''[[https://zettelstore.de]]''

|= Zettelmarkup | Rendered output | Remark
| ''[[00001007903000]]'' | [[00001007903000]] | If no description is given, the zettel identifier acts as a description
| ''[[First Steps|00001007903000]]'' | [[First Steps|00001007903000]] | The description should be chosen so that you are not confused later
| ''[[https://zettelstore.de]]'' | [[https://zettelstore.de]] | A link to an external URL is rendered differently
| ''[[Zettelstore|https://zettelstore.de]]'' | [[Zettelstore|https://zettelstore.de]] | You can use any URL your browser is able to support

Again, you probably see a principle.

=== Thematic Breaks
[[And now for something completely different|https://en.wikipedia.org/wiki/And_Now_for_Something_Completely_Different]].

Sometimes, you want to insert a thematic break into your text, because two paragraphs do not separate enough.
In Zettelmarkup is is done by entering three or more hyphen-minus characters at the beginning of a new line.
You must not include blank lines around this line, but it can be more readable if you want to look at the Zettelmarkup text.

```zmk
First paragraph.
---
Second paragraph.
```

```zmk
First paragraph.

---

Second paragraph.
```

Both are rendered as:
:::zs-example
First paragraph.
---
Second paragraph.
:::

Try it!

This might be the time to relax a rule about paragraphs.
You must not specify a blank line to end a paragraph.
Any Zettelmarkup that must start at the beginning of a new line will end a previous paragraph.
Similar, a blank line must not precede a paragraph.

This applies also to lists, as given in the first steps, as well as other [[similar markup|00001007030000]] you will probably later.

=== Headings
Headings explicitly structure a zettel, similar to thematic breaks, but gives the resulting part a name.

To specify a heading in Zettelmarkup, you must enter at least three equal signs, followed by a space, followed by the text of the heading.
Everything must be one the same line.

The number of equal signs determines the importance of the heading: less equal signs means more important.
Therefore, three equal signs treat a heading as most important.
It is a level-1 heading.
Zettelmarkup supports up to five levels.
To specify such a heading, you must enter seven equal signs, plus the space and the text.
If you enter more than seven equal signs, the resulting heading is still of level 5.

See the [[description of headings|00001007030300]] for more details and examples.

Changes to docs/manual/00001010070200.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010070200
title: Visibility rules for zettel
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
modified: 20220304114501

For every zettel you can specify under which condition the zettel is visible to others.
This is controlled with the metadata key [[''visibility''|00001006020000#visibility]].
The following values are supported:

; [!public|""public""]
: The zettel is visible to everybody, even if the user is not authenticated.





|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001010070200
title: Visibility rules for zettel
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
modified: 20220812145701

For every zettel you can specify under which condition the zettel is visible to others.
This is controlled with the metadata key [[''visibility''|00001006020000#visibility]].
The following values are supported:

; [!public|""public""]
: The zettel is visible to everybody, even if the user is not authenticated.
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
  This is for zettel with sensitive content, e.g. the [[configuration zettel|00001004020000]] or the various zettel that contains the templates for rendering zettel in HTML.
; [!expert|""expert""]
: Only the owner of the Zettelstore can access the zettel, if runtime configuration [[''expert-mode''|00001004020000#expert-mode]] is set to a [[boolean true value|00001006030500]].

  This is for zettel with sensitive content that might irritate the owner.
  Computed zettel with internal runtime information are examples for such a zettel.

When you install a Zettelstore, only [[some zettel|//h?visibility=public]] have visibility ""public"".
One is the zettel that contains [[CSS|00000000020001]] for displaying the [[web user interface|00001014000000]].
This is to ensure that the web interface looks nice even for not authenticated users.
Another is the zettel containing the Zettelstore [[license|00000000000004]].
The [[default image|00000000040001]], used if an image reference is invalid, is also public visible.

Please note: if [[authentication is not enabled|00001010040100]], every user has the same rights as the owner of a Zettelstore.
This is also true, if the Zettelstore runs additionally in [[read-only mode|00001004010000#read-only-mode]].
In this case, the [[runtime configuration zettel|00001004020000]] is shown (its visibility is ""owner"").
The [[startup configuration|00001004010000]] is not shown, because the associated computed zettel with identifier ''00000000000096'' is stored with the visibility ""expert"".
If you want to show such a zettel, you must set ''expert-mode'' to true.

=== Examples
Similar to the [[API|00001012051810]], you can easily create a zettel list based on the ''visibility'' metadata key:

| public  | [[//h?visibility=public]]
| login   | [[//h?visibility=login]]
| creator | [[//h?visibility=creator]]
| owner   | [[//h?visibility=owner]]
| expert  | [[//h?visibility=expert]][^Only if [[''expert-mode''|00001004020000#expert-mode]] is enabled, this list will show some zettel.]







|












|

|
|
|
|
|
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
  This is for zettel with sensitive content, e.g. the [[configuration zettel|00001004020000]] or the various zettel that contains the templates for rendering zettel in HTML.
; [!expert|""expert""]
: Only the owner of the Zettelstore can access the zettel, if runtime configuration [[''expert-mode''|00001004020000#expert-mode]] is set to a [[boolean true value|00001006030500]].

  This is for zettel with sensitive content that might irritate the owner.
  Computed zettel with internal runtime information are examples for such a zettel.

When you install a Zettelstore, only [[some zettel|search:visibility:public]] have visibility ""public"".
One is the zettel that contains [[CSS|00000000020001]] for displaying the [[web user interface|00001014000000]].
This is to ensure that the web interface looks nice even for not authenticated users.
Another is the zettel containing the Zettelstore [[license|00000000000004]].
The [[default image|00000000040001]], used if an image reference is invalid, is also public visible.

Please note: if [[authentication is not enabled|00001010040100]], every user has the same rights as the owner of a Zettelstore.
This is also true, if the Zettelstore runs additionally in [[read-only mode|00001004010000#read-only-mode]].
In this case, the [[runtime configuration zettel|00001004020000]] is shown (its visibility is ""owner"").
The [[startup configuration|00001004010000]] is not shown, because the associated computed zettel with identifier ''00000000000096'' is stored with the visibility ""expert"".
If you want to show such a zettel, you must set ''expert-mode'' to true.

=== Examples
Similar to the [[API|00001012051840]], you can easily create a zettel list based on the ''visibility'' metadata key:

| public  | [[search:visibility:public]]
| login   | [[search:visibility:login]]
| creator | [[search:visibility:creator]]
| owner   | [[search:visibility:owner]]
| expert  | [[search:visibility:expert]][^Only if [[''expert-mode''|00001004020000#expert-mode]] is enabled, this list will show some zettel.]

Changes to docs/manual/00001012000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
id: 00001012000000
title: API
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220627183444

The API (short for ""**A**pplication **P**rogramming **I**nterface"") is the primary way to communicate with a running Zettelstore.
Most integration with other systems and services is done through the API.
The [[web user interface|00001014000000]] is just an alternative, secondary way of interacting with a Zettelstore.

=== Background
The API is HTTP-based and uses plain text and JSON as its main encoding format for exchanging messages between a Zettelstore and its client software.

There is an [[overview zettel|00001012920000]] that shows the structure of the endpoints used by the API and gives an indication about its use.

=== Authentication
If [[authentication is enabled|00001010040100]], most API calls must include an [[access token|00001010040700]] that proves the identity of the caller.
* [[Authenticate an user|00001012050200]] to obtain an access token
* [[Renew an access token|00001012050400]] without costly re-authentication
* [[Provide an access token|00001012050600]] when doing an API call

=== Zettel lists
* [[List metadata of all zettel|00001012051200]]
* [[Shape the list of zettel metadata|00001012051800]]
** [[Selection of zettel|00001012051810]]
** [[Limit the list length|00001012051830]]
** [[Content search|00001012051840]]
** [[Sort the list of zettel metadata|00001012052000]]
* [[Map metadata values to lists of zettel identifier|00001012052400]]

=== Working with zettel
* [[Create a new zettel|00001012053200]]
* [[Retrieve metadata and content of an existing zettel|00001012053300]]
* [[Retrieve metadata of an existing zettel|00001012053400]]
* [[Retrieve evaluated metadata and content of an existing zettel in various encodings|00001012053500]]





|


















<
<
<
|
<







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24



25

26
27
28
29
30
31
32
id: 00001012000000
title: API
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220812145549

The API (short for ""**A**pplication **P**rogramming **I**nterface"") is the primary way to communicate with a running Zettelstore.
Most integration with other systems and services is done through the API.
The [[web user interface|00001014000000]] is just an alternative, secondary way of interacting with a Zettelstore.

=== Background
The API is HTTP-based and uses plain text and JSON as its main encoding format for exchanging messages between a Zettelstore and its client software.

There is an [[overview zettel|00001012920000]] that shows the structure of the endpoints used by the API and gives an indication about its use.

=== Authentication
If [[authentication is enabled|00001010040100]], most API calls must include an [[access token|00001010040700]] that proves the identity of the caller.
* [[Authenticate an user|00001012050200]] to obtain an access token
* [[Renew an access token|00001012050400]] without costly re-authentication
* [[Provide an access token|00001012050600]] when doing an API call

=== Zettel lists
* [[List metadata of all zettel|00001012051200]]



** [[Search expressions|00001012051840]] (includes content search)

* [[Map metadata values to lists of zettel identifier|00001012052400]]

=== Working with zettel
* [[Create a new zettel|00001012053200]]
* [[Retrieve metadata and content of an existing zettel|00001012053300]]
* [[Retrieve metadata of an existing zettel|00001012053400]]
* [[Retrieve evaluated metadata and content of an existing zettel in various encodings|00001012053500]]

Changes to docs/manual/00001012051200.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

23
24
25
26
27
28

29
30
31
32
33
34
35
id: 00001012051200
title: API: List metadata of all zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220201180649

To list the metadata of all zettel just send a HTTP GET request to the [[endpoint|00001012920000]] ''/j''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
If successful, the output is a JSON object:

```sh
# curl http://127.0.0.1:23123/j
{"query":"","list":[{"id":"00001012051200","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012050600","meta":{"title":"API: Provide an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012050400","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012050200","meta":{"title":"API: Authenticate a client","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012000000","meta":{"title":"API","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62}]}
```

The JSON object contains a key ''"list"'' where its value is a list of zettel JSON objects.
These zettel JSON objects themselves contains the keys ''"id"'' (value is a string containing the [[zettel identifier|00001006050000]]), ''"meta"'' (value as a JSON object), and ''"rights"'' (encodes the [[access rights|00001012921200]] for the given zettel).
The value of key ''"meta"'' effectively contains all metadata of the identified zettel, where metadata keys are encoded as JSON object keys and metadata values encoded as JSON strings.

Additionally, the JSON object contains a key ''"query"'' with a string value.
It will contain a textual description of the underlying query if you [[select only some zettel|00001012051810]].
Without a selection, the value is the empty string.


If you reformat the JSON output from the ''GET /j'' call, you'll see its structure better:

```json
{
  "query": "",

  "list": [
    {
      "id": "00001012051200",
      "meta": {
        "title": "API: List for all zettel some data",
        "tags": "#api #manual #zettelstore",
        "syntax": "zmk",





|













|
|
|
>






>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
id: 00001012051200
title: API: List metadata of all zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220812145500

To list the metadata of all zettel just send a HTTP GET request to the [[endpoint|00001012920000]] ''/j''[^If [[authentication is enabled|00001010040100]], you must include the a valid [[access token|00001012050200]] in the ''Authorization'' header].
If successful, the output is a JSON object:

```sh
# curl http://127.0.0.1:23123/j
{"query":"","list":[{"id":"00001012051200","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012050600","meta":{"title":"API: Provide an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012050400","meta":{"title":"API: Renew an access token","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012050200","meta":{"title":"API: Authenticate a client","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62},{"id":"00001012000000","meta":{"title":"API","tags":"#api #manual #zettelstore","syntax":"zmk","role":"manual"},"rights":62}]}
```

The JSON object contains a key ''"list"'' where its value is a list of zettel JSON objects.
These zettel JSON objects themselves contains the keys ''"id"'' (value is a string containing the [[zettel identifier|00001006050000]]), ''"meta"'' (value as a JSON object), and ''"rights"'' (encodes the [[access rights|00001012921200]] for the given zettel).
The value of key ''"meta"'' effectively contains all metadata of the identified zettel, where metadata keys are encoded as JSON object keys and metadata values encoded as JSON strings.

Additionally, the JSON object contains the keys ''"query"'' and ''"human"'' with a string value.
Both will contain a textual description of the underlying query if you select only some zettel with a [[search expression|00001012051840]].
Without a selection, the values are the empty string.
''"query"'' returns the normalized search expression itself, while ''"human"'' is the normalized search expression to be read by humans.

If you reformat the JSON output from the ''GET /j'' call, you'll see its structure better:

```json
{
  "query": "",
  "human": "",
  "list": [
    {
      "id": "00001012051200",
      "meta": {
        "title": "API: List for all zettel some data",
        "tags": "#api #manual #zettelstore",
        "syntax": "zmk",

Deleted docs/manual/00001012051800.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
id: 00001012051800
title: API: Shape the list of zettel metadata
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20211103162259

In most cases, it is not essential to list __all__ zettel.
Typically, you are interested only in a subset of the zettel maintained by your Zettelstore.
This is done by adding some query parameters to the general ''GET /j'' request.

* [[Select|00001012051810]] just some zettel, based on metadata.
* Only a specific amount of zettel will be selected by specifying [[a length and/or an offset|00001012051830]].
* [[Searching for specific content|00001012051840]], not just the metadata, is another way of selecting some zettel.
* The resulting list can be [[sorted|00001012052000]] according to various criteria.
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<






























Deleted docs/manual/00001012051810.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
id: 00001012051810
title: API: Select zettel based on their metadata
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220218133305

Every query parameter that does __not__ begin with the low line character (""_"", U+005F) is treated as the name of a [[metadata|00001006010000]] key.
According to the [[type|00001006030000]] of a metadata key, zettel are possibly selected.
All [[supported|00001006020000]] metadata keys have a well-defined type.
User-defined keys have the type ''e'' (string, possibly empty).

For example, if you want to retrieve all zettel that contain the string ""API"" in its title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/j?title=API'
{"query":"title MATCH API","list":[{"id":"00001012921000","meta":{"title":"API: JSON structure of an access token","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920500","meta":{"title":"Formats available by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920000","meta":{"title":"Endpoints used by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}}, ...
```

However, if you want all zettel that does not match a given value, you must prefix the value with the exclamation mark character (""!"", U+0021).
For example, if you want to retrieve all zettel that do not contain the string ""API"" in their title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/j?title=!API'
{"query":"title NOT MATCH API","list":[{"id":"00010000000000","meta":{"back":"00001003000000 00001005090000","backward":"00001003000000 00001005090000","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00000000000001 00000000000003 00000000000096 00000000000100","lang":"en","license":"EUPL-1.2-or-later","role":"zettel","syntax":"zmk","title":"Home"}},{"id":"00001014000000","meta":{"back":"00001000000000 00001004020000 00001012920510","backward":"00001000000000 00001004020000 00001012000000 00001012920510","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00001012000000","lang":"en","license":"EUPL-1.2-or-later","published":"00001014000000","role":"manual","syntax":"zmk","tags":"#manual #webui #zettelstore","title":"Web user interface"}},
...
```

In both cases, an implicit precondition is that the zettel must contain the given metadata key.
For a metadata key like [[''title''|00001006020000#title]], which has a default value, this precondition should always be true.
But the situation is different for a key like [[''url''|00001006020000#url]].
Both ``curl 'http://localhost:23123/j?url='`` and ``curl 'http://localhost:23123/j?url=!'`` may result in an empty list.

The empty query parameter values matches all zettel that contain the given metadata key.
Similar, if you specify just the exclamation mark character as a query parameter value, only those zettel match that does not contain the given metadata key.
This is in contrast to above rule that the metadata value must exist before a match is done.
For example ``curl 'http://localhost:23123/j?back=!&backward='`` returns all zettel that are reachable via other zettel, but also references these zettel.

As stated above, the exact rule for comparison depends on the [[type|00001006030000]] of the specified metadata key.
By using a [[simple search syntax|00001012051890]], you are able to specify other comparison operations.[^One is the already mentioned exclamation mark character.]

Above example shows that all sub-expressions of a select specification must be true so that no zettel is rejected from the final list.

If you specify the query parameter ''_negate'', either with or without a value, the whole selection will be negated.
Because of the precondition described above, ``curl 'http://127.0.0.1:23123/j?url=!com'`` and ``curl 'http://127.0.0.1:23123/j?url=com&_negate'`` may produce different lists.
The first query produces a zettel list, where each zettel does have a ''url'' metadata value, which does not contain the characters ""com"".
The second query produces a zettel list, that excludes any zettel containing a ''url'' metadata value that contains the characters ""com""; this also includes all zettel that do not contain the metadata key ''url''.

Alternatively, you also can use the [[endpoint|00001012920000]] ''/z'' for a simpler result format.
The first example translates to:
```sh
# curl 'http://127.0.0.1:23123/z?title=API'
00001012921000 API: JSON structure of an access token
00001012920500 Formats available by the API
00001012920000 Endpoints used by the API
...
```
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<














































































































Deleted docs/manual/00001012051830.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
id: 00001012051830
title: API: Shape the list of zettel metadata by limiting its length
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20211004124642

=== Limit
By using the query parameter ""''_limit''"", which must have an integer value, you specifying an upper limit of list elements:
```sh
# curl 'http://127.0.0.1:23123/j?title=API&_sort=id&_limit=2'
{"query":"title MATCH API LIMIT 2","list":[{"id":"00001012000000","meta":{"all-tags":"#api #manual #zettelstore","back":"00001000000000 00001004020000","backward":"00001000000000 00001004020000 00001012053200 00001012054000 00001014000000","box-number":"1","forward":"00001010040100 00001010040700 00001012050200 00001012050400 00001012050600 00001012051200 00001012051800 00001012051810 00001012051830 00001012051840 00001012052000 00001012052200 00001012052400 00001012052600 00001012053200 00001012053300 00001012053500 00001012053600 00001012053700 00001012053800 00001012054000 00001012054200 00001012054400 00001012054600 00001012920000 00001014000000","modified":"20210817160844","published":"20210817160844","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API"}},{"id":"00001012050200","meta":{"all-tags":"#api #manual #zettelstore","back":"00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600","backward":"00001010040700 00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600 00001012920000 00001012921000","box-number":"1","forward":"00001004010000 00001010040200 00001010040700 00001012920000 00001012921000","modified":"20210726123709","published":"20210726123709","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Authenticate a client"}}]}
```

```sh
# curl 'http://127.0.0.1:23123/z?title=API&_sort=id&_limit=2'
00001012000000 API
00001012050200 API: Authenticate a client
```

=== Offset
The query parameter ""''_offset''"" allows to list not only the first elements, but to begin at a specific element:
```sh
# curl 'http://127.0.0.1:23123/j?title=API&_sort=id&_limit=2&_offset=1'
{"query":"title MATCH API OFFSET 1 LIMIT 2","list":[{"id":"00001012050200","meta":{"all-tags":"#api #manual #zettelstore","back":"00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600","backward":"00001010040700 00001012000000 00001012050400 00001012050600 00001012051200 00001012053400 00001012053500 00001012053600 00001012920000 00001012921000","box-number":"1","forward":"00001004010000 00001010040200 00001010040700 00001012920000 00001012921000","modified":"20210726123709","published":"20210726123709","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Authenticate a client"}},{"id":"00001012050400","meta":{"all-tags":"#api #manual #zettelstore","back":"00001010040700 00001012000000","backward":"00001010040700 00001012000000 00001012920000 00001012921000","box-number":"1","forward":"00001010040100 00001012050200 00001012920000 00001012921000","modified":"20210726123745","published":"20210726123745","role":"manual","syntax":"zmk","tags":"#api #manual #zettelstore","title":"API: Renew an access token"}}]}
```

```sh
# curl 'http://127.0.0.1:23123/z?title=API&_sort=id&_limit=2&_offset=1'
00001012050200 API: Authenticate a client
00001012050400 API: Renew an access token
```
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
































































Changes to docs/manual/00001012051840.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17





























id: 00001012051840
title: API: Shape the list of zettel metadata by searching the content
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20211124182444

The query parameter ""''_s''"" allows to provide a string for a full-text search of all zettel.
The search string will be normalized according to Unicode NKFD, ignoring everything except letters and numbers.

If you want to search in a specific way, you must apply the [[simple search syntax|00001012051890]].
Otherwise, the content of each zettel is examined to just contain the words of the search string.

You are allowed to specify this query parameter more than once.
All results will be intersected, i.e. a zettel will be included into the list if all of the provided values match.

This parameter loosely resembles the search form of the [[web user interface|00001014000000]].






























|

|

|

<
<
|
<
<

|



>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7


8


9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
id: 00001012051840
title: API: Shape the list of zettel metadata by specifying a search expression
role: manual
tags: #api #manual #search #zettelstore
syntax: zmk
modified: 20220812144637



The query parameter ""''_s''"" allows you to specify [[search expressions|00001007700000]] for a full-text search of all zettel content and/or restricting the search according to specific metadata.



You are allowed to specify this query parameter more than once, as well as the other query parameters.
All results will be intersected, i.e. a zettel will be included into the list if all of the provided values match.

This parameter loosely resembles the search form of the [[web user interface|00001014000000]].

For example, if you want to retrieve all zettel that contain the string ""API"" in its title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/j?_s=title%3AAPI'
{"query":"title MATCH API","list":[{"id":"00001012921000","meta":{"title":"API: JSON structure of an access token","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920500","meta":{"title":"Formats available by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920000","meta":{"title":"Endpoints used by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}}, ...
```

However, if you want all zettel that does not match a given value, you must prefix the value with the exclamation mark character (""!"", U+0021).
For example, if you want to retrieve all zettel that do not contain the string ""API"" in their title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/j?_s=title!%3AAPI'
{"query":"title NOT MATCH API","list":[{"id":"00010000000000","meta":{"back":"00001003000000 00001005090000","backward":"00001003000000 00001005090000","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00000000000001 00000000000003 00000000000096 00000000000100","lang":"en","license":"EUPL-1.2-or-later","role":"zettel","syntax":"zmk","title":"Home"}},{"id":"00001014000000","meta":{"back":"00001000000000 00001004020000 00001012920510","backward":"00001000000000 00001004020000 00001012000000 00001012920510","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00001012000000","lang":"en","license":"EUPL-1.2-or-later","published":"00001014000000","role":"manual","syntax":"zmk","tags":"#manual #webui #zettelstore","title":"Web user interface"}},
...
```

In both cases, an implicit precondition is that the zettel must contain the given metadata key.
For a metadata key like [[''title''|00001006020000#title]], which has a default value, this precondition should always be true.
But the situation is different for a key like [[''url''|00001006020000#url]].
Both ``curl 'http://localhost:23123/j?_s=url%3A'`` and ``curl 'http://localhost:23123/j?_s=url%3A!'`` may result in an empty list.

Alternatively, you also can use the [[endpoint|00001012920000]] ''/z'' for a simpler result format.
The first example translates to:
```sh
# curl 'http://127.0.0.1:23123/z?_s=title%3AAPI'
00001012921000 API: JSON structure of an access token
00001012920500 Formats available by the API
00001012920000 Endpoints used by the API
...
```

Deleted docs/manual/00001012051890.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
id: 00001012051890
title: API: Search syntax (simple)
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220218130900

If the search string starts with the exclamation mark character (""!"", U+0021), it will be removed and the query matches all values that **do not match** the search string.

In the next step, the first character of the search string will be inspected.
If it contains one of the characters ""'':''"", ""''=''"", ""''>''"", ""''<''"", or ""''~''"", this will modify how the search will be performed.
The character will be removed from the start of the search string.

For example, assume the search string is ""def"":

; The colon character (""'':''"", U+003A) (or none of these characters)
: This is the __default__ comparison.
  The comparison depends on the type of the underlying values.
  For a content search, it is equal to the tilde character ""''~''"", which returns true if a word within the content just contains the search string.
  For metadata, it depends on the key [[type|00001006030000]].

  It you omit the the comparison character, the default comparison is also used.
; The tilde character (""''~''"", U+007E)
: The inspected text[^Either all words of the zettel content and/or some metadata values] contains the search string.
  ""def"", ""defghi"", and ""abcdefghi"" are matching the search string.
; The equal sign character (""''=''"", U+003D)
: The inspected text must contain a word that is equal to the search string.
  Only the word ""def"" matches the search string.
; The greater-than sign character (""''>''"", U+003E)
: The inspected text must contain a word with the search string as a prefix.
  A word like ""def"" or ""defghi"" matches the search string.
; The less-than sign character (""''<''"", U+003C)
: The inspected text must contain a word with the search string as a suffix.
  A word like ""def"" or ""abcdef"" matches the search string.

If you want to include an initial ""''!''"" into the search string, you must prefix that with the escape character ""''\\''"".
For example ""\\!abc"" will search for the string ""!abc"".
A similar rule applies to the characters that specify the way how the search will be done.
For example, ""!\\=abc"" will search for content that does not contains the string ""=abc"".
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<














































































Deleted docs/manual/00001012052000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
id: 00001012052000
title: API: Sort the list of zettel metadata
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220218131937

If not specified, the list of zettel is sorted descending by the value of the [[zettel identifier|00001006050000]].
The highest zettel identifier, which is a number, comes first.
You change that with the ""''_sort''"" query parameter.
Alternatively, you can also use the ""''_order''"" query parameter.
It is an alias.

Its value is the name of a metadata key, optionally prefixed with a hyphen-minus character (""''-''"", U+002D).
According to the [[type|00001006030000]] of a metadata key, the list of zettel is sorted.
If hyphen-minus is given, the order is descending, else ascending.

If you want a random list of zettel, specify the value ""_random"" in place of the metadata key.
""''_sort=_random''"" (or ""''_order=_random''"") is the query parameter in this case.
If can be combined with ""[[''_limit=1''|00001012051830]]"" to obtain just one random zettel.

Currently, only the first occurrence of ''_sort'' is recognized.
In the future it will be possible to specify a combined sort key.
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<














































Changes to docs/manual/00001012053900.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012053900
title: API: Retrieve unlinked references to an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220202112528

The value of a personal Zettelstore is determined in part by explicit connections between related zettel.
If the number of zettel grow, some of these connections are missing.
There are various reasons for this.
Maybe, you forgot that a zettel exists.
Or you add a zettel later, but forgot that previous zettel already mention its title.






|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012053900
title: API: Retrieve unlinked references to an existing zettel
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220812145045

The value of a personal Zettelstore is determined in part by explicit connections between related zettel.
If the number of zettel grow, some of these connections are missing.
There are various reasons for this.
Maybe, you forgot that a zettel exists.
Or you add a zettel later, but forgot that previous zettel already mention its title.

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
The title must not occur within a link (e.g. to another zettel), in a [[heading|00001007030300]], in a [[citation|00001007040340]], and must have a uniform formatting.
The match must be exact, but is case-insensitive.

If the title of the specified zettel contains some extra character that probably reduce the number of found unlinked references,
you can specify the title phase to be searched for as a query parameter ''_phrase'':

````
# curl 'http://127.0.0.1:23123/u/00001007000000?phrase=markdown'
{"id": "00001007000000","meta": {...},"list": [{"id": "00001008010000","meta": {...},"rights":62},{"id": "00001004020000","meta": {...},"rights":62}]}
````

In addition, you are allowed to specify all query parameter to [[select zettel based on their metadata|00001012051810]], to [[limit the length of the returned list|00001012051830]], and to [[sort the returned list|00001012052000]]. You are allowed to limit the search only for those zettel with some [[specific content|00001012051840]].

=== Keys
The following top-level JSON keys are returned:
; ''id''
: The [[zettel identifier|00001006050000]] for which the unlinked references were requested.
; ''meta'':
: The metadata of the zettel, encoded as a JSON object.







|



|







46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
The title must not occur within a link (e.g. to another zettel), in a [[heading|00001007030300]], in a [[citation|00001007040340]], and must have a uniform formatting.
The match must be exact, but is case-insensitive.

If the title of the specified zettel contains some extra character that probably reduce the number of found unlinked references,
you can specify the title phase to be searched for as a query parameter ''_phrase'':

````
# curl 'http://127.0.0.1:23123/u/00001007000000?_phrase=markdown'
{"id": "00001007000000","meta": {...},"list": [{"id": "00001008010000","meta": {...},"rights":62},{"id": "00001004020000","meta": {...},"rights":62}]}
````

%%TODO: In addition, you are allowed to limit the search by a [[search expression|00001012051840]], which may search for zettel content.

=== Keys
The following top-level JSON keys are returned:
; ''id''
: The [[zettel identifier|00001006050000]] for which the unlinked references were requested.
; ''meta'':
: The metadata of the zettel, encoded as a JSON object.

Changes to docs/manual/00001012070500.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012070500
title: Retrieve administrative data
role: zettel
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220304174027

The [[endpoint|00001012920000]] ''/x'' allows you to retrieve some (administrative) data.

Currently, you can only request Zettelstore version data.

````
# curl 'http://127.0.0.1:23123/x'


|


|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012070500
title: Retrieve administrative data
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220805174216

The [[endpoint|00001012920000]] ''/x'' allows you to retrieve some (administrative) data.

Currently, you can only request Zettelstore version data.

````
# curl 'http://127.0.0.1:23123/x'

Changes to docs/manual/00001012080100.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
id: 00001012080100
title: API: Execute commands
role: zettel
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220103225956

The [[endpoint|00001012920000]] ''/x'' allows you to execute some (administrative) commands.
To differentiate between the possible commands, you have to set the query parameter ''_cmd'' to a specific value:

; ''authenticated''
: [[Check for authentication|00001012080200]]
; ''refresh''
: [[Refresh internal data|00001012080500]]

Other commands will be defined in the future.


|


|










1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
id: 00001012080100
title: API: Execute commands
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220805174227

The [[endpoint|00001012920000]] ''/x'' allows you to execute some (administrative) commands.
To differentiate between the possible commands, you have to set the query parameter ''_cmd'' to a specific value:

; ''authenticated''
: [[Check for authentication|00001012080200]]
; ''refresh''
: [[Refresh internal data|00001012080500]]

Other commands will be defined in the future.

Changes to docs/manual/00001012080200.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012080200
title: API: Check for authentication
role: zettel
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220103235531

API clients typically wants to know, whether [[authentication is enabled|00001010040100]] or not.
If authentication is enabled, they present some form of user interface to get user name and password for the actual authentication.
Then they try to [[obtain an access token|00001012050200]].
If authentication is disabled, these steps are not needed.

To check for enabled authentication, you must send a HTTP POST request to the [[endpoint|00001012920000]] ''/x'' and you must specify the query parameter ''_cmd=authenticated''.


|


|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012080200
title: API: Check for authentication
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220805174236

API clients typically wants to know, whether [[authentication is enabled|00001010040100]] or not.
If authentication is enabled, they present some form of user interface to get user name and password for the actual authentication.
Then they try to [[obtain an access token|00001012050200]].
If authentication is disabled, these steps are not needed.

To check for enabled authentication, you must send a HTTP POST request to the [[endpoint|00001012920000]] ''/x'' and you must specify the query parameter ''_cmd=authenticated''.

Changes to docs/manual/00001012080500.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012080500
title: API: Refresh internal data
role: zettel
tags: #api #manual #zettelstore
syntax: zmk
modified: 20211230234431

Zettelstore maintains some internal data to allow faster operations.

One example is the [[content search|00001012051840]] for a term: Zettelstore does not need to scan all zettel to find all occurrences for the term.
Instead, all word are stored internally, with a list of zettel where they occur.

Another example is the way to determine which zettel are stored in a [[ZIP file|00001004011200]].


|


|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001012080500
title: API: Refresh internal data
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20220805174246

Zettelstore maintains some internal data to allow faster operations.

One example is the [[content search|00001012051840]] for a term: Zettelstore does not need to scan all zettel to find all occurrences for the term.
Instead, all word are stored internally, with a list of zettel where they occur.

Another example is the way to determine which zettel are stored in a [[ZIP file|00001004011200]].

Added docs/manual/00001017000000.zettel.









































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
id: 00001017000000
title: Tips and Tricks
role: manual
tags: #manual #zettelstore
syntax: zmk
modified: 20220805174255

=== Welcome Zettel
* **Problem:** You want to put your Zettelstore into the public and need a starting zettel for your users.
  In addition, you still want a ""home zettel"", with all your references to internal, non-public zettel.
  Zettelstore only allows to specify one [[''home-zettel''|00001004020000#home-zettel]].
* **Solution:**
*# Create a new zettel with all your references to internal, non-public zettel.
   Let's assume this zettel receives the zettel identifier ''20220803182600''.
*# Create the zettel that should serve as the starting zettel for your users.
   It must have syntax [[Zettelmarkup|00001008000000#zmk]], i.e. the syntax metadata must be set to ''zmk''.
   If needed, set the runtime configuration [[''home-zettel|00001004020000#home-zettel]] to the value of the identifier of this zettel.
*# At the beginning of the start zettel, add the following [[Zettelmarkup|00001007000000]] text in a separate paragraph: ``{{{20220803182600}}}`` (you have to adapt to the actual value of the zettel identifier for your non-public home zettel).
* **Discussion:** As stated in the description for a [[transclusion|00001007031100]], a transclusion will be ignored, if the transcluded zettel is not visible to the current user.
  In effect, the transclusion statement (above paragraph that contained ''{{{...}}}'') is ignored when rendering the zettel.

Changes to docs/manual/00001018000000.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001018000000
title: Troubleshooting
role: zettel
tags: #manual #zettelstore
syntax: zmk
modified: 20220218125940

This page lists some problems and their solutions that may occur when using your Zettelstore.

=== Installation
* **Problem:** When you double-click on the Zettelstore executable icon, macOS complains that Zettelstore is an application from an unknown developer.
  Therefore, it will not start Zettelstore.
** **Solution:** Press the ''Ctrl'' key while opening the context menu of the Zettelstore executable with a right-click.


|


|







1
2
3
4
5
6
7
8
9
10
11
12
13
id: 00001018000000
title: Troubleshooting
role: manual
tags: #manual #zettelstore
syntax: zmk
modified: 20220805174305

This page lists some problems and their solutions that may occur when using your Zettelstore.

=== Installation
* **Problem:** When you double-click on the Zettelstore executable icon, macOS complains that Zettelstore is an application from an unknown developer.
  Therefore, it will not start Zettelstore.
** **Solution:** Press the ''Ctrl'' key while opening the context menu of the Zettelstore executable with a right-click.

Changes to domain/meta/meta_test.go.

1
2
3
4
5
6
7
8
9
10
11
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package meta provides the domain specific type 'meta'.



|







1
2
3
4
5
6
7
8
9
10
11
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package meta provides the domain specific type 'meta'.
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
		}
	}
}

func TestTitleHeader(t *testing.T) {
	t.Parallel()
	m := New(testID)
	if got, ok := m.Get(api.KeyTitle); ok || got != "" {
		t.Errorf("Title is not empty, but %q", got)
	}
	addToMeta(m, api.KeyTitle, " ")
	if got, ok := m.Get(api.KeyTitle); ok || got != "" {
		t.Errorf("Title is not empty, but %q", got)
	}
	const st = "A simple text"
	addToMeta(m, api.KeyTitle, " "+st+"  ")
	if got, ok := m.Get(api.KeyTitle); !ok || got != st {
		t.Errorf("Title is not %q, but %q", st, got)
	}







|



|







36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
		}
	}
}

func TestTitleHeader(t *testing.T) {
	t.Parallel()
	m := New(testID)
	if got, ok := m.Get(api.KeyTitle); ok && got != "" {
		t.Errorf("Title is not empty, but %q", got)
	}
	addToMeta(m, api.KeyTitle, " ")
	if got, ok := m.Get(api.KeyTitle); ok && got != "" {
		t.Errorf("Title is not empty, but %q", got)
	}
	const st = "A simple text"
	addToMeta(m, api.KeyTitle, " "+st+"  ")
	if got, ok := m.Get(api.KeyTitle); !ok || got != st {
		t.Errorf("Title is not %q, but %q", st, got)
	}

Changes to domain/meta/parse.go.

149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
	switch key {
	case "", api.KeyID:
		// Empty key and 'id' key will be ignored
		return
	}

	switch Type(key) {
	case TypeString, TypeZettelmarkup:
		if v != "" {
			addData(m, key, v)
		}
	case TypeTagSet:
		addSet(m, key, strings.ToLower(v), func(s string) bool { return s[0] == '#' })
	case TypeWord:
		m.Set(key, strings.ToLower(v))
	case TypeWordSet:
		addSet(m, key, strings.ToLower(v), func(s string) bool { return true })
	case TypeID:







<
<
<
<







149
150
151
152
153
154
155




156
157
158
159
160
161
162
	switch key {
	case "", api.KeyID:
		// Empty key and 'id' key will be ignored
		return
	}

	switch Type(key) {




	case TypeTagSet:
		addSet(m, key, strings.ToLower(v), func(s string) bool { return s[0] == '#' })
	case TypeWord:
		m.Set(key, strings.ToLower(v))
	case TypeWordSet:
		addSet(m, key, strings.ToLower(v), func(s string) bool { return true })
	case TypeID:

Changes to domain/meta/parse_test.go.

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

82
83
84
85
86
87
88
	for i, tc := range td {
		m := parseMetaStr(tc.s)
		if got, ok := m.Get(api.KeyTitle); !ok || got != tc.e {
			t.Log(m)
			t.Errorf("TC=%d: expected %q, got %q", i, tc.e, got)
		}
	}

	m := parseMetaStr(api.KeyTitle + ": ")
	if title, ok := m.Get(api.KeyTitle); ok {
		t.Errorf("Expected a missing title key, but got %q (meta=%v)", title, m)
	}
}

func TestNewFromInput(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		input string
		exp   []meta.Pair
	}{
		{"", []meta.Pair{}},
		{" a:b", []meta.Pair{{"a", "b"}}},
		{"%a:b", []meta.Pair{}},
		{"a:b\r\n\r\nc:d", []meta.Pair{{"a", "b"}}},
		{"a:b\r\n%c:d", []meta.Pair{{"a", "b"}}},
		{"% a:b\r\n c:d", []meta.Pair{{"c", "d"}}},
		{"---\r\na:b\r\n", []meta.Pair{{"a", "b"}}},
		{"---\r\na:b\r\n--\r\nc:d", []meta.Pair{{"a", "b"}, {"c", "d"}}},
		{"---\r\na:b\r\n---\r\nc:d", []meta.Pair{{"a", "b"}}},
		{"---\r\na:b\r\n----\r\nc:d", []meta.Pair{{"a", "b"}}},

	}
	for i, tc := range testcases {
		meta := parseMetaStr(tc.input)
		if got := meta.Pairs(); !equalPairs(tc.exp, got) {
			t.Errorf("TC=%d: expected=%v, got=%v", i, tc.exp, got)
		}
	}







<
<
<
<
<


















>







52
53
54
55
56
57
58





59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
	for i, tc := range td {
		m := parseMetaStr(tc.s)
		if got, ok := m.Get(api.KeyTitle); !ok || got != tc.e {
			t.Log(m)
			t.Errorf("TC=%d: expected %q, got %q", i, tc.e, got)
		}
	}





}

func TestNewFromInput(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		input string
		exp   []meta.Pair
	}{
		{"", []meta.Pair{}},
		{" a:b", []meta.Pair{{"a", "b"}}},
		{"%a:b", []meta.Pair{}},
		{"a:b\r\n\r\nc:d", []meta.Pair{{"a", "b"}}},
		{"a:b\r\n%c:d", []meta.Pair{{"a", "b"}}},
		{"% a:b\r\n c:d", []meta.Pair{{"c", "d"}}},
		{"---\r\na:b\r\n", []meta.Pair{{"a", "b"}}},
		{"---\r\na:b\r\n--\r\nc:d", []meta.Pair{{"a", "b"}, {"c", "d"}}},
		{"---\r\na:b\r\n---\r\nc:d", []meta.Pair{{"a", "b"}}},
		{"---\r\na:b\r\n----\r\nc:d", []meta.Pair{{"a", "b"}}},
		{"new-title:\nnew-url:", []meta.Pair{{"new-title", ""}, {"new-url", ""}}},
	}
	for i, tc := range testcases {
		meta := parseMetaStr(tc.input)
		if got := meta.Pairs(); !equalPairs(tc.exp, got) {
			t.Errorf("TC=%d: expected=%v, got=%v", i, tc.exp, got)
		}
	}

Changes to encoder/encoder_inline_test.go.

122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
		},
	},
	{
		descr: "Quotes formatting (german)",
		zmk:   `""quotes""{lang=de}`,
		expect: expectMap{
			encoderZJSON: `[{"":"Quote","a":{"lang":"de"},"i":[{"":"Text","s":"quotes"}]}]`,
			encoderHTML:  `<q lang="de">quotes</q>`,
			encoderSexpr: `((FORMAT-QUOTE (("lang" "de")) (TEXT "quotes")))`,
			encoderText:  `quotes`,
			encoderZmk:   `""quotes""{lang="de"}`,
		},
	},
	{
		descr: "Span formatting",







|







122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
		},
	},
	{
		descr: "Quotes formatting (german)",
		zmk:   `""quotes""{lang=de}`,
		expect: expectMap{
			encoderZJSON: `[{"":"Quote","a":{"lang":"de"},"i":[{"":"Text","s":"quotes"}]}]`,
			encoderHTML:  `<span lang="de"><q>quotes</q></span>`,
			encoderSexpr: `((FORMAT-QUOTE (("lang" "de")) (TEXT "quotes")))`,
			encoderText:  `quotes`,
			encoderZmk:   `""quotes""{lang="de"}`,
		},
	},
	{
		descr: "Span formatting",
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
		},
	},
	{
		descr: "HTML in Code formatting",
		zmk:   "``<script `` abc",
		expect: expectMap{
			encoderZJSON: `[{"":"Code","s":"<script "},{"":"Space"},{"":"Text","s":"abc"}]`,
			encoderHTML:  "<code>&lt;script\u00a0</code> abc",
			encoderSexpr: `((LITERAL-CODE () "<script ") (SPACE) (TEXT "abc"))`,
			encoderText:  `<script  abc`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Input formatting",







|







166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
		},
	},
	{
		descr: "HTML in Code formatting",
		zmk:   "``<script `` abc",
		expect: expectMap{
			encoderZJSON: `[{"":"Code","s":"<script "},{"":"Space"},{"":"Text","s":"abc"}]`,
			encoderHTML:  "<code>&lt;script </code> abc",
			encoderSexpr: `((LITERAL-CODE () "<script ") (SPACE) (TEXT "abc"))`,
			encoderText:  `<script  abc`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Input formatting",
436
437
438
439
440
441
442






















443
444
445
446
447
448
449
			encoderZJSON: `[{"":"Link","q":"local","s":"../relative","i":[{"":"Text","s":"R"}]}]`,
			encoderHTML:  `<a href="../relative">R</a>`,
			encoderSexpr: `((LINK-HOSTED () "../relative" (TEXT "R")))`,
			encoderText:  `R`,
			encoderZmk:   useZmk,
		},
	},






















	{
		descr: "Dummy Embed",
		zmk:   `{{abc}}`,
		expect: expectMap{
			encoderZJSON: `[{"":"Embed","s":"abc"}]`,
			encoderHTML:  `<img src="abc">`,
			encoderSexpr: `((EMBED () (EXTERNAL "abc") ""))`,







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
			encoderZJSON: `[{"":"Link","q":"local","s":"../relative","i":[{"":"Text","s":"R"}]}]`,
			encoderHTML:  `<a href="../relative">R</a>`,
			encoderSexpr: `((LINK-HOSTED () "../relative" (TEXT "R")))`,
			encoderText:  `R`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Search link w/o text",
		zmk:   `[[search:title:syntax]]`,
		expect: expectMap{
			encoderZJSON: `[{"":"Link","q":"search","s":"title:syntax"}]`,
			encoderHTML:  `<a href="?_s=title%3Asyntax">title:syntax</a>`,
			encoderSexpr: `((LINK-SEARCH () "title:syntax"))`,
			encoderText:  ``,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Search link with text",
		zmk:   `[[S|search:title:syntax]]`,
		expect: expectMap{
			encoderZJSON: `[{"":"Link","q":"search","s":"title:syntax","i":[{"":"Text","s":"S"}]}]`,
			encoderHTML:  `<a href="?_s=title%3Asyntax">S</a>`,
			encoderSexpr: `((LINK-SEARCH () "title:syntax" (TEXT "S")))`,
			encoderText:  `S`,
			encoderZmk:   useZmk,
		},
	},
	{
		descr: "Dummy Embed",
		zmk:   `{{abc}}`,
		expect: expectMap{
			encoderZJSON: `[{"":"Embed","s":"abc"}]`,
			encoderHTML:  `<img src="abc">`,
			encoderSexpr: `((EMBED () (EXTERNAL "abc") ""))`,

Changes to encoder/sexprenc/transform.go.

303
304
305
306
307
308
309

310
311
312
313
314
315
316
	ast.RefStateInvalid:  sexpr.SymLinkInvalid,
	ast.RefStateZettel:   sexpr.SymLinkZettel,
	ast.RefStateSelf:     sexpr.SymLinkSelf,
	ast.RefStateFound:    sexpr.SymLinkFound,
	ast.RefStateBroken:   sexpr.SymLinkBroken,
	ast.RefStateHosted:   sexpr.SymLinkHosted,
	ast.RefStateBased:    sexpr.SymLinkBased,

	ast.RefStateExternal: sexpr.SymLinkExternal,
}

func (t *transformer) getLink(ln *ast.LinkNode) *sxpf.Pair {
	return sxpf.NewPair(
		mapGetS(mapRefStateLink, ln.Ref.State),
		sxpf.NewPair(







>







303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
	ast.RefStateInvalid:  sexpr.SymLinkInvalid,
	ast.RefStateZettel:   sexpr.SymLinkZettel,
	ast.RefStateSelf:     sexpr.SymLinkSelf,
	ast.RefStateFound:    sexpr.SymLinkFound,
	ast.RefStateBroken:   sexpr.SymLinkBroken,
	ast.RefStateHosted:   sexpr.SymLinkHosted,
	ast.RefStateBased:    sexpr.SymLinkBased,
	ast.RefStateSearch:   sexpr.SymLinkSearch,
	ast.RefStateExternal: sexpr.SymLinkExternal,
}

func (t *transformer) getLink(ln *ast.LinkNode) *sxpf.Pair {
	return sxpf.NewPair(
		mapGetS(mapRefStateLink, ln.Ref.State),
		sxpf.NewPair(
394
395
396
397
398
399
400

401
402
403
404
405
406
407
	ast.RefStateInvalid:  sexpr.SymRefStateInvalid,
	ast.RefStateZettel:   sexpr.SymRefStateZettel,
	ast.RefStateSelf:     sexpr.SymRefStateSelf,
	ast.RefStateFound:    sexpr.SymRefStateFound,
	ast.RefStateBroken:   sexpr.SymRefStateBroken,
	ast.RefStateHosted:   sexpr.SymRefStateHosted,
	ast.RefStateBased:    sexpr.SymRefStateBased,

	ast.RefStateExternal: sexpr.SymRefStateExternal,
}

func getReference(ref *ast.Reference) *sxpf.Pair {
	return sxpf.NewPair(
		mapGetS(mapRefStateS, ref.State),
		sxpf.NewPair(







>







395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
	ast.RefStateInvalid:  sexpr.SymRefStateInvalid,
	ast.RefStateZettel:   sexpr.SymRefStateZettel,
	ast.RefStateSelf:     sexpr.SymRefStateSelf,
	ast.RefStateFound:    sexpr.SymRefStateFound,
	ast.RefStateBroken:   sexpr.SymRefStateBroken,
	ast.RefStateHosted:   sexpr.SymRefStateHosted,
	ast.RefStateBased:    sexpr.SymRefStateBased,
	ast.RefStateSearch:   sexpr.SymRefStateSearch,
	ast.RefStateExternal: sexpr.SymRefStateExternal,
}

func getReference(ref *ast.Reference) *sxpf.Pair {
	return sxpf.NewPair(
		mapGetS(mapRefStateS, ref.State),
		sxpf.NewPair(
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
}

func mapGetS[T comparable](m map[T]*sxpf.Symbol, k T) *sxpf.Symbol {
	if result, found := m[k]; found {
		return result
	}
	log.Println("MISS", k, m)
	return sexpr.Smk.MakeSymbol(fmt.Sprintf("**%v:not-found**", k))
}

func getBase64String(data []byte) *sxpf.String {
	var buf bytes.Buffer
	encoder := base64.NewEncoder(base64.StdEncoding, &buf)
	_, err := encoder.Write(data)
	if err == nil {







|







455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
}

func mapGetS[T comparable](m map[T]*sxpf.Symbol, k T) *sxpf.Symbol {
	if result, found := m[k]; found {
		return result
	}
	log.Println("MISS", k, m)
	return sexpr.Smk.MakeSymbol(fmt.Sprintf("**%v:NOT-FOUND**", k))
}

func getBase64String(data []byte) *sxpf.String {
	var buf bytes.Buffer
	encoder := base64.NewEncoder(base64.StdEncoding, &buf)
	_, err := encoder.Write(data)
	if err == nil {

Changes to encoder/zjsonenc/zjsonenc.go.

136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
	case *ast.BreakNode:
		if n.Hard {
			v.writeNodeStart(zjson.TypeBreakHard)
		} else {
			v.writeNodeStart(zjson.TypeBreakSoft)
		}
	case *ast.LinkNode:
		v.writeNodeStart(zjson.TypeLink)
		v.visitAttributes(n.Attrs)
		v.writeContentStart(zjson.NameString2)
		writeEscaped(&v.b, mapRefState[n.Ref.State])
		v.writeContentStart(zjson.NameString)
		writeEscaped(&v.b, n.Ref.String())
		if len(n.Inlines) > 0 {
			v.writeContentStart(zjson.NameInline)
			ast.Walk(v, &n.Inlines)
		}
	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.EmbedBLOBNode:
		v.visitEmbedBLOB(n)
	case *ast.CiteNode:
		v.writeNodeStart(zjson.TypeCitation)
		v.visitAttributes(n.Attrs)







<
|
<
<
<
<
<
<
<
<







136
137
138
139
140
141
142

143








144
145
146
147
148
149
150
	case *ast.BreakNode:
		if n.Hard {
			v.writeNodeStart(zjson.TypeBreakHard)
		} else {
			v.writeNodeStart(zjson.TypeBreakSoft)
		}
	case *ast.LinkNode:

		v.visitLink(n)








	case *ast.EmbedRefNode:
		v.visitEmbedRef(n)
	case *ast.EmbedBLOBNode:
		v.visitEmbedBLOB(n)
	case *ast.CiteNode:
		v.writeNodeStart(zjson.TypeCitation)
		v.visitAttributes(n.Attrs)
362
363
364
365
366
367
368

369
370

















371
372
373
374
375
376
377
	ast.RefStateInvalid:  zjson.RefStateInvalid,
	ast.RefStateZettel:   zjson.RefStateZettel,
	ast.RefStateSelf:     zjson.RefStateSelf,
	ast.RefStateFound:    zjson.RefStateFound,
	ast.RefStateBroken:   zjson.RefStateBroken,
	ast.RefStateHosted:   zjson.RefStateHosted,
	ast.RefStateBased:    zjson.RefStateBased,

	ast.RefStateExternal: zjson.RefStateExternal,
}


















func (v *visitor) visitEmbedRef(en *ast.EmbedRefNode) {
	v.writeNodeStart(zjson.TypeEmbed)
	v.visitAttributes(en.Attrs)
	v.writeContentStart(zjson.NameString)
	writeEscaped(&v.b, en.Ref.String())








>


>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
	ast.RefStateInvalid:  zjson.RefStateInvalid,
	ast.RefStateZettel:   zjson.RefStateZettel,
	ast.RefStateSelf:     zjson.RefStateSelf,
	ast.RefStateFound:    zjson.RefStateFound,
	ast.RefStateBroken:   zjson.RefStateBroken,
	ast.RefStateHosted:   zjson.RefStateHosted,
	ast.RefStateBased:    zjson.RefStateBased,
	ast.RefStateSearch:   zjson.RefStateSearch,
	ast.RefStateExternal: zjson.RefStateExternal,
}

func (v *visitor) visitLink(ln *ast.LinkNode) {
	v.writeNodeStart(zjson.TypeLink)
	v.visitAttributes(ln.Attrs)
	v.writeContentStart(zjson.NameString2)
	writeEscaped(&v.b, mapRefState[ln.Ref.State])
	v.writeContentStart(zjson.NameString)
	if ln.Ref.State == ast.RefStateSearch {
		writeEscaped(&v.b, ln.Ref.Value)
	} else {
		writeEscaped(&v.b, ln.Ref.String())
	}
	if len(ln.Inlines) > 0 {
		v.writeContentStart(zjson.NameInline)
		ast.Walk(v, &ln.Inlines)
	}
}

func (v *visitor) visitEmbedRef(en *ast.EmbedRefNode) {
	v.writeNodeStart(zjson.TypeEmbed)
	v.visitAttributes(en.Attrs)
	v.writeContentStart(zjson.NameString)
	writeEscaped(&v.b, en.Ref.String())

Changes to evaluator/evaluator.go.

26
27
28
29
30
31
32

33
34
35
36
37
38

39
40
41
42
43
44
45
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/parser/cleaner"
	"zettelstore.de/z/parser/draw"

)

// Port contains all methods to retrieve zettel (or part of it) to evaluate a zettel.
type Port interface {
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)
	GetZettel(context.Context, id.Zid) (domain.Zettel, error)

}

// EvaluateZettel evaluates the given zettel in the given context, with the
// given ports, and the given environment.
func EvaluateZettel(ctx context.Context, port Port, rtConfig config.Config, zn *ast.ZettelNode) {
	if zn.Syntax == api.ValueSyntaxNone {
		// AST is empty, evaluate to a description list of metadata.







>






>







26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/parser/cleaner"
	"zettelstore.de/z/parser/draw"
	"zettelstore.de/z/search"
)

// Port contains all methods to retrieve zettel (or part of it) to evaluate a zettel.
type Port interface {
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)
	GetZettel(context.Context, id.Zid) (domain.Zettel, error)
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)
}

// EvaluateZettel evaluates the given zettel in the given context, with the
// given ports, and the given environment.
func EvaluateZettel(ctx context.Context, port Port, rtConfig config.Config, zn *ast.ZettelNode) {
	if zn.Syntax == api.ValueSyntaxNone {
		// AST is empty, evaluate to a description list of metadata.
112
113
114
115
116
117
118




119
120
121
122
123
124
125
	}
}

func transcludeNode(bln *ast.BlockSlice, i int, bn ast.BlockNode) int {
	if ln, ok := bn.(*ast.BlockSlice); ok {
		*bln = replaceWithBlockNodes(*bln, i, *ln)
		return len(*ln) - 1




	}
	(*bln)[i] = bn
	return 0
}

func replaceWithBlockNodes(bns []ast.BlockNode, i int, replaceBns []ast.BlockNode) []ast.BlockNode {
	if len(replaceBns) == 1 {







>
>
>
>







114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
	}
}

func transcludeNode(bln *ast.BlockSlice, i int, bn ast.BlockNode) int {
	if ln, ok := bn.(*ast.BlockSlice); ok {
		*bln = replaceWithBlockNodes(*bln, i, *ln)
		return len(*ln) - 1
	}
	if bn == nil {
		(*bln) = (*bln)[:i+copy((*bln)[i:], (*bln)[i+1:])]
		return -1
	}
	(*bln)[i] = bn
	return 0
}

func replaceWithBlockNodes(bns []ast.BlockNode, i int, replaceBns []ast.BlockNode) []ast.BlockNode {
	if len(replaceBns) == 1 {
183
184
185
186
187
188
189
190
191
192
193
194
195



196
197

198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213



214
215
216
217
218
219
220
221
222
223
224
225
226
227
228

































229
230
231
232
233
234
235
		return makeBlockNode(errText)
	}
	switch ref.State {
	case ast.RefStateZettel:
		// Only zettel references will be evaluated.
	case ast.RefStateInvalid, ast.RefStateBroken:
		e.transcludeCount++
		return makeBlockNode(createInlineErrorText(ref, "Invalid", "or", "broken", "transclusion", "reference:"))
	case ast.RefStateSelf:
		e.transcludeCount++
		return makeBlockNode(createInlineErrorText(ref, "Self", "transclusion", "reference:"))
	case ast.RefStateFound, ast.RefStateHosted, ast.RefStateBased, ast.RefStateExternal:
		return tn



	default:
		panic(fmt.Sprintf("Unknown state %v for reference %v", ref.State, ref))

	}

	zid, err := id.Parse(ref.URL.Path)
	if err != nil {
		panic(err)
	}

	cost, ok := e.costMap[zid]
	zn := cost.zn
	if zn == e.marker {
		e.transcludeCount++
		return makeBlockNode(createInlineErrorText(ref, "Recursive", "transclusion:"))
	}
	if !ok {
		zettel, err1 := e.port.GetZettel(box.NoEnrichContext(e.ctx), zid)
		if err1 != nil {



			e.transcludeCount++
			return makeBlockNode(createInlineErrorText(ref, "Unable", "to", "get", "zettel:"))
		}
		ec := e.transcludeCount
		e.costMap[zid] = transcludeCost{zn: e.marker, ec: ec}
		zn = e.evaluateEmbeddedZettel(zettel)
		e.costMap[zid] = transcludeCost{zn: zn, ec: e.transcludeCount - ec}
		e.transcludeCount = 0 // No stack needed, because embedding is done left-recursive, depth-first.
	}
	e.transcludeCount++
	if ec := cost.ec; ec > 0 {
		e.transcludeCount += cost.ec
	}
	return &zn.Ast
}


































func (e *evaluator) checkMaxTransclusions(ref *ast.Reference) ast.InlineNode {
	if maxTrans := e.transcludeMax; e.transcludeCount > maxTrans {
		e.transcludeCount = maxTrans + 1
		return createInlineErrorText(ref,
			"Too", "many", "transclusions", "(must", "be", "at", "most", strconv.Itoa(maxTrans)+",",
			"see", "runtime", "configuration", "key", "max-transclusions)")







|


|


>
>
>

<
>











|




>
>
>

|













>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205

206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
		return makeBlockNode(errText)
	}
	switch ref.State {
	case ast.RefStateZettel:
		// Only zettel references will be evaluated.
	case ast.RefStateInvalid, ast.RefStateBroken:
		e.transcludeCount++
		return makeBlockNode(createInlineErrorText(ref, "Invalid", "or", "broken", "transclusion", "reference"))
	case ast.RefStateSelf:
		e.transcludeCount++
		return makeBlockNode(createInlineErrorText(ref, "Self", "transclusion", "reference"))
	case ast.RefStateFound, ast.RefStateHosted, ast.RefStateBased, ast.RefStateExternal:
		return tn
	case ast.RefStateSearch:
		e.transcludeCount++
		return e.evalSearchTransclusion(tn.Ref.Value)
	default:

		return makeBlockNode(createInlineErrorText(ref, "Illegal", "block", "state", strconv.Itoa(int(ref.State))))
	}

	zid, err := id.Parse(ref.URL.Path)
	if err != nil {
		panic(err)
	}

	cost, ok := e.costMap[zid]
	zn := cost.zn
	if zn == e.marker {
		e.transcludeCount++
		return makeBlockNode(createInlineErrorText(ref, "Recursive", "transclusion"))
	}
	if !ok {
		zettel, err1 := e.port.GetZettel(box.NoEnrichContext(e.ctx), zid)
		if err1 != nil {
			if errors.Is(err1, &box.ErrNotAllowed{}) {
				return nil
			}
			e.transcludeCount++
			return makeBlockNode(createInlineErrorText(ref, "Unable", "to", "get", "zettel"))
		}
		ec := e.transcludeCount
		e.costMap[zid] = transcludeCost{zn: e.marker, ec: ec}
		zn = e.evaluateEmbeddedZettel(zettel)
		e.costMap[zid] = transcludeCost{zn: zn, ec: e.transcludeCount - ec}
		e.transcludeCount = 0 // No stack needed, because embedding is done left-recursive, depth-first.
	}
	e.transcludeCount++
	if ec := cost.ec; ec > 0 {
		e.transcludeCount += cost.ec
	}
	return &zn.Ast
}

func (e *evaluator) evalSearchTransclusion(expr string) ast.BlockNode {
	ml, err := e.port.SelectMeta(e.ctx, search.Parse(expr))
	if err != nil {
		if errors.Is(err, &box.ErrNotAllowed{}) {
			return nil
		}
		return makeBlockNode(createInlineErrorText(nil, "Unable", "to", "search", "zettel"))
	}
	if len(ml) == 0 {
		return nil
	}
	items := make([]ast.ItemSlice, 0, len(ml))
	for _, m := range ml {
		zid := m.Zid.String()
		title, found := m.Get(api.KeyTitle)
		if !found {
			title = zid
		}
		items = append(items, ast.ItemSlice{ast.CreateParaNode(&ast.LinkNode{
			Attrs:   nil,
			Ref:     ast.ParseReference(zid),
			Inlines: parser.ParseMetadataNoLink(title),
		})})
	}
	result := &ast.NestedListNode{
		Kind:  ast.NestedListUnordered,
		Items: items,
		Attrs: nil,
	}
	ast.Walk(e, result)
	return result
}

func (e *evaluator) checkMaxTransclusions(ref *ast.Reference) ast.InlineNode {
	if maxTrans := e.transcludeMax; e.transcludeCount > maxTrans {
		e.transcludeCount = maxTrans + 1
		return createInlineErrorText(ref,
			"Too", "many", "transclusions", "(must", "be", "at", "most", strconv.Itoa(maxTrans)+",",
			"see", "runtime", "configuration", "key", "max-transclusions)")
254
255
256
257
258
259
260




261
262
263
264
265
266
267
	}
}

func embedNode(is *ast.InlineSlice, i int, in ast.InlineNode) int {
	if ln, ok := in.(*ast.InlineSlice); ok {
		*is = replaceWithInlineNodes(*is, i, *ln)
		return len(*ln) - 1




	}
	(*is)[i] = in
	return 0
}

func replaceWithInlineNodes(ins ast.InlineSlice, i int, replaceIns ast.InlineSlice) ast.InlineSlice {
	if len(replaceIns) == 1 {







>
>
>
>







299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
	}
}

func embedNode(is *ast.InlineSlice, i int, in ast.InlineNode) int {
	if ln, ok := in.(*ast.InlineSlice); ok {
		*is = replaceWithInlineNodes(*is, i, *ln)
		return len(*ln) - 1
	}
	if in == nil {
		(*is) = (*is)[:i+copy((*is)[i:], (*is)[i+1:])]
		return -1
	}
	(*is)[i] = in
	return 0
}

func replaceWithInlineNodes(ins ast.InlineSlice, i int, replaceIns ast.InlineSlice) ast.InlineSlice {
	if len(replaceIns) == 1 {
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342



343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
	case ast.RefStateZettel:
		// Only zettel references will be evaluated.
	case ast.RefStateInvalid, ast.RefStateBroken:
		e.transcludeCount++
		return createInlineErrorImage(en)
	case ast.RefStateSelf:
		e.transcludeCount++
		return createInlineErrorText(ref, "Self", "embed", "reference:")
	case ast.RefStateFound, ast.RefStateHosted, ast.RefStateBased, ast.RefStateExternal:
		return en
	default:
		panic(fmt.Sprintf("Unknown state %v for reference %v", ref.State, ref))
	}

	zid := mustParseZid(ref)
	zettel, err := e.port.GetZettel(box.NoEnrichContext(e.ctx), zid)
	if err != nil {



		e.transcludeCount++
		return createInlineErrorImage(en)
	}

	if syntax := zettel.Meta.GetDefault(api.KeySyntax, ""); parser.IsImageFormat(syntax) {
		en.Syntax = syntax
		return en
	} else if !parser.IsTextParser(syntax) {
		// Not embeddable.
		e.transcludeCount++
		return createInlineErrorText(ref, "Not", "embeddable (syntax="+syntax+"):")
	}

	cost, ok := e.costMap[zid]
	zn := cost.zn
	if zn == e.marker {
		e.transcludeCount++
		return createInlineErrorText(ref, "Recursive", "transclusion:")
	}
	if !ok {
		ec := e.transcludeCount
		e.costMap[zid] = transcludeCost{zn: e.marker, ec: ec}
		zn = e.evaluateEmbeddedZettel(zettel)
		e.costMap[zid] = transcludeCost{zn: zn, ec: e.transcludeCount - ec}
		e.transcludeCount = 0 // No stack needed, because embedding is done left-recursive, depth-first.







|



|





>
>
>










|






|







375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
	case ast.RefStateZettel:
		// Only zettel references will be evaluated.
	case ast.RefStateInvalid, ast.RefStateBroken:
		e.transcludeCount++
		return createInlineErrorImage(en)
	case ast.RefStateSelf:
		e.transcludeCount++
		return createInlineErrorText(ref, "Self", "embed", "reference")
	case ast.RefStateFound, ast.RefStateHosted, ast.RefStateBased, ast.RefStateExternal:
		return en
	default:
		return createInlineErrorText(ref, "Illegal", "inline", "state", strconv.Itoa(int(ref.State)))
	}

	zid := mustParseZid(ref)
	zettel, err := e.port.GetZettel(box.NoEnrichContext(e.ctx), zid)
	if err != nil {
		if errors.Is(err, &box.ErrNotAllowed{}) {
			return nil
		}
		e.transcludeCount++
		return createInlineErrorImage(en)
	}

	if syntax := zettel.Meta.GetDefault(api.KeySyntax, ""); parser.IsImageFormat(syntax) {
		en.Syntax = syntax
		return en
	} else if !parser.IsTextParser(syntax) {
		// Not embeddable.
		e.transcludeCount++
		return createInlineErrorText(ref, "Not", "embeddable (syntax="+syntax+")")
	}

	cost, ok := e.costMap[zid]
	zn := cost.zn
	if zn == e.marker {
		e.transcludeCount++
		return createInlineErrorText(ref, "Recursive", "transclusion")
	}
	if !ok {
		ec := e.transcludeCount
		e.costMap[zid] = transcludeCost{zn: e.marker, ec: ec}
		zn = e.evaluateEmbeddedZettel(zettel)
		e.costMap[zid] = transcludeCost{zn: zn, ec: e.transcludeCount - ec}
		e.transcludeCount = 0 // No stack needed, because embedding is done left-recursive, depth-first.

Changes to go.mod.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
module zettelstore.de/z

go 1.18

require (
	codeberg.org/t73fde/sxpf v0.0.0-20220719090054-749a39d0a7a0
	github.com/fsnotify/fsnotify v1.5.4
	github.com/pascaldekloe/jwt v1.12.0
	github.com/yuin/goldmark v1.4.13
	golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa
	golang.org/x/term v0.0.0-20220722155259-a9ba230a4035
	golang.org/x/text v0.3.7
	zettelstore.de/c v0.0.0-20220729135959-532261810eac
)

require golang.org/x/sys v0.0.0-20220608164250-635b8c9b7f68 // indirect


|









|


|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
module zettelstore.de/z

go 1.19

require (
	codeberg.org/t73fde/sxpf v0.0.0-20220719090054-749a39d0a7a0
	github.com/fsnotify/fsnotify v1.5.4
	github.com/pascaldekloe/jwt v1.12.0
	github.com/yuin/goldmark v1.4.13
	golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa
	golang.org/x/term v0.0.0-20220722155259-a9ba230a4035
	golang.org/x/text v0.3.7
	zettelstore.de/c v0.6.1-0.20220812112501-f518deb0fc31
)

require golang.org/x/sys v0.0.0-20220804214406-8e32c043e418 // indirect

Changes to go.sum.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
codeberg.org/t73fde/sxpf v0.0.0-20220719090054-749a39d0a7a0 h1:viya/OgeF16+i8caBPJmcLQhGpZodPh+/nxtJzSSO1s=
codeberg.org/t73fde/sxpf v0.0.0-20220719090054-749a39d0a7a0/go.mod h1:4fAHEF3VH+ofbZkF6NzqiItTNy2X11tVCnZX99jXouA=
github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI=
github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU=
github.com/pascaldekloe/jwt v1.12.0 h1:imQSkPOtAIBAXoKKjL9ZVJuF/rVqJ+ntiLGpLyeqMUQ=
github.com/pascaldekloe/jwt v1.12.0/go.mod h1:LiIl7EwaglmH1hWThd/AmydNCnHf/mmfluBlNqHbk8U=
github.com/yuin/goldmark v1.4.13 h1:fVcFKWvrslecOb/tg+Cc05dkeYx540o0FuFt3nUVDoE=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa h1:zuSxTR4o9y82ebqCUJYNGJbGPo6sKVl54f/TVDObg1c=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220608164250-635b8c9b7f68 h1:z8Hj/bl9cOV2grsOpEaQFUaly0JWN3i97mo3jXKJNp0=
golang.org/x/sys v0.0.0-20220608164250-635b8c9b7f68/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20220722155259-a9ba230a4035 h1:Q5284mrmYTpACcm+eAKjKJH48BBwSyfJqmmGDTtT8Vc=
golang.org/x/term v0.0.0-20220722155259-a9ba230a4035/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
zettelstore.de/c v0.0.0-20220729135959-532261810eac h1:T8GP4P8pHNfjQ2lXtpRXBmNF685xI0kraoou3tKqH2Y=
zettelstore.de/c v0.0.0-20220729135959-532261810eac/go.mod h1:8wPlP6rYiXkfoCNE84K3bmhSAKHz15+xWSHMH26JAU0=











|
|




|
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
codeberg.org/t73fde/sxpf v0.0.0-20220719090054-749a39d0a7a0 h1:viya/OgeF16+i8caBPJmcLQhGpZodPh+/nxtJzSSO1s=
codeberg.org/t73fde/sxpf v0.0.0-20220719090054-749a39d0a7a0/go.mod h1:4fAHEF3VH+ofbZkF6NzqiItTNy2X11tVCnZX99jXouA=
github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI=
github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU=
github.com/pascaldekloe/jwt v1.12.0 h1:imQSkPOtAIBAXoKKjL9ZVJuF/rVqJ+ntiLGpLyeqMUQ=
github.com/pascaldekloe/jwt v1.12.0/go.mod h1:LiIl7EwaglmH1hWThd/AmydNCnHf/mmfluBlNqHbk8U=
github.com/yuin/goldmark v1.4.13 h1:fVcFKWvrslecOb/tg+Cc05dkeYx540o0FuFt3nUVDoE=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa h1:zuSxTR4o9y82ebqCUJYNGJbGPo6sKVl54f/TVDObg1c=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220804214406-8e32c043e418 h1:9vYwv7OjYaky/tlAeD7C4oC9EsPTlaFl1H2jS++V+ME=
golang.org/x/sys v0.0.0-20220804214406-8e32c043e418/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20220722155259-a9ba230a4035 h1:Q5284mrmYTpACcm+eAKjKJH48BBwSyfJqmmGDTtT8Vc=
golang.org/x/term v0.0.0-20220722155259-a9ba230a4035/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
zettelstore.de/c v0.6.1-0.20220812112501-f518deb0fc31 h1:AImZkrq6lKPdv2r6+5YRBkuGdSqCGE+pcv/YOqwIK4M=
zettelstore.de/c v0.6.1-0.20220812112501-f518deb0fc31/go.mod h1:+SoneUhKQ81A2Id/bC6FdDYYQAHYfVryh7wHFnnklew=

Changes to input/input.go.

34
35
36
37
38
39
40
41
42
43




44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
	inp.Next()
	return inp
}

// EOS = End of source
const EOS = rune(-1)

// Next reads the next rune into inp.Ch.
func (inp *Input) Next() {
	if inp.readPos < len(inp.Src) {




		inp.Pos = inp.readPos
		r, w := rune(inp.Src[inp.readPos]), 1
		if r >= utf8.RuneSelf {
			r, w = utf8.DecodeRune(inp.Src[inp.readPos:])
		}
		inp.readPos += w
		inp.Ch = r
	} else {
		inp.Pos = len(inp.Src)
		inp.Ch = EOS
	}
}

// Peek returns the rune following the most recently read rune without
// advancing. If end-of-source was already found peek returns EOS.
func (inp *Input) Peek() rune {
	return inp.PeekN(0)
}







|
|
|
>
>
>
>
|
|
|
|
|
|
|
|
<
<
<







34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55



56
57
58
59
60
61
62
	inp.Next()
	return inp
}

// EOS = End of source
const EOS = rune(-1)

// Next reads the next rune into inp.Ch and returns it too.
func (inp *Input) Next() rune {
	if inp.readPos >= len(inp.Src) {
		inp.Pos = len(inp.Src)
		inp.Ch = EOS
		return EOS
	}
	inp.Pos = inp.readPos
	r, w := rune(inp.Src[inp.readPos]), 1
	if r >= utf8.RuneSelf {
		r, w = utf8.DecodeRune(inp.Src[inp.readPos:])
	}
	inp.readPos += w
	inp.Ch = r
	return r



}

// Peek returns the rune following the most recently read rune without
// advancing. If end-of-source was already found peek returns EOS.
func (inp *Input) Peek() rune {
	return inp.PeekN(0)
}
72
73
74
75
76
77
78



















79
80
81
82
83
84
85
		if r == '\t' {
			return ' '
		}
		return r
	}
	return EOS
}




















// IsEOLEOS returns true if char is either EOS or EOL.
func IsEOLEOS(ch rune) bool {
	switch ch {
	case EOS, '\n', '\r':
		return true
	}







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
		if r == '\t' {
			return ' '
		}
		return r
	}
	return EOS
}

// Accept checks if the given string is a prefix of the text to be parsed.
// If successful, advance position and current character.
// String must only contain bytes < 128.
// If not successful, everything remains as it is.
func (inp *Input) Accept(s string) bool {
	pos := inp.Pos
	remaining := len(inp.Src) - pos
	if s == "" || len(s) > remaining {
		return false
	}
	// According to internal documentation of bytes.Equal, the string() will not allocate any memory.
	if readPos := pos + len(s); s == string(inp.Src[pos:readPos]) {
		inp.readPos = readPos
		inp.Next()
		return true
	}
	return false
}

// IsEOLEOS returns true if char is either EOS or EOL.
func IsEOLEOS(ch rune) bool {
	switch ch {
	case EOS, '\n', '\r':
		return true
	}
98
99
100
101
102
103
104

105
106

107
108
109
110
111
112
113
	case '\n':
		inp.Next()
	}
}

// SetPos allows to reset the read position.
func (inp *Input) SetPos(pos int) {

	inp.readPos = pos
	inp.Next()

}

// SkipToEOL reads until the next end-of-line.
func (inp *Input) SkipToEOL() {
	for {
		switch inp.Ch {
		case EOS, '\n', '\r':







>
|
|
>







118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
	case '\n':
		inp.Next()
	}
}

// SetPos allows to reset the read position.
func (inp *Input) SetPos(pos int) {
	if inp.Pos != pos {
		inp.readPos = pos
		inp.Next()
	}
}

// SkipToEOL reads until the next end-of-line.
func (inp *Input) SkipToEOL() {
	for {
		switch inp.Ch {
		case EOS, '\n', '\r':

Changes to input/input_test.go.

76
77
78
79
80
81
82


























		got, ok := inp.ScanEntity()
		if ok {
			t.Errorf("%d: scanning %q was unexpected successful, got %q", i, tc, got)
			continue
		}
	}
}

































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
		got, ok := inp.ScanEntity()
		if ok {
			t.Errorf("%d: scanning %q was unexpected successful, got %q", i, tc, got)
			continue
		}
	}
}

func TestAccept(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		accept string
		src    string
		acc    bool
		exp    rune
	}{
		{"", "", false, input.EOS},
		{"AB", "abc", false, 'a'},
		{"AB", "ABC", true, 'C'},
		{"AB", "AB", true, input.EOS},
		{"AB", "A", false, 'A'},
	}
	for i, tc := range testcases {
		inp := input.NewInput([]byte(tc.src))
		acc := inp.Accept(tc.accept)
		if acc != tc.acc {
			t.Errorf("%d: %q.Accept(%q) == %v, but got %v", i, tc.src, tc.accept, tc.acc, acc)
		}
		if got := inp.Ch; tc.exp != got {
			t.Errorf("%d: %q.Accept(%q) should result in run %v, but got %v", i, tc.src, tc.accept, tc.exp, got)
		}
	}
}

Changes to input/runes.go.

1
2
3
4
5
6
7
8
9
10
11


12
13
14
15
16
17


18
19
20
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package input



// IsSpace returns true if rune is a whitespace.
func IsSpace(ch rune) bool {
	switch ch {
	case ' ', '\t':
		return true


	}
	return false
}











>
>






>
>

|

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package input

import "unicode"

// IsSpace returns true if rune is a whitespace.
func IsSpace(ch rune) bool {
	switch ch {
	case ' ', '\t':
		return true
	case '\n', '\r', EOS:
		return false
	}
	return unicode.IsSpace(ch)
}

Changes to kernel/impl/box.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package impl

import (
	"context"
	"fmt"
	"io"
	"net/url"

	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
)


|

|













>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package impl

import (
	"context"
	"fmt"
	"io"
	"net/url"
	"strconv"
	"sync"

	"zettelstore.de/z/box"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/logger"
)

63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
	}
}

func (ps *boxService) GetLogger() *logger.Logger { return ps.logger }

func (ps *boxService) Start(kern *myKernel) error {
	boxURIs := make([]*url.URL, 0, 4)
	format := kernel.BoxURIs + "%d"
	for i := 1; ; i++ {
		u := ps.GetNextConfig(fmt.Sprintf(format, i))
		if u == nil {
			break
		}
		boxURIs = append(boxURIs, u.(*url.URL))
	}
	ps.mxService.Lock()
	defer ps.mxService.Unlock()







<

|







64
65
66
67
68
69
70

71
72
73
74
75
76
77
78
79
	}
}

func (ps *boxService) GetLogger() *logger.Logger { return ps.logger }

func (ps *boxService) Start(kern *myKernel) error {
	boxURIs := make([]*url.URL, 0, 4)

	for i := 1; ; i++ {
		u := ps.GetNextConfig(kernel.BoxURIs + strconv.Itoa(i))
		if u == nil {
			break
		}
		boxURIs = append(boxURIs, u.(*url.URL))
	}
	ps.mxService.Lock()
	defer ps.mxService.Unlock()
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134

func (ps *boxService) GetStatistics() []kernel.KeyValue {
	var st box.Stats
	ps.mxService.RLock()
	ps.manager.ReadStats(&st)
	ps.mxService.RUnlock()
	return []kernel.KeyValue{
		{Key: "Read-only", Value: fmt.Sprintf("%v", st.ReadOnly)},
		{Key: "Managed boxes", Value: fmt.Sprintf("%v", st.NumManagedBoxes)},
		{Key: "Zettel (total)", Value: fmt.Sprintf("%v", st.ZettelTotal)},
		{Key: "Zettel (indexed)", Value: fmt.Sprintf("%v", st.ZettelIndexed)},
		{Key: "Last re-index", Value: st.LastReload.Format("2006-01-02 15:04:05 -0700 MST")},
		{Key: "Duration last re-index", Value: fmt.Sprintf("%vms", st.DurLastReload.Milliseconds())},
		{Key: "Indexes since last re-index", Value: fmt.Sprintf("%v", st.IndexesSinceReload)},
		{Key: "Indexed words", Value: fmt.Sprintf("%v", st.IndexedWords)},
		{Key: "Indexed URLs", Value: fmt.Sprintf("%v", st.IndexedUrls)},
		{Key: "Zettel enrichments", Value: fmt.Sprintf("%v", st.IndexUpdates)},
	}
}

func (ps *boxService) DumpIndex(w io.Writer) {
	ps.manager.Dump(w)
}








|
|
|
|


|
|
|
|







111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134

func (ps *boxService) GetStatistics() []kernel.KeyValue {
	var st box.Stats
	ps.mxService.RLock()
	ps.manager.ReadStats(&st)
	ps.mxService.RUnlock()
	return []kernel.KeyValue{
		{Key: "Read-only", Value: strconv.FormatBool(st.ReadOnly)},
		{Key: "Managed boxes", Value: strconv.Itoa(st.NumManagedBoxes)},
		{Key: "Zettel (total)", Value: strconv.Itoa(st.ZettelTotal)},
		{Key: "Zettel (indexed)", Value: strconv.Itoa(st.ZettelIndexed)},
		{Key: "Last re-index", Value: st.LastReload.Format("2006-01-02 15:04:05 -0700 MST")},
		{Key: "Duration last re-index", Value: fmt.Sprintf("%vms", st.DurLastReload.Milliseconds())},
		{Key: "Indexes since last re-index", Value: strconv.FormatUint(st.IndexesSinceReload, 10)},
		{Key: "Indexed words", Value: strconv.FormatUint(st.IndexedWords, 10)},
		{Key: "Indexed URLs", Value: strconv.FormatUint(st.IndexedUrls, 10)},
		{Key: "Zettel enrichments", Value: strconv.FormatUint(st.IndexUpdates, 10)},
	}
}

func (ps *boxService) DumpIndex(w io.Writer) {
	ps.manager.Dump(w)
}

Changes to kernel/impl/cfg.go.

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
// under this license.
//-----------------------------------------------------------------------------

package impl

import (
	"context"
	"fmt"
	"strings"
	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"







<







8
9
10
11
12
13
14

15
16
17
18
19
20
21
// under this license.
//-----------------------------------------------------------------------------

package impl

import (
	"context"

	"strings"
	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
}
func (cs *configService) GetLogger() *logger.Logger { return cs.logger }

func (cs *configService) Start(*myKernel) error {
	cs.logger.Info().Msg("Start Service")
	data := meta.New(id.ConfigurationZid)
	for _, kv := range cs.GetNextConfigList() {
		data.Set(kv.Key, fmt.Sprintf("%v", kv.Value))
	}
	cs.mxService.Lock()
	cs.rtConfig = newConfig(cs.logger, data)
	cs.mxService.Unlock()
	return nil
}








|







95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
}
func (cs *configService) GetLogger() *logger.Logger { return cs.logger }

func (cs *configService) Start(*myKernel) error {
	cs.logger.Info().Msg("Start Service")
	data := meta.New(id.ConfigurationZid)
	for _, kv := range cs.GetNextConfigList() {
		data.Set(kv.Key, kv.Value)
	}
	cs.mxService.Lock()
	cs.rtConfig = newConfig(cs.logger, data)
	cs.mxService.Unlock()
	return nil
}

Changes to kernel/impl/cmd.go.

484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
			descr = descr[:pos]
		}
		value := samples[i].Value
		i++
		var sVal string
		switch value.Kind() {
		case metrics.KindUint64:
			sVal = fmt.Sprintf("%v", value.Uint64())
		case metrics.KindFloat64:
			sVal = fmt.Sprintf("%v", value.Float64())
		case metrics.KindFloat64Histogram:
			sVal = "(Histogramm)"
		case metrics.KindBad:
			sVal = "BAD"
		default:







|







484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
			descr = descr[:pos]
		}
		value := samples[i].Value
		i++
		var sVal string
		switch value.Kind() {
		case metrics.KindUint64:
			sVal = strconv.FormatUint(value.Uint64(), 10)
		case metrics.KindFloat64:
			sVal = fmt.Sprintf("%v", value.Float64())
		case metrics.KindFloat64Histogram:
			sVal = "(Histogramm)"
		case metrics.KindBad:
			sVal = "BAD"
		default:

Changes to kernel/impl/config.go.

81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
	defer cfg.mxConfig.Unlock()
	descr, ok := cfg.descr[key]
	if !ok {
		d, baseKey, num := cfg.getListDescription(key)
		if num < 0 {
			return false
		}
		format := baseKey + "%d"
		for i := num + 1; ; i++ {
			k := fmt.Sprintf(format, i)
			if _, ok = cfg.next[k]; !ok {
				break
			}
			delete(cfg.next, k)
		}
		if num == 0 {
			return true







<

|







81
82
83
84
85
86
87

88
89
90
91
92
93
94
95
96
	defer cfg.mxConfig.Unlock()
	descr, ok := cfg.descr[key]
	if !ok {
		d, baseKey, num := cfg.getListDescription(key)
		if num < 0 {
			return false
		}

		for i := num + 1; ; i++ {
			k := baseKey + strconv.Itoa(i)
			if _, ok = cfg.next[k]; !ok {
				break
			}
			delete(cfg.next, k)
		}
		if num == 0 {
			return true
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
	keys := make([]string, 0, len(cfg.descr))
	for k, descr := range cfg.descr {
		if all || descr.canList {
			if !strings.HasSuffix(k, "-") {
				keys = append(keys, k)
				continue
			}
			format := k + "%d"
			for i := 1; ; i++ {
				key := fmt.Sprintf(format, i)
				val := getConfig(key)
				if val == nil {
					break
				}
				keys = append(keys, key)
			}
		}







<

|







179
180
181
182
183
184
185

186
187
188
189
190
191
192
193
194
	keys := make([]string, 0, len(cfg.descr))
	for k, descr := range cfg.descr {
		if all || descr.canList {
			if !strings.HasSuffix(k, "-") {
				keys = append(keys, k)
				continue
			}

			for i := 1; ; i++ {
				key := k + strconv.Itoa(i)
				val := getConfig(key)
				if val == nil {
					break
				}
				keys = append(keys, key)
			}
		}

Changes to kernel/impl/web.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package impl

import (
	"fmt"
	"net"
	"strconv"
	"strings"
	"sync"
	"time"

	"zettelstore.de/z/kernel"













<







1
2
3
4
5
6
7
8
9
10
11
12
13

14
15
16
17
18
19
20
//-----------------------------------------------------------------------------
// Copyright (c) 2021-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package impl

import (

	"net"
	"strconv"
	"strings"
	"sync"
	"time"

	"zettelstore.de/z/kernel"
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
	ws.mxService.Unlock()

	if kern.cfg.GetConfig(kernel.ConfigSimpleMode).(bool) {
		listenAddr := ws.GetNextConfig(kernel.WebListenAddress).(string)
		if idx := strings.LastIndexByte(listenAddr, ':'); idx >= 0 {
			ws.logger.Mandatory().Msg(strings.Repeat("--------------------", 3))
			ws.logger.Mandatory().Msg("Open your browser and enter the following URL:")
			ws.logger.Mandatory().Msg(fmt.Sprintf("    http://localhost%v", listenAddr[idx:]))
		}
	}

	return nil
}

func (ws *webService) IsStarted() bool {







|







128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
	ws.mxService.Unlock()

	if kern.cfg.GetConfig(kernel.ConfigSimpleMode).(bool) {
		listenAddr := ws.GetNextConfig(kernel.WebListenAddress).(string)
		if idx := strings.LastIndexByte(listenAddr, ':'); idx >= 0 {
			ws.logger.Mandatory().Msg(strings.Repeat("--------------------", 3))
			ws.logger.Mandatory().Msg("Open your browser and enter the following URL:")
			ws.logger.Mandatory().Msg("    http://localhost%v" + listenAddr[idx:])
		}
	}

	return nil
}

func (ws *webService) IsStarted() bool {

Changes to logger/message.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package logger

import (
	"context"

	"strconv"
	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/z/domain/id"
)


|

|










>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package logger

import (
	"context"
	"net/http"
	"strconv"
	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/z/domain/id"
)

117
118
119
120
121
122
123











124
125
126
127
128
129
130
					m.buf = append(m.buf, user.Zid.Bytes()...)
				}
			}
		}
	}
	return m
}












// Zid adds a zettel identifier to the full message
func (m *Message) Zid(zid id.Zid) *Message {
	return m.Bytes("zid", zid.Bytes())
}

// Msg add the given text to the message and writes it to the log.







>
>
>
>
>
>
>
>
>
>
>







118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
					m.buf = append(m.buf, user.Zid.Bytes()...)
				}
			}
		}
	}
	return m
}

// HTTPIP adds the IP address of a HTTP request to the message.
func (m *Message) HTTPIP(r *http.Request) *Message {
	if r == nil {
		return m
	}
	if from := r.Header.Get("X-Forwarded-For"); from != "" {
		return m.Str("ip", from)
	}
	return m.Str("IP", r.RemoteAddr)
}

// Zid adds a zettel identifier to the full message
func (m *Message) Zid(zid id.Zid) *Message {
	return m.Bytes("zid", zid.Bytes())
}

// Msg add the given text to the message and writes it to the log.

Changes to parser/cleaner/cleaner.go.

76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
}

func (cv *cleanVisitor) visitMark(mn *ast.MarkNode) {
	if !cv.doMark {
		cv.hasMark = true
		return
	}
	// if mn.Mark == "" && len(mn.Inlines) > 0 {
	// 	var buf bytes.Buffer
	// 	_, err := cv.textEnc.WriteInlines(&buf, &mn.Inlines)
	// 	if err == nil {
	// 		mn.Mark = buf.String()
	// 	}
	// }
	if mn.Mark == "" {
		mn.Slug = ""
		mn.Fragment = cv.addIdentifier("*", mn)
		return
	}
	if mn.Slug == "" {
		mn.Slug = strfun.Slugify(mn.Mark)







<
<
<
<
<
<
<







76
77
78
79
80
81
82







83
84
85
86
87
88
89
}

func (cv *cleanVisitor) visitMark(mn *ast.MarkNode) {
	if !cv.doMark {
		cv.hasMark = true
		return
	}







	if mn.Mark == "" {
		mn.Slug = ""
		mn.Fragment = cv.addIdentifier("*", mn)
		return
	}
	if mn.Slug == "" {
		mn.Slug = strfun.Slugify(mn.Mark)
112
113
114
115
116
117
118









































				return newID
			}
		}
	}
	cv.ids[id] = node
	return id
}
















































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
				return newID
			}
		}
	}
	cv.ids[id] = node
	return id
}

// CleanInlineLinks removes all links and footnote node from the given inline slice.
func CleanInlineLinks(is *ast.InlineSlice) { ast.Walk(&cleanLinks{}, is) }

type cleanLinks struct{}

func (cl *cleanLinks) Visit(node ast.Node) ast.Visitor {
	ins, ok := node.(*ast.InlineSlice)
	if !ok {
		return cl
	}
	for _, in := range *ins {
		ast.Walk(cl, in)
	}
	if hasNoLinks(*ins) {
		return nil
	}

	result := make(ast.InlineSlice, 0, len(*ins))
	for _, in := range *ins {
		switch n := in.(type) {
		case *ast.LinkNode:
			result = append(result, n.Inlines...)
		case *ast.FootnoteNode: // Do nothing
		default:
			result = append(result, n)
		}
	}
	*ins = result
	return nil
}

func hasNoLinks(ins ast.InlineSlice) bool {
	for _, in := range ins {
		switch in.(type) {
		case *ast.LinkNode, *ast.FootnoteNode:
			return false
		}
	}
	return true
}

Changes to parser/markdown/markdown.go.

10
11
12
13
14
15
16

17
18
19
20
21
22
23

// Package markdown provides a parser for markdown.
package markdown

import (
	"bytes"
	"fmt"


	gm "github.com/yuin/goldmark"
	gmAst "github.com/yuin/goldmark/ast"
	gmText "github.com/yuin/goldmark/text"

	"zettelstore.de/c/attrs"
	"zettelstore.de/z/ast"







>







10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

// Package markdown provides a parser for markdown.
package markdown

import (
	"bytes"
	"fmt"
	"strconv"

	gm "github.com/yuin/goldmark"
	gmAst "github.com/yuin/goldmark/ast"
	gmText "github.com/yuin/goldmark/text"

	"zettelstore.de/c/attrs"
	"zettelstore.de/z/ast"
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189

func (p *mdP) acceptList(node *gmAst.List) ast.ItemNode {
	kind := ast.NestedListUnordered
	var a attrs.Attributes
	if node.IsOrdered() {
		kind = ast.NestedListOrdered
		if node.Start != 1 {
			a = a.Set("start", fmt.Sprintf("%d", node.Start))
		}
	}
	items := make([]ast.ItemSlice, 0, node.ChildCount())
	for child := node.FirstChild(); child != nil; child = child.NextSibling() {
		item, ok := child.(*gmAst.ListItem)
		if !ok {
			panic(fmt.Sprintf("Expected list item node, but got %v", child.Kind()))







|







176
177
178
179
180
181
182
183
184
185
186
187
188
189
190

func (p *mdP) acceptList(node *gmAst.List) ast.ItemNode {
	kind := ast.NestedListUnordered
	var a attrs.Attributes
	if node.IsOrdered() {
		kind = ast.NestedListOrdered
		if node.Start != 1 {
			a = a.Set("start", strconv.Itoa(node.Start))
		}
	}
	items := make([]ast.ItemSlice, 0, node.ChildCount())
	for child := node.FirstChild(); child != nil; child = child.NextSibling() {
		item, ok := child.(*gmAst.ListItem)
		if !ok {
			panic(fmt.Sprintf("Expected list item node, but got %v", child.Kind()))

Changes to parser/parser.go.

105
106
107
108
109
110
111








112
113
114
115
116
117
118
}

// ParseMetadata parses a string as Zettelmarkup, resulting in an inline slice.
// Typically used to parse the title or other metadata of type Zettelmarkup.
func ParseMetadata(value string) ast.InlineSlice {
	return ParseInlines(input.NewInput([]byte(value)), api.ValueSyntaxZmk)
}









// ParseZettel parses the zettel based on the syntax.
func ParseZettel(zettel domain.Zettel, syntax string, rtConfig config.Config) *ast.ZettelNode {
	m := zettel.Meta
	inhMeta := m
	if rtConfig != nil {
		inhMeta = rtConfig.AddDefaultValues(inhMeta)







>
>
>
>
>
>
>
>







105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
}

// ParseMetadata parses a string as Zettelmarkup, resulting in an inline slice.
// Typically used to parse the title or other metadata of type Zettelmarkup.
func ParseMetadata(value string) ast.InlineSlice {
	return ParseInlines(input.NewInput([]byte(value)), api.ValueSyntaxZmk)
}

// ParseMetadataNoLink parses a string as Zettelmarkup, resulting in an inline slice.
// All link and footnote nodes will be removed.
func ParseMetadataNoLink(value string) ast.InlineSlice {
	in := ParseMetadata(value)
	cleaner.CleanInlineLinks(&in)
	return in
}

// ParseZettel parses the zettel based on the syntax.
func ParseZettel(zettel domain.Zettel, syntax string, rtConfig config.Config) *ast.ZettelNode {
	m := zettel.Meta
	inhMeta := m
	if rtConfig != nil {
		inhMeta = rtConfig.AddDefaultValues(inhMeta)

Changes to parser/zettelmark/block.go.

636
637
638
639
640
641
642
643
644




645
646
647
648
649
650
651
		return nil, false
	}
	inp := cp.inp
	posA, posE := inp.Pos, 0
loop:
	for {
		switch inp.Ch {
		case input.EOS, '\n', '\r', ' ':
			return nil, false




		case '\\':
			inp.Next()
			switch inp.Ch {
			case input.EOS, '\n', '\r':
				return nil, false
			}
		case '}':







|

>
>
>
>







636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
		return nil, false
	}
	inp := cp.inp
	posA, posE := inp.Pos, 0
loop:
	for {
		switch inp.Ch {
		case input.EOS:
			return nil, false
		case '\n', '\r', ' ', '\t':
			if !hasSearchPrefix(inp.Src[posA:]) {
				return nil, false
			}
		case '\\':
			inp.Next()
			switch inp.Ch {
			case input.EOS, '\n', '\r':
				return nil, false
			}
		case '}':

Changes to parser/zettelmark/inline.go.

9
10
11
12
13
14
15

16
17
18
19
20
21
22
//-----------------------------------------------------------------------------

package zettelmark

import (
	"bytes"
	"fmt"


	"zettelstore.de/z/ast"
	"zettelstore.de/z/input"
)

// parseInlineSlice parses a sequence of Inlines until EOS.
func (cp *zmkP) parseInlineSlice() (ins ast.InlineSlice) {







>







9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
//-----------------------------------------------------------------------------

package zettelmark

import (
	"bytes"
	"fmt"
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/input"
)

// parseInlineSlice parses a sequence of Inlines until EOS.
func (cp *zmkP) parseInlineSlice() (ins ast.InlineSlice) {
160
161
162
163
164
165
166




167
168
169
170
171
172
173
				Inlines: is,
				Attrs:   attrs,
			}, true
		}
	}
	return nil, false
}





func (cp *zmkP) parseReference(closeCh rune) (ref string, is ast.InlineSlice, ok bool) {
	inp := cp.inp
	inp.Next()
	cp.skipSpace()
	pos := inp.Pos
	hasSpace, ok := cp.readReferenceToSep(closeCh)







>
>
>
>







161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
				Inlines: is,
				Attrs:   attrs,
			}, true
		}
	}
	return nil, false
}

func hasSearchPrefix(src []byte) bool {
	return len(src) > len(ast.SearchPrefix) && string(src[:len(ast.SearchPrefix)]) == ast.SearchPrefix
}

func (cp *zmkP) parseReference(closeCh rune) (ref string, is ast.InlineSlice, ok bool) {
	inp := cp.inp
	inp.Next()
	cp.skipSpace()
	pos := inp.Pos
	hasSpace, ok := cp.readReferenceToSep(closeCh)
184
185
186
187
188
189
190
191

192
193

194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
			if in == nil {
				break
			}
			is = append(is, in)
		}
		cp.inp = inp
		inp.Next()
	} else if hasSpace {

		return "", nil, false
	} else {

		inp.SetPos(pos)
	}

	cp.skipSpace()
	pos = inp.Pos
	if !cp.readReferenceToClose(closeCh) {
		return "", nil, false
	}
	ref = string(inp.Src[pos:inp.Pos])
	inp.Next()
	if inp.Ch != closeCh {
		return "", nil, false
	}
	inp.Next()
	if len(is) == 0 {
		return ref, nil, true







|
>
|
<
>








|







189
190
191
192
193
194
195
196
197
198

199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
			if in == nil {
				break
			}
			is = append(is, in)
		}
		cp.inp = inp
		inp.Next()
	} else {
		if hasSpace && !hasSearchPrefix(inp.Src[pos:]) {
			return "", nil, false

		}
		inp.SetPos(pos)
	}

	cp.skipSpace()
	pos = inp.Pos
	if !cp.readReferenceToClose(closeCh) {
		return "", nil, false
	}
	ref = strings.TrimSpace(string(inp.Src[pos:inp.Pos]))
	inp.Next()
	if inp.Ch != closeCh {
		return "", nil, false
	}
	inp.Next()
	if len(is) == 0 {
		return ref, nil, true
245
246
247
248
249
250
251

252
253
254
255




256
257
258
259
260
261
262
		}
		inp.Next()
	}
}

func (cp *zmkP) readReferenceToClose(closeCh rune) bool {
	inp := cp.inp

	for {
		switch inp.Ch {
		case input.EOS, '\n', '\r', ' ':
			return false




		case '\\':
			inp.Next()
			switch inp.Ch {
			case input.EOS, '\n', '\r':
				return false
			}
		case closeCh:







>


|

>
>
>
>







251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
		}
		inp.Next()
	}
}

func (cp *zmkP) readReferenceToClose(closeCh rune) bool {
	inp := cp.inp
	pos := inp.Pos
	for {
		switch inp.Ch {
		case input.EOS:
			return false
		case '\t', '\r', '\n', ' ':
			if !hasSearchPrefix(inp.Src[pos:]) {
				return false
			}
		case '\\':
			inp.Next()
			switch inp.Ch {
			case input.EOS, '\n', '\r':
				return false
			}
		case closeCh:

Changes to parser/zettelmark/post-processor.go.

355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
		ast.Walk(pp, in)
	}

	pp.processInlineSliceHead(is)
	toPos := pp.processInlineSliceCopy(is)
	toPos = pp.processInlineSliceTail(is, toPos)
	*is = (*is)[:toPos:toPos]
	pp.processInlineSliceInplace(is)
}

// processInlineSliceHead removes leading spaces and empty text.
func (pp *postProcessor) processInlineSliceHead(is *ast.InlineSlice) {
	ins := *is
	for i, in := range ins {
		switch in := in.(type) {







<







355
356
357
358
359
360
361

362
363
364
365
366
367
368
		ast.Walk(pp, in)
	}

	pp.processInlineSliceHead(is)
	toPos := pp.processInlineSliceCopy(is)
	toPos = pp.processInlineSliceTail(is, toPos)
	*is = (*is)[:toPos:toPos]

}

// processInlineSliceHead removes leading spaces and empty text.
func (pp *postProcessor) processInlineSliceHead(is *ast.InlineSlice) {
	ins := *is
	for i, in := range ins {
		switch in := in.(type) {
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
			return toPos
		}
		toPos--
		ins[toPos] = nil // Kill node to enable garbage collection
	}
	return toPos
}

func (*postProcessor) processInlineSliceInplace(is *ast.InlineSlice) {
	for _, in := range *is {
		if n, ok := in.(*ast.TextNode); ok {
			if n.Text == "..." {
				n.Text = "\u2026"
			} else if len(n.Text) == 4 && strings.IndexByte(",;:!?", n.Text[3]) >= 0 && n.Text[:3] == "..." {
				n.Text = "\u2026" + n.Text[3:]
			}
		}
	}
}







<
<
<
<
<
<
<
<
<
<
<
<
470
471
472
473
474
475
476












			return toPos
		}
		toPos--
		ins[toPos] = nil // Kill node to enable garbage collection
	}
	return toPos
}












Changes to parser/zettelmark/zettelmark_test.go.

82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
		{"\\\r\n", ""},
		{"\\\r\ndef", "(PARA HB def)"},
		{"\\a", "(PARA a)"},
		{"\\aa", "(PARA aa)"},
		{"a\\a", "(PARA aa)"},
		{"\\+", "(PARA +)"},
		{"\\ ", "(PARA \u00a0)"},
		{"...", "(PARA \u2026)"},
		{"...,", "(PARA \u2026,)"},
		{"...;", "(PARA \u2026;)"},
		{"...:", "(PARA \u2026:)"},
		{"...!", "(PARA \u2026!)"},
		{"...?", "(PARA \u2026?)"},
		{"...-", "(PARA ...-)"},
		{"a...b", "(PARA a...b)"},
		// {"http://a, http://b", "(PARA http://a, SP http://b)"},
	})
}

func TestSpace(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{" ", ""},







<
<
<
<
<
<
<
<
|







82
83
84
85
86
87
88








89
90
91
92
93
94
95
96
		{"\\\r\n", ""},
		{"\\\r\ndef", "(PARA HB def)"},
		{"\\a", "(PARA a)"},
		{"\\aa", "(PARA aa)"},
		{"a\\a", "(PARA aa)"},
		{"\\+", "(PARA +)"},
		{"\\ ", "(PARA \u00a0)"},








		{"http://a, http://b", "(PARA http://a, SP http://b)"},
	})
}

func TestSpace(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{" ", ""},
168
169
170
171
172
173
174




175
176
177
178
179
180
181
		{"[[\\]]]", "(PARA (LINK %5C%5D))"},
		{"[[\\]|a]]", "(PARA (LINK a ]))"},
		{"[[b\\]|a]]", "(PARA (LINK a b]))"},
		{"[[\\]\\||a]]", "(PARA (LINK a ]|))"},
		{"[[http://a]]", "(PARA (LINK http://a))"},
		{"[[http://a|http://a]]", "(PARA (LINK http://a http://a))"},
		{"[[[[a]]]]", "(PARA (LINK [[a) ]])"},




	})
}

func TestCite(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[@", "(PARA [@)"},







>
>
>
>







160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
		{"[[\\]]]", "(PARA (LINK %5C%5D))"},
		{"[[\\]|a]]", "(PARA (LINK a ]))"},
		{"[[b\\]|a]]", "(PARA (LINK a b]))"},
		{"[[\\]\\||a]]", "(PARA (LINK a ]|))"},
		{"[[http://a]]", "(PARA (LINK http://a))"},
		{"[[http://a|http://a]]", "(PARA (LINK http://a http://a))"},
		{"[[[[a]]]]", "(PARA (LINK [[a) ]])"},
		{"[[search:title]]", "(PARA (LINK search:title))"},
		{"[[search:title syntax]]", "(PARA (LINK search:title syntax))"},
		{"[[Text|search:title]]", "(PARA (LINK search:title Text))"},
		{"[[Text|search:title syntax]]", "(PARA (LINK search:title syntax Text))"},
	})
}

func TestCite(t *testing.T) {
	t.Parallel()
	checkTcs(t, TestCases{
		{"[@", "(PARA [@)"},

Added search/parser.go.





































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package search

import (
	"strconv"

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
)

// Parse the search specification and return a Search object.
func Parse(spec string) (s *Search) { return s.Parse(spec) }

// Parse the search string and update the search object.
func (s *Search) Parse(spec string) *Search {
	state := parserState{
		inp: input.NewInput([]byte(spec)),
	}
	return state.parse(s)
}

type parserState struct {
	inp *input.Input
}

func (ps *parserState) mustStop() bool { return ps.inp.Ch == input.EOS }
func (ps *parserState) acceptSingleKw(s string) bool {
	return ps.inp.Accept(s) && (ps.isSpace() || ps.mustStop())
}
func (ps *parserState) acceptKwArgs(s string) bool {
	if ps.inp.Accept(s) && ps.isSpace() {
		ps.skipSpace()
		return true
	}
	return false
}

const (
	kwLimit   = "LIMIT"
	kwNegate  = "NEGATE"
	kwOffset  = "OFFSET"
	kwOrder   = "ORDER"
	kwRandom  = "RANDOM"
	kwReverse = "REVERSE"
)

func (ps *parserState) parse(sea *Search) *Search {
	inp := ps.inp
	for {
		ps.skipSpace()
		if ps.mustStop() {
			break
		}
		pos := inp.Pos
		if ps.acceptSingleKw(kwNegate) {
			sea = createIfNeeded(sea)
			sea.negate = !sea.negate
			continue
		}
		if ps.acceptSingleKw(kwRandom) {
			sea = createIfNeeded(sea)
			if len(sea.order) == 0 {
				sea.order = []sortOrder{{"", false}}
			}
			continue
		}
		if ps.acceptKwArgs(kwOrder) {
			if s, ok := ps.parseOrder(sea); ok {
				sea = s
				continue
			}
		}
		if ps.acceptKwArgs(kwOffset) {
			if s, ok := ps.parseOffset(sea); ok {
				sea = s
				continue
			}
		}
		if ps.acceptKwArgs(kwLimit) {
			if s, ok := ps.parseLimit(sea); ok {
				sea = s
				continue
			}
		}
		inp.SetPos(pos)
		sea = ps.parseText(sea)
	}
	return sea
}
func (ps *parserState) parseOrder(s *Search) (*Search, bool) {
	reverse := false
	if ps.acceptKwArgs(kwReverse) {
		reverse = true
	}
	word := ps.scanWord()
	if len(word) == 0 {
		return s, false
	}
	if sWord := string(word); meta.KeyIsValid(sWord) {
		s = createIfNeeded(s)
		if len(s.order) == 1 && s.order[0].isRandom() {
			s.order = nil
		}
		s.order = append(s.order, sortOrder{sWord, reverse})
		return s, true
	}
	return s, false
}

func (ps *parserState) parseOffset(s *Search) (*Search, bool) {
	num, ok := ps.scanPosInt()
	if !ok {
		return s, false
	}
	s = createIfNeeded(s)
	if s.offset <= num {
		s.offset = num
	}
	return s, true
}

func (ps *parserState) parseLimit(s *Search) (*Search, bool) {
	num, ok := ps.scanPosInt()
	if !ok {
		return s, false
	}
	s = createIfNeeded(s)
	if s.limit == 0 || s.limit >= num {
		s.limit = num
	}
	return s, true
}

func (ps *parserState) parseText(s *Search) *Search {
	op, hasOp := ps.scanSearchOp()
	text, key := ps.scanSearchTextOrKey(hasOp)
	if len(key) > 0 {
		// Assert: hasOp == false
		op, hasOp = ps.scanSearchOp()
		// Assert hasOp == true
		text = ps.scanWord()
	} else if len(text) == 0 {
		// Only an empty search operation is found -> ignore it
		return s
	}
	s = createIfNeeded(s)
	if hasOp {
		if key == nil {
			s.addSearch(expValue{string(text), op})
		} else if s.mvals == nil {
			s.mvals = expMetaValues{string(key): {expValue{string(text), op}}}
		} else {
			sKey := string(key)
			s.mvals[sKey] = append(s.mvals[sKey], expValue{string(text), op})
		}
	} else {
		// Assert key == nil
		s.addSearch(expValue{string(text), cmpContains})
	}
	return s
}

func (ps *parserState) scanSearchTextOrKey(hasOp bool) ([]byte, []byte) {
	inp := ps.inp
	pos := inp.Pos
	allowKey := !hasOp

	for !ps.isSpace() && !ps.mustStop() {
		if allowKey {
			switch inp.Ch {
			case '!', ':', '=', '>', '<', '~':
				allowKey = false
				if key := inp.Src[pos:inp.Pos]; meta.KeyIsValid(string(key)) {
					return nil, key
				}
			}
		}
		inp.Next()
	}
	return inp.Src[pos:inp.Pos], nil
}

func (ps *parserState) scanWord() []byte {
	inp := ps.inp
	pos := inp.Pos
	for !ps.isSpace() && !ps.mustStop() {
		inp.Next()
	}
	return inp.Src[pos:inp.Pos]
}

func (ps *parserState) scanPosInt() (int, bool) {
	inp := ps.inp
	ch := inp.Ch
	if ch == '0' {
		ch = inp.Next()
		if isSpace(ch) || ps.mustStop() {
			return 0, true
		}
		return 0, false
	}
	word := ps.scanWord()
	if len(word) == 0 {
		return 0, false
	}
	uval, err := strconv.ParseUint(string(word), 10, 63)
	if err != nil {
		return 0, false
	}
	return int(uval), true
}

func (ps *parserState) scanSearchOp() (compareOp, bool) {
	inp := ps.inp
	ch := inp.Ch
	negate := false
	if ch == '!' {
		ch = inp.Next()
		negate = true
	}
	op := cmpUnknown
	switch ch {
	case ':':
		inp.Next()
		op = cmpEqual
	case '<':
		inp.Next()
		op = cmpSuffix
	case '>':
		inp.Next()
		op = cmpPrefix
	case '~':
		inp.Next()
		op = cmpContains
	default:
		if negate {
			return cmpNotContains, true
		}
		return cmpUnknown, false
	}
	if negate {
		return op.negate(), true
	}
	return op, true
}

func (ps *parserState) isSpace() bool {
	return isSpace(ps.inp.Ch)
}

func isSpace(ch rune) bool {
	switch ch {
	case input.EOS:
		return false
	case ' ', '\t', '\n', '\r':
		return true
	}
	return input.IsSpace(ch)
}

func (ps *parserState) skipSpace() {
	for ps.isSpace() {
		ps.inp.Next()
	}
}

Added search/parser_test.go.













































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
//-----------------------------------------------------------------------------
// Copyright (c) 2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package search_test

import (
	"testing"

	"zettelstore.de/z/search"
)

func TestParser(t *testing.T) {
	t.Parallel()
	testcases := []struct {
		spec string
		exp  string
	}{
		{"", ""}, {"!", ""}, {":", ""}, {"!:", ""}, {">", ""}, {"!>", ""}, {"<", ""}, {"!<", ""}, {"~", ""}, {"!~", ""},
		{`a`, `a`}, {`!a`, `!a`},
		{`:a`, `:a`}, {`!:a`, `!:a`},
		{`>a`, `>a`}, {`!>a`, `!>a`},
		{`<a`, `<a`}, {`!<a`, `!<a`},
		{`~a`, `a`}, {`!~a`, `!a`},
		{`key:`, `key:`}, {`key!:`, `key!:`},
		{`key>`, `key>`}, {`key!>`, `key!>`},
		{`key<`, `key<`}, {`key!<`, `key!<`},
		{`key~`, `key~`}, {`key!~`, `key!~`},
		{`key:a`, `key:a`}, {`key!:a`, `key!:a`},
		{`key>a`, `key>a`}, {`key!>a`, `key!>a`},
		{`key<a`, `key<a`}, {`key!<a`, `key!<a`},
		{`key~a`, `key~a`}, {`key!~a`, `key!~a`},
		{`key1:a key2:b`, `key1:a key2:b`},
		{`key1: key2:b`, `key1: key2:b`},
		{`NEGATE`, `NEGATE`}, {`NEGATE a`, `NEGATE a`}, {`a NEGATE`, `NEGATE a`},
		{`NEGATE NEGATE a`, `a`},
		{`NEGATENEGATE a`, `NEGATENEGATE a`},
		{`RANDOM`, `RANDOM`}, {`RANDOM a`, `a RANDOM`}, {`a RANDOM`, `a RANDOM`},
		{`RANDOM RANDOM a`, `a RANDOM`},
		{`RANDOMRANDOM a`, `RANDOMRANDOM a`}, {`a RANDOMRANDOM`, `a RANDOMRANDOM`},
		{`ORDER`, `ORDER`}, {"ORDER a b", "b ORDER a"}, {"a ORDER", "a ORDER"}, {"ORDER ?", "ORDER ?"},
		{"ORDER a ?", "? ORDER a"},
		{"ORDER REVERSE", "ORDER REVERSE"}, {"ORDER REVERSE a b", "b ORDER REVERSE a"},
		{"a RANDOM ORDER b", "a ORDER b"}, {"a ORDER b RANDOM", "a ORDER b"},
		{"OFFSET", "OFFSET"}, {"OFFSET a", "OFFSET a"}, {"OFFSET 10 a", "a OFFSET 10"},
		{"OFFSET 01 a", "OFFSET 01 a"}, {"OFFSET 0 a", "a"}, {"a OFFSET 0", "a"},
		{"OFFSET 4 OFFSET 8", "OFFSET 8"}, {"OFFSET 8 OFFSET 4", "OFFSET 8"},
		{"LIMIT", "LIMIT"}, {"LIMIT a", "LIMIT a"}, {"LIMIT 10 a", "a LIMIT 10"},
		{"LIMIT 01 a", "LIMIT 01 a"}, {"LIMIT 0 a", "a"}, {"a LIMIT 0", "a"},
		{"LIMIT 4 LIMIT 8", "LIMIT 4"}, {"LIMIT 8 LIMIT 4", "LIMIT 4"},
	}
	for i, tc := range testcases {
		got := search.Parse(tc.spec).String()
		if tc.exp != got {
			t.Errorf("%d: Parse(%q) does not yield %q, but got %q", i, tc.spec, tc.exp, got)
			continue
		}

		gotReparse := search.Parse(got).String()
		if gotReparse != got {
			t.Errorf("%d: Parse(%q) does not yield itself, but %q", i, got, gotReparse)
		}
	}
}

Changes to search/print.go.

1
2
3
4
5
6
7
8
9
10
11
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package search provides a zettel search.

|

|







1
2
3
4
5
6
7
8
9
10
11
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package search provides a zettel search.
22
23
24
25
26
27
28
29
30
31
32
33

34
35

36
37
38
39
40
41
42
43






44

45

46
47

48





49
50

51



52

53
















54







55






56





57


























58



59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80


81
82


83
84


85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100



101
102
103
104



105
106
107
108

109
110


111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133

func (s *Search) String() string {
	var sb strings.Builder
	s.Print(&sb)
	return sb.String()
}

// Print the search to a writer.
func (s *Search) Print(w io.Writer) {
	if s == nil {
		return
	}

	if s.negate {
		io.WriteString(w, "NOT (")

	}
	space := false
	if len(s.search) > 0 {
		io.WriteString(w, "ANY")
		printSelectExprValues(w, s.search)
		space = true
	}
	for _, name := range maps.Keys(s.mvals) {






		if space {

			io.WriteString(w, " AND ")

		}
		io.WriteString(w, name)

		printSelectExprValues(w, s.mvals[name])





		space = true
	}

	if s.negate {



		io.WriteString(w, ")")

		space = true
















	}














	space = printOrder(w, s.order, s.descending, space)





	space = printPosInt(w, "OFFSET", s.offset, space)


























	_ = printPosInt(w, "LIMIT", s.limit, space)



}

func printSelectExprValues(w io.Writer, values []expValue) {
	if len(values) == 0 {
		io.WriteString(w, " MATCH ANY")
		return
	}

	for j, val := range values {
		if j > 0 {
			io.WriteString(w, " AND")
		}
		if val.negate {
			io.WriteString(w, " NOT")
		}
		switch val.op {
		case cmpDefault:
			io.WriteString(w, " MATCH ")
		case cmpEqual:
			io.WriteString(w, " EQUAL ")
		case cmpPrefix:
			io.WriteString(w, " PREFIX ")


		case cmpSuffix:
			io.WriteString(w, " SUFFIX ")


		case cmpContains:
			io.WriteString(w, " CONTAINS ")


		default:
			io.WriteString(w, " MaTcH ")
		}
		if val.value == "" {
			io.WriteString(w, "ANY")
		} else {
			io.WriteString(w, val.value)
		}
	}
}

func printOrder(w io.Writer, order string, descending, withSpace bool) bool {
	if len(order) > 0 {
		switch order {
		case api.KeyID:
			// Ignore



		case RandomOrder:
			withSpace = printSpace(w, withSpace)
			io.WriteString(w, "RANDOM")
		default:



			withSpace = printSpace(w, withSpace)
			io.WriteString(w, "SORT ")
			io.WriteString(w, order)
			if descending {

				io.WriteString(w, " DESC")
			}


		}
	}
	return withSpace
}

func printPosInt(w io.Writer, key string, val int, space bool) bool {
	if val > 0 {
		space = printSpace(w, space)
		io.WriteString(w, key)
		w.Write(bsSpace)
		io.WriteString(w, strconv.Itoa(val))
	}
	return space
}

var bsSpace = []byte{' '}

func printSpace(w io.Writer, space bool) bool {
	if space {
		w.Write(bsSpace)
	}
	return true
}







|




>

|
>

<

<
|
<


>
>
>
>
>
>
|
>
|
>
|
|
>
|
>
>
>
>
>
|
|
>
|
>
>
>
|
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
|
>
>
>
>
>
>
|
>
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>


|

|





|

<
<
<

|
|
|
|

|
>
>

|
>
>

|
>
>

|


|

|



<
<
<
<
<
<
>
>
>
|
|
|
|
>
>
>
|
<
|
|
>
|
|
>
>
|
|
<
|
|
<

|
|
|
|

<
<
<
<
<
<
<
<
<
<

22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

39

40

41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151



152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178






179
180
181
182
183
184
185
186
187
188
189

190
191
192
193
194
195
196
197
198

199
200

201
202
203
204
205
206










207

func (s *Search) String() string {
	var sb strings.Builder
	s.Print(&sb)
	return sb.String()
}

// Print the search in a parseable form.
func (s *Search) Print(w io.Writer) {
	if s == nil {
		return
	}
	env := printEnv{w: w}
	if s.negate {
		io.WriteString(w, kwNegate)
		env.space = true
	}

	if len(s.search) > 0 {

		env.printExprValues("", s.search)

	}
	for _, name := range maps.Keys(s.mvals) {
		env.printExprValues(name, s.mvals[name])
	}
	env.printOrder(s.order)
	env.printPosInt(kwOffset, s.offset)
	env.printPosInt(kwLimit, s.limit)
}

type printEnv struct {
	w     io.Writer
	space bool
}

var bsSpace = []byte{' '}

func (pe *printEnv) printSpace() {
	if pe.space {
		pe.w.Write(bsSpace)
		return
	}
	pe.space = true
}
func (pe *printEnv) writeString(s string) { io.WriteString(pe.w, s) }

func (pe *printEnv) printExprValues(key string, values []expValue) {
	for _, val := range values {
		pe.printSpace()
		pe.writeString(key)
		switch val.op {
		case cmpEqual:
			pe.writeString(":")
		case cmpNotEqual:
			pe.writeString("!:")
		case cmpPrefix:
			pe.writeString(">")
		case cmpNoPrefix:
			pe.writeString("!>")
		case cmpSuffix:
			pe.writeString("<")
		case cmpNoSuffix:
			pe.writeString("!<")
		case cmpContains:
			// An empty key signals a full-text search. Since "~" is the default op in this case,
			// it can be ignored. Therefore, print only "~" if there is a key.
			if key != "" {
				pe.writeString("~")
			}
		case cmpNotContains:
			// An empty key signals a full-text search. Since "!" is the shortcut for "!~",
			// it can be ignored. Therefore, print only "!~" if there is a key.
			if key == "" {
				pe.writeString("!")
			} else {
				pe.writeString("!~")
			}
		}
		if s := val.value; s != "" {
			pe.writeString(s)
		}
	}
}

func (s *Search) Human() string {
	var sb strings.Builder
	s.PrintHuman(&sb)
	return sb.String()
}

// PrintHuman the search to a writer in a human readable form.
func (s *Search) PrintHuman(w io.Writer) {
	if s == nil {
		return
	}
	env := printEnv{w: w}
	if s.negate {
		env.writeString("NOT (")
	}
	if len(s.search) > 0 {
		env.writeString("ANY")
		env.printHumanSelectExprValues(s.search)
		env.space = true
	}
	for _, name := range maps.Keys(s.mvals) {
		if env.space {
			env.writeString(" AND ")
		}
		env.writeString(name)
		env.printHumanSelectExprValues(s.mvals[name])
		env.space = true
	}
	if s.negate {
		env.writeString(")")
		env.space = true
	}

	env.printOrder(s.order)
	env.printPosInt(kwOffset, s.offset)
	env.printPosInt(kwLimit, s.limit)
}

func (pe *printEnv) printHumanSelectExprValues(values []expValue) {
	if len(values) == 0 {
		pe.writeString(" MATCH ANY")
		return
	}

	for j, val := range values {
		if j > 0 {
			pe.writeString(" AND")
		}



		switch val.op {
		case cmpEqual:
			pe.writeString(" MATCH ")
		case cmpNotEqual:
			pe.writeString(" NOT MATCH ")
		case cmpPrefix:
			pe.writeString(" PREFIX ")
		case cmpNoPrefix:
			pe.writeString(" NOT PREFIX ")
		case cmpSuffix:
			pe.writeString(" SUFFIX ")
		case cmpNoSuffix:
			pe.writeString(" NOT SUFFIX ")
		case cmpContains:
			pe.writeString(" CONTAINS ")
		case cmpNotContains:
			pe.writeString(" NOT CONTAINS ")
		default:
			pe.writeString(" MaTcH ")
		}
		if val.value == "" {
			pe.writeString("ANY")
		} else {
			pe.writeString(val.value)
		}
	}
}







func (pe *printEnv) printOrder(order []sortOrder) {
	for _, o := range order {
		if o.isRandom() {
			pe.printSpace()
			pe.writeString(kwRandom)
			continue
		} else if o.key == api.KeyID && o.descending {
			continue
		}
		pe.printSpace()

		pe.writeString(kwOrder)
		if o.descending {
			pe.printSpace()
			pe.writeString(kwReverse)
		}
		pe.printSpace()
		pe.writeString(o.key)
	}
}


func (pe *printEnv) printPosInt(key string, val int) {

	if val > 0 {
		pe.printSpace()
		pe.writeString(key)
		pe.writeString(" ")
		pe.writeString(strconv.Itoa(val))
	}










}

Changes to search/retrieve.go.

64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

func prepareRetrieveCalls(searcher Searcher, search []expValue) (normCalls, plainCalls, negCalls searchCallMap) {
	normCalls = make(searchCallMap, len(search))
	negCalls = make(searchCallMap, len(search))
	for _, val := range search {
		for _, word := range strfun.NormalizeWords(val.value) {
			sf := getSearchFunc(searcher, val.op)
			if val.negate {
				negCalls.addSearch(word, val.op, sf)
			} else {
				normCalls.addSearch(word, val.op, sf)
			}
		}
	}

	plainCalls = make(searchCallMap, len(search))
	for _, val := range search {
		word := strings.ToLower(strings.TrimSpace(val.value))
		sf := getSearchFunc(searcher, val.op)
		if val.negate {
			negCalls.addSearch(word, val.op, sf)
		} else {
			plainCalls.addSearch(word, val.op, sf)
		}
	}
	return normCalls, plainCalls, negCalls
}







|











|







64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

func prepareRetrieveCalls(searcher Searcher, search []expValue) (normCalls, plainCalls, negCalls searchCallMap) {
	normCalls = make(searchCallMap, len(search))
	negCalls = make(searchCallMap, len(search))
	for _, val := range search {
		for _, word := range strfun.NormalizeWords(val.value) {
			sf := getSearchFunc(searcher, val.op)
			if val.op.isNegated() {
				negCalls.addSearch(word, val.op, sf)
			} else {
				normCalls.addSearch(word, val.op, sf)
			}
		}
	}

	plainCalls = make(searchCallMap, len(search))
	for _, val := range search {
		word := strings.ToLower(strings.TrimSpace(val.value))
		sf := getSearchFunc(searcher, val.op)
		if val.op.isNegated() {
			negCalls.addSearch(word, val.op, sf)
		} else {
			plainCalls.addSearch(word, val.op, sf)
		}
	}
	return normCalls, plainCalls, negCalls
}

Changes to search/search.go.

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

// Package search provides a zettel search.
package search

import (
	"math/rand"
	"sort"
	"strings"
	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)








<







10
11
12
13
14
15
16

17
18
19
20
21
22
23

// Package search provides a zettel search.
package search

import (
	"math/rand"
	"sort"

	"sync"

	"zettelstore.de/c/api"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

57
58
59
60
61
62
63
64
65
66
67
68
69







70







71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126





127


128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
	// Fields to be used for selecting
	preMatch MetaMatchFunc // Match that must be true
	mvals    expMetaValues // Expected values for a meta datum
	search   []expValue    // Search string
	negate   bool          // Negate the result of the whole selecting process

	// Fields to be used for sorting
	order      string // Name of meta key. None given: use "id"
	descending bool   // Sort by order, but descending
	offset     int    // <= 0: no offset
	limit      int    // <= 0: no limit
}








type expMetaValues map[string][]expValue








// Clone the search value.
func (s *Search) Clone() *Search {
	if s == nil {
		return nil
	}
	c := new(Search)
	c.preMatch = s.preMatch
	c.mvals = make(expMetaValues, len(s.mvals))
	for k, v := range s.mvals {
		c.mvals[k] = v
	}
	c.search = append([]expValue{}, s.search...)
	c.negate = s.negate
	c.order = s.order
	c.descending = s.descending
	c.offset = s.offset
	c.limit = s.limit
	return c
}

// RandomOrder is a pseudo metadata key that selects a random order.
const RandomOrder = "_random"

type compareOp uint8

const (
	cmpUnknown compareOp = iota
	cmpDefault
	cmpNotDefault
	cmpEqual
	cmpNotEqual
	cmpPrefix
	cmpNoPrefix
	cmpSuffix
	cmpNoSuffix
	cmpContains
	cmpNotContains
)

var negateMap = map[compareOp]compareOp{
	cmpUnknown:     cmpUnknown,
	cmpDefault:     cmpNotDefault,
	cmpNotDefault:  cmpDefault,
	cmpEqual:       cmpNotEqual,
	cmpNotEqual:    cmpEqual,
	cmpPrefix:      cmpNoPrefix,
	cmpNoPrefix:    cmpPrefix,
	cmpSuffix:      cmpNoSuffix,
	cmpNoSuffix:    cmpSuffix,
	cmpContains:    cmpNotContains,
	cmpNotContains: cmpContains,
}

func (op compareOp) negate() compareOp {
	return negateMap[op]





}



type expValue struct {
	value  string
	op     compareOp
	negate bool
}

// AddExpr adds a match expression to the search.
func (s *Search) AddExpr(key, value string) *Search {
	val := parseOp(strings.TrimSpace(value))
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	if key == "" {
		s.addSearch(val)
	} else if s.mvals == nil {
		s.mvals = expMetaValues{key: {val}}
	} else {
		s.mvals[key] = append(s.mvals[key], val)
	}
	return s
}

func (s *Search) addSearch(val expValue) {
	if val.negate {
		val.op = val.op.negate()
		val.negate = false
	}
	switch val.op {
	case cmpDefault:
		val.op = cmpContains
	case cmpNotDefault:
		val.op = cmpContains
		val.negate = true
	case cmpNotEqual, cmpNoPrefix, cmpNoSuffix, cmpNotContains:
		val.op = val.op.negate()
		val.negate = true
	}
	s.search = append(s.search, val)
}

func parseOp(s string) expValue {
	if s == "" {
		return expValue{value: s, op: cmpDefault, negate: false}
	}
	if s[0] == '\\' {
		return expValue{value: s[1:], op: cmpDefault, negate: false}
	}
	negate := false
	if s[0] == '!' {
		negate = true
		s = s[1:]
	}
	if s == "" {
		return expValue{value: s, op: cmpDefault, negate: negate}
	}
	if s[0] == '\\' {
		return expValue{value: s[1:], op: cmpDefault, negate: negate}
	}
	switch s[0] {
	case ':':
		return expValue{value: s[1:], op: cmpDefault, negate: negate}
	case '=':
		return expValue{value: s[1:], op: cmpEqual, negate: negate}
	case '>':
		return expValue{value: s[1:], op: cmpPrefix, negate: negate}
	case '<':
		return expValue{value: s[1:], op: cmpSuffix, negate: negate}
	case '~':
		return expValue{value: s[1:], op: cmpContains, negate: negate}
	}
	return expValue{value: s, op: cmpDefault, negate: negate}
}

// SetNegate changes the search to reverse its selection.
func (s *Search) SetNegate() *Search {
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	s.negate = true
	return s
}

// AddPreMatch adds the pre-selection predicate.
func (s *Search) AddPreMatch(preMatch MetaMatchFunc) *Search {
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	if pre := s.preMatch; pre == nil {
		s.preMatch = preMatch
	} else {
		s.preMatch = func(m *meta.Meta) bool {
			return preMatch(m) && pre(m)
		}
	}
	return s
}

// AddOrder adds the given order to the search object.
func (s *Search) AddOrder(key string, descending bool) *Search {
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	if s.order != "" {
		panic("order field already set: " + s.order)
	}
	s.order = key
	s.descending = descending
	return s
}

// SetOffset sets the given offset of the search object.
func (s *Search) SetOffset(offset int) *Search {
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	if offset < 0 {
		offset = 0
	}
	s.offset = offset
	return s
}

// GetOffset returns the current offset value.
func (s *Search) GetOffset() int {
	if s == nil {
		return 0
	}
	s.mx.RLock()
	defer s.mx.RUnlock()
	return s.offset
}

// SetLimit sets the given limit of the search object.
func (s *Search) SetLimit(limit int) *Search {
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	if limit < 0 {
		limit = 0
	}
	s.limit = limit
	return s







|
<
|
|


>
>
>
>
>
>
>

>
>
>
>
>
>
>














|
<












<
<












<
<










|
|
>
>
>
>
>

>
>


|
|
<


<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
|
<
<
<
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<


<
|
<












<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<


<
|
<







56
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97

98
99
100
101
102
103
104
105
106
107
108
109


110
111
112
113
114
115
116
117
118
119
120
121


122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145

146
147


















148



149

























































150
151

152

153
154
155
156
157
158
159
160
161
162
163
164







































165
166

167

168
169
170
171
172
173
174
	// Fields to be used for selecting
	preMatch MetaMatchFunc // Match that must be true
	mvals    expMetaValues // Expected values for a meta datum
	search   []expValue    // Search string
	negate   bool          // Negate the result of the whole selecting process

	// Fields to be used for sorting
	order  []sortOrder

	offset int // <= 0: no offset
	limit  int // <= 0: no limit
}

type sortOrder struct {
	key        string
	descending bool
}

func (so *sortOrder) isRandom() bool { return so.key == "" }

type expMetaValues map[string][]expValue

func createIfNeeded(s *Search) *Search {
	if s == nil {
		return new(Search)
	}
	return s
}

// Clone the search value.
func (s *Search) Clone() *Search {
	if s == nil {
		return nil
	}
	c := new(Search)
	c.preMatch = s.preMatch
	c.mvals = make(expMetaValues, len(s.mvals))
	for k, v := range s.mvals {
		c.mvals[k] = v
	}
	c.search = append([]expValue{}, s.search...)
	c.negate = s.negate
	c.order = append([]sortOrder{}, s.order...)

	c.offset = s.offset
	c.limit = s.limit
	return c
}

// RandomOrder is a pseudo metadata key that selects a random order.
const RandomOrder = "_random"

type compareOp uint8

const (
	cmpUnknown compareOp = iota


	cmpEqual
	cmpNotEqual
	cmpPrefix
	cmpNoPrefix
	cmpSuffix
	cmpNoSuffix
	cmpContains
	cmpNotContains
)

var negateMap = map[compareOp]compareOp{
	cmpUnknown:     cmpUnknown,


	cmpEqual:       cmpNotEqual,
	cmpNotEqual:    cmpEqual,
	cmpPrefix:      cmpNoPrefix,
	cmpNoPrefix:    cmpPrefix,
	cmpSuffix:      cmpNoSuffix,
	cmpNoSuffix:    cmpSuffix,
	cmpContains:    cmpNotContains,
	cmpNotContains: cmpContains,
}

func (op compareOp) negate() compareOp { return negateMap[op] }

var negativeMap = map[compareOp]bool{
	cmpNotEqual:    true,
	cmpNoPrefix:    true,
	cmpNoSuffix:    true,
	cmpNotContains: true,
}

func (op compareOp) isNegated() bool { return negativeMap[op] }

type expValue struct {
	value string
	op    compareOp

}



















func (s *Search) addSearch(val expValue) { s.search = append(s.search, val) }





























































// AddPreMatch adds the pre-selection predicate.
func (s *Search) AddPreMatch(preMatch MetaMatchFunc) *Search {

	s = createIfNeeded(s)

	s.mx.Lock()
	defer s.mx.Unlock()
	if pre := s.preMatch; pre == nil {
		s.preMatch = preMatch
	} else {
		s.preMatch = func(m *meta.Meta) bool {
			return preMatch(m) && pre(m)
		}
	}
	return s
}








































// SetLimit sets the given limit of the search object.
func (s *Search) SetLimit(limit int) *Search {

	s = createIfNeeded(s)

	s.mx.Lock()
	defer s.mx.Unlock()
	if limit < 0 {
		limit = 0
	}
	s.limit = limit
	return s
301
302
303
304
305
306
307

308




309
310
311
312
313
314
315
	s.mx.RLock()
	defer s.mx.RUnlock()
	for key := range s.mvals {
		if meta.IsComputed(key) {
			return true
		}
	}

	return meta.IsComputed(s.order)




}

// RetrieveAndCompileMatch queries the search index and returns a predicate
// for its results and returns a matching predicate.
func (s *Search) RetrieveAndCompileMatch(searcher Searcher) (RetrievePredicate, MetaMatchFunc) {
	if s == nil {
		return alwaysIncluded, matchAlways







>
|
>
>
>
>







193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
	s.mx.RLock()
	defer s.mx.RUnlock()
	for key := range s.mvals {
		if meta.IsComputed(key) {
			return true
		}
	}
	for _, o := range s.order {
		if meta.IsComputed(o.key) {
			return true
		}
	}
	return false
}

// RetrieveAndCompileMatch queries the search index and returns a predicate
// for its results and returns a matching predicate.
func (s *Search) RetrieveAndCompileMatch(searcher Searcher) (RetrievePredicate, MetaMatchFunc) {
	if s == nil {
		return alwaysIncluded, matchAlways
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
	}

	if s == nil {
		sort.Slice(metaList, func(i, j int) bool { return metaList[i].Zid > metaList[j].Zid })
		return metaList
	}

	if s.order == "" {
		sort.Slice(metaList, createSortFunc(api.KeyID, true, metaList))
	} else if s.order == RandomOrder {
		rand.Shuffle(len(metaList), func(i, j int) {
			metaList[i], metaList[j] = metaList[j], metaList[i]
		})
	} else {
		sort.Slice(metaList, createSortFunc(s.order, s.descending, metaList))
	}

	if s.offset > 0 {
		if s.offset > len(metaList) {
			return nil
		}
		metaList = metaList[s.offset:]







|

|




|







304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
	}

	if s == nil {
		sort.Slice(metaList, func(i, j int) bool { return metaList[i].Zid > metaList[j].Zid })
		return metaList
	}

	if len(s.order) == 0 {
		sort.Slice(metaList, createSortFunc(api.KeyID, true, metaList))
	} else if s.order[0].isRandom() {
		rand.Shuffle(len(metaList), func(i, j int) {
			metaList[i], metaList[j] = metaList[j], metaList[i]
		})
	} else {
		sort.Slice(metaList, createSortFunc(s.order[0].key, s.order[0].descending, metaList))
	}

	if s.offset > 0 {
		if s.offset > len(metaList) {
			return nil
		}
		metaList = metaList[s.offset:]

Changes to search/select.go.

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
				negSpecs = append(negSpecs, matchSpec{key, nil})
				continue
			}
			// value must match always AND never, at the same time. This results in a no-match.
			nomatch = append(nomatch, key)
			continue
		}
		posMatch, negMatch := createPosNegMatchFunc(
			key, values,
			func(val string, op compareOp) { s.addSearch(expValue{value: val, op: op, negate: false}) })
		if posMatch != nil {
			posSpecs = append(posSpecs, matchSpec{key, posMatch})
		}
		if negMatch != nil {
			negSpecs = append(negSpecs, matchSpec{key, negMatch})
		}
	}
	return posSpecs, negSpecs, nomatch
}

func countEmptyValues(values []expValue) (always, never int) {
	for _, v := range values {
		if v.value == "" {
			if v.negate {
				never++
			} else {
				always++
			}
		}
	}
	return always, never
}

type addSearchFunc func(val string, op compareOp)

func createPosNegMatchFunc(key string, values []expValue, addSearch addSearchFunc) (posMatch, negMatch matchValueFunc) {
	posValues := make([]opValue, 0, len(values))
	negValues := make([]opValue, 0, len(values))
	for _, val := range values {
		if val.negate {
			negValues = append(negValues, opValue{value: val.value, op: val.op.negate()})
		} else {
			posValues = append(posValues, opValue{value: val.value, op: val.op})
		}
	}
	return createMatchFunc(key, posValues, addSearch), createMatchFunc(key, negValues, addSearch)
}

// opValue is an expValue, but w/o the field "negate"
type opValue struct {
	value string
	op    compareOp
}

func createMatchFunc(key string, values []opValue, addSearch addSearchFunc) matchValueFunc {
	if len(values) == 0 {
		return nil
	}
	switch meta.Type(key) {
	case meta.TypeCredential:
		return matchValueNever
	case meta.TypeID, meta.TypeTimestamp: // ID and timestamp use the same layout
		return createMatchIDFunc(values, addSearch)
	case meta.TypeIDSet:
		return createMatchIDSetFunc(values, addSearch)
	case meta.TypeTagSet:
		return createMatchTagSetFunc(values, addSearch)
	case meta.TypeWord:
		return createMatchWordFunc(values, addSearch)
	case meta.TypeWordSet:
		return createMatchWordSetFunc(values, addSearch)
	}
	return createMatchStringFunc(values, addSearch)
}

func createMatchIDFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(values, cmpPrefix, addSearch)
	return func(value string) bool {
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func createMatchIDSetFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	predList := valuesToStringSetPredicates(preprocessSet(values), cmpPrefix, addSearch)
	return func(value string) bool {
		ids := meta.ListFromValue(value)
		for _, preds := range predList {
			for _, pred := range preds {
				if !pred(ids) {
					return false
				}
			}
		}
		return true
	}
}

func createMatchTagSetFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	predList := valuesToStringSetPredicates(processTagSet(preprocessSet(sliceToLower(values))), cmpEqual, addSearch)
	return func(value string) bool {
		tags := meta.ListFromValue(value)
		// Remove leading '#' from each tag
		for i, tag := range tags {
			tags[i] = meta.CleanTag(tag)
		}
		for _, preds := range predList {
			for _, pred := range preds {
				if !pred(tags) {
					return false
				}
			}
		}
		return true
	}
}

func processTagSet(valueSet [][]opValue) [][]opValue {
	result := make([][]opValue, len(valueSet))
	for i, values := range valueSet {
		tags := make([]opValue, len(values))
		for j, val := range values {
			if tval := val.value; tval != "" && tval[0] == '#' {
				tval = meta.CleanTag(tval)
				tags[j] = opValue{value: tval, op: resolveDefaultOp(val.op, cmpEqual)}
			} else {
				tags[j] = opValue{value: tval, op: resolveDefaultOp(val.op, cmpPrefix)}
			}
		}
		result[i] = tags
	}
	return result
}

func createMatchWordFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpEqual, addSearch)
	return func(value string) bool {
		value = strings.ToLower(value)
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func createMatchWordSetFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	predsList := valuesToStringSetPredicates(preprocessSet(sliceToLower(values)), cmpEqual, addSearch)
	return func(value string) bool {
		words := meta.ListFromValue(value)
		for _, preds := range predsList {
			for _, pred := range preds {
				if !pred(words) {
					return false
				}
			}
		}
		return true
	}
}

func createMatchStringFunc(values []opValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpContains, addSearch)
	return func(value string) bool {
		value = strings.ToLower(value)
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func sliceToLower(sl []opValue) []opValue {
	result := make([]opValue, 0, len(sl))
	for _, s := range sl {
		result = append(result, opValue{
			value: strings.ToLower(s.value),
			op:    s.op,
		})
	}
	return result
}

func preprocessSet(set []opValue) [][]opValue {
	result := make([][]opValue, 0, len(set))
	for _, elem := range set {
		splitElems := strings.Split(elem.value, ",")
		valueElems := make([]opValue, 0, len(splitElems))
		for _, se := range splitElems {
			e := strings.TrimSpace(se)
			if len(e) > 0 {
				valueElems = append(valueElems, opValue{value: e, op: elem.op})
			}
		}
		if len(valueElems) > 0 {
			result = append(result, valueElems)
		}
	}
	return result
}

type stringPredicate func(string) bool

func valuesToStringPredicates(values []opValue, defOp compareOp, addSearch addSearchFunc) []stringPredicate {
	result := make([]stringPredicate, len(values))
	for i, v := range values {
		opVal := v.value // loop variable is used in closure --> save needed value
		op := resolveDefaultOp(v.op, defOp)
		switch op {
		case cmpEqual:
			addSearch(opVal, op) // addSearch only for positive selections
			result[i] = func(metaVal string) bool { return metaVal == opVal }
		case cmpNotEqual:
			result[i] = func(metaVal string) bool { return metaVal != opVal }
		case cmpPrefix:
			addSearch(opVal, op)
			result[i] = func(metaVal string) bool { return strings.HasPrefix(metaVal, opVal) }
		case cmpNoPrefix:
			result[i] = func(metaVal string) bool { return !strings.HasPrefix(metaVal, opVal) }
		case cmpSuffix:
			addSearch(opVal, op)
			result[i] = func(metaVal string) bool { return strings.HasSuffix(metaVal, opVal) }
		case cmpNoSuffix:
			result[i] = func(metaVal string) bool { return !strings.HasSuffix(metaVal, opVal) }
		case cmpContains:
			addSearch(opVal, op)
			result[i] = func(metaVal string) bool { return strings.Contains(metaVal, opVal) }
		case cmpNotContains:
			result[i] = func(metaVal string) bool { return !strings.Contains(metaVal, opVal) }
		default:
			panic(fmt.Sprintf("Unknown compare operation %d/%d with value %q", op, v.op, opVal))
		}
	}
	return result
}

type stringSetPredicate func(value []string) bool

func valuesToStringSetPredicates(values [][]opValue, defOp compareOp, addSearch addSearchFunc) [][]stringSetPredicate {
	result := make([][]stringSetPredicate, len(values))
	for i, val := range values {
		elemPreds := make([]stringSetPredicate, len(val))
		for j, v := range val {
			opVal := v.value // loop variable is used in closure --> save needed value
			op := resolveDefaultOp(v.op, defOp)
			switch op {
			case cmpEqual:
				addSearch(opVal, op) // addSearch only for positive selections
				elemPreds[j] = makeStringSetPredicate(opVal, stringEqual, true)
			case cmpNotEqual:
				elemPreds[j] = makeStringSetPredicate(opVal, stringEqual, false)
			case cmpPrefix:
				addSearch(opVal, op)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasPrefix, true)
			case cmpNoPrefix:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasPrefix, false)
			case cmpSuffix:
				addSearch(opVal, op)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasSuffix, true)
			case cmpNoSuffix:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasSuffix, false)
			case cmpContains:
				addSearch(opVal, op)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.Contains, true)
			case cmpNotContains:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.Contains, false)
			default:
				panic(fmt.Sprintf("Unknown compare operation %d/%d with value %q", op, v.op, opVal))
			}
		}







|
<
<













|









|


|
|

|
|

|





<
<
<
<
<
<
|




















|











|














|


















|
|

|



|

|







|












|














|












|
|

|







|
|


|



|











|



|


|




|




|




|












|





|


|




|




|




|







52
53
54
55
56
57
58
59


60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97






98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
				negSpecs = append(negSpecs, matchSpec{key, nil})
				continue
			}
			// value must match always AND never, at the same time. This results in a no-match.
			nomatch = append(nomatch, key)
			continue
		}
		posMatch, negMatch := createPosNegMatchFunc(key, values, s.addSearch)


		if posMatch != nil {
			posSpecs = append(posSpecs, matchSpec{key, posMatch})
		}
		if negMatch != nil {
			negSpecs = append(negSpecs, matchSpec{key, negMatch})
		}
	}
	return posSpecs, negSpecs, nomatch
}

func countEmptyValues(values []expValue) (always, never int) {
	for _, v := range values {
		if v.value == "" {
			if v.op.isNegated() {
				never++
			} else {
				always++
			}
		}
	}
	return always, never
}

type addSearchFunc func(val expValue)

func createPosNegMatchFunc(key string, values []expValue, addSearch addSearchFunc) (posMatch, negMatch matchValueFunc) {
	posValues := make([]expValue, 0, len(values))
	negValues := make([]expValue, 0, len(values))
	for _, val := range values {
		if val.op.isNegated() {
			negValues = append(negValues, val)
		} else {
			posValues = append(posValues, val)
		}
	}
	return createMatchFunc(key, posValues, addSearch), createMatchFunc(key, negValues, addSearch)
}







func createMatchFunc(key string, values []expValue, addSearch addSearchFunc) matchValueFunc {
	if len(values) == 0 {
		return nil
	}
	switch meta.Type(key) {
	case meta.TypeCredential:
		return matchValueNever
	case meta.TypeID, meta.TypeTimestamp: // ID and timestamp use the same layout
		return createMatchIDFunc(values, addSearch)
	case meta.TypeIDSet:
		return createMatchIDSetFunc(values, addSearch)
	case meta.TypeTagSet:
		return createMatchTagSetFunc(values, addSearch)
	case meta.TypeWord:
		return createMatchWordFunc(values, addSearch)
	case meta.TypeWordSet:
		return createMatchWordSetFunc(values, addSearch)
	}
	return createMatchStringFunc(values, addSearch)
}

func createMatchIDFunc(values []expValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(values, cmpPrefix, addSearch)
	return func(value string) bool {
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func createMatchIDSetFunc(values []expValue, addSearch addSearchFunc) matchValueFunc {
	predList := valuesToStringSetPredicates(preprocessSet(values), cmpPrefix, addSearch)
	return func(value string) bool {
		ids := meta.ListFromValue(value)
		for _, preds := range predList {
			for _, pred := range preds {
				if !pred(ids) {
					return false
				}
			}
		}
		return true
	}
}

func createMatchTagSetFunc(values []expValue, addSearch addSearchFunc) matchValueFunc {
	predList := valuesToStringSetPredicates(processTagSet(preprocessSet(sliceToLower(values))), cmpEqual, addSearch)
	return func(value string) bool {
		tags := meta.ListFromValue(value)
		// Remove leading '#' from each tag
		for i, tag := range tags {
			tags[i] = meta.CleanTag(tag)
		}
		for _, preds := range predList {
			for _, pred := range preds {
				if !pred(tags) {
					return false
				}
			}
		}
		return true
	}
}

func processTagSet(valueSet [][]expValue) [][]expValue {
	result := make([][]expValue, len(valueSet))
	for i, values := range valueSet {
		tags := make([]expValue, len(values))
		for j, val := range values {
			if tval := val.value; tval != "" && tval[0] == '#' {
				tval = meta.CleanTag(tval)
				tags[j] = expValue{value: tval, op: val.op}
			} else {
				tags[j] = expValue{value: tval, op: val.op}
			}
		}
		result[i] = tags
	}
	return result
}

func createMatchWordFunc(values []expValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpEqual, addSearch)
	return func(value string) bool {
		value = strings.ToLower(value)
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func createMatchWordSetFunc(values []expValue, addSearch addSearchFunc) matchValueFunc {
	predsList := valuesToStringSetPredicates(preprocessSet(sliceToLower(values)), cmpEqual, addSearch)
	return func(value string) bool {
		words := meta.ListFromValue(value)
		for _, preds := range predsList {
			for _, pred := range preds {
				if !pred(words) {
					return false
				}
			}
		}
		return true
	}
}

func createMatchStringFunc(values []expValue, addSearch addSearchFunc) matchValueFunc {
	preds := valuesToStringPredicates(sliceToLower(values), cmpContains, addSearch)
	return func(value string) bool {
		value = strings.ToLower(value)
		for _, pred := range preds {
			if !pred(value) {
				return false
			}
		}
		return true
	}
}

func sliceToLower(sl []expValue) []expValue {
	result := make([]expValue, 0, len(sl))
	for _, s := range sl {
		result = append(result, expValue{
			value: strings.ToLower(s.value),
			op:    s.op,
		})
	}
	return result
}

func preprocessSet(set []expValue) [][]expValue {
	result := make([][]expValue, 0, len(set))
	for _, elem := range set {
		splitElems := strings.Split(elem.value, ",")
		valueElems := make([]expValue, 0, len(splitElems))
		for _, se := range splitElems {
			e := strings.TrimSpace(se)
			if len(e) > 0 {
				valueElems = append(valueElems, expValue{value: e, op: elem.op})
			}
		}
		if len(valueElems) > 0 {
			result = append(result, valueElems)
		}
	}
	return result
}

type stringPredicate func(string) bool

func valuesToStringPredicates(values []expValue, defOp compareOp, addSearch addSearchFunc) []stringPredicate {
	result := make([]stringPredicate, len(values))
	for i, v := range values {
		opVal := v.value // loop variable is used in closure --> save needed value
		op := v.op
		switch op {
		case cmpEqual:
			addSearch(v) // addSearch only for positive selections
			result[i] = func(metaVal string) bool { return metaVal == opVal }
		case cmpNotEqual:
			result[i] = func(metaVal string) bool { return metaVal != opVal }
		case cmpPrefix:
			addSearch(v)
			result[i] = func(metaVal string) bool { return strings.HasPrefix(metaVal, opVal) }
		case cmpNoPrefix:
			result[i] = func(metaVal string) bool { return !strings.HasPrefix(metaVal, opVal) }
		case cmpSuffix:
			addSearch(v)
			result[i] = func(metaVal string) bool { return strings.HasSuffix(metaVal, opVal) }
		case cmpNoSuffix:
			result[i] = func(metaVal string) bool { return !strings.HasSuffix(metaVal, opVal) }
		case cmpContains:
			addSearch(v)
			result[i] = func(metaVal string) bool { return strings.Contains(metaVal, opVal) }
		case cmpNotContains:
			result[i] = func(metaVal string) bool { return !strings.Contains(metaVal, opVal) }
		default:
			panic(fmt.Sprintf("Unknown compare operation %d/%d with value %q", op, v.op, opVal))
		}
	}
	return result
}

type stringSetPredicate func(value []string) bool

func valuesToStringSetPredicates(values [][]expValue, defOp compareOp, addSearch addSearchFunc) [][]stringSetPredicate {
	result := make([][]stringSetPredicate, len(values))
	for i, val := range values {
		elemPreds := make([]stringSetPredicate, len(val))
		for j, v := range val {
			opVal := v.value // loop variable is used in closure --> save needed value
			op := v.op
			switch op {
			case cmpEqual:
				addSearch(v) // addSearch only for positive selections
				elemPreds[j] = makeStringSetPredicate(opVal, stringEqual, true)
			case cmpNotEqual:
				elemPreds[j] = makeStringSetPredicate(opVal, stringEqual, false)
			case cmpPrefix:
				addSearch(v)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasPrefix, true)
			case cmpNoPrefix:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasPrefix, false)
			case cmpSuffix:
				addSearch(v)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasSuffix, true)
			case cmpNoSuffix:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.HasSuffix, false)
			case cmpContains:
				addSearch(v)
				elemPreds[j] = makeStringSetPredicate(opVal, strings.Contains, true)
			case cmpNotContains:
				elemPreds[j] = makeStringSetPredicate(opVal, strings.Contains, false)
			default:
				panic(fmt.Sprintf("Unknown compare operation %d/%d with value %q", op, v.op, opVal))
			}
		}
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
				return foundResult
			}
		}
		return !foundResult
	}
}

func resolveDefaultOp(op, defOp compareOp) compareOp {
	if op == cmpDefault {
		return defOp
	}
	if op == cmpNotDefault {
		return defOp.negate()
	}
	return op
}

func makeSearchMetaMatchFunc(posSpecs, negSpecs []matchSpec, nomatch []string) MetaMatchFunc {
	if len(nomatch) == 0 {
		// Optimize for simple cases: only negative or only positive matching

		if len(posSpecs) == 0 {
			return func(m *meta.Meta) bool { return matchMetaNegSpecs(m, negSpecs) }
		}







<
<
<
<
<
<
<
<
<
<







334
335
336
337
338
339
340










341
342
343
344
345
346
347
				return foundResult
			}
		}
		return !foundResult
	}
}











func makeSearchMetaMatchFunc(posSpecs, negSpecs []matchSpec, nomatch []string) MetaMatchFunc {
	if len(nomatch) == 0 {
		// Optimize for simple cases: only negative or only positive matching

		if len(posSpecs) == 0 {
			return func(m *meta.Meta) bool { return matchMetaNegSpecs(m, negSpecs) }
		}

Changes to search/select_test.go.

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
	"zettelstore.de/c/api"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

func TestMatchZidNegate(t *testing.T) {
	var s *search.Search
	s = s.AddExpr(api.KeyID, "!="+string(api.ZidVersion))
	s = s.AddExpr(api.KeyID, "!="+string(api.ZidLicense))
	_, matchFunc := s.RetrieveAndCompileMatch(nil)

	testCases := []struct {
		zid api.ZettelID
		exp bool
	}{
		{api.ZidVersion, false},







<
<
|







16
17
18
19
20
21
22


23
24
25
26
27
28
29
30
	"zettelstore.de/c/api"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

func TestMatchZidNegate(t *testing.T) {


	s := search.Parse(api.KeyID + "!:" + string(api.ZidVersion) + " " + api.KeyID + "!:" + string(api.ZidLicense))
	_, matchFunc := s.RetrieveAndCompileMatch(nil)

	testCases := []struct {
		zid api.ZettelID
		exp bool
	}{
		{api.ZidVersion, false},

Changes to template/mustache.go.

125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
}

type parseError struct {
	line    int
	message string
}

func (p parseError) Error() string {
	return fmt.Sprintf("line %d: %s", p.line, p.message)
}

// Tags returns the mustache tags for the given template
func (tmpl *Template) Tags() []Tag {
	return extractTags(tmpl.nodes)
}

func extractTags(nodes []node) []Tag {







|
<
<







125
126
127
128
129
130
131
132


133
134
135
136
137
138
139
}

type parseError struct {
	line    int
	message string
}

func (p parseError) Error() string { return fmt.Sprintf("line %d: %s", p.line, p.message) }



// Tags returns the mustache tags for the given template
func (tmpl *Template) Tags() []Tag {
	return extractTags(tmpl.nodes)
}

func extractTags(nodes []node) []Tag {

Changes to tests/client/client_test.go.

66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84



85
86
87
88
89
90
91
92
93
94
95
96
97
98
99




100
101
102
103
104
105
106
107
108
109
110
111
112
		{"reader", readerZettel},
		{"writer", writerZettel},
		{"owner", ownerZettel},
	}

	t.Parallel()
	c := getClient()
	query := url.Values{api.QueryKeyEncoding: {api.EncodingHTML}} // Client must remove "html"
	for i, tc := range testdata {
		t.Run(fmt.Sprintf("User %d/%q", i, tc.user), func(tt *testing.T) {
			c.SetAuth(tc.user, tc.user)
			q, l, err := c.ListZettelJSON(context.Background(), query)
			if err != nil {
				tt.Error(err)
				return
			}
			if q != "" {
				tt.Errorf("Query should be empty, but is %q", q)
			}



			got := len(l)
			if got != tc.exp {
				tt.Errorf("List of length %d expected, but got %d\n%v", tc.exp, got, l)
			}
		})
	}
	q, l, err := c.ListZettelJSON(context.Background(), url.Values{api.KeyRole: {api.ValueRoleConfiguration}})
	if err != nil {
		t.Error(err)
		return
	}
	expQ := "role MATCH configuration"
	if q != expQ {
		t.Errorf("Query should be %q, but is %q", expQ, q)
	}




	got := len(l)
	if got != configRoleZettel {
		t.Errorf("List of length %d expected, but got %d\n%v", configRoleZettel, got, l)
	}

	pl, err := c.ListZettel(context.Background(), url.Values{api.KeyRole: {api.ValueRoleConfiguration}})
	if err != nil {
		t.Error(err)
		return
	}
	compareZettelList(t, pl, l)
}








<



|







>
>
>






|




|



>
>
>
>





|







66
67
68
69
70
71
72

73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
		{"reader", readerZettel},
		{"writer", writerZettel},
		{"owner", ownerZettel},
	}

	t.Parallel()
	c := getClient()

	for i, tc := range testdata {
		t.Run(fmt.Sprintf("User %d/%q", i, tc.user), func(tt *testing.T) {
			c.SetAuth(tc.user, tc.user)
			q, h, l, err := c.ListZettelJSON(context.Background(), "")
			if err != nil {
				tt.Error(err)
				return
			}
			if q != "" {
				tt.Errorf("Query should be empty, but is %q", q)
			}
			if h != "" {
				tt.Errorf("Human should be empty, but is %q", q)
			}
			got := len(l)
			if got != tc.exp {
				tt.Errorf("List of length %d expected, but got %d\n%v", tc.exp, got, l)
			}
		})
	}
	q, h, l, err := c.ListZettelJSON(context.Background(), api.KeyRole+":"+api.ValueRoleConfiguration)
	if err != nil {
		t.Error(err)
		return
	}
	expQ := "role:configuration"
	if q != expQ {
		t.Errorf("Query should be %q, but is %q", expQ, q)
	}
	expH := "role MATCH configuration"
	if h != expH {
		t.Errorf("Human should be %q, but is %q", expH, h)
	}
	got := len(l)
	if got != configRoleZettel {
		t.Errorf("List of length %d expected, but got %d\n%v", configRoleZettel, got, l)
	}

	pl, err := c.ListZettel(context.Background(), api.KeyRole+":"+api.ValueRoleConfiguration)
	if err != nil {
		t.Error(err)
		return
	}
	compareZettelList(t, pl, l)
}

Changes to tests/client/embed_test.go.

65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
	}

	content, err = c.GetEvaluatedZettel(context.Background(), abc10000Zid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	checkContentContains(t, abc10000Zid, string(content), "Too\u00a0many\u00a0transclusions")
}

func TestZettelTransclusionNoPrivilegeEscalation(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("reader", "reader")








|







65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
	}

	content, err = c.GetEvaluatedZettel(context.Background(), abc10000Zid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	checkContentContains(t, abc10000Zid, string(content), "Too many transclusions")
}

func TestZettelTransclusionNoPrivilegeEscalation(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("reader", "reader")

88
89
90
91
92
93
94
95


96
97
98
99
100
101
102
	}

	content, err := c.GetEvaluatedZettel(context.Background(), abc10Zid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	checkContentContains(t, abc10Zid, string(content), "Error placeholder")


}

func stringHead(s string) string {
	const maxLen = 40
	if len(s) <= maxLen {
		return s
	}







|
>
>







88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
	}

	content, err := c.GetEvaluatedZettel(context.Background(), abc10Zid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	if exp, got := "<p></p>", string(content); exp != got {
		t.Errorf("Zettel %q must contain %q, but got %q", abc10Zid, exp, got)
	}
}

func stringHead(s string) string {
	const maxLen = 40
	if len(s) <= maxLen {
		return s
	}
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
	for zid, errZid := range recursiveZettel {
		content, err := c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML)
		if err != nil {
			t.Error(err)
			continue
		}
		sContent := string(content)
		checkContentContains(t, zid, sContent, "Recursive\u00a0transclusion")
		checkContentContains(t, zid, sContent, string(errZid))
	}
}
func TestNothingToTransclude(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")







|







131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
	for zid, errZid := range recursiveZettel {
		content, err := c.GetEvaluatedZettel(context.Background(), zid, api.EncoderHTML)
		if err != nil {
			t.Error(err)
			continue
		}
		sContent := string(content)
		checkContentContains(t, zid, sContent, "Recursive transclusion")
		checkContentContains(t, zid, sContent, string(errZid))
	}
}
func TestNothingToTransclude(t *testing.T) {
	t.Parallel()
	c := getClient()
	c.SetAuth("owner", "owner")
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178

	const selfEmbedZid = api.ZettelID("20211020185400")
	content, err := c.GetEvaluatedZettel(context.Background(), selfEmbedZid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	checkContentContains(t, selfEmbedZid, string(content), "Self\u00a0embed\u00a0reference")
}

func checkContentContains(t *testing.T, zid api.ZettelID, content, expected string) {
	if !strings.Contains(content, expected) {
		t.Helper()
		t.Errorf("Zettel %q should contain %q, but does not: %q", zid, expected, content)
	}
}







|








165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180

	const selfEmbedZid = api.ZettelID("20211020185400")
	content, err := c.GetEvaluatedZettel(context.Background(), selfEmbedZid, api.EncoderHTML)
	if err != nil {
		t.Error(err)
		return
	}
	checkContentContains(t, selfEmbedZid, string(content), "Self embed reference")
}

func checkContentContains(t *testing.T, zid api.ZettelID, content, expected string) {
	if !strings.Contains(content, expected) {
		t.Helper()
		t.Errorf("Zettel %q should contain %q, but does not: %q", zid, expected, content)
	}
}

Changes to usecase/authenticate.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

16
17
18
19
20
21
22
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase

import (
	"context"
	"math/rand"

	"time"

	"zettelstore.de/c/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"



|











>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package usecase

import (
	"context"
	"math/rand"
	"net/http"
	"time"

	"zettelstore.de/c/api"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
45
46
47
48
49
50
51



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
		token:     token,
		port:      port,
		ucGetUser: NewGetUser(authz, port),
	}
}

// Run executes the use case.



func (uc *Authenticate) Run(ctx context.Context, ident, credential string, d time.Duration, k auth.TokenKind) ([]byte, error) {
	identMeta, err := uc.ucGetUser.Run(ctx, ident)
	defer addDelay(time.Now(), 500*time.Millisecond, 100*time.Millisecond)

	if identMeta == nil || err != nil {
		uc.log.Info().Str("ident", ident).Err(err).Msg("No user with given ident found")
		compensateCompare()
		return nil, err
	}

	if hashCred, ok := identMeta.Get(api.KeyCredential); ok {
		ok, err = cred.CompareHashAndCredential(hashCred, identMeta.Zid, ident, credential)
		if err != nil {
			uc.log.Info().Str("ident", ident).Err(err).Msg("Error while comparing credentials")
			return nil, err
		}
		if ok {
			token, err2 := uc.token.GetToken(identMeta, d, k)
			if err2 != nil {
				uc.log.Info().Str("ident", ident).Err(err).Msg("Unable to produce authentication token")
				return nil, err2
			}
			uc.log.Info().Str("user", ident).Msg("Successful")
			return token, nil
		}
		uc.log.Info().Str("ident", ident).Msg("Credentials don't match")
		return nil, nil
	}
	uc.log.Info().Str("ident", ident).Msg("No credential stored")
	compensateCompare()
	return nil, nil
}








>
>
>
|




|







|











|







46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
		token:     token,
		port:      port,
		ucGetUser: NewGetUser(authz, port),
	}
}

// Run executes the use case.
//
// Parameter "r" is just included to produce better logging messages. It may be nil. Do not use it
// for other purposes.
func (uc *Authenticate) Run(ctx context.Context, r *http.Request, ident, credential string, d time.Duration, k auth.TokenKind) ([]byte, error) {
	identMeta, err := uc.ucGetUser.Run(ctx, ident)
	defer addDelay(time.Now(), 500*time.Millisecond, 100*time.Millisecond)

	if identMeta == nil || err != nil {
		uc.log.Info().Str("ident", ident).Err(err).HTTPIP(r).Msg("No user with given ident found")
		compensateCompare()
		return nil, err
	}

	if hashCred, ok := identMeta.Get(api.KeyCredential); ok {
		ok, err = cred.CompareHashAndCredential(hashCred, identMeta.Zid, ident, credential)
		if err != nil {
			uc.log.Info().Str("ident", ident).Err(err).HTTPIP(r).Msg("Error while comparing credentials")
			return nil, err
		}
		if ok {
			token, err2 := uc.token.GetToken(identMeta, d, k)
			if err2 != nil {
				uc.log.Info().Str("ident", ident).Err(err).Msg("Unable to produce authentication token")
				return nil, err2
			}
			uc.log.Info().Str("user", ident).Msg("Successful")
			return token, nil
		}
		uc.log.Info().Str("ident", ident).HTTPIP(r).Msg("Credentials don't match")
		return nil, nil
	}
	uc.log.Info().Str("ident", ident).Msg("No credential stored")
	compensateCompare()
	return nil, nil
}

Changes to usecase/evaluate.go.

16
17
18
19
20
21
22

23
24
25
26
27
28
29

30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/parser"

)

// Evaluate is the data for this use case.
type Evaluate struct {
	rtConfig  config.Config
	getZettel GetZettel
	getMeta   GetMeta

}

// NewEvaluate creates a new use case.
func NewEvaluate(rtConfig config.Config, getZettel GetZettel, getMeta GetMeta) Evaluate {
	return Evaluate{
		rtConfig:  rtConfig,
		getZettel: getZettel,
		getMeta:   getMeta,

	}
}

// Run executes the use case.
func (uc *Evaluate) Run(ctx context.Context, zid id.Zid, syntax string) (*ast.ZettelNode, error) {
	zettel, err := uc.getZettel.Run(ctx, zid)
	if err != nil {







>







>



|




>







16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/evaluator"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/search"
)

// Evaluate is the data for this use case.
type Evaluate struct {
	rtConfig  config.Config
	getZettel GetZettel
	getMeta   GetMeta
	listMeta  ListMeta
}

// NewEvaluate creates a new use case.
func NewEvaluate(rtConfig config.Config, getZettel GetZettel, getMeta GetMeta, listMeta ListMeta) Evaluate {
	return Evaluate{
		rtConfig:  rtConfig,
		getZettel: getZettel,
		getMeta:   getMeta,
		listMeta:  listMeta,
	}
}

// Run executes the use case.
func (uc *Evaluate) Run(ctx context.Context, zid id.Zid, syntax string) (*ast.ZettelNode, error) {
	zettel, err := uc.getZettel.Run(ctx, zid)
	if err != nil {
55
56
57
58
59
60
61







62
63
64
65
66
67
68
69
70
71






// RunMetadata executes the use case for a metadata value.
func (uc *Evaluate) RunMetadata(ctx context.Context, value string) ast.InlineSlice {
	is := parser.ParseMetadata(value)
	evaluator.EvaluateInline(ctx, uc, uc.rtConfig, &is)
	return is
}








// GetMeta retrieves the metadata of a given zettel identifier.
func (uc *Evaluate) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	return uc.getMeta.Run(ctx, zid)
}

// GetZettel retrieves the full zettel of a given zettel identifier.
func (uc *Evaluate) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	return uc.getZettel.Run(ctx, zid)
}












>
>
>
>
>
>
>










>
>
>
>
>
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

// RunMetadata executes the use case for a metadata value.
func (uc *Evaluate) RunMetadata(ctx context.Context, value string) ast.InlineSlice {
	is := parser.ParseMetadata(value)
	evaluator.EvaluateInline(ctx, uc, uc.rtConfig, &is)
	return is
}

// RunMetadataNoLink executes the use case for a metadata value, but ignores link and footnote nodes.
func (uc *Evaluate) RunMetadataNoLink(ctx context.Context, value string) ast.InlineSlice {
	is := parser.ParseMetadataNoLink(value)
	evaluator.EvaluateInline(ctx, uc, uc.rtConfig, &is)
	return is
}

// GetMeta retrieves the metadata of a given zettel identifier.
func (uc *Evaluate) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	return uc.getMeta.Run(ctx, zid)
}

// GetZettel retrieves the full zettel of a given zettel identifier.
func (uc *Evaluate) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	return uc.getZettel.Run(ctx, zid)
}

// SelectMeta returns a list of metadata that comply to the given selection criteria.
func (uc *Evaluate) SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) {
	return uc.listMeta.Run(ctx, s)
}

Changes to usecase/get_user.go.

49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
	// could give herself the same ''ident''. Second, in most cases the owner
	// will authenticate.
	identMeta, err := uc.port.GetMeta(ctx, uc.authz.Owner())
	if err == nil && identMeta.GetDefault(api.KeyUserID, "") == ident {
		return identMeta, nil
	}
	// Owner was not found or has another ident. Try via list search.
	var s *search.Search
	s = s.AddExpr("", "="+ident)
	s = s.AddExpr(api.KeyUserID, ident)
	metaList, err := uc.port.SelectMeta(ctx, s)
	if err != nil {
		return nil, err
	}
	if len(metaList) < 1 {
		return nil, nil
	}







<
<
|







49
50
51
52
53
54
55


56
57
58
59
60
61
62
63
	// could give herself the same ''ident''. Second, in most cases the owner
	// will authenticate.
	identMeta, err := uc.port.GetMeta(ctx, uc.authz.Owner())
	if err == nil && identMeta.GetDefault(api.KeyUserID, "") == ident {
		return identMeta, nil
	}
	// Owner was not found or has another ident. Try via list search.


	s := search.Parse(api.KeyUserID + ":" + ident + " :" + ident)
	metaList, err := uc.port.SelectMeta(ctx, s)
	if err != nil {
		return nil, err
	}
	if len(metaList) < 1 {
		return nil, nil
	}

Changes to usecase/lists.go.

57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
// NewListSyntax creates a new use case.
func NewListSyntax(port ListSyntaxPort) ListSyntax {
	return ListSyntax{port: port}
}

// Run executes the use case.
func (uc ListSyntax) Run(ctx context.Context) (meta.Arrangement, error) {
	var s *search.Search
	s = s.AddExpr(api.KeySyntax, "") // We look for all metadata with a syntax key
	metas, err := uc.port.SelectMeta(box.NoEnrichContext(ctx), s)
	if err != nil {
		return nil, err
	}
	result := meta.CreateArrangement(metas, api.KeySyntax)
	for _, syn := range parser.GetSyntaxes() {
		if _, found := result[syn]; !found {







<
|







57
58
59
60
61
62
63

64
65
66
67
68
69
70
71
// NewListSyntax creates a new use case.
func NewListSyntax(port ListSyntaxPort) ListSyntax {
	return ListSyntax{port: port}
}

// Run executes the use case.
func (uc ListSyntax) Run(ctx context.Context) (meta.Arrangement, error) {

	s := search.Parse(api.KeySyntax + ":") // We look for all metadata with a syntax key
	metas, err := uc.port.SelectMeta(box.NoEnrichContext(ctx), s)
	if err != nil {
		return nil, err
	}
	result := meta.CreateArrangement(metas, api.KeySyntax)
	for _, syn := range parser.GetSyntaxes() {
		if _, found := result[syn]; !found {
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
// NewListRoles creates a new use case.
func NewListRoles(port ListRolesPort) ListRoles {
	return ListRoles{port: port}
}

// Run executes the use case.
func (uc ListRoles) Run(ctx context.Context) (meta.Arrangement, error) {
	var s *search.Search
	s = s.AddExpr(api.KeyRole, "") // We look for all metadata with a role key
	metas, err := uc.port.SelectMeta(box.NoEnrichContext(ctx), s)
	if err != nil {
		return nil, err
	}
	return meta.CreateArrangement(metas, api.KeyRole), nil
}








<
|







91
92
93
94
95
96
97

98
99
100
101
102
103
104
105
// NewListRoles creates a new use case.
func NewListRoles(port ListRolesPort) ListRoles {
	return ListRoles{port: port}
}

// Run executes the use case.
func (uc ListRoles) Run(ctx context.Context) (meta.Arrangement, error) {

	s := search.Parse(api.KeyRole + ":") // We look for all metadata with a role key
	metas, err := uc.port.SelectMeta(box.NoEnrichContext(ctx), s)
	if err != nil {
		return nil, err
	}
	return meta.CreateArrangement(metas, api.KeyRole), nil
}

121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
// NewListTags creates a new use case.
func NewListTags(port ListTagsPort) ListTags {
	return ListTags{port: port}
}

// Run executes the use case.
func (uc ListTags) Run(ctx context.Context, minCount int) (meta.Arrangement, error) {
	var s *search.Search
	s = s.AddExpr(api.KeyTags, "") // We look for all metadata with a tag
	metas, err := uc.port.SelectMeta(ctx, s)
	if err != nil {
		return nil, err
	}
	result := meta.CreateArrangement(metas, api.KeyAllTags)
	if minCount > 1 {
		for t, ms := range result {







<
|







119
120
121
122
123
124
125

126
127
128
129
130
131
132
133
// NewListTags creates a new use case.
func NewListTags(port ListTagsPort) ListTags {
	return ListTags{port: port}
}

// Run executes the use case.
func (uc ListTags) Run(ctx context.Context, minCount int) (meta.Arrangement, error) {

	s := search.Parse(api.KeyAllTags + ":") // We look for all metadata with a tag
	metas, err := uc.port.SelectMeta(ctx, s)
	if err != nil {
		return nil, err
	}
	result := meta.CreateArrangement(metas, api.KeyAllTags)
	if minCount > 1 {
		for t, ms := range result {

Changes to usecase/unlinked_refs.go.

47
48
49
50
51
52
53
54
55
56
57
58

59

60
61

62
63
64
65
66
67
68
		port:     port,
		rtConfig: rtConfig,
		encText:  textenc.Create(),
	}
}

// Run executes the usecase with already evaluated title value.
func (uc *UnlinkedReferences) Run(ctx context.Context, title string, s *search.Search) ([]*meta.Meta, error) {
	words := makeWords(title)
	if len(words) == 0 {
		return nil, nil
	}

	for _, word := range words {

		s = s.AddExpr("", "="+word)
	}


	// Limit applies to the filtering process, not to SelectMeta
	limit := s.GetLimit()
	s = s.SetLimit(0)

	candidates, err := uc.port.SelectMeta(ctx, s)
	if err != nil {







|
|



>

>
|

>







47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
		port:     port,
		rtConfig: rtConfig,
		encText:  textenc.Create(),
	}
}

// Run executes the usecase with already evaluated title value.
func (uc *UnlinkedReferences) Run(ctx context.Context, phrase string, s *search.Search) ([]*meta.Meta, error) {
	words := makeWords(phrase)
	if len(words) == 0 {
		return nil, nil
	}
	var sb strings.Builder
	for _, word := range words {
		sb.WriteString(" :")
		sb.WriteString(word)
	}
	s = s.Parse(sb.String())

	// Limit applies to the filtering process, not to SelectMeta
	limit := s.GetLimit()
	s = s.SetLimit(0)

	candidates, err := uc.port.SelectMeta(ctx, s)
	if err != nil {

Changes to web/adapter/api/get_parsed_zettel.go.

49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
func (a *API) writeEncodedZettelPart(
	w http.ResponseWriter, zn *ast.ZettelNode,
	evalMeta encoder.EvalMetaFunc,
	enc api.EncodingEnum, encStr string, part partType,
) {
	encdr := encoder.Create(enc)
	if encdr == nil {
		adapter.BadRequest(w, fmt.Sprintf("Zettel %q not available in encoding %q", zn.Meta.Zid.String(), encStr))
		return
	}
	var err error
	var buf bytes.Buffer
	switch part {
	case partZettel:
		_, err = encdr.WriteZettel(&buf, zn, evalMeta)







|







49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
func (a *API) writeEncodedZettelPart(
	w http.ResponseWriter, zn *ast.ZettelNode,
	evalMeta encoder.EvalMetaFunc,
	enc api.EncodingEnum, encStr string, part partType,
) {
	encdr := encoder.Create(enc)
	if encdr == nil {
		adapter.BadRequest(w, fmt.Sprintf("Zettel %q not available in encoding %q", zn.Meta.Zid, encStr))
		return
	}
	var err error
	var buf bytes.Buffer
	switch part {
	case partZettel:
		_, err = encdr.WriteZettel(&buf, zn, evalMeta)

Changes to web/adapter/api/get_zettel_list.go.

41
42
43
44
45
46
47

48
49
50
51
52
53
54
				Rights: a.getRights(ctx, m),
			})
		}

		var buf bytes.Buffer
		err = encodeJSONData(&buf, api.ZettelListJSON{
			Query: s.String(),

			List:  result,
		})
		if err != nil {
			a.log.Fatal().Err(err).Msg("Unable to store meta list in buffer")
			http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
			return
		}







>







41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
				Rights: a.getRights(ctx, m),
			})
		}

		var buf bytes.Buffer
		err = encodeJSONData(&buf, api.ZettelListJSON{
			Query: s.String(),
			Human: s.Human(),
			List:  result,
		})
		if err != nil {
			a.log.Fatal().Err(err).Msg("Unable to store meta list in buffer")
			http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
			return
		}

Changes to web/adapter/api/login.go.

1
2
3
4
5
6
7
8
9
10
11
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package api



|







1
2
3
4
5
6
7
8
9
10
11
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2022 Detlef Stern
//
// This file is part of Zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package api
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
			err := a.writeJSONToken(w, "freeaccess", 24*366*10*time.Hour)
			a.log.IfErr(err).Msg("Login/free")
			return
		}
		var token []byte
		if ident, cred := retrieveIdentCred(r); ident != "" {
			var err error
			token, err = ucAuth.Run(r.Context(), ident, cred, a.tokenLifetime, auth.KindJSON)
			if err != nil {
				a.reportUsecaseError(w, err)
				return
			}
		}
		if len(token) == 0 {
			w.Header().Set("WWW-Authenticate", `Bearer realm="Default"`)







|







29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
			err := a.writeJSONToken(w, "freeaccess", 24*366*10*time.Hour)
			a.log.IfErr(err).Msg("Login/free")
			return
		}
		var token []byte
		if ident, cred := retrieveIdentCred(r); ident != "" {
			var err error
			token, err = ucAuth.Run(r.Context(), r, ident, cred, a.tokenLifetime, auth.KindJSON)
			if err != nil {
				a.reportUsecaseError(w, err)
				return
			}
		}
		if len(token) == 0 {
			w.Header().Set("WWW-Authenticate", `Bearer realm="Default"`)

Changes to web/adapter/request.go.

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129



130
131
132
133



134
135
136

137


138
139
140
141
142
			return val, true
		}
	}
	return 0, false
}

// GetSearch retrieves the specified search and sorting options from a query.
func GetSearch(q url.Values) (s *search.Search) {
	for key, values := range q {
		switch key {
		case api.QueryKeySort, api.QueryKeyOrder:
			s = extractOrderFromQuery(values, s)
		case api.QueryKeyOffset:
			s = extractOffsetFromQuery(values, s)
		case api.QueryKeyLimit:
			s = extractLimitFromQuery(values, s)
		case api.QueryKeyNegate:
			s = s.SetNegate()
		case api.QueryKeySearch:
			s = setCleanedQueryValues(s, "", values)
		default:
			if meta.KeyIsValid(key) {
				s = setCleanedQueryValues(s, key, values)
			}
		}
	}
	return s
}

func extractOrderFromQuery(values []string, s *search.Search) *search.Search {
	if len(values) > 0 {
		descending := false
		sortkey := values[0]
		if strings.HasPrefix(sortkey, "-") {
			descending = true
			sortkey = sortkey[1:]
		}
		if meta.KeyIsValid(sortkey) || sortkey == search.RandomOrder {
			s = s.AddOrder(sortkey, descending)
		}
	}
	return s
}

func extractOffsetFromQuery(values []string, s *search.Search) *search.Search {
	if len(values) > 0 {
		if offset, err := strconv.Atoi(values[0]); err == nil && offset > 0 {
			s = s.SetOffset(offset)
		}
	}
	return s
}

func extractLimitFromQuery(values []string, s *search.Search) *search.Search {
	if len(values) > 0 {
		if limit, err := strconv.Atoi(values[0]); err == nil && limit > 0 {
			s = s.SetLimit(limit)
		}
	}
	return s
}

func setCleanedQueryValues(s *search.Search, key string, values []string) *search.Search {
	for _, val := range values {
		s = s.AddExpr(key, val)
	}
	return s
}

// GetZCDirection returns a direction value for a given string.
func GetZCDirection(s string) usecase.ZettelContextDirection {
	switch s {
	case api.DirBackward:
		return usecase.ZettelContextBackward
	case api.DirForward:
		return usecase.ZettelContextForward
	}
	return usecase.ZettelContextBoth
}

// AddUnlinkedRefsToSearch inspects metadata and enhances the given search to ignore
// some zettel identifier.
func AddUnlinkedRefsToSearch(s *search.Search, m *meta.Meta) *search.Search {



	s = s.AddExpr(api.KeyID, "!="+m.Zid.String())
	for _, pair := range m.ComputedPairsRest() {
		switch meta.Type(pair.Key) {
		case meta.TypeID:



			s = s.AddExpr(api.KeyID, "!="+pair.Value)
		case meta.TypeIDSet:
			for _, value := range meta.ListFromValue(pair.Value) {

				s = s.AddExpr(api.KeyID, "!="+value)


			}
		}
	}
	return s
}







|
<
<
<
<
<
<
<
<
<
<
|
<
<
<
<
<
<
<
|
|
<
<
<
<
<
<
<
<
<
<
<
<
<
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
















>
>
>
|



>
>
>
|


>
|
>
>



|

47
48
49
50
51
52
53
54










55







56
57













58

























59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
			return val, true
		}
	}
	return 0, false
}

// GetSearch retrieves the specified search and sorting options from a query.
func GetSearch(q url.Values) *search.Search {










	if exprs, found := q[api.QueryKeySearch]; found {







		return search.Parse(strings.Join(exprs, " "))
	}













	return nil

























}

// GetZCDirection returns a direction value for a given string.
func GetZCDirection(s string) usecase.ZettelContextDirection {
	switch s {
	case api.DirBackward:
		return usecase.ZettelContextBackward
	case api.DirForward:
		return usecase.ZettelContextForward
	}
	return usecase.ZettelContextBoth
}

// AddUnlinkedRefsToSearch inspects metadata and enhances the given search to ignore
// some zettel identifier.
func AddUnlinkedRefsToSearch(s *search.Search, m *meta.Meta) *search.Search {
	var sb strings.Builder
	sb.WriteString(api.KeyID)
	sb.WriteString("!:")
	sb.WriteString(m.Zid.String())
	for _, pair := range m.ComputedPairsRest() {
		switch meta.Type(pair.Key) {
		case meta.TypeID:
			sb.WriteByte(' ')
			sb.WriteString(api.KeyID)
			sb.WriteString("!:")
			sb.WriteString(pair.Value)
		case meta.TypeIDSet:
			for _, value := range meta.ListFromValue(pair.Value) {
				sb.WriteByte(' ')
				sb.WriteString(api.KeyID)
				sb.WriteString("!:")
				sb.WriteString(value)
			}
		}
	}
	return s.Parse(sb.String())
}

Changes to web/adapter/webui/create_zettel.go.

55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
			m := origZettel.Meta
			title := parser.ParseMetadata(m.GetTitle())
			textTitle, err2 := encodeInlinesText(&title, wui.gentext)
			if err2 != nil {
				wui.reportError(ctx, w, err2)
				return
			}
			htmlTitle, err2 := wui.getSimpleHTMLEncoder().InlinesString(&title, false)
			if err2 != nil {
				wui.reportError(ctx, w, err2)
				return
			}
			wui.renderZettelForm(ctx, w, createZettel.PrepareNew(origZettel), textTitle, htmlTitle, roleData, syntaxData)
		}
	}







|







55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
			m := origZettel.Meta
			title := parser.ParseMetadata(m.GetTitle())
			textTitle, err2 := encodeInlinesText(&title, wui.gentext)
			if err2 != nil {
				wui.reportError(ctx, w, err2)
				return
			}
			htmlTitle, err2 := wui.getSimpleHTMLEncoder().InlinesString(&title)
			if err2 != nil {
				wui.reportError(ctx, w, err2)
				return
			}
			wui.renderZettelForm(ctx, w, createZettel.PrepareNew(origZettel), textTitle, htmlTitle, roleData, syntaxData)
		}
	}

Changes to web/adapter/webui/get_info.go.

74
75
76
77
78
79
80
81





82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
				func(val string) ast.InlineSlice {
					return evaluate.RunMetadata(ctx, val)
				},
				enc)
			metaData[i] = metaDataInfo{p.Key, buf.String()}
		}
		summary := collect.References(zn)
		locLinks, extLinks := splitLocExtLinks(append(summary.Links, summary.Embeds...))






		textTitle := wui.encodeTitleAsText(ctx, zn.InhMeta, evaluate)
		phrase := q.Get(api.QueryKeyPhrase)
		if phrase == "" {
			phrase = textTitle
		}
		phrase = strings.TrimSpace(phrase)
		unlinkedMeta, err := unlinkedRefs.Run(
			ctx, phrase, adapter.AddUnlinkedRefsToSearch(nil, zn.InhMeta))
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		unLinks := wui.buildHTMLMetaList(ctx, unlinkedMeta, evaluate)

		shadowLinks := getShadowLinks(ctx, zid, getAllMeta)
		endnotes, err := enc.BlocksString(&ast.BlockSlice{})
		if err != nil {
			endnotes = ""
		}








|
>
>
>
>
>













|







74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
				func(val string) ast.InlineSlice {
					return evaluate.RunMetadata(ctx, val)
				},
				enc)
			metaData[i] = metaDataInfo{p.Key, buf.String()}
		}
		summary := collect.References(zn)
		locLinks, searchQuery, extLinks := splitLocSeaExtLinks(append(summary.Links, summary.Embeds...))
		searchLinks := make([]simpleLink, len(searchQuery))
		for i, sq := range searchQuery {
			searchLinks[i].Text = sq
			searchLinks[i].URL = wui.NewURLBuilder('h').AppendSearch(sq).String()
		}

		textTitle := wui.encodeTitleAsText(ctx, zn.InhMeta, evaluate)
		phrase := q.Get(api.QueryKeyPhrase)
		if phrase == "" {
			phrase = textTitle
		}
		phrase = strings.TrimSpace(phrase)
		unlinkedMeta, err := unlinkedRefs.Run(
			ctx, phrase, adapter.AddUnlinkedRefsToSearch(nil, zn.InhMeta))
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		unLinks := wui.buildHTMLMetaList(unlinkedMeta, func(val string) ast.InlineSlice { return evaluate.RunMetadata(ctx, val) })

		shadowLinks := getShadowLinks(ctx, zid, getAllMeta)
		endnotes, err := enc.BlocksString(&ast.BlockSlice{})
		if err != nil {
			endnotes = ""
		}

118
119
120
121
122
123
124


125
126
127
128
129
130
131
			CanRename      bool
			RenameURL      string
			CanDelete      bool
			DeleteURL      string
			MetaData       []metaDataInfo
			HasLocLinks    bool
			LocLinks       []localLink


			HasExtLinks    bool
			ExtLinks       []string
			ExtNewWindow   string
			UnLinks        []simpleLink
			UnLinksPhrase  string
			QueryKeyPhrase string
			EvalMatrix     []matrixLine







>
>







123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
			CanRename      bool
			RenameURL      string
			CanDelete      bool
			DeleteURL      string
			MetaData       []metaDataInfo
			HasLocLinks    bool
			LocLinks       []localLink
			HasSearchLinks bool
			SearchLinks    []simpleLink
			HasExtLinks    bool
			ExtLinks       []string
			ExtNewWindow   string
			UnLinks        []simpleLink
			UnLinksPhrase  string
			QueryKeyPhrase string
			EvalMatrix     []matrixLine
146
147
148
149
150
151
152


153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181

182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
			CanRename:      wui.canRename(ctx, user, zn.Meta),
			RenameURL:      wui.NewURLBuilder('b').SetZid(apiZid).String(),
			CanDelete:      wui.canDelete(ctx, user, zn.Meta),
			DeleteURL:      wui.NewURLBuilder('d').SetZid(apiZid).String(),
			MetaData:       metaData,
			HasLocLinks:    len(locLinks) > 0,
			LocLinks:       locLinks,


			HasExtLinks:    len(extLinks) > 0,
			ExtLinks:       extLinks,
			ExtNewWindow:   htmlAttrNewWindow(len(extLinks) > 0),
			UnLinks:        unLinks,
			UnLinksPhrase:  phrase,
			QueryKeyPhrase: api.QueryKeyPhrase,
			EvalMatrix:     wui.infoAPIMatrix('v', zid),
			ParseMatrix:    wui.infoAPIMatrixPlain('p', zid),
			HasShadowLinks: len(shadowLinks) > 0,
			ShadowLinks:    shadowLinks,
			Endnotes:       endnotes,
		})
	}
}

type localLink struct {
	Valid bool
	Zid   string
}

func splitLocExtLinks(links []*ast.Reference) (locLinks []localLink, extLinks []string) {
	if len(links) == 0 {
		return nil, nil
	}
	for _, ref := range links {
		if ref.State == ast.RefStateSelf {
			continue
		}
		if ref.IsZettel() {

			continue
		}
		if ref.IsExternal() {
			extLinks = append(extLinks, ref.String())
			continue
		}
		locLinks = append(locLinks, localLink{ref.IsValid(), ref.String()})
	}
	return locLinks, extLinks
}

func (wui *WebUI) infoAPIMatrix(key byte, zid id.Zid) []matrixLine {
	encodings := encoder.GetEncodings()
	encTexts := make([]string, 0, len(encodings))
	for _, f := range encodings {
		encTexts = append(encTexts, f.String())







>
>




















|

|


|


|
>








|







153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
			CanRename:      wui.canRename(ctx, user, zn.Meta),
			RenameURL:      wui.NewURLBuilder('b').SetZid(apiZid).String(),
			CanDelete:      wui.canDelete(ctx, user, zn.Meta),
			DeleteURL:      wui.NewURLBuilder('d').SetZid(apiZid).String(),
			MetaData:       metaData,
			HasLocLinks:    len(locLinks) > 0,
			LocLinks:       locLinks,
			HasSearchLinks: len(searchQuery) > 0,
			SearchLinks:    searchLinks,
			HasExtLinks:    len(extLinks) > 0,
			ExtLinks:       extLinks,
			ExtNewWindow:   htmlAttrNewWindow(len(extLinks) > 0),
			UnLinks:        unLinks,
			UnLinksPhrase:  phrase,
			QueryKeyPhrase: api.QueryKeyPhrase,
			EvalMatrix:     wui.infoAPIMatrix('v', zid),
			ParseMatrix:    wui.infoAPIMatrixPlain('p', zid),
			HasShadowLinks: len(shadowLinks) > 0,
			ShadowLinks:    shadowLinks,
			Endnotes:       endnotes,
		})
	}
}

type localLink struct {
	Valid bool
	Zid   string
}

func splitLocSeaExtLinks(links []*ast.Reference) (locLinks []localLink, searchQuery, extLinks []string) {
	if len(links) == 0 {
		return nil, nil, nil
	}
	for _, ref := range links {
		if ref.State == ast.RefStateSelf || ref.IsZettel() {
			continue
		}
		if ref.State == ast.RefStateSearch {
			searchQuery = append(searchQuery, ref.Value)
			continue
		}
		if ref.IsExternal() {
			extLinks = append(extLinks, ref.String())
			continue
		}
		locLinks = append(locLinks, localLink{ref.IsValid(), ref.String()})
	}
	return locLinks, searchQuery, extLinks
}

func (wui *WebUI) infoAPIMatrix(key byte, zid id.Zid) []matrixLine {
	encodings := encoder.GetEncodings()
	encTexts := make([]string, 0, len(encodings))
	for _, f := range encodings {
		encTexts = append(encTexts, f.String())

Changes to web/adapter/webui/get_zettel.go.

39
40
41
42
43
44
45
46
47
48


49
50
51
52
53
54
55
56
57

		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}

		enc := wui.createZettelEncoder()
		metaHeader := enc.MetaString(zn.InhMeta, func(value string) ast.InlineSlice {
			return evaluate.RunMetadata(ctx, value)
		})


		textTitle := wui.encodeTitleAsText(ctx, zn.InhMeta, evaluate)
		htmlTitle := encodeTitleAsHTML(ctx, zn.InhMeta, evaluate, enc, false)
		htmlContent, err := enc.BlocksString(&zn.Ast)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		var roleCSSURL string
		cssZid, err := wui.retrieveCSSZidFromRole(ctx, *zn.InhMeta)







|

<
>
>

|







39
40
41
42
43
44
45
46
47

48
49
50
51
52
53
54
55
56
57
58

		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}

		enc := wui.createZettelEncoder()
		evalMetadata := func(value string) ast.InlineSlice {
			return evaluate.RunMetadata(ctx, value)

		}
		metaHeader := enc.MetaString(zn.InhMeta, evalMetadata)
		textTitle := wui.encodeTitleAsText(ctx, zn.InhMeta, evaluate)
		htmlTitle := encodeZmkMetadata(zn.InhMeta.GetTitle(), evalMetadata, enc)
		htmlContent, err := enc.BlocksString(&zn.Ast)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		var roleCSSURL string
		cssZid, err := wui.retrieveCSSZidFromRole(ctx, *zn.InhMeta)
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
			HTMLTitle:     htmlTitle,
			RoleCSS:       roleCSSURL,
			CanWrite:      wui.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:       wui.NewURLBuilder('e').SetZid(apiZid).String(),
			Zid:           zid.String(),
			InfoURL:       wui.NewURLBuilder('i').SetZid(apiZid).String(),
			RoleText:      roleText,
			RoleURL:       wui.NewURLBuilder('h').AppendQuery("role", roleText).String(),
			HasTags:       len(tags) > 0,
			Tags:          tags,
			CanCopy:       canCreate && !zn.Content.IsBinary(),
			CopyURL:       wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionCopy).String(),
			CanFolge:      canCreate,
			FolgeURL:      wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionFolge).String(),
			PrecursorRefs: wui.encodeIdentifierSet(zn.InhMeta, api.KeyPrecursor, getTextTitle),







|







103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
			HTMLTitle:     htmlTitle,
			RoleCSS:       roleCSSURL,
			CanWrite:      wui.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:       wui.NewURLBuilder('e').SetZid(apiZid).String(),
			Zid:           zid.String(),
			InfoURL:       wui.NewURLBuilder('i').SetZid(apiZid).String(),
			RoleText:      roleText,
			RoleURL:       wui.NewURLBuilder('h').AppendSearch("role:" + roleText).String(),
			HasTags:       len(tags) > 0,
			Tags:          tags,
			CanCopy:       canCreate && !zn.Content.IsBinary(),
			CopyURL:       wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionCopy).String(),
			CanFolge:      canCreate,
			FolgeURL:      wui.NewURLBuilder('c').SetZid(apiZid).AppendQuery(queryKeyAction, valueActionFolge).String(),
			PrecursorRefs: wui.encodeIdentifierSet(zn.InhMeta, api.KeyPrecursor, getTextTitle),
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155

func (wui *WebUI) buildTagInfos(m *meta.Meta) []simpleLink {
	var tagInfos []simpleLink
	if tags, ok := m.GetList(api.KeyTags); ok {
		ub := wui.NewURLBuilder('h')
		tagInfos = make([]simpleLink, len(tags))
		for i, tag := range tags {
			tagInfos[i] = simpleLink{Text: tag, URL: ub.AppendQuery(api.KeyAllTags, tag).String()}
			ub.ClearQuery()
		}
	}
	return tagInfos
}

func (wui *WebUI) encodeIdentifierSet(m *meta.Meta, key string, getTextTitle getTextTitleFunc) string {







|







142
143
144
145
146
147
148
149
150
151
152
153
154
155
156

func (wui *WebUI) buildTagInfos(m *meta.Meta) []simpleLink {
	var tagInfos []simpleLink
	if tags, ok := m.GetList(api.KeyTags); ok {
		ub := wui.NewURLBuilder('h')
		tagInfos = make([]simpleLink, len(tags))
		for i, tag := range tags {
			tagInfos[i] = simpleLink{Text: tag, URL: ub.AppendSearch(api.KeyAllTags + ":" + tag).String()}
			ub.ClearQuery()
		}
	}
	return tagInfos
}

func (wui *WebUI) encodeIdentifierSet(m *meta.Meta, key string, getTextTitle getTextTitleFunc) string {

Changes to web/adapter/webui/htmlgen.go.

19
20
21
22
23
24
25

26
27
28
29
30
31
32
	"zettelstore.de/c/html"
	"zettelstore.de/c/sexpr"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/sexprenc"
	"zettelstore.de/z/encoder/textenc"

	"zettelstore.de/z/strfun"
)

// Builder allows to build new URLs for the web service.
type urlBuilder interface {
	GetURLPrefix() string
	NewURLBuilder(key byte) *api.URLBuilder







>







19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
	"zettelstore.de/c/html"
	"zettelstore.de/c/sexpr"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/sexprenc"
	"zettelstore.de/z/encoder/textenc"
	"zettelstore.de/z/search"
	"zettelstore.de/z/strfun"
)

// Builder allows to build new URLs for the web service.
type urlBuilder interface {
	GetURLPrefix() string
	NewURLBuilder(key byte) *api.URLBuilder
50
51
52
53
54
55
56

57
58
59
60
61
62
63
		env:       env,
	}

	env.Builtins.Set(sexpr.SymTag, sxpf.NewBuiltin("tag", true, 0, -1, gen.generateTag))
	env.Builtins.Set(sexpr.SymLinkZettel, sxpf.NewBuiltin("linkZ", true, 2, -1, gen.generateLinkZettel))
	env.Builtins.Set(sexpr.SymLinkFound, sxpf.NewBuiltin("linkZ", true, 2, -1, gen.generateLinkZettel))
	env.Builtins.Set(sexpr.SymLinkBased, sxpf.NewBuiltin("linkB", true, 2, -1, gen.generateLinkBased))

	env.Builtins.Set(sexpr.SymLinkExternal, sxpf.NewBuiltin("linkE", true, 2, -1, gen.generateLinkExternal))

	f, err := env.Builtins.LookupForm(sexpr.SymEmbed)
	if err != nil {
		panic(err)
	}
	b := f.(*sxpf.Builtin)







>







51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
		env:       env,
	}

	env.Builtins.Set(sexpr.SymTag, sxpf.NewBuiltin("tag", true, 0, -1, gen.generateTag))
	env.Builtins.Set(sexpr.SymLinkZettel, sxpf.NewBuiltin("linkZ", true, 2, -1, gen.generateLinkZettel))
	env.Builtins.Set(sexpr.SymLinkFound, sxpf.NewBuiltin("linkZ", true, 2, -1, gen.generateLinkZettel))
	env.Builtins.Set(sexpr.SymLinkBased, sxpf.NewBuiltin("linkB", true, 2, -1, gen.generateLinkBased))
	env.Builtins.Set(sexpr.SymLinkSearch, sxpf.NewBuiltin("linkS", true, 2, -1, gen.generateLinkSearch))
	env.Builtins.Set(sexpr.SymLinkExternal, sxpf.NewBuiltin("linkE", true, 2, -1, gen.generateLinkExternal))

	f, err := env.Builtins.LookupForm(sexpr.SymEmbed)
	if err != nil {
		panic(err)
	}
	b := f.(*sxpf.Builtin)
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
		g.env.WriteEndnotes()
	}
	g.env.ReplaceWriter(nil)
	return buf.String(), g.env.GetError()
}

// InlinesString writes an inline slice to the writer
func (g *htmlGenerator) InlinesString(is *ast.InlineSlice, noLink bool) (string, error) {
	if is == nil || len(*is) == 0 {
		return "", nil
	}
	return html.EvaluateInline(g.env, sexprenc.GetSexpr(is), !noLink, noLink), nil
}

func (g *htmlGenerator) generateTag(senv sxpf.Environment, args *sxpf.Pair, _ int) (sxpf.Value, error) {
	if !sxpf.IsNil(args) {
		env := senv.(*html.EncEnvironment)
		s := env.GetString(args)
		if env.IgnoreLinks() {
			env.WriteEscaped(s)
		} else {
			u := g.builder.NewURLBuilder('h').AppendQuery(api.KeyAllTags, "#"+strings.ToLower(s))
			env.WriteStrings(`<a href="`, u.String(), `">#`)
			env.WriteEscaped(s)
			env.WriteString("</a>")
		}
	}
	return nil, nil
}







|



|









|







130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
		g.env.WriteEndnotes()
	}
	g.env.ReplaceWriter(nil)
	return buf.String(), g.env.GetError()
}

// InlinesString writes an inline slice to the writer
func (g *htmlGenerator) InlinesString(is *ast.InlineSlice) (string, error) {
	if is == nil || len(*is) == 0 {
		return "", nil
	}
	return html.EvaluateInline(g.env, sexprenc.GetSexpr(is), true, false), nil
}

func (g *htmlGenerator) generateTag(senv sxpf.Environment, args *sxpf.Pair, _ int) (sxpf.Value, error) {
	if !sxpf.IsNil(args) {
		env := senv.(*html.EncEnvironment)
		s := env.GetString(args)
		if env.IgnoreLinks() {
			env.WriteEscaped(s)
		} else {
			u := g.builder.NewURLBuilder('h').AppendSearch(api.KeyAllTags + ":#" + strings.ToLower(s))
			env.WriteStrings(`<a href="`, u.String(), `">#`)
			env.WriteEscaped(s)
			env.WriteString("</a>")
		}
	}
	return nil, nil
}
172
173
174
175
176
177
178










179
180
181
182
183
184
185
	env := senv.(*html.EncEnvironment)
	if a, refValue, ok := html.PrepareLink(env, args); ok {
		u := g.builder.NewURLBuilder('/').SetRawLocal(refValue)
		html.WriteLink(env, args, a.Set("href", u.String()), refValue, "")
	}
	return nil, nil
}











func (g *htmlGenerator) generateLinkExternal(senv sxpf.Environment, args *sxpf.Pair, _ int) (sxpf.Value, error) {
	env := senv.(*html.EncEnvironment)
	if a, refValue, ok := html.PrepareLink(env, args); ok {
		a = a.Set("href", refValue).
			AddClass("external").
			Set("target", "_blank").







>
>
>
>
>
>
>
>
>
>







174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
	env := senv.(*html.EncEnvironment)
	if a, refValue, ok := html.PrepareLink(env, args); ok {
		u := g.builder.NewURLBuilder('/').SetRawLocal(refValue)
		html.WriteLink(env, args, a.Set("href", u.String()), refValue, "")
	}
	return nil, nil
}

func (g *htmlGenerator) generateLinkSearch(senv sxpf.Environment, args *sxpf.Pair, _ int) (sxpf.Value, error) {
	env := senv.(*html.EncEnvironment)
	if a, refValue, ok := html.PrepareLink(env, args); ok {
		searchExpr := search.Parse(refValue).String()
		u := g.builder.NewURLBuilder('h').AppendSearch(searchExpr)
		html.WriteLink(env, args, a.Set("href", u.String()), refValue, "")
	}
	return nil, nil
}

func (g *htmlGenerator) generateLinkExternal(senv sxpf.Environment, args *sxpf.Pair, _ int) (sxpf.Value, error) {
	env := senv.(*html.EncEnvironment)
	if a, refValue, ok := html.PrepareLink(env, args); ok {
		a = a.Set("href", refValue).
			AddClass("external").
			Set("target", "_blank").

Changes to web/adapter/webui/htmlmeta.go.

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
	case meta.TypeURL:
		writeURL(w, value)
	case meta.TypeWord:
		wui.writeWord(w, key, value)
	case meta.TypeWordSet:
		wui.writeWordSet(w, key, meta.ListFromValue(value))
	case meta.TypeZettelmarkup:
		io.WriteString(w, encodeZmkMetadata(value, evalMetadata, gen, false))
	default:
		html.Escape(w, value)
		fmt.Fprintf(w, " <b>(Unhandled type: %v, key: %v)</b>", kt, key)
	}
}

func writeCredential(w io.Writer, val string) { html.Escape(w, val) }







|







60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
	case meta.TypeURL:
		writeURL(w, value)
	case meta.TypeWord:
		wui.writeWord(w, key, value)
	case meta.TypeWordSet:
		wui.writeWordSet(w, key, meta.ListFromValue(value))
	case meta.TypeZettelmarkup:
		io.WriteString(w, encodeZmkMetadata(value, evalMetadata, gen))
	default:
		html.Escape(w, value)
		fmt.Fprintf(w, " <b>(Unhandled type: %v, key: %v)</b>", kt, key)
	}
}

func writeCredential(w io.Writer, val string) { html.Escape(w, val) }
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
			w.Write(space)
		}
		wui.writeWord(w, key, word)
	}
}

func (wui *WebUI) writeLink(w io.Writer, key, value, text string) {
	fmt.Fprintf(w, "<a href=\"%v?%v=%v\">", wui.NewURLBuilder('h'), url.QueryEscape(key), url.QueryEscape(value))
	html.Escape(w, text)
	io.WriteString(w, "</a>")
}

type getTextTitleFunc func(id.Zid) (string, int)

func (wui *WebUI) makeGetTextTitle(







|







145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
			w.Write(space)
		}
		wui.writeWord(w, key, word)
	}
}

func (wui *WebUI) writeLink(w io.Writer, key, value, text string) {
	fmt.Fprintf(w, `<a href="%v">`, wui.NewURLBuilder('h').AppendSearch(key+":"+value))
	html.Escape(w, text)
	io.WriteString(w, "</a>")
}

type getTextTitleFunc func(id.Zid) (string, int)

func (wui *WebUI) makeGetTextTitle(
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
			}
			return "", 0
		}
		return wui.encodeTitleAsText(ctx, m, evaluate), 1
	}
}

func encodeTitleAsHTML(
	ctx context.Context, m *meta.Meta,
	evaluate *usecase.Evaluate,
	gen *htmlGenerator, noLink bool,
) string {
	plainTitle := m.GetTitle()
	return encodeZmkMetadata(
		plainTitle,
		func(val string) ast.InlineSlice { return evaluate.RunMetadata(ctx, val) },
		gen, noLink)
}

func (wui *WebUI) encodeTitleAsText(ctx context.Context, m *meta.Meta, evaluate *usecase.Evaluate) string {
	plainTitle := m.GetTitle()
	is := evaluate.RunMetadata(ctx, plainTitle)
	result, err := encodeInlinesText(&is, wui.gentext)
	if err != nil {
		return err.Error()
	}
	return result
}

func encodeZmkMetadata(value string, evalMetadata evalMetadataFunc, gen *htmlGenerator, noLink bool) string {
	is := evalMetadata(value)
	result, err := gen.InlinesString(&is, noLink)
	if err != nil {
		return err.Error()
	}
	return result
}







<
<
<
<
<
<
<
<
<
<
<
<

<
|







|

|





168
169
170
171
172
173
174












175

176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
			}
			return "", 0
		}
		return wui.encodeTitleAsText(ctx, m, evaluate), 1
	}
}













func (wui *WebUI) encodeTitleAsText(ctx context.Context, m *meta.Meta, evaluate *usecase.Evaluate) string {

	is := evaluate.RunMetadata(ctx, m.GetTitle())
	result, err := encodeInlinesText(&is, wui.gentext)
	if err != nil {
		return err.Error()
	}
	return result
}

func encodeZmkMetadata(value string, evalMetadata evalMetadataFunc, gen *htmlGenerator) string {
	is := evalMetadata(value)
	result, err := gen.InlinesString(&is)
	if err != nil {
		return err.Error()
	}
	return result
}

Changes to web/adapter/webui/lists.go.

8
9
10
11
12
13
14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
// under this license.
//-----------------------------------------------------------------------------

package webui

import (
	"bytes"
	"context"
	"net/http"
	"net/url"
	"sort"
	"strconv"

	"zettelstore.de/c/api"

	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)







<






>







8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
// under this license.
//-----------------------------------------------------------------------------

package webui

import (
	"bytes"

	"net/http"
	"net/url"
	"sort"
	"strconv"

	"zettelstore.de/c/api"
	"zettelstore.de/z/ast"
	"zettelstore.de/z/box"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

73


74
75
76



77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
func (wui *WebUI) renderZettelList(
	w http.ResponseWriter, r *http.Request,
	listMeta usecase.ListMeta, evaluate *usecase.Evaluate,
) {
	query := r.URL.Query()
	s := adapter.GetSearch(query)
	ctx := r.Context()
	title := wui.listTitleSearch(s)

	if !s.EnrichNeeded() {
		ctx = box.NoEnrichContext(ctx)
	}
	metaList, err := listMeta.Run(ctx, s)
	if err != nil {
		wui.reportError(ctx, w, err)
		return
	}
	user := wui.getUser(ctx)
	metas := wui.buildHTMLMetaList(ctx, metaList, evaluate)
	var base baseData
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), "", user, &base)
	wui.renderTemplate(ctx, w, id.ListTemplateZid, &base, struct {

		Title string


		Metas []simpleLink
	}{
		Title: title,



		Metas: metas,
	})
}

type roleInfo struct {
	Text string
	URL  string
}

func (wui *WebUI) renderRolesList(w http.ResponseWriter, r *http.Request, listRole usecase.ListRoles) {
	ctx := r.Context()
	roleArrangement, err := listRole.Run(ctx)
	if err != nil {
		wui.reportError(ctx, w, err)
		return
	}
	roleList := roleArrangement.Counted()
	roleList.SortByName()

	roleInfos := make([]roleInfo, len(roleList))
	for i, role := range roleList {
		roleInfos[i] = roleInfo{role.Name, wui.NewURLBuilder('h').AppendQuery("role", role.Name).String()}
	}

	user := wui.getUser(ctx)
	var base baseData
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), "", user, &base)
	wui.renderTemplate(ctx, w, id.RolesTemplateZid, &base, struct {
		Roles []roleInfo







<
<









|



>
|
>
>
|

|
>
>
>
|




















|







51
52
53
54
55
56
57


58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
func (wui *WebUI) renderZettelList(
	w http.ResponseWriter, r *http.Request,
	listMeta usecase.ListMeta, evaluate *usecase.Evaluate,
) {
	query := r.URL.Query()
	s := adapter.GetSearch(query)
	ctx := r.Context()


	if !s.EnrichNeeded() {
		ctx = box.NoEnrichContext(ctx)
	}
	metaList, err := listMeta.Run(ctx, s)
	if err != nil {
		wui.reportError(ctx, w, err)
		return
	}
	user := wui.getUser(ctx)
	metas := wui.buildHTMLMetaList(metaList, func(val string) ast.InlineSlice { return evaluate.RunMetadataNoLink(ctx, val) })
	var base baseData
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), "", user, &base)
	wui.renderTemplate(ctx, w, id.ListTemplateZid, &base, struct {
		Title          string
		SearchURL      string
		SearchValue    string
		QueryKeySearch string
		Metas          []simpleLink
	}{
		Title:          wui.listTitleSearch(s),
		SearchURL:      base.SearchURL,
		SearchValue:    s.String(),
		QueryKeySearch: base.QueryKeySearch,
		Metas:          metas,
	})
}

type roleInfo struct {
	Text string
	URL  string
}

func (wui *WebUI) renderRolesList(w http.ResponseWriter, r *http.Request, listRole usecase.ListRoles) {
	ctx := r.Context()
	roleArrangement, err := listRole.Run(ctx)
	if err != nil {
		wui.reportError(ctx, w, err)
		return
	}
	roleList := roleArrangement.Counted()
	roleList.SortByName()

	roleInfos := make([]roleInfo, len(roleList))
	for i, role := range roleList {
		roleInfos[i] = roleInfo{role.Name, wui.NewURLBuilder('h').AppendSearch("role:" + role.Name).String()}
	}

	user := wui.getUser(ctx)
	var base baseData
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), "", user, &base)
	wui.renderTemplate(ctx, w, id.RolesTemplateZid, &base, struct {
		Roles []roleInfo
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
	countMap := make(map[int]int)
	baseTagListURL := wui.NewURLBuilder('h')
	for tag, ml := range tagData {
		count := len(ml)
		countMap[count]++
		tagsList = append(
			tagsList,
			tagInfo{tag, baseTagListURL.AppendQuery(api.KeyAllTags, tag).String(), count, "", ""})
		baseTagListURL.ClearQuery()
	}
	sort.Slice(tagsList, func(i, j int) bool { return tagsList[i].Name < tagsList[j].Name })

	countList := make([]int, 0, len(countMap))
	for count := range countMap {
		countList = append(countList, count)







|







144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
	countMap := make(map[int]int)
	baseTagListURL := wui.NewURLBuilder('h')
	for tag, ml := range tagData {
		count := len(ml)
		countMap[count]++
		tagsList = append(
			tagsList,
			tagInfo{tag, baseTagListURL.AppendSearch(api.KeyAllTags + ":" + tag).String(), count, "", ""})
		baseTagListURL.ClearQuery()
	}
	sort.Slice(tagsList, func(i, j int) bool { return tagsList[i].Name < tagsList[j].Name })

	countList := make([]int, 0, len(countMap))
	for count := range countMap {
		countList = append(countList, count)
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
		limit := getIntParameter(q, api.QueryKeyLimit, 200)
		metaList, err := getContext.Run(ctx, zid, dir, depth, limit)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		apiZid := api.ZettelID(zid.String())
		metaLinks := wui.buildHTMLMetaList(ctx, metaList, evaluate)
		depths := []string{"2", "3", "4", "5", "6", "7", "8", "9", "10"}
		depthLinks := make([]simpleLink, len(depths))
		depthURL := wui.NewURLBuilder('k').SetZid(apiZid)
		for i, depth := range depths {
			depthURL.ClearQuery()
			switch dir {
			case usecase.ZettelContextBackward:







|







201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
		limit := getIntParameter(q, api.QueryKeyLimit, 200)
		metaList, err := getContext.Run(ctx, zid, dir, depth, limit)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		apiZid := api.ZettelID(zid.String())
		metaLinks := wui.buildHTMLMetaList(metaList, func(val string) ast.InlineSlice { return evaluate.RunMetadataNoLink(ctx, val) })
		depths := []string{"2", "3", "4", "5", "6", "7", "8", "9", "10"}
		depthLinks := make([]simpleLink, len(depths))
		depthURL := wui.NewURLBuilder('k').SetZid(apiZid)
		for i, depth := range depths {
			depthURL.ClearQuery()
			switch dir {
			case usecase.ZettelContextBackward:
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
}

func (wui *WebUI) listTitleSearch(s *search.Search) string {
	if s == nil {
		return wui.rtConfig.GetSiteName()
	}
	var buf bytes.Buffer
	s.Print(&buf)
	return buf.String()
}

// buildHTMLMetaList builds a zettel list based on a meta list for HTML rendering.
func (wui *WebUI) buildHTMLMetaList(ctx context.Context, metaList []*meta.Meta, evaluate *usecase.Evaluate) []simpleLink {
	metas := make([]simpleLink, 0, len(metaList))
	encHTML := wui.getSimpleHTMLEncoder()
	for _, m := range metaList {
		metas = append(metas, simpleLink{
			Text: encodeTitleAsHTML(ctx, m, evaluate, encHTML, true),
			URL:  wui.NewURLBuilder('h').SetZid(api.ZettelID(m.Zid.String())).String(),
		})
	}
	return metas
}







|




|




|





249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
}

func (wui *WebUI) listTitleSearch(s *search.Search) string {
	if s == nil {
		return wui.rtConfig.GetSiteName()
	}
	var buf bytes.Buffer
	s.PrintHuman(&buf)
	return buf.String()
}

// buildHTMLMetaList builds a zettel list based on a meta list for HTML rendering.
func (wui *WebUI) buildHTMLMetaList(metaList []*meta.Meta, evalMetadata evalMetadataFunc) []simpleLink {
	metas := make([]simpleLink, 0, len(metaList))
	encHTML := wui.getSimpleHTMLEncoder()
	for _, m := range metaList {
		metas = append(metas, simpleLink{
			Text: encodeZmkMetadata(m.GetTitle(), evalMetadata, encHTML),
			URL:  wui.NewURLBuilder('h').SetZid(api.ZettelID(m.Zid.String())).String(),
		})
	}
	return metas
}

Changes to web/adapter/webui/login.go.

55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
		}
		ctx := r.Context()
		ident, cred, ok := adapter.GetCredentialsViaForm(r)
		if !ok {
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read login form"))
			return
		}
		token, err := ucAuth.Run(ctx, ident, cred, wui.tokenLifetime, auth.KindHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		if token == nil {
			wui.renderLoginForm(wui.clearToken(ctx, w), w, true)
			return
		}

		wui.setToken(w, token)
		wui.redirectFound(w, r, wui.NewURLBuilder('/'))
	}
}







|













55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
		}
		ctx := r.Context()
		ident, cred, ok := adapter.GetCredentialsViaForm(r)
		if !ok {
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read login form"))
			return
		}
		token, err := ucAuth.Run(ctx, r, ident, cred, wui.tokenLifetime, auth.KindHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		if token == nil {
			wui.renderLoginForm(wui.clearToken(ctx, w), w, true)
			return
		}

		wui.setToken(w, token)
		wui.redirectFound(w, r, wui.NewURLBuilder('/'))
	}
}

Changes to web/adapter/webui/webui.go.

332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
		if err2 != nil {
			continue
		}
		if !wui.policy.CanRead(user, m) {
			continue
		}
		title := m.GetTitle()
		astTitle := parser.ParseMetadata(title)
		menuTitle, err2 := wui.getSimpleHTMLEncoder().InlinesString(&astTitle, false)
		if err2 != nil {
			menuTitle, err2 = encodeInlinesText(&astTitle, wui.gentext)
			if err2 != nil {
				menuTitle = title
			}
		}
		result = append(result, simpleLink{







|
|







332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
		if err2 != nil {
			continue
		}
		if !wui.policy.CanRead(user, m) {
			continue
		}
		title := m.GetTitle()
		astTitle := parser.ParseMetadataNoLink(title)
		menuTitle, err2 := wui.getSimpleHTMLEncoder().InlinesString(&astTitle)
		if err2 != nil {
			menuTitle, err2 = encodeInlinesText(&astTitle, wui.gentext)
			if err2 != nil {
				menuTitle = title
			}
		}
		result = append(result, simpleLink{

Changes to web/server/impl/router.go.

106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
	rt.mux.Handle(pattern, handler)
}

func (rt *httpRouter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	// Something may panic. Ensure a kernel log.
	defer func() {
		if reco := recover(); reco != nil {
			rt.log.Error().Str("Method", r.Method).Str("URL", r.URL.String()).Str("ip", getCallerIP(r)).Msg("Recover context")
			kernel.Main.LogRecover("Web", reco)
		}
	}()

	var withDebug bool
	if msg := rt.log.Debug(); msg.Enabled() {
		withDebug = true
		w = &traceResponseWriter{original: w}
		msg.Str("method", r.Method).Str("uri", r.RequestURI).Str("ip", getCallerIP(r)).Msg("ServeHTTP")
	}

	if prefixLen := len(rt.urlPrefix); prefixLen > 1 {
		if len(r.URL.Path) < prefixLen || r.URL.Path[:prefixLen] != rt.urlPrefix {
			http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
			if withDebug {
				rt.log.Debug().Int("sc", int64(w.(*traceResponseWriter).statusCode)).Msg("/ServeHTTP/prefix")







|








|







106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
	rt.mux.Handle(pattern, handler)
}

func (rt *httpRouter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	// Something may panic. Ensure a kernel log.
	defer func() {
		if reco := recover(); reco != nil {
			rt.log.Error().Str("Method", r.Method).Str("URL", r.URL.String()).HTTPIP(r).Msg("Recover context")
			kernel.Main.LogRecover("Web", reco)
		}
	}()

	var withDebug bool
	if msg := rt.log.Debug(); msg.Enabled() {
		withDebug = true
		w = &traceResponseWriter{original: w}
		msg.Str("method", r.Method).Str("uri", r.RequestURI).HTTPIP(r).Msg("ServeHTTP")
	}

	if prefixLen := len(rt.urlPrefix); prefixLen > 1 {
		if len(r.URL.Path) < prefixLen || r.URL.Path[:prefixLen] != rt.urlPrefix {
			http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
			if withDebug {
				rt.log.Debug().Int("sc", int64(w.(*traceResponseWriter).statusCode)).Msg("/ServeHTTP/prefix")
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
	if rt.ur == nil {
		// No auth needed
		return r
	}
	k := auth.KindJSON
	t := getHeaderToken(r)
	if len(t) == 0 {
		rt.log.Debug().Str("ip", getCallerIP(r)).Msg("no jwt token found")
		k = auth.KindHTML
		t = getSessionToken(r)
	}
	if len(t) == 0 {
		rt.log.Sense().Str("ip", getCallerIP(r)).Msg("no auth token found in request")
		return r
	}
	tokenData, err := rt.auth.CheckToken(t, k)
	if err != nil {
		rt.log.Sense().Err(err).Str("ip", getCallerIP(r)).Msg("invalid auth token")
		return r
	}
	ctx := r.Context()
	user, err := rt.ur.GetUser(ctx, tokenData.Zid, tokenData.Ident)
	if err != nil {
		rt.log.Sense().Zid(tokenData.Zid).Str("ident", tokenData.Ident).Err(err).Str("ip", getCallerIP(r)).Msg("auth user not found")
		return r
	}
	return r.WithContext(updateContext(ctx, user, &tokenData))
}

func getCallerIP(r *http.Request) string {
	if from := r.Header.Get("X-Forwarded-For"); from != "" {
		return from
	}
	return r.RemoteAddr
}

func getSessionToken(r *http.Request) []byte {
	cookie, err := r.Cookie(sessionName)
	if err != nil {
		return nil
	}
	return []byte(cookie.Value)
}







|




|




|





|





<
<
<
<
<
<
<







173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201







202
203
204
205
206
207
208
	if rt.ur == nil {
		// No auth needed
		return r
	}
	k := auth.KindJSON
	t := getHeaderToken(r)
	if len(t) == 0 {
		rt.log.Debug().Msg("no jwt token found") // IP already logged: ServeHTTP
		k = auth.KindHTML
		t = getSessionToken(r)
	}
	if len(t) == 0 {
		rt.log.Debug().Msg("no auth token found in request") // IP already logged: ServeHTTP
		return r
	}
	tokenData, err := rt.auth.CheckToken(t, k)
	if err != nil {
		rt.log.Sense().Err(err).HTTPIP(r).Msg("invalid auth token")
		return r
	}
	ctx := r.Context()
	user, err := rt.ur.GetUser(ctx, tokenData.Zid, tokenData.Ident)
	if err != nil {
		rt.log.Sense().Zid(tokenData.Zid).Str("ident", tokenData.Ident).Err(err).HTTPIP(r).Msg("auth user not found")
		return r
	}
	return r.WithContext(updateContext(ctx, user, &tokenData))
}








func getSessionToken(r *http.Request) []byte {
	cookie, err := r.Cookie(sessionName)
	if err != nil {
		return nil
	}
	return []byte(cookie.Value)
}

Changes to www/changes.wiki.

1
2






3
4






























5
6








7
8
9
10
11
12
13
<title>Change Log</title>







<a name="0_6_0"></a>
<h2>Changes for Version 0.6.0 (pending)</h2>































<a name="0_5_0"></a>








<h2>Changes for Version 0.5.0 (2022-07-29)</h2>
  *  Removed zettel syntax &ldquo;draw&rdquo;. The new default syntax for
     inline zettel is now &ldquo;text&rdquo;. A drawing can now be made by
     using the &ldquo;evaluation block&rdquo; syntax (see below) by setting
     the generic attribute to &ldquo;draw&rdquo;.
     (breaking: zettelmarkup, api, webui)
  *  If authentication is enabled, a secret of at least 16 bytes must be set in


>
>
>
>
>
>
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


>
>
>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
<title>Change Log</title>

<a name="0_7"></a>
<h2>Changes for Version 0.7.0 (pending)</h2>
  *  Removes support for URL query parameter to search for metadata values,
     sorting, offset, and limit a zettel list. Deprecated in version 0.6.0
     (breaking: api, webui)

<a name="0_6"></a>
<h2>Changes for Version 0.6.0 (2022-08-11)</h2>
  *  Translating of "..." into horizontal ellipsis is no longer supported. Use
     &amp;hellip; instead.
     (breaking: zettelmarkup)
  *  Allow to specify search expressions, which allow to specify search
     criterias by using a simple syntax. Can be specified in WebUI's search box
     and via the API by using query parameter "_s".
     (major: api, webui)
  *  A link reference is allowed to be a search expression. The WebUI will
     render this as a link to a list of zettel that satisfy the search
     expression.
     (major: zettelmarkup, webui)
  *  A block transclusion is allowed to specify a search expression. When
     evaluated, the transclusion is replaced by a list of zettel that satisfy
     the search expression.
     (major: zettelmarkup)
  *  When presenting a zettel list, allow to change the search expression.
     (minor: webui)
  *  When evaluating a zettel, ignore transclusions if current user is not
     allowed to read transcluded zettel.
     (minor)
  *  Added a small tutorial for Zettelmarkup.
     (minor: manual)
  *  Using URL query parameter to search for metdata values, specify an
     ordering, an offset, and a limit for the resulting list, will be removed
     in version 0.7. Replace these with the more useable search expressions.
     Please be aware that the = search operator is also deprecated. It was only
     introduced to help the migration.
     (deprecated: api, webui)
  *  Some smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_5_0"></a>
<h2>Changes for Version 0.5.1 (2022-08-02)</h2>
  *  Log missing authentication tokens in debug level (was: sense level)
     (major)
  *  Allow to use empty metadata values of string and zmk types.
     (minor)
  *  Add IP address to some log messages, esp. when authentication fails.
     (minor)

<h2>Changes for Version 0.5.0 (2022-07-29)</h2>
  *  Removed zettel syntax &ldquo;draw&rdquo;. The new default syntax for
     inline zettel is now &ldquo;text&rdquo;. A drawing can now be made by
     using the &ldquo;evaluation block&rdquo; syntax (see below) by setting
     the generic attribute to &ldquo;draw&rdquo;.
     (breaking: zettelmarkup, api, webui)
  *  If authentication is enabled, a secret of at least 16 bytes must be set in
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
     (minor)
  *  Log authentication issues in level "sense"; add caller IP address to some
     web server log messages.
     (minor: web server)
  *  New startup configuration key <kbd>max-request-size</kbd> to limit a web
     request body to prevent client sending too large requests.
     (minor: web server)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_4"></a>
<h2>Changes for Version 0.4 (2022-03-08)</h2>
  *  Encoding &ldquo;djson&rdquo; renamed to &ldquo;zjson&rdquo; (<em>zettel
     json</em>).
     (breaking: api; minor: webui)







|







97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
     (minor)
  *  Log authentication issues in level "sense"; add caller IP address to some
     web server log messages.
     (minor: web server)
  *  New startup configuration key <kbd>max-request-size</kbd> to limit a web
     request body to prevent client sending too large requests.
     (minor: web server)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_4"></a>
<h2>Changes for Version 0.4 (2022-03-08)</h2>
  *  Encoding &ldquo;djson&rdquo; renamed to &ldquo;zjson&rdquo; (<em>zettel
     json</em>).
     (breaking: api; minor: webui)
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
     (minor: api)
  *  Limit the amount of zettel and bytes to be stored in a memory box. Allows
     to use it with public access.
     (minor: box)
  *  Disallow to cache the authentication cookie. Will remove most unexpected
     log-outs when using a mobile device.
     (minor: webui)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_3"></a>
<h2>Changes for Version 0.3 (2022-02-09)</h2>
  *  Zettel files with extension <tt>.meta</tt> are now treated as content
     files. Previoulsy, they were interpreted as metadata files. The
     interpretation as metadata files was deprecated in version 0.2.







|







153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
     (minor: api)
  *  Limit the amount of zettel and bytes to be stored in a memory box. Allows
     to use it with public access.
     (minor: box)
  *  Disallow to cache the authentication cookie. Will remove most unexpected
     log-outs when using a mobile device.
     (minor: webui)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_3"></a>
<h2>Changes for Version 0.3 (2022-02-09)</h2>
  *  Zettel files with extension <tt>.meta</tt> are now treated as content
     files. Previoulsy, they were interpreted as metadata files. The
     interpretation as metadata files was deprecated in version 0.2.
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
     (major: Zettelmarkup)
  *  Metadata-returning API calls additionally return an indication about
     access rights for the given zettel.
     (minor: api)
  *  A previously duplicate file that is now useful (because another file was
     deleted) is now logged as such.
     (minor: directory and file/zip box)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_2"></a>
<h2>Changes for Version 0.2 (2022-01-19)</h2>
  *  v0.2.1 (2021-02-01) updates the license year in some documents
  *  Remove support for <tt>;;small text;;</tt> Zettelmarkup.
     (breaking: Zettelmarkup)







|







175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
     (major: Zettelmarkup)
  *  Metadata-returning API calls additionally return an indication about
     access rights for the given zettel.
     (minor: api)
  *  A previously duplicate file that is now useful (because another file was
     deleted) is now logged as such.
     (minor: directory and file/zip box)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_2"></a>
<h2>Changes for Version 0.2 (2022-01-19)</h2>
  *  v0.2.1 (2021-02-01) updates the license year in some documents
  *  Remove support for <tt>;;small text;;</tt> Zettelmarkup.
     (breaking: Zettelmarkup)
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
     <tt>Content-Length: 0</tt>.
     (minor: api)
  *  Metadata key <tt>duplicates</tt> stores the duplicate file names, instead
     of just a boolean value that there were duplicate file names.
     (minor)
  *  Document autostarting Zettelstore on Windows, macOS, and Linux.
     (minor)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_1"></a><a name="0_1_0"></a>
<h2>Changes for Version 0.1 (2021-11-11)</h2>
  *  v0.1.3 (2021-12-15) fixes a bug where the modification date could be set
     when a new zettel is created.
  *  v0.1.2 (2021-11-18) fixes a bug when selecting zettel from a list when







|







241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
     <tt>Content-Length: 0</tt>.
     (minor: api)
  *  Metadata key <tt>duplicates</tt> stores the duplicate file names, instead
     of just a boolean value that there were duplicate file names.
     (minor)
  *  Document autostarting Zettelstore on Windows, macOS, and Linux.
     (minor)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_1"></a><a name="0_1_0"></a>
<h2>Changes for Version 0.1 (2021-11-11)</h2>
  *  v0.1.3 (2021-12-15) fixes a bug where the modification date could be set
     when a new zettel is created.
  *  v0.1.2 (2021-11-18) fixes a bug when selecting zettel from a list when
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
  *  Fix: when a very long word (longer than width of browser window) is given,
     still allow to scroll horizontally.
     (minor: webui)
  *  Separate repository for [https://zettelstore.de/contrib/|contributed]
     software. First entry is a software for creating a presentation by using
     zettel.
     (info)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_0_15"></a>
<h2>Changes for Version 0.0.15 (2021-09-17)</h2>
  *  Move again endpoint characters for authentication to make room for future
     features. WebUI authentication moves from <tt>/a</tt> to <tt>/i</tt>
     (login) and <tt>/i?logout</tt> (logout). API authentication moves from







|







306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
  *  Fix: when a very long word (longer than width of browser window) is given,
     still allow to scroll horizontally.
     (minor: webui)
  *  Separate repository for [https://zettelstore.de/contrib/|contributed]
     software. First entry is a software for creating a presentation by using
     zettel.
     (info)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_0_15"></a>
<h2>Changes for Version 0.0.15 (2021-09-17)</h2>
  *  Move again endpoint characters for authentication to make room for future
     features. WebUI authentication moves from <tt>/a</tt> to <tt>/i</tt>
     (login) and <tt>/i?logout</tt> (logout). API authentication moves from
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
  *  Folgezettel are now displayed with full title at the bottom of a page.
     (minor: webui)
  *  Add API endpoint <tt>/p/{ID}</tt> to retrieve a parsed, but not evaluated
     zettel in various encodings.
     (minor: api)
  *  Fix: do not list a shadowed zettel that matches the select criteria.
     (minor)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_0_14"></a>
<h2>Changes for Version 0.0.14 (2021-07-23)</h2>
  *  Rename &ldquo;place&rdquo; into &ldquo;box&rdquo;. This also affects the
     configuration keys to specify boxes <tt>box-uri<em>X</em></tt> (previously
     <tt>place-uri-<em>X</em></tt>. Older changes documented here are renamed







|







361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
  *  Folgezettel are now displayed with full title at the bottom of a page.
     (minor: webui)
  *  Add API endpoint <tt>/p/{ID}</tt> to retrieve a parsed, but not evaluated
     zettel in various encodings.
     (minor: api)
  *  Fix: do not list a shadowed zettel that matches the select criteria.
     (minor)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_0_14"></a>
<h2>Changes for Version 0.0.14 (2021-07-23)</h2>
  *  Rename &ldquo;place&rdquo; into &ldquo;box&rdquo;. This also affects the
     configuration keys to specify boxes <tt>box-uri<em>X</em></tt> (previously
     <tt>place-uri-<em>X</em></tt>. Older changes documented here are renamed
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
     (minor: webui)
  *  When a user moves a zettel file with additional characters into the box
     directory, these characters are preserved when zettel is updated.
     (bug)
  *  The phase &ldquo;filtering a zettel list&rdquo; is more precise
     &ldquo;selecting zettel&rdquo;
     (documentation)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_0_13"></a>
<h2>Changes for Version 0.0.13 (2021-06-01)</h2>
  *  Startup configuration <tt>box-<em>X</em>-uri</tt> (where <em>X</em> is a
     number greater than zero) has been renamed to
     <tt>box-uri-<em>X</em></tt>.







|







410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
     (minor: webui)
  *  When a user moves a zettel file with additional characters into the box
     directory, these characters are preserved when zettel is updated.
     (bug)
  *  The phase &ldquo;filtering a zettel list&rdquo; is more precise
     &ldquo;selecting zettel&rdquo;
     (documentation)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_0_13"></a>
<h2>Changes for Version 0.0.13 (2021-06-01)</h2>
  *  Startup configuration <tt>box-<em>X</em>-uri</tt> (where <em>X</em> is a
     number greater than zero) has been renamed to
     <tt>box-uri-<em>X</em></tt>.
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
     substituted with the new default image, a spinning emoji.
     See [/file?name=box/constbox/emoji_spin.gif].
     (minor: webui)
  *  Add zettelmarkup syntax for a table row that should be ignored:
     <tt>|%</tt>. This allows to paste output of the administrator console into
     a zettel.
     (minor: zmk)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_0_12"></a>
<h2>Changes for Version 0.0.12 (2021-04-16)</h2>
  *  Raise the per-process limit of open files on macOS to 1.048.576. This
     allows most macOS users to use at least 500.000 zettel. That should be
     enough for the near future.







|







454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
     substituted with the new default image, a spinning emoji.
     See [/file?name=box/constbox/emoji_spin.gif].
     (minor: webui)
  *  Add zettelmarkup syntax for a table row that should be ignored:
     <tt>|%</tt>. This allows to paste output of the administrator console into
     a zettel.
     (minor: zmk)
  *  Many smaller bug fixes and improvements, to the software and to the
     documentation.

<a name="0_0_12"></a>
<h2>Changes for Version 0.0.12 (2021-04-16)</h2>
  *  Raise the per-process limit of open files on macOS to 1.048.576. This
     allows most macOS users to use at least 500.000 zettel. That should be
     enough for the near future.
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
     into erratic, non-deterministic context lists.
     (minor: api, webui)
  *  Selecting zettel depending on tag values can be both by comparing only the
     prefix or the whole string. If a search value begins with '#', only zettel
     with the exact tag will be returned. Otherwise a zettel will be returned
     if the search string just matches the prefix of only one of its tags.
     (minor: api, webui)
  *  Many smaller bug fixes and inprovements, to the software and to the documentation.

A note for users of macOS: in the current release and with macOS's default
values, a zettel directory must not contain more than approx. 250 files. There
are three options to mitigate this limitation temporarily:
  #  You update the per-process limit of open files on macOS.
  #  You setup a virtualization environment to run Zettelstore on Linux or Windows.
  #  You wait for version 0.0.12 which addresses this issue.







|







508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
     into erratic, non-deterministic context lists.
     (minor: api, webui)
  *  Selecting zettel depending on tag values can be both by comparing only the
     prefix or the whole string. If a search value begins with '#', only zettel
     with the exact tag will be returned. Otherwise a zettel will be returned
     if the search string just matches the prefix of only one of its tags.
     (minor: api, webui)
  *  Many smaller bug fixes and improvements, to the software and to the documentation.

A note for users of macOS: in the current release and with macOS's default
values, a zettel directory must not contain more than approx. 250 files. There
are three options to mitigate this limitation temporarily:
  #  You update the per-process limit of open files on macOS.
  #  You setup a virtualization environment to run Zettelstore on Linux or Windows.
  #  You wait for version 0.0.12 which addresses this issue.
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
     &ldquo;&amp;#10138;&rdquo; (&ldquo;&#10138;&rdquo;). It is more beautiful
     than the previous &ldquo;&amp;#8599;&amp;#xfe0e;&rdquo;
     (&ldquo;&#8599;&#65038;&rdquo;), which also needed the additional
     &ldquo;&amp;#xfe0e;&rdquo; to disable the conversion to an emoji on iPadOS.
     (minor: webui)
  *  A pre-build binary for macOS ARM64 (also known as Apple silicon) is available.
     (minor: infrastructure)
  *  Many smaller bug fixes and inprovements, to the software and to the documentation.

<a name="0_0_9"></a>
<h2>Changes for Version 0.0.9 (2021-01-29)</h2>
This is the first version that is managed by [https://fossil-scm.org|Fossil]
instead of GitHub. To access older versions, use the Git repository under
[https://github.com/zettelstore/zettelstore-github|zettelstore-github].








|







559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
     &ldquo;&amp;#10138;&rdquo; (&ldquo;&#10138;&rdquo;). It is more beautiful
     than the previous &ldquo;&amp;#8599;&amp;#xfe0e;&rdquo;
     (&ldquo;&#8599;&#65038;&rdquo;), which also needed the additional
     &ldquo;&amp;#xfe0e;&rdquo; to disable the conversion to an emoji on iPadOS.
     (minor: webui)
  *  A pre-build binary for macOS ARM64 (also known as Apple silicon) is available.
     (minor: infrastructure)
  *  Many smaller bug fixes and improvements, to the software and to the documentation.

<a name="0_0_9"></a>
<h2>Changes for Version 0.0.9 (2021-01-29)</h2>
This is the first version that is managed by [https://fossil-scm.org|Fossil]
instead of GitHub. To access older versions, use the Git repository under
[https://github.com/zettelstore/zettelstore-github|zettelstore-github].

Changes to www/download.wiki.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
<title>Download</title>
<h1>Download of Zettelstore Software</h1>
<h2>Foreword</h2>
  *  Zettelstore is free/libre open source software, licensed under EUPL-1.2-or-later.
  *  The software is provided as-is.
  *  There is no guarantee that it will not damage your system.
  *  However, it is in use by the main developer since March 2020 without any damage.
  *  It may be useful for you. It is useful for me.
  *  Take a look at the [https://zettelstore.de/manual/|manual] to know how to start and use it.

<h2>ZIP-ped Executables</h2>
Build: <code>v0.5.0</code> (2022-07-29).

  *  [/uv/zettelstore-0.5.0-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.5.0-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.5.0-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.5.0-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.5.0-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)

Unzip the appropriate file, install and execute Zettelstore according to the manual.

<h2>Zettel for the manual</h2>
As a starter, you can download the zettel for the manual
[/uv/manual-0.5.0.zip|here].
Just unzip the contained files and put them into your zettel folder or
configure a file box to read the zettel directly from the ZIP file.











|

|
|
|
|
|





|


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
<title>Download</title>
<h1>Download of Zettelstore Software</h1>
<h2>Foreword</h2>
  *  Zettelstore is free/libre open source software, licensed under EUPL-1.2-or-later.
  *  The software is provided as-is.
  *  There is no guarantee that it will not damage your system.
  *  However, it is in use by the main developer since March 2020 without any damage.
  *  It may be useful for you. It is useful for me.
  *  Take a look at the [https://zettelstore.de/manual/|manual] to know how to start and use it.

<h2>ZIP-ped Executables</h2>
Build: <code>v0.6.0</code> (2022-08-11).

  *  [/uv/zettelstore-0.6.0-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.6.0-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.6.0-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.6.0-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.6.0-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)

Unzip the appropriate file, install and execute Zettelstore according to the manual.

<h2>Zettel for the manual</h2>
As a starter, you can download the zettel for the manual
[/uv/manual-0.6.0.zip|here].
Just unzip the contained files and put them into your zettel folder or
configure a file box to read the zettel directly from the ZIP file.

Changes to www/index.wiki.

18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

[https://zettelstore.de/client|Zettelstore Client] provides client software to
access Zettelstore via its API more easily,
[https://zettelstore.de/contrib|Zettelstore Contrib] contains contributed
software, which often connects to Zettelstore via its API. Some of the software
packages may be experimental.

[https://twitter.com/zettelstore|Stay tuned]&hellip;
<hr>
<h3>Latest Release: 0.5.0 (2022-07-29)</h3>
  *  [./download.wiki|Download]
  *  [./changes.wiki#0_5_0|Change summary]
  *  [/timeline?p=v0.5.0&bt=v0.4&y=ci|Check-ins for version 0.5.0],
     [/vdiff?to=v0.5.0&from=v0.4|content diff]
  *  [/timeline?df=v0.5.0&y=ci|Check-ins derived from the 0.5.0 release],
     [/vdiff?from=v0.5.0&to=trunk|content diff]
  *  [./plan.wiki|Limitations and planned improvements]
  *  [/timeline?t=release|Timeline of all past releases]

<hr>
<h2>Build instructions</h2>
Just install [https://golang.org/dl/|Go] and some Go-based tools. Please read
the [./build.md|instructions] for details.

  *  [/dir?ci=trunk|Source code]
  *  [/download|Download the source code] as a tarball or a ZIP file
     (you must [/login|login] as user &quot;anonymous&quot;).







|

|

|
|
|
|
|











18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

[https://zettelstore.de/client|Zettelstore Client] provides client software to
access Zettelstore via its API more easily,
[https://zettelstore.de/contrib|Zettelstore Contrib] contains contributed
software, which often connects to Zettelstore via its API. Some of the software
packages may be experimental.

[https://twitter.com/zettelstore|Stay tuned]&nbsp;&hellip;
<hr>
<h3>Latest Release: 0.6.0 (2022-08-11)</h3>
  *  [./download.wiki|Download]
  *  [./changes.wiki#0_6|Change summary]
  *  [/timeline?p=v0.6.0&bt=v0.5.0&y=ci|Check-ins for version 0.6.0],
     [/vdiff?to=v0.6.0&from=v0.5.0|content diff]
  *  [/timeline?df=v0.6.0&y=ci|Check-ins derived from the 0.6.0 release],
     [/vdiff?from=v0.6.0&to=trunk|content diff]
  *  [./plan.wiki|Limitations and planned improvements]
  *  [/timeline?t=release|Timeline of all past releases]

<hr>
<h2>Build instructions</h2>
Just install [https://golang.org/dl/|Go] and some Go-based tools. Please read
the [./build.md|instructions] for details.

  *  [/dir?ci=trunk|Source code]
  *  [/download|Download the source code] as a tarball or a ZIP file
     (you must [/login|login] as user &quot;anonymous&quot;).

Changes to www/plan.wiki.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<title>Limitations and planned improvements</title>

Here is a list of some shortcomings of Zettelstore.
They are planned to be solved.

<h3>Serious limitations</h3>
  *  Content with binary data (e.g. a GIF, PNG, or JPG file) cannot be created
     nor modified via the standard web interface. As a workaround, you should
     put your file into the directory where your zettel are stored. Make sure
     that the file name starts with unique 14 digits that make up the zettel
     identifier.
  *  Automatic lists are not supported in Zettelmarkup.
  *  &hellip;

<h3>Smaller limitations</h3>
  *  Quoted attribute values are not yet supported in Zettelmarkup:
     <code>{key="value with space"}</code>.
  *  The horizontal tab character (<tt>U+0009</tt>) is not supported.
  *  Missing support for citation keys.











<







1
2
3
4
5
6
7
8
9
10
11

12
13
14
15
16
17
18
<title>Limitations and planned improvements</title>

Here is a list of some shortcomings of Zettelstore.
They are planned to be solved.

<h3>Serious limitations</h3>
  *  Content with binary data (e.g. a GIF, PNG, or JPG file) cannot be created
     nor modified via the standard web interface. As a workaround, you should
     put your file into the directory where your zettel are stored. Make sure
     that the file name starts with unique 14 digits that make up the zettel
     identifier.

  *  &hellip;

<h3>Smaller limitations</h3>
  *  Quoted attribute values are not yet supported in Zettelmarkup:
     <code>{key="value with space"}</code>.
  *  The horizontal tab character (<tt>U+0009</tt>) is not supported.
  *  Missing support for citation keys.