Zettelstore

Check-in Differences
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Difference From version-0.0.12 To version-0.0.13

2021-06-07
09:11
Increase version to 0.0.14-dev to begin next development cycle ... (check-in: 7dd6f4dd5c user: stern tags: trunk)
2021-06-01
12:35
Version 0.0.13 ... (check-in: 11d9b6da63 user: stern tags: trunk, release, version-0.0.13)
10:14
Log output while starting Command Line Server ... (check-in: 968a91bbaa user: stern tags: trunk)
2021-04-17
12:34
Increase version to 0.0.13-dev to begin next development cycle ... (check-in: 5f0c8f2d4c user: stern tags: trunk)
2021-04-16
16:16
Version 0.0.12 ... (check-in: 86f8bc8a70 user: stern tags: trunk, release, version-0.0.12)
16:08
Show default dir place type in startup values zettel ... (check-in: 8269a7cbc4 user: stern tags: trunk)

Changes to VERSION.

1


1
-
+
0.0.12
0.0.13

Changes to ast/ast.go.

79
80
81
82
83
84
85
86

87
88
89
90
91
92
93
94
79
80
81
82
83
84
85

86
87
88
89
90
91
92
93
94







-
+








}

// RefState indicates the state of the reference.
type RefState int

// Constants for RefState
const (
	RefStateInvalid  RefState = iota // Invalid Referende
	RefStateInvalid  RefState = iota // Invalid Reference
	RefStateZettel                   // Reference to an internal zettel
	RefStateSelf                     // Reference to same zettel with a fragment
	RefStateFound                    // Reference to an existing internal zettel
	RefStateBroken                   // Reference to a non-existing internal zettel
	RefStateHosted                   // Reference to local hosted non-Zettel, without URL change
	RefStateBased                    // Reference to local non-Zettel, to be prefixed
	RefStateExternal                 // Reference to external material
)

Changes to ast/ref.go.

15
16
17
18
19
20
21
22


23
24
25
26
27
28
29
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30







-
+
+







	"net/url"

	"zettelstore.de/z/domain/id"
)

// ParseReference parses a string and returns a reference.
func ParseReference(s string) *Reference {
	if s == "" {
	switch s {
	case "", "00000000000000":
		return &Reference{URL: nil, Value: s, State: RefStateInvalid}
	}
	if state, ok := localState(s); ok {
		if state == RefStateBased {
			s = s[1:]
		}
		u, err := url.Parse(s)

Changes to ast/ref_test.go.

44
45
46
47
48
49
50

51
52
53
54
55
56
57
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58







+







	testcases := []struct {
		link       string
		isZettel   bool
		isExternal bool
		isLocal    bool
	}{
		{"", false, false, false},
		{"00000000000000", false, false, false},
		{"http://zettelstore.de/z/ast", false, true, false},
		{"12345678901234", true, false, false},
		{"12345678901234#local", true, false, false},
		{"http://12345678901234", false, true, false},
		{"http://zettelstore.de/z/12345678901234", false, true, false},
		{"http://zettelstore.de/12345678901234", false, true, false},
		{"/12345678901234", false, false, true},

Added auth/auth.go.






































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package auth provides services for authentification / authorization.
package auth

import (
	"time"

	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/web/server"
)

// BaseManager allows to check some base auth modes.
type BaseManager interface {
	// IsReadonly returns true, if the systems is configured to run in read-only-mode.
	IsReadonly() bool
}

// TokenManager provides methods to create authentication
type TokenManager interface {

	// GetToken produces a authentication token.
	GetToken(ident *meta.Meta, d time.Duration, kind TokenKind) ([]byte, error)

	// CheckToken checks the validity of the token and returns relevant data.
	CheckToken(token []byte, k TokenKind) (TokenData, error)
}

// TokenKind specifies for which application / usage a token is/was requested.
type TokenKind int

// Allowed values of token kind
const (
	_ TokenKind = iota
	KindJSON
	KindHTML
)

// TokenData contains some important elements from a token.
type TokenData struct {
	Token   []byte
	Now     time.Time
	Issued  time.Time
	Expires time.Time
	Ident   string
	Zid     id.Zid
}

// AuthzManager provides methods for authorization.
type AuthzManager interface {
	BaseManager

	// Owner returns the zettel identifier of the owner.
	Owner() id.Zid

	// IsOwner returns true, if the given zettel identifier is that of the owner.
	IsOwner(zid id.Zid) bool

	// Returns true if authentication is enabled.
	WithAuth() bool

	// GetUserRole role returns the user role of the given user zettel.
	GetUserRole(user *meta.Meta) meta.UserRole
}

// Manager is the main interface for providing the service.
type Manager interface {
	TokenManager
	AuthzManager

	PlaceWithPolicy(auth server.Auth, unprotectedPlace place.Place, rtConfig config.Config) (place.Place, Policy)
}

// Policy is an interface for checking access authorization.
type Policy interface {
	// User is allowed to create a new zettel.
	CanCreate(user, newMeta *meta.Meta) bool

	// User is allowed to read zettel
	CanRead(user, m *meta.Meta) bool

	// User is allowed to write zettel.
	CanWrite(user, oldMeta, newMeta *meta.Meta) bool

	// User is allowed to rename zettel
	CanRename(user, m *meta.Meta) bool

	// User is allowed to delete zettel
	CanDelete(user, m *meta.Meta) bool
}

Added auth/impl/impl.go.

























































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides services for authentification / authorization.
package impl

import (
	"errors"
	"hash/fnv"
	"io"
	"time"

	"github.com/pascaldekloe/jwt"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/policy"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/place"
	"zettelstore.de/z/web/server"
)

type myAuth struct {
	readonly bool
	owner    id.Zid
	secret   []byte
}

// New creates a new auth object.
func New(readonly bool, owner id.Zid, extSecret string) auth.Manager {
	return &myAuth{
		readonly: readonly,
		owner:    owner,
		secret:   calcSecret(extSecret),
	}
}

var configKeys = []string{
	kernel.CoreProgname,
	kernel.CoreGoVersion,
	kernel.CoreHostname,
	kernel.CoreGoOS,
	kernel.CoreGoArch,
	kernel.CoreVersion,
}

func calcSecret(extSecret string) []byte {
	h := fnv.New128()
	if extSecret != "" {
		io.WriteString(h, extSecret)
	}
	for _, key := range configKeys {
		io.WriteString(h, kernel.Main.GetConfig(kernel.CoreService, key).(string))
	}
	return h.Sum(nil)
}

// IsReadonly returns true, if the systems is configured to run in read-only-mode.
func (a *myAuth) IsReadonly() bool { return a.readonly }

const reqHash = jwt.HS512

// ErrNoUser signals that the meta data has no role value 'user'.
var ErrNoUser = errors.New("auth: meta is no user")

// ErrNoIdent signals that the 'ident' key is missing.
var ErrNoIdent = errors.New("auth: missing ident")

// ErrOtherKind signals that the token was defined for another token kind.
var ErrOtherKind = errors.New("auth: wrong token kind")

// ErrNoZid signals that the 'zid' key is missing.
var ErrNoZid = errors.New("auth: missing zettel id")

// GetToken returns a token to be used for authentification.
func (a *myAuth) GetToken(ident *meta.Meta, d time.Duration, kind auth.TokenKind) ([]byte, error) {
	if role, ok := ident.Get(meta.KeyRole); !ok || role != meta.ValueRoleUser {
		return nil, ErrNoUser
	}
	subject, ok := ident.Get(meta.KeyUserID)
	if !ok || subject == "" {
		return nil, ErrNoIdent
	}

	now := time.Now().Round(time.Second)
	claims := jwt.Claims{
		Registered: jwt.Registered{
			Subject: subject,
			Expires: jwt.NewNumericTime(now.Add(d)),
			Issued:  jwt.NewNumericTime(now),
		},
		Set: map[string]interface{}{
			"zid": ident.Zid.String(),
			"_tk": int(kind),
		},
	}
	token, err := claims.HMACSign(reqHash, a.secret)
	if err != nil {
		return nil, err
	}
	return token, nil
}

// ErrTokenExpired signals an exired token
var ErrTokenExpired = errors.New("auth: token expired")

// CheckToken checks the validity of the token and returns relevant data.
func (a *myAuth) CheckToken(token []byte, k auth.TokenKind) (auth.TokenData, error) {
	h, err := jwt.NewHMAC(reqHash, a.secret)
	if err != nil {
		return auth.TokenData{}, err
	}
	claims, err := h.Check(token)
	if err != nil {
		return auth.TokenData{}, err
	}
	now := time.Now().Round(time.Second)
	expires := claims.Expires.Time()
	if expires.Before(now) {
		return auth.TokenData{}, ErrTokenExpired
	}
	ident := claims.Subject
	if ident == "" {
		return auth.TokenData{}, ErrNoIdent
	}
	if zidS, ok := claims.Set["zid"].(string); ok {
		if zid, err := id.Parse(zidS); err == nil {
			if kind, ok := claims.Set["_tk"].(float64); ok {
				if auth.TokenKind(kind) == k {
					return auth.TokenData{
						Token:   token,
						Now:     now,
						Issued:  claims.Issued.Time(),
						Expires: expires,
						Ident:   ident,
						Zid:     zid,
					}, nil
				}
			}
			return auth.TokenData{}, ErrOtherKind
		}
	}
	return auth.TokenData{}, ErrNoZid
}

func (a *myAuth) Owner() id.Zid { return a.owner }

func (a *myAuth) IsOwner(zid id.Zid) bool {
	return zid.IsValid() && zid == a.owner
}

func (a *myAuth) WithAuth() bool { return a.owner != id.Invalid }

// GetUserRole role returns the user role of the given user zettel.
func (a *myAuth) GetUserRole(user *meta.Meta) meta.UserRole {
	if user == nil {
		if a.WithAuth() {
			return meta.UserRoleUnknown
		}
		return meta.UserRoleOwner
	}
	if a.IsOwner(user.Zid) {
		return meta.UserRoleOwner
	}
	if val, ok := user.Get(meta.KeyUserRole); ok {
		if ur := meta.GetUserRole(val); ur != meta.UserRoleUnknown {
			return ur
		}
	}
	return meta.UserRoleReader
}

func (a *myAuth) PlaceWithPolicy(auth server.Auth, unprotectedPlace place.Place, rtConfig config.Config) (place.Place, auth.Policy) {
	return policy.PlaceWithPolicy(auth, a, unprotectedPlace, rtConfig)
}

Changes to auth/policy/anon.go.

8
9
10
11
12
13
14


15
16
17
18
19

20
21
22

23
24
25
26
27
28
29
8
9
10
11
12
13
14
15
16
17
18
19
20

21



22
23
24
25
26
27
28
29







+
+




-
+
-
-
-
+







// under this license.
//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorization policies.
package policy

import (
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/meta"
)

type anonPolicy struct {
	simpleMode    bool
	authConfig config.AuthConfig
	expertMode    func() bool
	getVisibility func(*meta.Meta) meta.Visibility
	pre           Policy
	pre        auth.Policy
}

func (ap *anonPolicy) CanCreate(user, newMeta *meta.Meta) bool {
	return ap.pre.CanCreate(user, newMeta)
}

func (ap *anonPolicy) CanRead(user, m *meta.Meta) bool {
39
40
41
42
43
44
45
46
47
48
49
50


51
52
53
39
40
41
42
43
44
45





46
47
48
49
50







-
-
-
-
-
+
+



}

func (ap *anonPolicy) CanDelete(user, m *meta.Meta) bool {
	return ap.pre.CanDelete(user, m) && ap.checkVisibility(m)
}

func (ap *anonPolicy) checkVisibility(m *meta.Meta) bool {
	switch ap.getVisibility(m) {
	case meta.VisibilitySimple:
		return ap.simpleMode || ap.expertMode()
	case meta.VisibilityExpert:
		return ap.expertMode()
	if ap.authConfig.GetVisibility(m) == meta.VisibilityExpert {
		return ap.authConfig.GetExpertMode()
	}
	return true
}

Changes to auth/policy/default.go.

8
9
10
11
12
13
14
15

16
17
18
19



20
21
22
23
24
25
26
8
9
10
11
12
13
14

15
16
17
18

19
20
21
22
23
24
25
26
27
28







-
+



-
+
+
+







// under this license.
//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/domain/meta"
)

type defaultPolicy struct{}
type defaultPolicy struct {
	manager auth.AuthzManager
}

func (d *defaultPolicy) CanCreate(user, newMeta *meta.Meta) bool { return true }
func (d *defaultPolicy) CanRead(user, m *meta.Meta) bool         { return true }
func (d *defaultPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool {
	return d.canChange(user, oldMeta)
}
func (d *defaultPolicy) CanRename(user, m *meta.Meta) bool { return d.canChange(user, m) }
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52
53
38
39
40
41
42
43
44

45
46
47
48
49
50
51
52
53
54
55







-
+










		// See owner.go:CanWrite.

		// No authentication: check for owner-like restriction, because the user
		// acts as an owner
		return metaRo != meta.ValueUserRoleOwner && !meta.BoolValue(metaRo)
	}

	userRole := runtime.GetUserRole(user)
	userRole := d.manager.GetUserRole(user)
	switch metaRo {
	case meta.ValueUserRoleReader:
		return userRole > meta.UserRoleReader
	case meta.ValueUserRoleWriter:
		return userRole > meta.UserRoleWriter
	case meta.ValueUserRoleOwner:
		return userRole > meta.UserRoleOwner
	}
	return !meta.BoolValue(metaRo)
}

Changes to auth/policy/owner.go.

8
9
10
11
12
13
14
15
16


17
18
19
20
21
22

23
24


25
26
27
28
29
30
31
32
33
34
35

36
37
38
39
40
41
42
43
44
45
46
47

48
49
50
51
52
53
54
55
56

57
58
59
60
61
62
63
8
9
10
11
12
13
14


15
16
17
18
19
20


21


22
23
24
25
26
27
28
29
30
31
32
33

34
35
36
37
38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53
54

55
56
57
58
59
60
61
62







-
-
+
+




-
-
+
-
-
+
+










-
+











-
+








-
+







// under this license.
//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/meta"
)

type ownerPolicy struct {
	expertMode    func() bool
	isOwner       func(id.Zid) bool
	manager    auth.AuthzManager
	getVisibility func(*meta.Meta) meta.Visibility
	pre           Policy
	authConfig config.AuthConfig
	pre        auth.Policy
}

func (o *ownerPolicy) CanCreate(user, newMeta *meta.Meta) bool {
	if user == nil || !o.pre.CanCreate(user, newMeta) {
		return false
	}
	return o.userIsOwner(user) || o.userCanCreate(user, newMeta)
}

func (o *ownerPolicy) userCanCreate(user, newMeta *meta.Meta) bool {
	if runtime.GetUserRole(user) == meta.UserRoleReader {
	if o.manager.GetUserRole(user) == meta.UserRoleReader {
		return false
	}
	if role, ok := newMeta.Get(meta.KeyRole); ok && role == meta.ValueRoleUser {
		return false
	}
	return true
}

func (o *ownerPolicy) CanRead(user, m *meta.Meta) bool {
	// No need to call o.pre.CanRead(user, meta), because it will always return true.
	// Both the default and the readonly policy allow to read a zettel.
	vis := o.getVisibility(m)
	vis := o.authConfig.GetVisibility(m)
	if res, ok := o.checkVisibility(user, vis); ok {
		return res
	}
	return o.userIsOwner(user) || o.userCanRead(user, m, vis)
}

func (o *ownerPolicy) userCanRead(user, m *meta.Meta, vis meta.Visibility) bool {
	switch vis {
	case meta.VisibilityOwner, meta.VisibilitySimple, meta.VisibilityExpert:
	case meta.VisibilityOwner, meta.VisibilityExpert:
		return false
	case meta.VisibilityPublic:
		return true
	}
	if user == nil {
		return false
	}
75
76
77
78
79
80
81
82

83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

103
104
105
106
107
108
109
110
111
112

113
114
115
116
117
118
119
120
121
122

123
124
125
126
127
128
129
130
131


132
133
134
135
136
137
138
139
140

141
142
143
144
145
146
147
74
75
76
77
78
79
80

81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

101
102
103
104
105
106
107
108
109
110

111
112
113
114
115
116
117
118
119
120

121
122
123
124
125
126
127



128
129
130
131
132
133
134
135
136
137

138
139
140
141
142
143
144
145







-
+



















-
+









-
+









-
+






-
-
-
+
+








-
+







	meta.KeyUserRole,
}

func (o *ownerPolicy) CanWrite(user, oldMeta, newMeta *meta.Meta) bool {
	if user == nil || !o.pre.CanWrite(user, oldMeta, newMeta) {
		return false
	}
	vis := o.getVisibility(oldMeta)
	vis := o.authConfig.GetVisibility(oldMeta)
	if res, ok := o.checkVisibility(user, vis); ok {
		return res
	}
	if o.userIsOwner(user) {
		return true
	}
	if !o.userCanRead(user, oldMeta, vis) {
		return false
	}
	if role, ok := oldMeta.Get(meta.KeyRole); ok && role == meta.ValueRoleUser {
		// Here we know, that user.Zid == newMeta.Zid (because of userCanRead) and
		// user.Zid == newMeta.Zid (because oldMeta.Zid == newMeta.Zid)
		for _, key := range noChangeUser {
			if oldMeta.GetDefault(key, "") != newMeta.GetDefault(key, "") {
				return false
			}
		}
		return true
	}
	if runtime.GetUserRole(user) == meta.UserRoleReader {
	if o.manager.GetUserRole(user) == meta.UserRoleReader {
		return false
	}
	return o.userCanCreate(user, newMeta)
}

func (o *ownerPolicy) CanRename(user, m *meta.Meta) bool {
	if user == nil || !o.pre.CanRename(user, m) {
		return false
	}
	if res, ok := o.checkVisibility(user, o.getVisibility(m)); ok {
	if res, ok := o.checkVisibility(user, o.authConfig.GetVisibility(m)); ok {
		return res
	}
	return o.userIsOwner(user)
}

func (o *ownerPolicy) CanDelete(user, m *meta.Meta) bool {
	if user == nil || !o.pre.CanDelete(user, m) {
		return false
	}
	if res, ok := o.checkVisibility(user, o.getVisibility(m)); ok {
	if res, ok := o.checkVisibility(user, o.authConfig.GetVisibility(m)); ok {
		return res
	}
	return o.userIsOwner(user)
}

func (o *ownerPolicy) checkVisibility(user *meta.Meta, vis meta.Visibility) (bool, bool) {
	switch vis {
	case meta.VisibilitySimple, meta.VisibilityExpert:
		return o.userIsOwner(user) && o.expertMode(), true
	if vis == meta.VisibilityExpert {
		return o.userIsOwner(user) && o.authConfig.GetExpertMode(), true
	}
	return false, false
}

func (o *ownerPolicy) userIsOwner(user *meta.Meta) bool {
	if user == nil {
		return false
	}
	if o.isOwner(user.Zid) {
	if o.manager.IsOwner(user.Zid) {
		return true
	}
	if val, ok := user.Get(meta.KeyUserRole); ok && val == meta.ValueUserRoleOwner {
		return true
	}
	return false
}

Changes to auth/policy/place.go.

9
10
11
12
13
14
15

16


17
18
19
20
21
22

23
24
25
26


27
28
29

30
31
32
33
34
35
36



37
38
39
40

41
42

43
44
45
46

47

48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

63
64
65
66
67
68
69
70
71
72
73
74

75
76
77
78
79
80
81
82
83
84
85
86

87
88
89
90
91
92
93
94

95
96
97
98

99
100
101
102
103
104
105
106
107
108
109
110

111
112
113
114
115
116
117
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32


33







34
35
36
37
38
39
40
41
42

43
44
45
46

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71
72
73
74
75

76
77
78
79
80
81
82
83
84
85
86
87

88
89
90
91
92
93
94
95

96
97
98
99

100
101
102
103
104
105
106
107
108
109
110
111

112
113
114
115
116
117
118
119







+

+
+





-
+




+
+

-
-
+
-
-
-
-
-
-
-
+
+
+




+

-
+



-
+

+














-
+











-
+











-
+







-
+



-
+











-
+







//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"context"
	"io"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
	"zettelstore.de/z/web/session"
	"zettelstore.de/z/web/server"
)

// PlaceWithPolicy wraps the given place inside a policy place.
func PlaceWithPolicy(
	auth server.Auth,
	manager auth.AuthzManager,
	place place.Place,
	simpleMode bool,
	withAuth func() bool,
	authConfig config.AuthConfig,
	isReadOnlyMode bool,
	expertMode func() bool,
	isOwner func(id.Zid) bool,
	getVisibility func(*meta.Meta) meta.Visibility,
) (place.Place, Policy) {
	pol := newPolicy(simpleMode, withAuth, isReadOnlyMode, expertMode, isOwner, getVisibility)
	return newPlace(place, pol), pol
) (place.Place, auth.Policy) {
	pol := newPolicy(manager, authConfig)
	return newPlace(auth, place, pol), pol
}

// polPlace implements a policy place.
type polPlace struct {
	auth   server.Auth
	place  place.Place
	policy Policy
	policy auth.Policy
}

// newPlace creates a new policy place.
func newPlace(place place.Place, policy Policy) place.Place {
func newPlace(auth server.Auth, place place.Place, policy auth.Policy) place.Place {
	return &polPlace{
		auth:   auth,
		place:  place,
		policy: policy,
	}
}

func (pp *polPlace) Location() string {
	return pp.place.Location()
}

func (pp *polPlace) CanCreateZettel(ctx context.Context) bool {
	return pp.place.CanCreateZettel(ctx)
}

func (pp *polPlace) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	user := session.GetUser(ctx)
	user := pp.auth.GetUser(ctx)
	if pp.policy.CanCreate(user, zettel.Meta) {
		return pp.place.CreateZettel(ctx, zettel)
	}
	return id.Invalid, place.NewErrNotAllowed("Create", user, id.Invalid)
}

func (pp *polPlace) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	zettel, err := pp.place.GetZettel(ctx, zid)
	if err != nil {
		return domain.Zettel{}, err
	}
	user := session.GetUser(ctx)
	user := pp.auth.GetUser(ctx)
	if pp.policy.CanRead(user, zettel.Meta) {
		return zettel, nil
	}
	return domain.Zettel{}, place.NewErrNotAllowed("GetZettel", user, zid)
}

func (pp *polPlace) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	m, err := pp.place.GetMeta(ctx, zid)
	if err != nil {
		return nil, err
	}
	user := session.GetUser(ctx)
	user := pp.auth.GetUser(ctx)
	if pp.policy.CanRead(user, m) {
		return m, nil
	}
	return nil, place.NewErrNotAllowed("GetMeta", user, zid)
}

func (pp *polPlace) FetchZids(ctx context.Context) (id.Set, error) {
	return nil, place.NewErrNotAllowed("fetch-zids", session.GetUser(ctx), id.Invalid)
	return nil, place.NewErrNotAllowed("fetch-zids", pp.auth.GetUser(ctx), id.Invalid)
}

func (pp *polPlace) SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) {
	user := session.GetUser(ctx)
	user := pp.auth.GetUser(ctx)
	canRead := pp.policy.CanRead
	s = s.AddPreMatch(func(m *meta.Meta) bool { return canRead(user, m) })
	return pp.place.SelectMeta(ctx, s)
}

func (pp *polPlace) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
	return pp.place.CanUpdateZettel(ctx, zettel)
}

func (pp *polPlace) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	zid := zettel.Meta.Zid
	user := session.GetUser(ctx)
	user := pp.auth.GetUser(ctx)
	if !zid.IsValid() {
		return &place.ErrInvalidID{Zid: zid}
	}
	// Write existing zettel
	oldMeta, err := pp.place.GetMeta(ctx, zid)
	if err != nil {
		return err
127
128
129
130
131
132
133
134

135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150

151
152
153
154
155
156
157
158
159




129
130
131
132
133
134
135

136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151

152
153
154
155
156
157
158
159
160
161
162
163
164
165







-
+















-
+









+
+
+
+
}

func (pp *polPlace) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	meta, err := pp.place.GetMeta(ctx, curZid)
	if err != nil {
		return err
	}
	user := session.GetUser(ctx)
	user := pp.auth.GetUser(ctx)
	if pp.policy.CanRename(user, meta) {
		return pp.place.RenameZettel(ctx, curZid, newZid)
	}
	return place.NewErrNotAllowed("Rename", user, curZid)
}

func (pp *polPlace) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	return pp.place.CanDeleteZettel(ctx, zid)
}

func (pp *polPlace) DeleteZettel(ctx context.Context, zid id.Zid) error {
	meta, err := pp.place.GetMeta(ctx, zid)
	if err != nil {
		return err
	}
	user := session.GetUser(ctx)
	user := pp.auth.GetUser(ctx)
	if pp.policy.CanDelete(user, meta) {
		return pp.place.DeleteZettel(ctx, zid)
	}
	return place.NewErrNotAllowed("Delete", user, zid)
}

func (pp *polPlace) ReadStats(st *place.Stats) {
	pp.place.ReadStats(st)
}

func (pp *polPlace) Dump(w io.Writer) {
	pp.place.Dump(w)
}

Changes to auth/policy/policy.go.

8
9
10
11
12
13
14
15


16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

39
40
41
42
43
44
45
46
47


48
49
50

51
52

53
54
55

56
57


58
59
60
61

62
63
64

65
66
67
68
69
70
71

72
73
74
75
76
77
78
8
9
10
11
12
13
14

15
16
17
18
19


















20

21









22
23
24
25

26
27

28
29


30


31
32
33
34
35

36



37
38
39
40
41
42
43

44
45
46
47
48
49
50
51







-
+
+



-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-

-
+
-
-
-
-
-
-
-
-
-
+
+


-
+

-
+

-
-
+
-
-
+
+



-
+
-
-
-
+






-
+







// under this license.
//-----------------------------------------------------------------------------

// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/meta"
)

// Policy is an interface for checking access authorization.
type Policy interface {
	// User is allowed to create a new zettel.
	CanCreate(user, newMeta *meta.Meta) bool

	// User is allowed to read zettel
	CanRead(user, m *meta.Meta) bool

	// User is allowed to write zettel.
	CanWrite(user, oldMeta, newMeta *meta.Meta) bool

	// User is allowed to rename zettel
	CanRename(user, m *meta.Meta) bool

	// User is allowed to delete zettel
	CanDelete(user, m *meta.Meta) bool
}

// newPolicy creates a policy based on given constraints.
func newPolicy(
func newPolicy(manager auth.AuthzManager, authConfig config.AuthConfig) auth.Policy {
	simpleMode bool,
	withAuth func() bool,
	isReadOnlyMode bool,
	expertMode func() bool,
	isOwner func(id.Zid) bool,
	getVisibility func(*meta.Meta) meta.Visibility,
) Policy {
	var pol Policy
	if isReadOnlyMode {
	var pol auth.Policy
	if manager.IsReadonly() {
		pol = &roPolicy{}
	} else {
		pol = &defaultPolicy{}
		pol = &defaultPolicy{manager}
	}
	if withAuth() {
	if manager.WithAuth() {
		pol = &ownerPolicy{
			expertMode:    expertMode,
			isOwner:       isOwner,
			manager:    manager,
			getVisibility: getVisibility,
			pre:           pol,
			authConfig: authConfig,
			pre:        pol,
		}
	} else {
		pol = &anonPolicy{
			simpleMode:    simpleMode,
			authConfig: authConfig,
			expertMode:    expertMode,
			getVisibility: getVisibility,
			pre:           pol,
			pre:        pol,
		}
	}
	return &prePolicy{pol}
}

type prePolicy struct {
	post Policy
	post auth.Policy
}

func (p *prePolicy) CanCreate(user, newMeta *meta.Meta) bool {
	return newMeta != nil && p.post.CanCreate(user, newMeta)
}

func (p *prePolicy) CanRead(user, m *meta.Meta) bool {

Changes to auth/policy/policy_test.go.

11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28
29

30
31

32
33

34
35

36
37

38
39

40
41

42
43

44
45
46
47
48



49
50
51
52
53
54

55
56
57
58
59
60
61


62
63
64
65
66
67





68
69
70
71


72
73
74
75
76
77
































78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

94
95
96
97
98
99
100
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28

29


30


31


32


33


34


35


36

37
38


39
40
41



42


43







44
45
46





47
48
49
50
51
52
53
54
55
56
57






58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97


98
99
100
101
102

103
104
105
106
107
108
109
110







+






-




-
+
-
-
+
-
-
+
-
-
+
-
-
+
-
-
+
-
-
+
-
-
+
-


-
-
+
+
+
-
-
-

-
-
+
-
-
-
-
-
-
-
+
+

-
-
-
-
-
+
+
+
+
+




+
+
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+








-
-





-
+







// Package policy provides some interfaces and implementation for authorizsation policies.
package policy

import (
	"fmt"
	"testing"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

func TestPolicies(t *testing.T) {
	testScene := []struct {
		simple   bool
		readonly bool
		withAuth bool
		expert   bool
	}{
		{true, true, true, true},
		{true, true, true},
		{true, true, true, false},
		{true, true, false, true},
		{true, true, false},
		{true, true, false, false},
		{true, false, true, true},
		{true, false, true},
		{true, false, true, false},
		{true, false, false, true},
		{true, false, false},
		{true, false, false, false},
		{false, true, true, true},
		{false, true, true},
		{false, true, true, false},
		{false, true, false, true},
		{false, true, false},
		{false, true, false, false},
		{false, false, true, true},
		{false, false, true},
		{false, false, true, false},
		{false, false, false, true},
		{false, false, false},
		{false, false, false, false},
	}
	for _, ts := range testScene {
		var authFunc func() bool
		if ts.withAuth {
		authzManager := &testAuthzManager{
			readOnly: ts.readonly,
			withAuth: ts.withAuth,
			authFunc = withAuth
		} else {
			authFunc = withoutAuth
		}
		var expertFunc func() bool
		if ts.expert {
		pol := newPolicy(authzManager, &authConfig{ts.expert})
			expertFunc = expertMode
		} else {
			expertFunc = noExpertMode
		}
		pol := newPolicy(ts.simple, authFunc, ts.readonly, expertFunc, isOwner, getVisibility)
		name := fmt.Sprintf("simple=%v/readonly=%v/withauth=%v/expert=%v",
			ts.simple, ts.readonly, ts.withAuth, ts.expert)
		name := fmt.Sprintf("readonly=%v/withauth=%v/expert=%v",
			ts.readonly, ts.withAuth, ts.expert)
		t.Run(name, func(tt *testing.T) {
			testCreate(tt, pol, ts.simple, ts.withAuth, ts.readonly, ts.expert)
			testRead(tt, pol, ts.simple, ts.withAuth, ts.readonly, ts.expert)
			testWrite(tt, pol, ts.simple, ts.withAuth, ts.readonly, ts.expert)
			testRename(tt, pol, ts.simple, ts.withAuth, ts.readonly, ts.expert)
			testDelete(tt, pol, ts.simple, ts.withAuth, ts.readonly, ts.expert)
			testCreate(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testRead(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testWrite(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testRename(tt, pol, ts.withAuth, ts.readonly, ts.expert)
			testDelete(tt, pol, ts.withAuth, ts.readonly, ts.expert)
		})
	}
}

type testAuthzManager struct {
	readOnly bool
func withAuth() bool          { return true }
func withoutAuth() bool       { return false }
func expertMode() bool        { return true }
func noExpertMode() bool      { return false }
func isOwner(zid id.Zid) bool { return zid == ownerZid }
func getVisibility(m *meta.Meta) meta.Visibility {
	withAuth bool
}

func (a *testAuthzManager) IsReadonly() bool        { return a.readOnly }
func (a *testAuthzManager) Owner() id.Zid           { return ownerZid }
func (a *testAuthzManager) IsOwner(zid id.Zid) bool { return zid == ownerZid }

func (a *testAuthzManager) WithAuth() bool { return a.withAuth }

func (a *testAuthzManager) GetUserRole(user *meta.Meta) meta.UserRole {
	if user == nil {
		if a.WithAuth() {
			return meta.UserRoleUnknown
		}
		return meta.UserRoleOwner
	}
	if a.IsOwner(user.Zid) {
		return meta.UserRoleOwner
	}
	if val, ok := user.Get(meta.KeyUserRole); ok {
		if ur := meta.GetUserRole(val); ur != meta.UserRoleUnknown {
			return ur
		}
	}
	return meta.UserRoleReader
}

type authConfig struct{ expert bool }

func (ac *authConfig) GetExpertMode() bool { return ac.expert }

func (ac *authConfig) GetVisibility(m *meta.Meta) meta.Visibility {
	if vis, ok := m.Get(meta.KeyVisibility); ok {
		switch vis {
		case meta.ValueVisibilityPublic:
			return meta.VisibilityPublic
		case meta.ValueVisibilityOwner:
			return meta.VisibilityOwner
		case meta.ValueVisibilityExpert:
			return meta.VisibilityExpert
		case meta.ValueVisibilitySimple:
			return meta.VisibilitySimple
		}
	}
	return meta.VisibilityLogin
}

func testCreate(t *testing.T, pol Policy, simple, withAuth, readonly, isExpert bool) {
func testCreate(t *testing.T, pol auth.Policy, withAuth, readonly, isExpert bool) {
	t.Helper()
	anonUser := newAnon()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()
	zettel := newZettel()
129
130
131
132
133
134
135
136

137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
139
140
141
142
143
144
145

146
147
148
149
150
151
152
153
154
155
156
157

158
159
160
161
162
163
164







-
+











-







			if tc.exp != got {
				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testRead(t *testing.T, pol Policy, simple, withAuth, readonly, expert bool) {
func testRead(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {
	t.Helper()
	anonUser := newAnon()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()
	zettel := newZettel()
	publicZettel := newPublicZettel()
	loginZettel := newLoginZettel()
	ownerZettel := newOwnerZettel()
	expertZettel := newExpertZettel()
	simpleZettel := newSimpleZettel()
	userZettel := newUserZettel()
	testCases := []struct {
		user *meta.Meta
		meta *meta.Meta
		exp  bool
	}{
		// No meta
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
193
194
195
196
197
198
199






200
201
202
203
204
205
206







-
-
-
-
-
-







		{owner2, ownerZettel, true},
		// Expert zettel
		{anonUser, expertZettel, !withAuth && expert},
		{reader, expertZettel, !withAuth && expert},
		{writer, expertZettel, !withAuth && expert},
		{owner, expertZettel, expert},
		{owner2, expertZettel, expert},
		// Simple expert zettel
		{anonUser, simpleZettel, !withAuth && (simple || expert)},
		{reader, simpleZettel, !withAuth && (simple || expert)},
		{writer, simpleZettel, !withAuth && (simple || expert)},
		{owner, simpleZettel, (!withAuth && simple) || expert},
		{owner2, simpleZettel, (!withAuth && simple) || expert},
		// Other user zettel
		{anonUser, userZettel, !withAuth},
		{reader, userZettel, !withAuth},
		{writer, userZettel, !withAuth},
		{owner, userZettel, true},
		{owner2, userZettel, true},
		// Own user zettel
214
215
216
217
218
219
220
221

222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241

242
243
244
245
246
247
248
217
218
219
220
221
222
223

224
225
226
227
228
229
230
231
232
233
234
235

236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251







-
+











-








+







			if tc.exp != got {
				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testWrite(t *testing.T, pol Policy, simple, withAuth, readonly, expert bool) {
func testWrite(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {
	t.Helper()
	anonUser := newAnon()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()
	zettel := newZettel()
	publicZettel := newPublicZettel()
	loginZettel := newLoginZettel()
	ownerZettel := newOwnerZettel()
	expertZettel := newExpertZettel()
	simpleZettel := newSimpleZettel()
	userZettel := newUserZettel()
	writerNew := writer.Clone()
	writerNew.Set(meta.KeyUserRole, owner.GetDefault(meta.KeyUserRole, ""))
	roFalse := newRoFalseZettel()
	roTrue := newRoTrueZettel()
	roReader := newRoReaderZettel()
	roWriter := newRoWriterZettel()
	roOwner := newRoOwnerZettel()
	notAuthNotReadonly := !withAuth && !readonly
	testCases := []struct {
		user *meta.Meta
		old  *meta.Meta
		new  *meta.Meta
		exp  bool
	}{
		// No old and new meta
266
267
268
269
270
271
272
273
274


275
276
277
278
279
280


281
282
283
284
285
286


287
288
289
290
291
292
293



294
295
296
297
298
299



300
301
302
303
304
305
306
307
308
309
310
311



312
313
314
315
316
317
318
319
320

321
322
323


324
325
326
327
328
329
330
269
270
271
272
273
274
275


276
277
278
279
280
281


282
283
284
285
286
287


288
289
290
291
292
293



294
295
296
297
298
299



300
301
302
303
304






305



306
307
308
309
310
311
312
313
314
315
316

317
318


319
320
321
322
323
324
325
326
327







-
-
+
+




-
-
+
+




-
-
+
+




-
-
-
+
+
+



-
-
-
+
+
+


-
-
-
-
-
-

-
-
-
+
+
+








-
+

-
-
+
+







		// Old an new zettel have different zettel identifier
		{anonUser, zettel, publicZettel, false},
		{reader, zettel, publicZettel, false},
		{writer, zettel, publicZettel, false},
		{owner, zettel, publicZettel, false},
		{owner2, zettel, publicZettel, false},
		// Overwrite a normal zettel
		{anonUser, zettel, zettel, !withAuth && !readonly},
		{reader, zettel, zettel, !withAuth && !readonly},
		{anonUser, zettel, zettel, notAuthNotReadonly},
		{reader, zettel, zettel, notAuthNotReadonly},
		{writer, zettel, zettel, !readonly},
		{owner, zettel, zettel, !readonly},
		{owner2, zettel, zettel, !readonly},
		// Public zettel
		{anonUser, publicZettel, publicZettel, !withAuth && !readonly},
		{reader, publicZettel, publicZettel, !withAuth && !readonly},
		{anonUser, publicZettel, publicZettel, notAuthNotReadonly},
		{reader, publicZettel, publicZettel, notAuthNotReadonly},
		{writer, publicZettel, publicZettel, !readonly},
		{owner, publicZettel, publicZettel, !readonly},
		{owner2, publicZettel, publicZettel, !readonly},
		// Login zettel
		{anonUser, loginZettel, loginZettel, !withAuth && !readonly},
		{reader, loginZettel, loginZettel, !withAuth && !readonly},
		{anonUser, loginZettel, loginZettel, notAuthNotReadonly},
		{reader, loginZettel, loginZettel, notAuthNotReadonly},
		{writer, loginZettel, loginZettel, !readonly},
		{owner, loginZettel, loginZettel, !readonly},
		{owner2, loginZettel, loginZettel, !readonly},
		// Owner zettel
		{anonUser, ownerZettel, ownerZettel, !withAuth && !readonly},
		{reader, ownerZettel, ownerZettel, !withAuth && !readonly},
		{writer, ownerZettel, ownerZettel, !withAuth && !readonly},
		{anonUser, ownerZettel, ownerZettel, notAuthNotReadonly},
		{reader, ownerZettel, ownerZettel, notAuthNotReadonly},
		{writer, ownerZettel, ownerZettel, notAuthNotReadonly},
		{owner, ownerZettel, ownerZettel, !readonly},
		{owner2, ownerZettel, ownerZettel, !readonly},
		// Expert zettel
		{anonUser, expertZettel, expertZettel, !withAuth && !readonly && expert},
		{reader, expertZettel, expertZettel, !withAuth && !readonly && expert},
		{writer, expertZettel, expertZettel, !withAuth && !readonly && expert},
		{anonUser, expertZettel, expertZettel, notAuthNotReadonly && expert},
		{reader, expertZettel, expertZettel, notAuthNotReadonly && expert},
		{writer, expertZettel, expertZettel, notAuthNotReadonly && expert},
		{owner, expertZettel, expertZettel, !readonly && expert},
		{owner2, expertZettel, expertZettel, !readonly && expert},
		// Simple expert zettel
		{anonUser, simpleZettel, expertZettel, !withAuth && !readonly && (simple || expert)},
		{reader, simpleZettel, expertZettel, !withAuth && !readonly && (simple || expert)},
		{writer, simpleZettel, expertZettel, !withAuth && !readonly && (simple || expert)},
		{owner, simpleZettel, expertZettel, !readonly && ((!withAuth && simple) || expert)},
		{owner2, simpleZettel, expertZettel, !readonly && ((!withAuth && simple) || expert)},
		// Other user zettel
		{anonUser, userZettel, userZettel, !withAuth && !readonly},
		{reader, userZettel, userZettel, !withAuth && !readonly},
		{writer, userZettel, userZettel, !withAuth && !readonly},
		{anonUser, userZettel, userZettel, notAuthNotReadonly},
		{reader, userZettel, userZettel, notAuthNotReadonly},
		{writer, userZettel, userZettel, notAuthNotReadonly},
		{owner, userZettel, userZettel, !readonly},
		{owner2, userZettel, userZettel, !readonly},
		// Own user zettel
		{reader, reader, reader, !readonly},
		{writer, writer, writer, !readonly},
		{owner, owner, owner, !readonly},
		{owner2, owner2, owner2, !readonly},
		// Writer cannot change importand metadata of its own user zettel
		{writer, writer, writerNew, !withAuth && !readonly},
		{writer, writer, writerNew, notAuthNotReadonly},
		// No r/o zettel
		{anonUser, roFalse, roFalse, !withAuth && !readonly},
		{reader, roFalse, roFalse, !withAuth && !readonly},
		{anonUser, roFalse, roFalse, notAuthNotReadonly},
		{reader, roFalse, roFalse, notAuthNotReadonly},
		{writer, roFalse, roFalse, !readonly},
		{owner, roFalse, roFalse, !readonly},
		{owner2, roFalse, roFalse, !readonly},
		// Reader r/o zettel
		{anonUser, roReader, roReader, false},
		{reader, roReader, roReader, false},
		{writer, roReader, roReader, !readonly},
355
356
357
358
359
360
361
362

363
364
365
366
367
368
369
370
371
372
373
374
375
376

377
378
379
380
381
382
383
384
385
386
387
388
389
390
391



392
393
394
395
396
397



398
399
400
401
402
403
404
405
406
407
408
409



410
411
412
413
414
415

416
417
418
419
420
421
422
352
353
354
355
356
357
358

359
360
361
362
363
364
365
366
367

368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385



386
387
388
389
390
391



392
393
394
395
396






397



398
399
400
401
402
403
404
405

406
407
408
409
410
411
412
413







-
+








-





+












-
-
-
+
+
+



-
-
-
+
+
+


-
-
-
-
-
-

-
-
-
+
+
+





-
+







			if tc.exp != got {
				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testRename(t *testing.T, pol Policy, simple, withAuth, readonly, expert bool) {
func testRename(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {
	t.Helper()
	anonUser := newAnon()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()
	zettel := newZettel()
	expertZettel := newExpertZettel()
	simpleZettel := newSimpleZettel()
	roFalse := newRoFalseZettel()
	roTrue := newRoTrueZettel()
	roReader := newRoReaderZettel()
	roWriter := newRoWriterZettel()
	roOwner := newRoOwnerZettel()
	notAuthNotReadonly := !withAuth && !readonly
	testCases := []struct {
		user *meta.Meta
		meta *meta.Meta
		exp  bool
	}{
		// No meta
		{anonUser, nil, false},
		{reader, nil, false},
		{writer, nil, false},
		{owner, nil, false},
		{owner2, nil, false},
		// Any zettel
		{anonUser, zettel, !withAuth && !readonly},
		{reader, zettel, !withAuth && !readonly},
		{writer, zettel, !withAuth && !readonly},
		{anonUser, zettel, notAuthNotReadonly},
		{reader, zettel, notAuthNotReadonly},
		{writer, zettel, notAuthNotReadonly},
		{owner, zettel, !readonly},
		{owner2, zettel, !readonly},
		// Expert zettel
		{anonUser, expertZettel, !withAuth && !readonly && expert},
		{reader, expertZettel, !withAuth && !readonly && expert},
		{writer, expertZettel, !withAuth && !readonly && expert},
		{anonUser, expertZettel, notAuthNotReadonly && expert},
		{reader, expertZettel, notAuthNotReadonly && expert},
		{writer, expertZettel, notAuthNotReadonly && expert},
		{owner, expertZettel, !readonly && expert},
		{owner2, expertZettel, !readonly && expert},
		// Simple expert zettel
		{anonUser, simpleZettel, !withAuth && !readonly && (simple || expert)},
		{reader, simpleZettel, !withAuth && !readonly && (simple || expert)},
		{writer, simpleZettel, !withAuth && !readonly && (simple || expert)},
		{owner, simpleZettel, !readonly && ((!withAuth && simple) || expert)},
		{owner2, simpleZettel, !readonly && ((!withAuth && simple) || expert)},
		// No r/o zettel
		{anonUser, roFalse, !withAuth && !readonly},
		{reader, roFalse, !withAuth && !readonly},
		{writer, roFalse, !withAuth && !readonly},
		{anonUser, roFalse, notAuthNotReadonly},
		{reader, roFalse, notAuthNotReadonly},
		{writer, roFalse, notAuthNotReadonly},
		{owner, roFalse, !readonly},
		{owner2, roFalse, !readonly},
		// Reader r/o zettel
		{anonUser, roReader, false},
		{reader, roReader, false},
		{writer, roReader, !withAuth && !readonly},
		{writer, roReader, notAuthNotReadonly},
		{owner, roReader, !readonly},
		{owner2, roReader, !readonly},
		// Writer r/o zettel
		{anonUser, roWriter, false},
		{reader, roWriter, false},
		{writer, roWriter, false},
		{owner, roWriter, !readonly},
440
441
442
443
444
445
446
447

448
449
450
451
452
453
454
455
456
457
458
459
460
461

462
463
464
465
466
467
468
469
470
471
472
473
474
475
476



477
478
479
480
481
482



483
484
485
486
487
488
489
490
491
492
493
494



495
496
497
498
499
500

501
502
503
504
505
506
507
431
432
433
434
435
436
437

438
439
440
441
442
443
444
445
446

447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464



465
466
467
468
469
470



471
472
473
474
475






476



477
478
479
480
481
482
483
484

485
486
487
488
489
490
491
492







-
+








-





+












-
-
-
+
+
+



-
-
-
+
+
+


-
-
-
-
-
-

-
-
-
+
+
+





-
+







			if tc.exp != got {
				tt.Errorf("exp=%v, but got=%v", tc.exp, got)
			}
		})
	}
}

func testDelete(t *testing.T, pol Policy, simple, withAuth, readonly, expert bool) {
func testDelete(t *testing.T, pol auth.Policy, withAuth, readonly, expert bool) {
	t.Helper()
	anonUser := newAnon()
	reader := newReader()
	writer := newWriter()
	owner := newOwner()
	owner2 := newOwner2()
	zettel := newZettel()
	expertZettel := newExpertZettel()
	simpleZettel := newSimpleZettel()
	roFalse := newRoFalseZettel()
	roTrue := newRoTrueZettel()
	roReader := newRoReaderZettel()
	roWriter := newRoWriterZettel()
	roOwner := newRoOwnerZettel()
	notAuthNotReadonly := !withAuth && !readonly
	testCases := []struct {
		user *meta.Meta
		meta *meta.Meta
		exp  bool
	}{
		// No meta
		{anonUser, nil, false},
		{reader, nil, false},
		{writer, nil, false},
		{owner, nil, false},
		{owner2, nil, false},
		// Any zettel
		{anonUser, zettel, !withAuth && !readonly},
		{reader, zettel, !withAuth && !readonly},
		{writer, zettel, !withAuth && !readonly},
		{anonUser, zettel, notAuthNotReadonly},
		{reader, zettel, notAuthNotReadonly},
		{writer, zettel, notAuthNotReadonly},
		{owner, zettel, !readonly},
		{owner2, zettel, !readonly},
		// Expert zettel
		{anonUser, expertZettel, !withAuth && !readonly && expert},
		{reader, expertZettel, !withAuth && !readonly && expert},
		{writer, expertZettel, !withAuth && !readonly && expert},
		{anonUser, expertZettel, notAuthNotReadonly && expert},
		{reader, expertZettel, notAuthNotReadonly && expert},
		{writer, expertZettel, notAuthNotReadonly && expert},
		{owner, expertZettel, !readonly && expert},
		{owner2, expertZettel, !readonly && expert},
		// Simple expert zettel
		{anonUser, simpleZettel, !withAuth && !readonly && (simple || expert)},
		{reader, simpleZettel, !withAuth && !readonly && (simple || expert)},
		{writer, simpleZettel, !withAuth && !readonly && (simple || expert)},
		{owner, simpleZettel, !readonly && ((!withAuth && simple) || expert)},
		{owner2, simpleZettel, !readonly && ((!withAuth && simple) || expert)},
		// No r/o zettel
		{anonUser, roFalse, !withAuth && !readonly},
		{reader, roFalse, !withAuth && !readonly},
		{writer, roFalse, !withAuth && !readonly},
		{anonUser, roFalse, notAuthNotReadonly},
		{reader, roFalse, notAuthNotReadonly},
		{writer, roFalse, notAuthNotReadonly},
		{owner, roFalse, !readonly},
		{owner2, roFalse, !readonly},
		// Reader r/o zettel
		{anonUser, roReader, false},
		{reader, roReader, false},
		{writer, roReader, !withAuth && !readonly},
		{writer, roReader, notAuthNotReadonly},
		{owner, roReader, !readonly},
		{owner2, roReader, !readonly},
		// Writer r/o zettel
		{anonUser, roWriter, false},
		{reader, roWriter, false},
		{writer, roWriter, false},
		{owner, roWriter, !readonly},
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
578
579
580
581
582
583
584






585
586
587
588
589
590
591







-
-
-
-
-
-







}
func newExpertZettel() *meta.Meta {
	m := meta.New(visZid)
	m.Set(meta.KeyTitle, "Expert Zettel")
	m.Set(meta.KeyVisibility, meta.ValueVisibilityExpert)
	return m
}
func newSimpleZettel() *meta.Meta {
	m := meta.New(visZid)
	m.Set(meta.KeyTitle, "Simple Zettel")
	m.Set(meta.KeyVisibility, meta.ValueVisibilitySimple)
	return m
}
func newRoFalseZettel() *meta.Meta {
	m := meta.New(zettelZid)
	m.Set(meta.KeyTitle, "No r/o Zettel")
	m.Set(meta.KeyReadOnly, "false")
	return m
}
func newRoTrueZettel() *meta.Meta {

Deleted auth/token/token.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package token provides some function for handling auth token.
package token

import (
	"errors"
	"time"

	"github.com/pascaldekloe/jwt"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

const reqHash = jwt.HS512

// ErrNoUser signals that the meta data has no role value 'user'.
var ErrNoUser = errors.New("auth: meta is no user")

// ErrNoIdent signals that the 'ident' key is missing.
var ErrNoIdent = errors.New("auth: missing ident")

// ErrOtherKind signals that the token was defined for another token kind.
var ErrOtherKind = errors.New("auth: wrong token kind")

// ErrNoZid signals that the 'zid' key is missing.
var ErrNoZid = errors.New("auth: missing zettel id")

// Kind specifies for which application / usage a token is/was requested.
type Kind int

// Allowed values of token kind
const (
	_ Kind = iota
	KindJSON
	KindHTML
)

// GetToken returns a token to be used for authentification.
func GetToken(ident *meta.Meta, d time.Duration, kind Kind) ([]byte, error) {
	if role, ok := ident.Get(meta.KeyRole); !ok || role != meta.ValueRoleUser {
		return nil, ErrNoUser
	}
	subject, ok := ident.Get(meta.KeyUserID)
	if !ok || subject == "" {
		return nil, ErrNoIdent
	}

	now := time.Now().Round(time.Second)
	claims := jwt.Claims{
		Registered: jwt.Registered{
			Subject: subject,
			Expires: jwt.NewNumericTime(now.Add(d)),
			Issued:  jwt.NewNumericTime(now),
		},
		Set: map[string]interface{}{
			"zid": ident.Zid.String(),
			"_tk": int(kind),
		},
	}
	token, err := claims.HMACSign(reqHash, startup.Secret())
	if err != nil {
		return nil, err
	}
	return token, nil
}

// ErrTokenExpired signals an exired token
var ErrTokenExpired = errors.New("auth: token expired")

// Data contains some important elements from a token.
type Data struct {
	Token   []byte
	Now     time.Time
	Issued  time.Time
	Expires time.Time
	Ident   string
	Zid     id.Zid
}

// CheckToken checks the validity of the token and returns relevant data.
func CheckToken(token []byte, k Kind) (Data, error) {
	h, err := jwt.NewHMAC(reqHash, startup.Secret())
	if err != nil {
		return Data{}, err
	}
	claims, err := h.Check(token)
	if err != nil {
		return Data{}, err
	}
	now := time.Now().Round(time.Second)
	expires := claims.Expires.Time()
	if expires.Before(now) {
		return Data{}, ErrTokenExpired
	}
	ident := claims.Subject
	if ident == "" {
		return Data{}, ErrNoIdent
	}
	if zidS, ok := claims.Set["zid"].(string); ok {
		if zid, err := id.Parse(zidS); err == nil {
			if kind, ok := claims.Set["_tk"].(float64); ok {
				if Kind(kind) == k {
					return Data{
						Token:   token,
						Now:     now,
						Issued:  claims.Issued.Time(),
						Expires: expires,
						Ident:   ident,
						Zid:     zid,
					}, nil
				}
			}
			return Data{}, ErrOtherKind
		}
	}
	return Data{}, ErrNoZid
}

Deleted cmd/cmd_config.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40








































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"fmt"

	"zettelstore.de/z/config/startup"
)

// ---------- Subcommand: config ---------------------------------------------

func cmdConfig(*flag.FlagSet) (int, error) {
	fmtVersion()
	fmt.Println("Stores")
	fmt.Printf("  Read-only mode    = %v\n", startup.IsReadOnlyMode())
	fmt.Println("Web")
	fmt.Printf("  Listen address    = %q\n", startup.ListenAddress())
	fmt.Printf("  URL prefix        = %q\n", startup.URLPrefix())
	if startup.WithAuth() {
		fmt.Println("Auth")
		fmt.Printf("  Owner             = %v\n", startup.Owner())
		fmt.Printf("  Secure cookie     = %v\n", startup.SecureCookie())
		fmt.Printf("  Persistent cookie = %v\n", startup.PersistentCookie())
		htmlLifetime, apiLifetime := startup.TokenLifetime()
		fmt.Printf("  HTML lifetime     = %v\n", htmlLifetime)
		fmt.Printf("  API lifetime      = %v\n", apiLifetime)
	}

	return 0, nil
}

Changes to cmd/cmd_file.go.

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

31
32
33


34
35
36
37
38

39
40
41


42
43

44
45
46
47
48
49
50
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28

29
30


31
32
33
34
35
36

37
38
39

40
41
42

43
44
45
46
47
48
49
50







-










-
+

-
-
+
+




-
+


-
+
+

-
+








import (
	"flag"
	"fmt"
	"io"
	"os"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
)

// ---------- Subcommand: file -----------------------------------------------

func cmdFile(fs *flag.FlagSet) (int, error) {
func cmdFile(fs *flag.FlagSet, cfg *meta.Meta) (int, error) {
	format := fs.Lookup("t").Value.String()
	meta, inp, err := getInput(fs.Args())
	if meta == nil {
	m, inp, err := getInput(fs.Args())
	if m == nil {
		return 2, err
	}
	z := parser.ParseZettel(
		domain.Zettel{
			Meta:    meta,
			Meta:    m,
			Content: domain.NewContent(inp.Src[inp.Pos:]),
		},
		runtime.GetSyntax(meta),
		m.GetDefault(meta.KeySyntax, meta.ValueSyntaxZmk),
		nil,
	)
	enc := encoder.Create(format, &encoder.Environment{Lang: runtime.GetLang(meta)})
	enc := encoder.Create(format, &encoder.Environment{Lang: m.GetDefault(meta.KeyLang, meta.ValueLangEN)})
	if enc == nil {
		fmt.Fprintf(os.Stderr, "Unknown format %q\n", format)
		return 2, nil
	}
	_, err = enc.WriteZettel(os.Stdout, z, format != "raw")
	if err != nil {
		return 2, err

Changes to cmd/cmd_password.go.

20
21
22
23
24
25
26
27

28
29
30
31
32
33
34
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34







-
+







	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// ---------- Subcommand: password -------------------------------------------

func cmdPassword(fs *flag.FlagSet) (int, error) {
func cmdPassword(fs *flag.FlagSet, cfg *meta.Meta) (int, error) {
	if fs.NArg() == 0 {
		fmt.Fprintln(os.Stderr, "User name and user zettel identification missing")
		return 2, nil
	}
	if fs.NArg() == 1 {
		fmt.Fprintln(os.Stderr, "User zettel identification missing")
		return 2, nil

Changes to cmd/cmd_run.go.

8
9
10
11
12
13
14
15
16
17
18
19
20




21
22
23
24
25
26
27
28
29
30
31
32
33
34

35

36
37
38
39
40
41
42
43



44
45
46
47
48








49
50
51

52
53
54
55



56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

72
73
74

75
76
77


78
79
80
81
82
83
84
85
86









87
88
89
90
91



92
93
94
95
96
97





98
99
100
101
102
103





104
105

106
107
108
109



110
111
112
113
114
115





116
117
118
119



120
121
122
123
124
125
126
127
128







129
130

131
132
133
134
135
136
137
138
139
140









141


142


143
8
9
10
11
12
13
14

15
16



17
18
19
20
21
22
23
24
25

26

27
28
29
30
31
32
33

34
35
36
37
38
39
40


41
42
43





44
45
46
47
48
49
50
51



52




53
54
55
56
57
58
59
60











61



62



63
64
65








66
67
68
69
70
71
72
73
74
75




76
77
78
79





80
81
82
83
84






85
86
87
88
89


90




91
92
93






94
95
96
97
98




99
100
101

102







103
104
105
106
107
108
109


110
111









112
113
114
115
116
117
118
119
120
121
122
123

124
125
126







-


-
-
-
+
+
+
+





-

-






+
-
+






-
-
+
+
+
-
-
-
-
-
+
+
+
+
+
+
+
+
-
-
-
+
-
-
-
-
+
+
+





-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
+
-
-
-
+
+

-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+

-
-
-
-
+
+
+

-
-
-
-
-
+
+
+
+
+
-
-
-
-
-
-
+
+
+
+
+
-
-
+
-
-
-
-
+
+
+
-
-
-
-
-
-
+
+
+
+
+
-
-
-
-
+
+
+
-

-
-
-
-
-
-
-
+
+
+
+
+
+
+
-
-
+

-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+

+
+
-
+
+

// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"log"
	"net/http"

	"zettelstore.de/z/auth/policy"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/adapter/api"
	"zettelstore.de/z/web/adapter/webui"
	"zettelstore.de/z/web/router"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/web/session"
)

// ---------- Subcommand: run ------------------------------------------------

func flgRun(fs *flag.FlagSet) {
	fs.String("c", defConfigfile, "configuration file")
	fs.Uint("a", 0, "port number kernel service (0=disable)")
	fs.Uint("p", 23123, "port number")
	fs.Uint("p", 23123, "port number web service")
	fs.String("d", "", "zettel directory")
	fs.Bool("r", false, "system-wide read-only mode")
	fs.Bool("v", false, "verbose mode")
	fs.Bool("debug", false, "debug mode")
}

func enableDebug(fs *flag.FlagSet, srv *server.Server) {
	if dbg := fs.Lookup("debug"); dbg != nil && dbg.Value.String() == "true" {
func withDebug(fs *flag.FlagSet) bool {
	dbg := fs.Lookup("debug")
	return dbg != nil && dbg.Value.String() == "true"
		srv.SetDebug()
	}
}

func runFunc(fs *flag.FlagSet) (int, error) {
}

func runFunc(fs *flag.FlagSet, cfg *meta.Meta) (int, error) {
	exitCode, err := doRun(withDebug(fs))
	kernel.Main.WaitForShutdown()
	return exitCode, err
}

	listenAddr := startup.ListenAddress()
	readonlyMode := startup.IsReadOnlyMode()
	logBeforeRun(listenAddr, readonlyMode)
func doRun(debug bool) (int, error) {
	handler := setupRouting(startup.PlaceManager(), readonlyMode)
	srv := server.New(listenAddr, handler)
	enableDebug(fs, srv)
	if err := srv.Run(); err != nil {
	kern := kernel.Main
	kern.SetDebug(debug)
	if err := kern.StartService(kernel.WebService); err != nil {
		return 1, err
	}
	return 0, nil
}

func logBeforeRun(listenAddr string, readonlyMode bool) {
	v := startup.GetVersion()
	log.Printf("%v %v (%v@%v/%v)", v.Prog, v.Build, v.GoVersion, v.Os, v.Arch)
	log.Println("Licensed under the latest version of the EUPL (European Union Public License)")
	log.Printf("Listening on %v", listenAddr)
	log.Printf("Zettel location [%v]", startup.PlaceManager().Location())
	if readonlyMode {
		log.Println("Read-only mode")
	}
}

func setupRouting(webSrv server.Server, placeManager place.Manager, authManager auth.Manager, rtConfig config.Config) {
func setupRouting(mgr place.Manager, readonlyMode bool) http.Handler {
	var up place.Place = mgr
	pp, pol := policy.PlaceWithPolicy(
	protectedPlaceManager, authPolicy := authManager.PlaceWithPolicy(webSrv, placeManager, rtConfig)
		up, startup.IsSimple(), startup.WithAuth, readonlyMode, runtime.GetExpertMode,
		startup.IsOwner, runtime.GetVisibility)
	te := webui.NewTemplateEngine(mgr, pol)
	api := api.New(webSrv, authManager, authManager, webSrv, rtConfig)
	wui := webui.New(webSrv, authManager, rtConfig, authManager, placeManager, authPolicy)

	ucAuthenticate := usecase.NewAuthenticate(up)
	ucGetMeta := usecase.NewGetMeta(pp)
	ucGetZettel := usecase.NewGetZettel(pp)
	ucParseZettel := usecase.NewParseZettel(ucGetZettel)
	ucListMeta := usecase.NewListMeta(pp)
	ucListRoles := usecase.NewListRole(pp)
	ucListTags := usecase.NewListTags(pp)
	ucZettelContext := usecase.NewZettelContext(pp)
	ucAuthenticate := usecase.NewAuthenticate(authManager, authManager, placeManager)
	ucCreateZettel := usecase.NewCreateZettel(rtConfig, protectedPlaceManager)
	ucGetMeta := usecase.NewGetMeta(protectedPlaceManager)
	ucGetZettel := usecase.NewGetZettel(protectedPlaceManager)
	ucParseZettel := usecase.NewParseZettel(rtConfig, ucGetZettel)
	ucListMeta := usecase.NewListMeta(protectedPlaceManager)
	ucListRoles := usecase.NewListRole(protectedPlaceManager)
	ucListTags := usecase.NewListTags(protectedPlaceManager)
	ucZettelContext := usecase.NewZettelContext(protectedPlaceManager)

	router := router.NewRouter()
	router.Handle("/", webui.MakeGetRootHandler(te, pp))
	router.AddListRoute('a', http.MethodGet, webui.MakeGetLoginHandler(te))
	router.AddListRoute('a', http.MethodPost, adapter.MakePostLoginHandler(
	webSrv.Handle("/", wui.MakeGetRootHandler(protectedPlaceManager))
	webSrv.AddListRoute('a', http.MethodGet, wui.MakeGetLoginHandler())
	webSrv.AddListRoute('a', http.MethodPost, adapter.MakePostLoginHandler(
		api.MakePostLoginHandlerAPI(ucAuthenticate),
		webui.MakePostLoginHandlerHTML(te, ucAuthenticate)))
	router.AddListRoute('a', http.MethodPut, api.MakeRenewAuthHandler())
	router.AddZettelRoute('a', http.MethodGet, webui.MakeGetLogoutHandler(te))
	if !readonlyMode {
		router.AddZettelRoute('b', http.MethodGet, webui.MakeGetRenameZettelHandler(
		wui.MakePostLoginHandlerHTML(ucAuthenticate)))
	webSrv.AddListRoute('a', http.MethodPut, api.MakeRenewAuthHandler())
	webSrv.AddZettelRoute('a', http.MethodGet, wui.MakeGetLogoutHandler())
	if !authManager.IsReadonly() {
		webSrv.AddZettelRoute('b', http.MethodGet, wui.MakeGetRenameZettelHandler(ucGetMeta))
			te, ucGetMeta))
		router.AddZettelRoute('b', http.MethodPost, webui.MakePostRenameZettelHandler(
			te, usecase.NewRenameZettel(pp)))
		router.AddZettelRoute('c', http.MethodGet, webui.MakeGetCopyZettelHandler(
			te, ucGetZettel, usecase.NewCopyZettel()))
		router.AddZettelRoute('c', http.MethodPost, webui.MakePostCreateZettelHandler(
		webSrv.AddZettelRoute('b', http.MethodPost, wui.MakePostRenameZettelHandler(
			usecase.NewRenameZettel(protectedPlaceManager)))
		webSrv.AddZettelRoute('c', http.MethodGet, wui.MakeGetCopyZettelHandler(
			ucGetZettel, usecase.NewCopyZettel()))
		webSrv.AddZettelRoute('c', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
			te, usecase.NewCreateZettel(pp)))
		router.AddZettelRoute('d', http.MethodGet, webui.MakeGetDeleteZettelHandler(
		webSrv.AddZettelRoute('d', http.MethodGet, wui.MakeGetDeleteZettelHandler(ucGetZettel))
			te, ucGetZettel))
		router.AddZettelRoute('d', http.MethodPost, webui.MakePostDeleteZettelHandler(
			te, usecase.NewDeleteZettel(pp)))
		router.AddZettelRoute('e', http.MethodGet, webui.MakeEditGetZettelHandler(
		webSrv.AddZettelRoute('d', http.MethodPost, wui.MakePostDeleteZettelHandler(
			usecase.NewDeleteZettel(protectedPlaceManager)))
		webSrv.AddZettelRoute('e', http.MethodGet, wui.MakeEditGetZettelHandler(ucGetZettel))
			te, ucGetZettel))
		router.AddZettelRoute('e', http.MethodPost, webui.MakeEditSetZettelHandler(
			te, usecase.NewUpdateZettel(pp)))
		router.AddZettelRoute('f', http.MethodGet, webui.MakeGetFolgeZettelHandler(
			te, ucGetZettel, usecase.NewFolgeZettel()))
		router.AddZettelRoute('f', http.MethodPost, webui.MakePostCreateZettelHandler(
		webSrv.AddZettelRoute('e', http.MethodPost, wui.MakeEditSetZettelHandler(
			usecase.NewUpdateZettel(protectedPlaceManager)))
		webSrv.AddZettelRoute('f', http.MethodGet, wui.MakeGetFolgeZettelHandler(
			ucGetZettel, usecase.NewFolgeZettel(rtConfig)))
		webSrv.AddZettelRoute('f', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
			te, usecase.NewCreateZettel(pp)))
		router.AddZettelRoute('g', http.MethodGet, webui.MakeGetNewZettelHandler(
			te, ucGetZettel, usecase.NewNewZettel()))
		router.AddZettelRoute('g', http.MethodPost, webui.MakePostCreateZettelHandler(
		webSrv.AddZettelRoute('g', http.MethodGet, wui.MakeGetNewZettelHandler(
			ucGetZettel, usecase.NewNewZettel()))
		webSrv.AddZettelRoute('g', http.MethodPost, wui.MakePostCreateZettelHandler(ucCreateZettel))
			te, usecase.NewCreateZettel(pp)))
	}
	router.AddListRoute('f', http.MethodGet, webui.MakeSearchHandler(
		te, usecase.NewSearch(pp), ucGetMeta, ucGetZettel))
	router.AddListRoute('h', http.MethodGet, webui.MakeListHTMLMetaHandler(
		te, ucListMeta, ucListRoles, ucListTags))
	router.AddZettelRoute('h', http.MethodGet, webui.MakeGetHTMLZettelHandler(
		te, ucParseZettel, ucGetMeta))
	router.AddZettelRoute('i', http.MethodGet, webui.MakeGetInfoHandler(
	webSrv.AddListRoute('f', http.MethodGet, wui.MakeSearchHandler(
		usecase.NewSearch(protectedPlaceManager), ucGetMeta, ucGetZettel))
	webSrv.AddListRoute('h', http.MethodGet, wui.MakeListHTMLMetaHandler(
		ucListMeta, ucListRoles, ucListTags))
	webSrv.AddZettelRoute('h', http.MethodGet, wui.MakeGetHTMLZettelHandler(
		ucParseZettel, ucGetMeta))
	webSrv.AddZettelRoute('i', http.MethodGet, wui.MakeGetInfoHandler(ucParseZettel, ucGetMeta))
		te, ucParseZettel, ucGetMeta))
	router.AddZettelRoute('j', http.MethodGet, webui.MakeZettelContextHandler(te, ucZettelContext))
	webSrv.AddZettelRoute('j', http.MethodGet, wui.MakeZettelContextHandler(ucZettelContext))

	router.AddZettelRoute('l', http.MethodGet, api.MakeGetLinksHandler(ucParseZettel))
	router.AddZettelRoute('o', http.MethodGet, api.MakeGetOrderHandler(
		usecase.NewZettelOrder(pp, ucParseZettel)))
	router.AddListRoute('r', http.MethodGet, api.MakeListRoleHandler(ucListRoles))
	router.AddListRoute('t', http.MethodGet, api.MakeListTagsHandler(ucListTags))
	router.AddZettelRoute('y', http.MethodGet, api.MakeZettelContextHandler(ucZettelContext))
	router.AddListRoute('z', http.MethodGet, api.MakeListMetaHandler(
		usecase.NewListMeta(pp), ucGetMeta, ucParseZettel))
	router.AddZettelRoute('z', http.MethodGet, api.MakeGetZettelHandler(
	webSrv.AddZettelRoute('l', http.MethodGet, api.MakeGetLinksHandler(ucParseZettel))
	webSrv.AddZettelRoute('o', http.MethodGet, api.MakeGetOrderHandler(
		usecase.NewZettelOrder(protectedPlaceManager, ucParseZettel)))
	webSrv.AddListRoute('r', http.MethodGet, api.MakeListRoleHandler(ucListRoles))
	webSrv.AddListRoute('t', http.MethodGet, api.MakeListTagsHandler(ucListTags))
	webSrv.AddZettelRoute('y', http.MethodGet, api.MakeZettelContextHandler(ucZettelContext))
	webSrv.AddListRoute('z', http.MethodGet, api.MakeListMetaHandler(
		usecase.NewListMeta(protectedPlaceManager), ucGetMeta, ucParseZettel))
	webSrv.AddZettelRoute('z', http.MethodGet, api.MakeGetZettelHandler(
		ucParseZettel, ucGetMeta))

	if authManager.WithAuth() {
	return session.NewHandler(router, usecase.NewGetUserByZid(up))
		webSrv.SetUserRetriever(usecase.NewGetUserByZid(placeManager))
	}
}

Changes to cmd/cmd_run_simple.go.

9
10
11
12
13
14
15
16
17
18
19
20
21


22
23
24
25
26
27
28
29



30
31

32
33
34
35
36
37






38
39

40
41
42
43
44


45
46

47
48
49
50

51
52
53
54
55
56

57
9
10
11
12
13
14
15

16
17
18


19
20
21
22
23
24
25
26


27
28
29


30
31





32
33
34
35
36
37
38

39





40
41


42

43
44

45
46
47
48
49
50

51
52







-



-
-
+
+






-
-
+
+
+
-
-
+

-
-
-
-
-
+
+
+
+
+
+

-
+
-
-
-
-
-
+
+
-
-
+
-


-
+





-
+

//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"fmt"
	"log"
	"os"
	"strings"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func flgSimpleRun(fs *flag.FlagSet) {
	fs.String("d", "", "zettel directory")
}

func runSimpleFunc(*flag.FlagSet) (int, error) {
	listenAddr := startup.ListenAddress()
func runSimpleFunc(fs *flag.FlagSet, cfg *meta.Meta) (int, error) {
	kern := kernel.Main
	listenAddr := kern.GetConfig(kernel.WebService, kernel.WebListenAddress).(string)
	readonlyMode := startup.IsReadOnlyMode()
	logBeforeRun(listenAddr, readonlyMode)
	exitCode, err := doRun(false)
	if idx := strings.LastIndexByte(listenAddr, ':'); idx >= 0 {
		log.Println()
		log.Println("--------------------------")
		log.Printf("Open your browser and enter the following URL:")
		log.Println()
		log.Printf("    http://localhost%v", listenAddr[idx:])
		kern.Log()
		kern.Log("--------------------------")
		kern.Log("Open your browser and enter the following URL:")
		kern.Log()
		kern.Log(fmt.Sprintf("    http://localhost%v", listenAddr[idx:]))
		kern.Log()
	}

	kern.WaitForShutdown()
	handler := setupRouting(startup.PlaceManager(), readonlyMode)
	srv := server.New(listenAddr, handler)
	if err := srv.Run(); err != nil {
		return 1, err
	}
	return exitCode, err
}
	return 0, nil
}


// runSimple is called, when the user just starts the software via a double click
// or via a simple call ``./zettelstore`` on the command line.
func runSimple() {
func runSimple() int {
	dir := "./zettel"
	if err := os.MkdirAll(dir, 0750); err != nil {
		fmt.Fprintf(os.Stderr, "Unable to create zettel directory %q (%s)\n", dir, err)
		os.Exit(1)
	}
	executeCommand("run-simple", "-d", dir)
	return executeCommand("run-simple", "-d", dir)
}

Changes to cmd/command.go.

1
2

3
4
5
6
7
8
9
10
11
12
13
14
15


16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32

33
34
35
36
37
38
39
1

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32
33

34
35
36
37
38
39
40
41

-
+













+
+







-
+








-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"flag"
	"sort"

	"zettelstore.de/z/domain/meta"
)

// Command stores information about commands / sub-commands.
type Command struct {
	Name   string              // command name as it appears on the command line
	Func   CommandFunc         // function that executes a command
	Places bool                // if true then places will be set up
	Simple bool                // Should start in simple mode
	Header bool                // Print a heading on startup
	Flags  func(*flag.FlagSet) // function to set up flag.FlagSet
	flags  *flag.FlagSet       // flags that belong to the command

}

// CommandFunc is the function that executes the command.
// It accepts the parsed command line parameters.
// It returns the exit code and an error.
type CommandFunc func(*flag.FlagSet) (int, error)
type CommandFunc func(*flag.FlagSet, *meta.Meta) (int, error)

// GetFlags return the flag.FlagSet defined for the command.
func (c *Command) GetFlags() *flag.FlagSet { return c.flags }

var commands = make(map[string]Command)

// RegisterCommand registers the given command.

Changes to cmd/main.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

15
16
17


18

19
20
21
22
23
24
25
26
27







28
29
30

31
32
33
34
35
36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51



52
53
54
55
56
57
58

59
60
61
62
63
64
65

66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

94
95
96
97
98
99
100
101
102

103
104
105






106
107
108
109







110
111
112
113
114
115
116
117

118
119

120
121

122
123
124
125
126
127

128
129
130
131
132
133
134
135
136
137
























138
139
140
141


142
143
144
145
146
147
148
149
150
151
152
















153
154

155
156
157
158
159











160
161
162
163
164
165
166

167
168
169
170
171
172









173
174

175
176
177


178
179
180
181
182










183


184








185



186
187
188

189
190
191
192

193
194
195
196
197

198
199
200

201
202

203
204
205



206
207
208
209
210

211
212

213
214
215


216
217
218

219


220
221

222
223




224
225
1
2
3
4
5
6
7
8
9
10
11
12
13

14
15
16

17
18
19
20
21
22







23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

43
44
45
46
47
48
49
50
51



52
53
54


55
56
57
58
59
60
61
62
63
64
65
66

67
68
69





70
71
72
73
74
75
76
77
78
79
80
81
82








83
84
85
86
87
88
89
90
91

92



93
94
95
96
97
98
99
100
101

102
103
104
105
106
107
108
109
110
111
112
113
114
115

116
117

118
119

120
121
122
123
124
125

126










127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150




151
152











153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169

170


171


172
173
174
175
176
177
178
179
180
181
182
183






184






185
186
187
188
189
190
191
192
193
194

195
196
197

198
199
200




201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222

223
224
225
226
227

228
229
230
231

232
233
234
235
236

237
238
239

240
241

242
243


244
245
246
247
248
249


250


251



252
253

254
255
256

257
258
259

260
261

262
263
264
265
266
267













-
+


-
+
+

+


-
-
-
-
-
-
-
+
+
+
+
+
+
+



+









-
+








-
-
-
+
+
+
-
-





+






-
+


-
-
-
-
-













-
-
-
-
-
-
-
-
+








-
+
-
-
-
+
+
+
+
+
+



-
+
+
+
+
+
+
+







-
+

-
+

-
+





-
+
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
+
-
-

-
-
+
+
+
+
+
+
+
+
+
+
+

-
-
-
-
-
-
+
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+

-
+


-
+
+

-
-
-
-
+
+
+
+
+
+
+
+
+
+

+
+

+
+
+
+
+
+
+
+
-
+
+
+


-
+



-
+




-
+


-
+

-
+

-
-
+
+
+



-
-
+
-
-
+
-
-
-
+
+
-


+
-
+
+

-
+

-
+
+
+
+


//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

package cmd

import (
	"context"
	"errors"
	"flag"
	"fmt"
	"log"
	"net"
	"net/url"
	"os"
	"strconv"
	"strings"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/index/indexer"
	"zettelstore.de/z/input"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/impl"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/manager"
	"zettelstore.de/z/place/progplace"
	"zettelstore.de/z/web/server"
)

const (
	defConfigfile = ".zscfg"
)

func init() {
	RegisterCommand(Command{
		Name: "help",
		Func: func(*flag.FlagSet) (int, error) {
		Func: func(*flag.FlagSet, *meta.Meta) (int, error) {
			fmt.Println("Available commands:")
			for _, name := range List() {
				fmt.Printf("- %q\n", name)
			}
			return 0, nil
		},
	})
	RegisterCommand(Command{
		Name: "version",
		Func: func(*flag.FlagSet) (int, error) {
			fmtVersion()
		Name:   "version",
		Func:   func(*flag.FlagSet, *meta.Meta) (int, error) { return 0, nil },
		Header: true,
			return 0, nil
		},
	})
	RegisterCommand(Command{
		Name:   "run",
		Func:   runFunc,
		Places: true,
		Header: true,
		Flags:  flgRun,
	})
	RegisterCommand(Command{
		Name:   "run-simple",
		Func:   runSimpleFunc,
		Places: true,
		Simple: true,
		Header: true,
		Flags:  flgSimpleRun,
	})
	RegisterCommand(Command{
		Name:  "config",
		Func:  cmdConfig,
		Flags: flgRun,
	})
	RegisterCommand(Command{
		Name: "file",
		Func: cmdFile,
		Flags: func(fs *flag.FlagSet) {
			fs.String("t", "html", "target output format")
		},
	})
	RegisterCommand(Command{
		Name: "password",
		Func: cmdPassword,
	})
}

func fmtVersion() {
	version := startup.GetVersion()
	fmt.Printf("%v (%v/%v) running on %v (%v/%v)\n",
		version.Prog, version.Build, version.GoVersion,
		version.Hostname, version.Os, version.Arch)
}

func getConfig(fs *flag.FlagSet) (cfg *meta.Meta) {
func readConfig(fs *flag.FlagSet) (cfg *meta.Meta) {
	var configFile string
	if configFlag := fs.Lookup("c"); configFlag != nil {
		configFile = configFlag.Value.String()
	} else {
		configFile = defConfigfile
	}
	content, err := os.ReadFile(configFile)
	if err != nil {
		cfg = meta.New(id.Invalid)
		return meta.New(id.Invalid)
	} else {
		cfg = meta.NewFromInput(id.Invalid, input.NewInput(string(content)))
	}
	}
	return meta.NewFromInput(id.Invalid, input.NewInput(string(content)))
}

func getConfig(fs *flag.FlagSet) *meta.Meta {
	cfg := readConfig(fs)
	fs.Visit(func(flg *flag.Flag) {
		switch flg.Name {
		case "p":
			cfg.Set(startup.KeyListenAddress, "127.0.0.1:"+flg.Value.String())
			if portStr, err := parsePort(flg.Value.String()); err == nil {
				cfg.Set(keyListenAddr, net.JoinHostPort("127.0.0.1", portStr))
			}
		case "a":
			if portStr, err := parsePort(flg.Value.String()); err == nil {
				cfg.Set(keyAdminPort, portStr)
			}
		case "d":
			val := flg.Value.String()
			if strings.HasPrefix(val, "/") {
				val = "dir://" + val
			} else {
				val = "dir:" + val
			}
			cfg.Set(startup.KeyPlaceOneURI, val)
			cfg.Set(keyPlaceOneURI, val)
		case "r":
			cfg.Set(startup.KeyReadOnlyMode, flg.Value.String())
			cfg.Set(keyReadOnly, flg.Value.String())
		case "v":
			cfg.Set(startup.KeyVerbose, flg.Value.String())
			cfg.Set(keyVerbose, flg.Value.String())
		}
	})
	return cfg
}

func setupOperations(cfg *meta.Meta, withPlaces, simple bool) error {
func parsePort(s string) (string, error) {
	var mgr place.Manager
	var idx index.Indexer
	if withPlaces {
		err := raiseFdLimit()
		if err != nil {
			log.Println("Raising some limitions did not work:", err)
			log.Println("Prepare to encounter errors. Most of them can be mitigated. See the manual for details")
			cfg.Set(startup.KeyDefaultDirPlaceType, startup.ValueDirPlaceTypeSimple)
		}
		startup.SetupStartupConfig(cfg)
	port, err := net.LookupPort("tcp", s)
	if err != nil {
		fmt.Fprintf(os.Stderr, "Wrong port specification: %q", s)
		return "", err
	}
	return strconv.Itoa(port), nil
}

const (
	keyAdminPort           = "admin-port"
	keyDefaultDirPlaceType = "default-dir-place-type"
	keyInsecureCookie      = "insecure-cookie"
	keyListenAddr          = "listen-addr"
	keyOwner               = "owner"
	keyPersistentCookie    = "persistent-cookie"
	keyPlaceOneURI         = kernel.PlaceURIs + "1"
	keyReadOnly            = "read-only-mode"
	keyTokenLifetimeHTML   = "token-lifetime-html"
	keyTokenLifetimeAPI    = "token-lifetime-api"
	keyURLPrefix           = "url-prefix"
	keyVerbose             = "verbose"
)

func setServiceConfig(cfg *meta.Meta) error {
		idx = indexer.New()
		filter := index.NewMetaFilter(idx)
		mgr, err = manager.New(getPlaces(cfg), cfg.GetBool(startup.KeyReadOnlyMode), filter)
		if err != nil {
	ok := setConfigValue(true, kernel.CoreService, kernel.CoreVerbose, cfg.GetBool(keyVerbose))
	if val, found := cfg.Get(keyAdminPort); found {
			return err
		}
	} else {
		startup.SetupStartupConfig(cfg)
	}

	startup.SetupStartupService(mgr, idx, simple)
	if withPlaces {
		if err := mgr.Start(context.Background()); err != nil {
			fmt.Fprintln(os.Stderr, "Unable to start zettel place")
			return err
		ok = setConfigValue(ok, kernel.CoreService, kernel.CorePort, val)
	}

	ok = setConfigValue(ok, kernel.AuthService, kernel.AuthOwner, cfg.GetDefault(keyOwner, ""))
	ok = setConfigValue(ok, kernel.AuthService, kernel.AuthReadonly, cfg.GetBool(keyReadOnly))

	ok = setConfigValue(
		ok, kernel.PlaceService, kernel.PlaceDefaultDirType,
		cfg.GetDefault(keyDefaultDirPlaceType, kernel.PlaceDirTypeNotify))
	ok = setConfigValue(ok, kernel.PlaceService, kernel.PlaceURIs+"1", "dir:./zettel")
	format := kernel.PlaceURIs + "%v"
	for i := 1; ; i++ {
		key := fmt.Sprintf(format, i)
		val, found := cfg.Get(key)
		if !found {
			break
		}
		runtime.SetupConfiguration(mgr)
		ok = setConfigValue(ok, kernel.PlaceService, key, val)
		progplace.Setup(cfg, mgr, idx)
		idx.Start(mgr)
	}
	return nil
}

	ok = setConfigValue(
		ok, kernel.WebService, kernel.WebListenAddress,
		cfg.GetDefault(keyListenAddr, "127.0.0.1:23123"))
	ok = setConfigValue(ok, kernel.WebService, kernel.WebURLPrefix, cfg.GetDefault(keyURLPrefix, "/"))
	ok = setConfigValue(ok, kernel.WebService, kernel.WebSecureCookie, !cfg.GetBool(keyInsecureCookie))
	ok = setConfigValue(ok, kernel.WebService, kernel.WebPersistentCookie, cfg.GetBool(keyPersistentCookie))
	ok = setConfigValue(
		ok, kernel.WebService, kernel.WebTokenLifetimeAPI, cfg.GetDefault(keyTokenLifetimeAPI, ""))
	ok = setConfigValue(
		ok, kernel.WebService, kernel.WebTokenLifetimeHTML, cfg.GetDefault(keyTokenLifetimeHTML, ""))

func getPlaces(cfg *meta.Meta) []string {
	var result []string = nil
	for cnt := 1; ; cnt++ {
		key := fmt.Sprintf("place-%v-uri", cnt)
		uri, ok := cfg.Get(key)
		if !ok || uri == "" {
	if !ok {
			if cnt > 1 {
				break
			}
			uri = "dir:./zettel"
		}
		result = append(result, uri)
		return errors.New("unable to set configuration")
	}
	return nil
}

func setConfigValue(ok bool, subsys kernel.Service, key string, val interface{}) bool {
	done := kernel.Main.SetConfig(subsys, key, fmt.Sprintf("%v", val))
	if !done {
		kernel.Main.Log("unable to set configuration:", key, val)
	}
	return result
	return ok && done
}

func cleanupOperations(withPlaces bool) error {
func setupOperations(cfg *meta.Meta, withPlaces bool) {
	var createManager kernel.CreatePlaceManagerFunc
	if withPlaces {
		startup.Indexer().Stop()
		if err := startup.PlaceManager().Stop(context.Background()); err != nil {
			fmt.Fprintln(os.Stderr, "Unable to stop zettel place")
			return err
		err := raiseFdLimit()
		if err != nil {
			srvm := kernel.Main
			srvm.Log("Raising some limitions did not work:", err)
			srvm.Log("Prepare to encounter errors. Most of them can be mitigated. See the manual for details")
			srvm.SetConfig(kernel.PlaceService, kernel.PlaceDefaultDirType, kernel.PlaceDirTypeSimple)
		}
		createManager = func(placeURIs []*url.URL, authManager auth.Manager, rtConfig config.Config) (place.Manager, error) {
			progplace.Setup(cfg)
			return manager.New(placeURIs, authManager, rtConfig)
		}
	} else {
		createManager = func([]*url.URL, auth.Manager, config.Config) (place.Manager, error) { return nil, nil }
	}

	kernel.Main.SetCreators(
		func(readonly bool, owner id.Zid) (auth.Manager, error) {
			return impl.New(readonly, owner, cfg.GetDefault("secret", "")), nil
		},
		createManager,
		func(srv server.Server, plMgr place.Manager, authMgr auth.Manager, rtConfig config.Config) error {
			setupRouting(srv, plMgr, authMgr, rtConfig)
	return nil
			return nil
		},
	)
}

func executeCommand(name string, args ...string) {
func executeCommand(name string, args ...string) int {
	command, ok := Get(name)
	if !ok {
		fmt.Fprintf(os.Stderr, "Unknown command %q\n", name)
		os.Exit(1)
		return 1
	}
	fs := command.GetFlags()
	if err := fs.Parse(args); err != nil {
		fmt.Fprintf(os.Stderr, "%s: unable to parse flags: %v %v\n", name, args, err)
		os.Exit(1)
		return 1
	}
	cfg := getConfig(fs)
	if err := setupOperations(cfg, command.Places, command.Simple); err != nil {
	if err := setServiceConfig(cfg); err != nil {
		fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
		os.Exit(2)
		return 2
	}

	exitCode, err := command.Func(fs)
	setupOperations(cfg, command.Places)
	kernel.Main.Start(command.Header)
	exitCode, err := command.Func(fs, cfg)
	if err != nil {
		fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
	}
	if err := cleanupOperations(command.Places); err != nil {
		fmt.Fprintf(os.Stderr, "%s: %v\n", name, err)
	kernel.Main.Shutdown(true)
	}
	if exitCode != 0 {
	return exitCode
		os.Exit(exitCode)
	}
}
}


// Main is the real entrypoint of the zettelstore.
func Main(progName, buildVersion string) {
	kernel.Main.SetConfig(kernel.CoreService, kernel.CoreProgname, progName)
	startup.SetupVersion(progName, buildVersion)
	kernel.Main.SetConfig(kernel.CoreService, kernel.CoreVersion, buildVersion)
	var exitCode int
	if len(os.Args) <= 1 {
		runSimple()
		exitCode = runSimple()
	} else {
		executeCommand(os.Args[1], os.Args[2:]...)
		exitCode = executeCommand(os.Args[1], os.Args[2:]...)
	}
	if exitCode != 0 {
		os.Exit(exitCode)
	}
}

Changes to cmd/register.go.

15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30

31
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33







+









+

import (
	_ "zettelstore.de/z/encoder/htmlenc"   // Allow to use HTML encoder.
	_ "zettelstore.de/z/encoder/jsonenc"   // Allow to use JSON encoder.
	_ "zettelstore.de/z/encoder/nativeenc" // Allow to use native encoder.
	_ "zettelstore.de/z/encoder/rawenc"    // Allow to use raw encoder.
	_ "zettelstore.de/z/encoder/textenc"   // Allow to use text encoder.
	_ "zettelstore.de/z/encoder/zmkenc"    // Allow to use zmk encoder.
	_ "zettelstore.de/z/kernel/impl"       // Allow kernel implementation to create itself
	_ "zettelstore.de/z/parser/blob"       // Allow to use BLOB parser.
	_ "zettelstore.de/z/parser/markdown"   // Allow to use markdown parser.
	_ "zettelstore.de/z/parser/none"       // Allow to use none parser.
	_ "zettelstore.de/z/parser/plain"      // Allow to use plain parser.
	_ "zettelstore.de/z/parser/zettelmark" // Allow to use zettelmark parser.
	_ "zettelstore.de/z/place/constplace"  // Allow to use global internal place.
	_ "zettelstore.de/z/place/dirplace"    // Allow to use directory place.
	_ "zettelstore.de/z/place/fileplace"   // Allow to use file place.
	_ "zettelstore.de/z/place/memplace"    // Allow to use memory place.
	_ "zettelstore.de/z/place/progplace"   // Allow to use computed place.
)

Changes to collect/split.go.

10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28
29
30

31
32

33
34

35
36
37
38
39
40

41
42
43
44
45
46
47
48
49
50
51
52
53





54
55
56
57
10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28
29

30
31

32
33

34
35
36
37
38
39

40













41
42
43
44
45


46
47







-
+












-
+

-
+

-
+





-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
-
-



// Package collect provides functions to collect items from a syntax tree.
package collect

import "zettelstore.de/z/ast"

// DivideReferences divides the given list of rederences into zettel, local, and external References.
func DivideReferences(all []*ast.Reference, duplicates bool) (zettel, local, external []*ast.Reference) {
func DivideReferences(all []*ast.Reference) (zettel, local, external []*ast.Reference) {
	if len(all) == 0 {
		return nil, nil, nil
	}

	mapZettel := make(map[string]bool)
	mapLocal := make(map[string]bool)
	mapExternal := make(map[string]bool)
	for _, ref := range all {
		if ref.State == ast.RefStateSelf {
			continue
		}
		if ref.IsZettel() {
			zettel = appendRefToList(zettel, mapZettel, ref, duplicates)
			zettel = appendRefToList(zettel, mapZettel, ref)
		} else if ref.IsExternal() {
			external = appendRefToList(external, mapExternal, ref, duplicates)
			external = appendRefToList(external, mapExternal, ref)
		} else {
			local = appendRefToList(local, mapLocal, ref, duplicates)
			local = appendRefToList(local, mapLocal, ref)
		}
	}
	return zettel, local, external
}

func appendRefToList(
func appendRefToList(reflist []*ast.Reference, refSet map[string]bool, ref *ast.Reference) []*ast.Reference {
	reflist []*ast.Reference,
	refSet map[string]bool,
	ref *ast.Reference,
	duplicates bool,
) []*ast.Reference {
	if duplicates {
		reflist = append(reflist, ref)
	} else {
		s := ref.String()
		if _, ok := refSet[s]; !ok {
			reflist = append(reflist, ref)
			refSet[s] = true
		}
	s := ref.String()
	if _, ok := refSet[s]; !ok {
		reflist = append(reflist, ref)
		refSet[s] = true
	}
	}

	return reflist
}

Added config/config.go.













































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package config provides functions to retrieve runtime configuration data.
package config

import (
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// Config allows to retrieve all defined configuration values that can be changed during runtime.
type Config interface {
	AuthConfig

	// AddDefaultValues enriches the given meta data with its default values.
	AddDefaultValues(m *meta.Meta) *meta.Meta

	// GetDefaultTitle returns the current value of the "default-title" key.
	GetDefaultTitle() string

	// GetDefaultRole returns the current value of the "default-role" key.
	GetDefaultRole() string

	// GetDefaultSyntax returns the current value of the "default-syntax" key.
	GetDefaultSyntax() string

	// GetDefaultLang returns the current value of the "default-lang" key.
	GetDefaultLang() string

	// GetSiteName returns the current value of the "site-name" key.
	GetSiteName() string

	// GetHomeZettel returns the value of the "home-zettel" key.
	GetHomeZettel() id.Zid

	// GetDefaultVisibility returns the default value for zettel visibility.
	GetDefaultVisibility() meta.Visibility

	// GetYAMLHeader returns the current value of the "yaml-header" key.
	GetYAMLHeader() bool

	// GetZettelFileSyntax returns the current value of the "zettel-file-syntax" key.
	GetZettelFileSyntax() []string

	// GetMarkerExternal returns the current value of the "marker-external" key.
	GetMarkerExternal() string

	// GetFooterHTML returns HTML code that should be embedded into the footer
	// of each WebUI page.
	GetFooterHTML() string

	// GetListPageSize returns the maximum length of a list to be returned in WebUI.
	// A value less or equal to zero signals no limit.
	GetListPageSize() int
}

// AuthConfig are relevant configuration values for authentication.
type AuthConfig interface {
	// GetExpertMode returns the current value of the "expert-mode" key
	GetExpertMode() bool

	// GetVisibility returns the visibility value of the metadata.
	GetVisibility(m *meta.Meta) meta.Visibility
}

// GetTitle returns the value of the "title" key of the given meta. If there
// is no such value, GetDefaultTitle is returned.
func GetTitle(m *meta.Meta, cfg Config) string {
	if val, ok := m.Get(meta.KeyTitle); ok {
		return val
	}
	return cfg.GetDefaultTitle()
}

// GetRole returns the value of the "role" key of the given meta. If there
// is no such value, GetDefaultRole is returned.
func GetRole(m *meta.Meta, cfg Config) string {
	if val, ok := m.Get(meta.KeyRole); ok {
		return val
	}
	return cfg.GetDefaultRole()
}

// GetSyntax returns the value of the "syntax" key of the given meta. If there
// is no such value, GetDefaultSyntax is returned.
func GetSyntax(m *meta.Meta, cfg Config) string {
	if val, ok := m.Get(meta.KeySyntax); ok {
		return val
	}
	return cfg.GetDefaultSyntax()
}

// GetLang returns the value of the "lang" key of the given meta. If there is
// no such value, GetDefaultLang is returned.
func GetLang(m *meta.Meta, cfg Config) string {
	if val, ok := m.Get(meta.KeyLang); ok {
		return val
	}
	return cfg.GetDefaultLang()
}

Deleted config/runtime/meta.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107











































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package runtime provides functions to retrieve runtime configuration data.
package runtime

import (
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/meta"
)

var mapDefaultKeys = map[string]func() string{
	meta.KeyCopyright: GetDefaultCopyright,
	meta.KeyLang:      GetDefaultLang,
	meta.KeyLicense:   GetDefaultLicense,
	meta.KeyRole:      GetDefaultRole,
	meta.KeySyntax:    GetDefaultSyntax,
	meta.KeyTitle:     GetDefaultTitle,
}

// AddDefaultValues enriches the given meta data with its default values.
func AddDefaultValues(m *meta.Meta) *meta.Meta {
	result := m
	for k, f := range mapDefaultKeys {
		if _, ok := result.Get(k); !ok {
			if result == m {
				result = m.Clone()
			}
			if val := f(); len(val) > 0 || m.Type(k) == meta.TypeEmpty {
				result.Set(k, val)
			}
		}
	}
	return result
}

// GetTitle returns the value of the "title" key of the given meta. If there
// is no such value, GetDefaultTitle is returned.
func GetTitle(m *meta.Meta) string {
	if syntax, ok := m.Get(meta.KeyTitle); ok && len(syntax) > 0 {
		return syntax
	}
	return GetDefaultTitle()
}

// GetRole returns the value of the "role" key of the given meta. If there
// is no such value, GetDefaultRole is returned.
func GetRole(m *meta.Meta) string {
	if syntax, ok := m.Get(meta.KeyRole); ok && len(syntax) > 0 {
		return syntax
	}
	return GetDefaultRole()
}

// GetSyntax returns the value of the "syntax" key of the given meta. If there
// is no such value, GetDefaultSyntax is returned.
func GetSyntax(m *meta.Meta) string {
	if syntax, ok := m.Get(meta.KeySyntax); ok && len(syntax) > 0 {
		return syntax
	}
	return GetDefaultSyntax()
}

// GetLang returns the value of the "lang" key of the given meta. If there is
// no such value, GetDefaultLang is returned.
func GetLang(m *meta.Meta) string {
	if lang, ok := m.Get(meta.KeyLang); ok && len(lang) > 0 {
		return lang
	}
	return GetDefaultLang()
}

// GetVisibility returns the visibility value, or "login" if none is given.
func GetVisibility(m *meta.Meta) meta.Visibility {
	if val, ok := m.Get(meta.KeyVisibility); ok {
		if vis := meta.GetVisibility(val); vis != meta.VisibilityUnknown {
			return vis
		}
	}
	return GetDefaultVisibility()
}

// GetUserRole role returns the user role of the given user zettel.
func GetUserRole(user *meta.Meta) meta.UserRole {
	if user == nil {
		if startup.WithAuth() {
			return meta.UserRoleUnknown
		}
		return meta.UserRoleOwner
	}
	if startup.IsOwner(user.Zid) {
		return meta.UserRoleOwner
	}
	if val, ok := user.Get(meta.KeyUserRole); ok {
		if ur := meta.GetUserRole(val); ur != meta.UserRoleUnknown {
			return ur
		}
	}
	return meta.UserRoleReader
}

Deleted config/runtime/runtime.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206














































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package runtime provides functions to retrieve runtime configuration data.
package runtime

import (
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/stock"
)

// --- Configuration zettel --------------------------------------------------

var configStock stock.Stock

// SetupConfiguration enables the configuration data.
func SetupConfiguration(mgr place.Manager) {
	if configStock != nil {
		panic("configStock already set")
	}
	configStock = stock.NewStock(mgr)
	if err := configStock.Subscribe(id.ConfigurationZid); err != nil {
		panic(err)
	}
}

// getConfigurationMeta returns the meta data of the configuration zettel.
func getConfigurationMeta() *meta.Meta {
	if configStock == nil {
		panic("configStock not set")
	}
	return configStock.GetMeta(id.ConfigurationZid)
}

// GetDefaultTitle returns the current value of the "default-title" key.
func GetDefaultTitle() string {
	if configStock != nil {
		if config := getConfigurationMeta(); config != nil {
			if title, ok := config.Get(meta.KeyDefaultTitle); ok {
				return title
			}
		}
	}
	return "Untitled"
}

// GetDefaultSyntax returns the current value of the "default-syntax" key.
func GetDefaultSyntax() string {
	if configStock != nil {
		if config := getConfigurationMeta(); config != nil {
			if syntax, ok := config.Get(meta.KeyDefaultSyntax); ok {
				return syntax
			}
		}
	}
	return meta.ValueSyntaxZmk
}

// GetDefaultRole returns the current value of the "default-role" key.
func GetDefaultRole() string {
	if configStock != nil {
		if config := getConfigurationMeta(); config != nil {
			if role, ok := config.Get(meta.KeyDefaultRole); ok {
				return role
			}
		}
	}
	return meta.ValueRoleZettel
}

// GetDefaultLang returns the current value of the "default-lang" key.
func GetDefaultLang() string {
	if configStock != nil {
		if config := getConfigurationMeta(); config != nil {
			if lang, ok := config.Get(meta.KeyDefaultLang); ok {
				return lang
			}
		}
	}
	return "en"
}

// GetDefaultCopyright returns the current value of the "default-copyright" key.
func GetDefaultCopyright() string {
	if configStock != nil {
		if config := getConfigurationMeta(); config != nil {
			if copyright, ok := config.Get(meta.KeyDefaultCopyright); ok {
				return copyright
			}
		}
	}
	return ""
}

// GetDefaultLicense returns the current value of the "default-license" key.
func GetDefaultLicense() string {
	if configStock != nil {
		if config := getConfigurationMeta(); config != nil {
			if license, ok := config.Get(meta.KeyDefaultLicense); ok {
				return license
			}
		}
	}
	return ""
}

// GetExpertMode returns the current value of the "expert-mode" key
func GetExpertMode() bool {
	if config := getConfigurationMeta(); config != nil {
		if mode, ok := config.Get(meta.KeyExpertMode); ok {
			return meta.BoolValue(mode)
		}
	}
	return false
}

// GetSiteName returns the current value of the "site-name" key.
func GetSiteName() string {
	if config := getConfigurationMeta(); config != nil {
		if name, ok := config.Get(meta.KeySiteName); ok {
			return name
		}
	}
	return "Zettelstore"
}

// GetHomeZettel returns the value of the "home-zettel" key.
func GetHomeZettel() id.Zid {
	if config := getConfigurationMeta(); config != nil {
		if start, ok := config.Get(meta.KeyHomeZettel); ok {
			if startID, err := id.Parse(start); err == nil {
				return startID
			}
		}
	}
	return id.DefaultHomeZid
}

// GetDefaultVisibility returns the default value for zettel visibility.
func GetDefaultVisibility() meta.Visibility {
	if config := getConfigurationMeta(); config != nil {
		if value, ok := config.Get(meta.KeyDefaultVisibility); ok {
			if vis := meta.GetVisibility(value); vis != meta.VisibilityUnknown {
				return vis
			}
		}
	}
	return meta.VisibilityLogin
}

// GetYAMLHeader returns the current value of the "yaml-header" key.
func GetYAMLHeader() bool {
	if config := getConfigurationMeta(); config != nil {
		return config.GetBool(meta.KeyYAMLHeader)
	}
	return false
}

// GetZettelFileSyntax returns the current value of the "zettel-file-syntax" key.
func GetZettelFileSyntax() []string {
	if config := getConfigurationMeta(); config != nil {
		return config.GetListOrNil(meta.KeyZettelFileSyntax)
	}
	return nil
}

// GetMarkerExternal returns the current value of the "marker-external" key.
func GetMarkerExternal() string {
	if config := getConfigurationMeta(); config != nil {
		if html, ok := config.Get(meta.KeyMarkerExternal); ok {
			return html
		}
	}
	return "&#10138;"
}

// GetFooterHTML returns HTML code that should be embedded into the footer
// of each WebUI page.
func GetFooterHTML() string {
	if config := getConfigurationMeta(); config != nil {
		if data, ok := config.Get(meta.KeyFooterHTML); ok {
			return data
		}
	}
	return ""
}

// GetListPageSize returns the maximum length of a list to be returned in WebUI.
// A value less or equal to zero signals no limit.
func GetListPageSize() int {
	if config := getConfigurationMeta(); config != nil {
		if value, ok := config.GetNumber(meta.KeyListPageSize); ok && value > 0 {
			return value
		}
	}
	return 0
}

Deleted config/startup/startup.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209

















































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package startup provides functions to retrieve startup configuration data.
package startup

import (
	"hash/fnv"
	"io"
	"strconv"
	"time"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
)

var config struct {
	// Set in SetupStartupConfig
	verbose             bool
	readonlyMode        bool
	urlPrefix           string
	listenAddress       string
	defaultDirPlaceType string
	owner               id.Zid
	withAuth            bool
	secret              []byte
	insecCookie         bool
	persistCookie       bool
	htmlLifetime        time.Duration
	apiLifetime         time.Duration

	// Set in SetupStartupService
	simple  bool // was started without run command
	manager place.Manager
	indexer index.Indexer
}

// Predefined keys for startup zettel
const (
	KeyDefaultDirPlaceType = "default-dir-place-type"
	KeyInsecureCookie      = "insecure-cookie"
	KeyListenAddress       = "listen-addr"
	KeyOwner               = "owner"
	KeyPersistentCookie    = "persistent-cookie"
	KeyPlaceOneURI         = "place-1-uri"
	KeyReadOnlyMode        = "read-only-mode"
	KeyTokenLifetimeHTML   = "token-lifetime-html"
	KeyTokenLifetimeAPI    = "token-lifetime-api"
	KeyURLPrefix           = "url-prefix"
	KeyVerbose             = "verbose"
)

// Important values for some keys.
const (
	ValueDirPlaceTypeNotify = "notify"
	ValueDirPlaceTypeSimple = "simple"
)

// SetupStartupConfig initializes the startup data with content of config file.
func SetupStartupConfig(cfg *meta.Meta) {
	if config.urlPrefix != "" {
		panic("startup.config already set")
	}
	config.verbose = cfg.GetBool(KeyVerbose)
	config.readonlyMode = cfg.GetBool(KeyReadOnlyMode)
	config.urlPrefix = cfg.GetDefault(KeyURLPrefix, "/")
	if prefix, ok := cfg.Get(KeyURLPrefix); ok &&
		len(prefix) > 0 && prefix[0] == '/' && prefix[len(prefix)-1] == '/' {
		config.urlPrefix = prefix
	} else {
		config.urlPrefix = "/"
	}
	if val, ok := cfg.Get(KeyListenAddress); ok {
		config.listenAddress = val // TODO: check for valid string
	} else {
		config.listenAddress = "127.0.0.1:23123"
	}
	if defaultType, ok := cfg.Get(KeyDefaultDirPlaceType); ok {
		switch defaultType {
		case ValueDirPlaceTypeNotify:
		case ValueDirPlaceTypeSimple:
		default:
			defaultType = ValueDirPlaceTypeNotify
		}
		config.defaultDirPlaceType = defaultType
	} else {
		config.defaultDirPlaceType = ValueDirPlaceTypeNotify
	}
	config.owner = id.Invalid
	if owner, ok := cfg.Get(KeyOwner); ok {
		if zid, err := id.Parse(owner); err == nil {
			config.owner = zid
			config.withAuth = true
		}
	}
	if config.withAuth {
		config.insecCookie = cfg.GetBool(KeyInsecureCookie)
		config.persistCookie = cfg.GetBool(KeyPersistentCookie)
		config.secret = calcSecret(cfg)
		config.htmlLifetime = getDuration(
			cfg, KeyTokenLifetimeHTML, 1*time.Hour, 1*time.Minute, 30*24*time.Hour)
		config.apiLifetime = getDuration(
			cfg, KeyTokenLifetimeAPI, 10*time.Minute, 0, 1*time.Hour)
	}
}

// SetupStartupService initializes the startup data with internal services.
func SetupStartupService(manager place.Manager, idx index.Indexer, simple bool) {
	if config.urlPrefix == "" {
		panic("startup.config not set")
	}
	config.simple = simple && !config.withAuth
	config.manager = manager
	config.indexer = idx
}

func calcSecret(cfg *meta.Meta) []byte {
	h := fnv.New128()
	if secret, ok := cfg.Get("secret"); ok {
		io.WriteString(h, secret)
	}
	io.WriteString(h, version.Prog)
	io.WriteString(h, version.Build)
	io.WriteString(h, version.Hostname)
	io.WriteString(h, version.Os)
	io.WriteString(h, version.Arch)
	return h.Sum(nil)
}

func getDuration(
	cfg *meta.Meta, key string, defDur, minDur, maxDur time.Duration) time.Duration {
	if s, ok := cfg.Get(key); ok && len(s) > 0 {
		if d, err := strconv.ParseUint(s, 10, 64); err == nil {
			secs := time.Duration(d) * time.Minute
			if secs < minDur {
				return minDur
			}
			if secs > maxDur {
				return maxDur
			}
			return secs
		}
	}
	return defDur
}

// IsSimple returns true if Zettelstore was not started with command "run"
// and authentication is disabled.
func IsSimple() bool { return config.simple }

// IsVerbose returns whether the system should be more chatty about its operations.
func IsVerbose() bool { return config.verbose }

// IsReadOnlyMode returns whether the system is in read-only mode or not.
func IsReadOnlyMode() bool { return config.readonlyMode }

// URLPrefix returns the configured prefix to be used when providing URL to
// the service.
func URLPrefix() string { return config.urlPrefix }

// ListenAddress returns the string that specifies the the network card and the ip port
// where the server listens for requests
func ListenAddress() string { return config.listenAddress }

// DefaultDirPlaceType returns the default value for a directory place type.
func DefaultDirPlaceType() string { return config.defaultDirPlaceType }

// WithAuth returns true if user authentication is enabled.
func WithAuth() bool { return config.withAuth }

// SecureCookie returns whether the web app should set cookies to secure mode.
func SecureCookie() bool { return config.withAuth && !config.insecCookie }

// PersistentCookie returns whether the web app should set persistent cookies
// (instead of temporary).
func PersistentCookie() bool { return config.persistCookie }

// Owner returns the zid of the zettelkasten's owner.
// If there is no owner defined, the value ZettelID(0) is returned.
func Owner() id.Zid { return config.owner }

// IsOwner returns true, if the given user is the owner of the Zettelstore.
func IsOwner(zid id.Zid) bool { return zid.IsValid() && zid == config.owner }

// Secret returns the interal application secret. It is typically used to
// encrypt session values.
func Secret() []byte { return config.secret }

// TokenLifetime return the token lifetime for the web/HTML access and for the
// API access. If lifetime for API access is equal to zero, no API access is
// possible.
func TokenLifetime() (htmlLifetime, apiLifetime time.Duration) {
	return config.htmlLifetime, config.apiLifetime
}

// PlaceManager returns the managing place.
func PlaceManager() place.Manager { return config.manager }

// Indexer returns the current indexer.
func Indexer() index.Indexer { return config.indexer }

Deleted config/startup/version.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51



















































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package startup provides functions to retrieve startup configuration data.
package startup

import (
	"os"
	"runtime"
)

// Version describes all elements of a software version.
type Version struct {
	Prog      string // Name of the software
	Build     string // Representation of build process
	Hostname  string // Host name a reported by the kernel
	GoVersion string // Version of go
	Os        string // GOOS
	Arch      string // GOARCH
	// More to come
}

var version Version

// SetupVersion initializes the version data.
func SetupVersion(progName, buildVersion string) {
	version.Prog = progName
	if buildVersion == "" {
		version.Build = "unknown"
	} else {
		version.Build = buildVersion
	}
	if hn, err := os.Hostname(); err == nil {
		version.Hostname = hn
	} else {
		version.Hostname = "*unknown host*"
	}
	version.GoVersion = runtime.Version()
	version.Os = runtime.GOOS
	version.Arch = runtime.GOARCH
}

// GetVersion returns the current software version data.
func GetVersion() Version { return version }

Changes to docs/manual/00001004000000.zettel.

1
2
3
4
5

6
7
8
9
10








11
12
13
14



15
16
17


1
2
3
4
5
6
7




8
9
10
11
12
13
14
15




16
17
18
19


20
21





+

-
-
-
-
+
+
+
+
+
+
+
+
-
-
-
-
+
+
+

-
-
+
+
id: 00001004000000
title: Configuration of Zettelstore
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20210510153233

There are two levels to change the behavior and/or the appearance of Zettelstore.
The first level is the configuration that is needed to start the services provided by Zettelstore.
For example, this includes the URI under which your Zettelstore is accessible.
* [[Zettelstore start-up configuration|00001004010000]]
There are some levels to change the behavior and/or the appearance of Zettelstore.

# The first level is the way to start Zettelstore services and to manage it via command line (and, in part, via a graphical user interface).
#* [[Command line parameters|00001004050000]]
# As an intermediate user, you usually want to have more control over how Zettelstore is started.
  This may include the URI under which your Zettelstore is accessible, or the directories in which your Zettel are stored.
  You may want to permanently store the command line parameters so that you don't have to specify them every time you start Zettelstore.
#* [[Zettelstore startup configuration|00001004010000]]

The second level is configuring the running Zettelstore.
For example, you can configure the default language of your Zettelstore.
* [[Configure a running Zettelstore|00001004020000]]
# The last level is configuring the running Zettelstore.
  For example, you can configure the default language of your Zettelstore.
#* [[Configure a running Zettelstore|00001004020000]]

The third level is the way to start Zettelstore services and to manage it.
* [[Command line parameters|00001004050000]]
If you have enabled the administrator console, either via [[command-line parameters|00001004050000#a]] or via the [[startup configuration file|00001004010000#admin-port]], you can control the inner workings of Zettelstore even further.
* [[Zettelstore Administrator Console|00001004100000]]

Changes to docs/manual/00001004010000.zettel.

1
2

3
4
5

6
7

8
9
10
11
12
13

14
15
16
17







18
19
20
21
22
23
24
1

2
3
4
5
6
7

8
9
10
11
12
13

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

-
+



+

-
+





-
+




+
+
+
+
+
+
+







id: 00001004010000
title: Zettelstore start-up configuration
title: Zettelstore startup configuration
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20210525121644

The configuration file, as specified by the ''-c CONFIGFILE'' [[command line option|00001004051000]], allows you to specify some start-up options.
The configuration file, as specified by the ''-c CONFIGFILE'' [[command line option|00001004051000]], allows you to specify some startup options.
These options cannot be stored in a [[configuration zettel|00001004020000]] because either they are needed before Zettelstore can start or because of security reasons.
For example, Zettelstore need to know in advance, on which network address is must listen or where zettel are placed.
An attacker that is able to change the owner can do anything.
Therefore only the owner of the computer on which Zettelstore runs can change this information.

The file for start-up configuration must be created via a text editor in advance.
The file for startup configuration must be created via a text editor in advance.

The syntax of the configuration file is the same as for any zettel metadata.
The following keys are supported:

; [!admin-port]''admin-port''
: Specifies the TCP port through which you can reach the [[administrator console|00001004100000]].
  A value of ''0'' (the default) disables the administrators console.

  On most operating systems, the value must be greater than ''1024'' unless you start Zettelstore with the full privileges of a system administrator (which is not recommended).

  Default: ''0''
; [!default-dir-place-type]''default-dir-place-type''
: Specifies the default value for the (sub-) type of [[directory places|00001004011400#type]].
  Zettel are typically stored in such places.

  Default: ''notify''
; [!insecure-cookie]''insecure-cookie''
: Must be set to ''true'', if authentication is enabled and Zettelstore is not accessible not via HTTPS (but via HTTP).
40
41
42
43
44
45
46
47

48
49

50
51
52
53


54
55
56
57
58
59
60
48
49
50
51
52
53
54

55
56

57
58
59


60
61
62
63
64
65
66
67
68







-
+

-
+


-
-
+
+







  On these devices, the operating system is free to stop the web browser and to remove temporary cookies.
  Therefore, an authenticated user will be logged off.

  If ''true'', a persistent cookie is used.
  Its lifetime exceeds the lifetime of the authentication token (see option ''token-lifetime-html'') by 30 seconds.

  Default: ''false''
; [!place-X-uri]''place-//X//-uri'', where //X// is a number greater or equal to one
; [!place-uri-X]''place-uri-//X//'', where //X// is a number greater or equal to one
: Specifies a [[place|00001004011200]] where zettel are stored.
  During start-up //X// is counted, starting with one, until no key is found.
  During startup //X// is counted up, starting with one, until no key is found.
  This allows to configure more than one place.

  If no ''place-1-uri'' key is given, the overall effect will be the same as if only ''place-1-uri'' was specified with the value ''dir://.zettel''.
  In this case, even a key ''place-2-uri'' will be ignored.
  If no ''place-uri-1'' key is given, the overall effect will be the same as if only ''place-uri-1'' was specified with the value ''dir://.zettel''.
  In this case, even a key ''place-uri-2'' will be ignored.
; [!read-only-mode]''read-only-mode''
: Puts the Zettelstore web service into a read-only mode.
  No changes are possible.
  Default: false.
; [!token-lifetime-api]''token-lifetime-api'', [!token-lifetime-html]''token-lifetime-html''
: Define lifetime of access tokens in minutes.
  Values are only valid if authentication is enabled, i.e. key ''owner'' is set.

Changes to docs/manual/00001004011200.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16

17
18
19
20
21
22
23
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

17
18
19
20
21
22
23
24





+










-
+







id: 00001004011200
title: Zettelstore places
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20210525121452

A Zettelstore must store its zettel somehow and somewhere.
In most cases you want to store your zettel as files in a directory.
Under certain circumstances you may want to store your zettel in other places.

An example are the [[predefined zettel|00001005090000]] that come with a Zettelstore.
They are stored within the software itself.
In another situation you may want to store your zettel volatile, e.g. if you want to provide a sandbox for experimenting.

To cope with these (and more) situations, you configure Zettelstore to use one or more places.
This is done via the ''place-X-uri'' keys of the [[start-up configuration|00001004010000#place-X-uri]] (X is a number).
This is done via the ''place-uri-X'' keys of the [[startup configuration|00001004010000#place-uri-X]] (X is a number).
Places are specified using special [[URIs|https://en.wikipedia.org/wiki/Uniform_Resource_Identifier]], somehow similar to web addresses.

The following place URIs are supported:

; ''dir:\//DIR''
: Specifies a directory where zettel files are stored.
  ''DIR'' is the file path.
31
32
33
34
35
36
37
38
39
40



41
42
43
44
45
46
47


48
32
33
34
35
36
37
38



39
40
41
42
43





44
45
46







-
-
-
+
+
+


-
-
-
-
-
+
+

  You can create such a ZIP file, if you zip a directory full of zettel files.

  This place is always read-only.
; ''mem:''
: Stores all its zettel in volatile memory.
  If you stop the Zettelstore, all changes are lost.

All places that you configure via the ''store-X-uri'' keys form a chain of places.
If a zettel should be retrieved, a search starts in the place specified with the ''place-1-uri'' key, then ''place-2-uri'' and so on.
If a zettel is created or changed, it is always stored in the place specified with the ''place-1-uri'' key.
All places that you configure via the ''store-uri-X'' keys form a chain of places.
If a zettel should be retrieved, a search starts in the place specified with the ''place-uri-2'' key, then ''place-uri-3'' and so on.
If a zettel is created or changed, it is always stored in the place specified with the ''place-uri-1'' key.
This allows to overwrite zettel from other places, e.g. the predefined zettel.

If you did not configure the place of the predefined zettel (''const:'') they will automatically be appended as a last place.
Otherwise Zettelstore will not work in certain situations.

If you use the ''mem:'' place, where zettel are stored in volatile memory, it makes only sense if you configure it as ''place-1-uri''.
Such a place will be empty when Zettelstore starts and only the place 1 will receive updates.
If you use the ''mem:'' place, where zettel are stored in volatile memory, it makes only sense if you configure it as ''place-uri-1''.
Such a place will be empty when Zettelstore starts and only the first place will receive updates.
You must make sure that your computer has enough RAM to store all zettel.

Changes to docs/manual/00001004011400.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
1
2
3
4
5
6
7
8
9
10
11
12
13





+







id: 00001004011400
title: Configure file directory places
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20210525121232

Under certain circumstances, it is preferable to further configure a file directory place.
This is done by appending query parameters after the base place URI ''dir:\//DIR''.

The following parameters are supported:

|= Parameter:|Description|Default value:|
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51







-
+







When the parameter ''type'' is set to ""notify"", Zettelstore automatically detects changes to zettel files that originates from other software.
It is done on a ""best-effort"" basis.
Under certain circumstances it is possible that Zettelstore does not detect a change done by another software.

To cope with this unlikely, but still possible event, Zettelstore re-scans periodically the file directory.
The time interval is configured by the ''rescan'' parameter, e.g.
```
place-1-uri: dir:///home/zettel?rescan=300
place-uri-1: dir:///home/zettel?rescan=300
```
This makes Zettelstore to re-scan the directory ''/home/zettel/'' every 300 seconds, i.e. 5 minutes.

For a Zettelstore with many zettel, re-scanning the directory may take a while, especially if it is stored on a remote computer (e.g. via CIFS/SMB or NFS).
In this case, you should adjust the parameter value.

Please note that a directory re-scan invalidates all internal data of a Zettelstore.
72
73
74
75
76
77
78
79

80
81
73
74
75
76
77
78
79

80
81
82







-
+



For various reasons, the value should be a prime number, with a maximum value of 1499.

=== Readonly
Sometimes you may want to provide zettel from a file directory place, but you want to disallow any changes.
If you provide the query parameter ''readonly'' (with or without a corresponding value), the place will disallow any changes.
```
place-1-uri: dir:///home/zettel?readonly
place-uri-1: dir:///home/zettel?readonly
```
If you put the whole Zettelstore in [[read-only|00001004010000]] [[mode|00001004051000]], all configured file directory places will be in read-only mode too, even if not explicitly configured.

Changes to docs/manual/00001004050000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28





+


















-




id: 00001004050000
title: Command line parameters
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20210511140731

Zettelstore is not just a web service that provides services of a zettelkasten.
It allows to some tasks to be executed at the command line.
Typically, the task (""sub-command"") will be given at the command line as the first parameter.

If no parameter is given, the Zettelstore is called as
```
zettelstore
```
This is equivalent to call it this way:
```sh
mkdir -p ./zettel
zettelstore run -d ./zettel -c ./.zscfg
```
Typically this is done by starting Zettelstore via a graphical user interface by double-clicking to its file icon.
=== Sub-commands
* [[``zettelstore help``|00001004050200]] lists all available sub-commands.
* [[``zettelstore version``|00001004050400]] to display version information of Zettelstore.
* [[``zettelstore config``|00001004050600]] to show the currently active [[configuration|00001004000000]].
* [[``zettelstore run``|00001004051000]] to start the web-based Zettelstore service.
* [[``zettelstore run-simple``|00001004051100]] is typically called, when you start Zettelstore by a double.click in your GUI.
* [[``zettelstore file``|00001004051200]] to render files manually without activated/running Zettelstore services.
* [[``zettelstore password``|00001004051400]] to calculate data for user authentication.

Deleted docs/manual/00001004050600.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23























-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
id: 00001004050600
title: The ''config'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
precursor: 00001004050000

Shows the Zettelstore configuration, for debugging purposes.
Currently, only the [[start-up configuration|00001004010000]] is shown.

This sub-command uses the same command line parameters as [[``zettelstore run``|00001004051000]].

An example for an unconfigured Zettelstore:
```
# zettelstore config
Zettelstore (v0.0.4/go1.15) running on mycomputer (linux/amd64)
Stores
  Read only         = false
Web
  Listen Addr       = "127.0.0.1:23123"
  URL prefix        = "/"
```
The first line is identical to the output of the [[``zettelkasten version``|00001004050400]] sub-command.

Changes to docs/manual/00001004051000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14



15
16


17
18
19
20

21
22
23
24
25
26
27
28
29
30
31


32
33
34
35
36
37


38
39
40
41
42

43
44
45
46

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18


19
20
21
22
23

24
25
26
27
28
29
30
31
32
33


34
35
36
37
38
39


40
41
42
43
44
45

46
47

48

49





+









+
+
+
-
-
+
+



-
+









-
-
+
+




-
-
+
+




-
+

-

-
+
id: 00001004051000
title: The ''run'' sub-command
role: manual
tags: #command #configuration #manual #zettelstore
syntax: zmk
modified: 20210510153318
precursor: 00001004050000

=== ``zettelstore run``
This starts the web service.

```
zettelstore run [-c CONFIGFILE] [-d DIR] [-p PORT] [-r] [-v]
```

; [!a]''-a PORT''
: Specifies the TCP port through which you can reach the [[administrator console|00001004100000]].
  See the explanation of [[''admin-port''|00001004010000#admin-port]] for more details.
; ''-c CONFIGFILE''
: Specifies ''CONFIGFILE'' as a file, where [[start-up configuration data|00001004010000]] is read.
; [!c]''-c CONFIGFILE''
: Specifies ''CONFIGFILE'' as a file, where [[startup configuration data|00001004010000]] is read.
  It is ignored, when the given file is not available, nor readable.

  Default: ''./.zscfg''. (''.\\.zscfg'' on Windows)), where ''.'' denotes the ""current directory"".
; ''-d DIR''
; [!d]''-d DIR''
: Specifies ''DIR'' as the directory that contains all zettel.

  Default is ''./zettel'' (''.\\zettel'' on Windows), where ''.'' denotes the ""current directory"".
; [!debug]''-debug''
: Allows better debugging of the internal web server by disabling any timeout values.
  You should specify this only as a developer.
  Especially do not enable it for a production server.

  [[https://blog.cloudflare.com/exposing-go-on-the-internet/#timeouts]] contains a good explanation for the usefulness of sensitive timeout values.
; ''-p PORT''
: Specifies the integer value ''PORT'' as the TCP port, where the Zettelstore service listens for requests.
; [!p]''-p PORT''
: Specifies the integer value ''PORT'' as the TCP port, where the Zettelstore web server listens for requests.

  Default: 23123.

  Zettelstore listens only on ''127.0.0.1'', e.g. only requests from the current computer will be processed.
  If you want to listen on network card to process requests from other computer, please use ''listen-addr'' of the configuration file as described below.
; ''-r''
  If you want to listen on network card to process requests from other computer, please use [[''listen-addr''|00001004010000#listen-addr]] of the configuration file as described below.
; [!r]''-r''
: Puts the Zettelstore in read-only mode.
  No changes are possible via the web interface / via the API.

  This allows to publish your content without any risks of unauthorized changes.
; ''-v''
; [!v]''-v''
: Be more verbose in writing logs.
  Writes the start-up configuration to stderr.

Command line options take precedence over configuration file options.
Command line options take precedence over [[configuration file|00001004010000]] options.

Added docs/manual/00001004100000.zettel.


























1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001004100000
title: Zettelstore Administrator Console
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20210510155859

The administrator console is a service accessible only on the same computer on which Zettelstore is running.
It allows an experienced user to monitor and control some of the inner workings of Zettelstore.

You enable the administrator console by specifying a TCP port number greater than zero (better: greater than 1024) for it, either via the [[command-line parameter ''-a''|00001004051000#a]] or via the ''admin-port'' key of the [[startup configuration file|00001004010000#admin-port]].

After you enable the administrator console, you can use tools such as [[PuTTY|https://www.chiark.greenend.org.uk/~sgtatham/putty/]] or other telnet software to connect to the administrator console.
In fact, the administrator console is //not// a full telnet service.
It is merely a simple line-oriented service where each input line is interpreted separately.
Therefore, you can also use tools like [[netcat|https://nc110.sourceforge.io/]], [[socat|http://www.dest-unreach.org/socat/]], etc.

After connecting to the administrator console, there is no further authentication.
It is not needed because you must be logged in on the same computer where Zettelstore is running.
You cannot connect to the administrator console if you are on a different computer.
Of course, on multi-user systems with untrusted users, you should not enable the administrator console.

* Enable via [[command line|00001004051000#a]]
* Enable via [[configuration file|00001004010000#admin-port]]
* [[List of supported commands|00001004101000]]

Added docs/manual/00001004101000.zettel.








































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001004101000
title: List of supported commands of the administrator console
role: manual
tags: #configuration #manual #zettelstore
syntax: zmk
modified: 20210525161623

; ''bye''
: Closes the connection to the administrator console.
; ''config SERVICE''
: Displays all valid configuration keys for the given service.

  If a key ends with the hyphen-minus character (""''-''"", ''U+002D''), the key denotes a list value.
  Keys of list elements are specified by appending a number greater than zero to the key.
; ''crlf''
: Toggles CRLF mode for console output.
  Changes end of line sequences between Windows mode (==\\r\\n==) and non-Windows mode (==\\n==, initial value).
  Often used on Windows telnet clients that otherwise scramble the output of commands.
; ''dump-index''
: Displays the content of the internal search index.
; ''dump-recover RECOVER''
: Displays data about the last given recovered internal activity.

  The value for ''RECOVER'' can be obtained via the command ``stat core``, which lists all overview data about all recoveries.
; ''echo''
: Toggles the echo mode, where each command is printed before execution
; ''env''
: Display environment values.
; ''help''
: Displays a list of all available commands.
; ''get-config''
: Displays current configuration data.

  ``get-config`` shows all current configuration data.

  ``get-config SERVICE`` shows only the current configuration data of the given service.

  ``get-config SERVICE KEY`` shows the current configuration data for the given service and key.
; ''header''
: Toggles the header mode, where each table is show with a header nor not. 
; ''metrics''
: Displays some values that reflect the inner workings of Zettelstore.
  See [[here|https://golang.org/pkg/runtime/metrics/]] for a technical description of these values.
; ''next-config''
: Displays next configuration data.
  It will be the current configuration, if the corresponding services is restarted.

  ``next-config`` shows all next configuration data.

  ``next-config SERVICE`` shows only the next configuration data of the given service.

  ``next-config SERVICE KEY`` shows the next configuration data for the given service and key.
; ''restart SERVICE''
: Restart the given service and all other that depend on this.
; ''services''
: Displays s list of all available services and their current status.
; ''set-config SERVICE KEY VALUE''
: Sets a single configuration value for the next configuration of a given service.
  It will become effective if the service is restarted.

  If the key specifies a list value, all other list values with a number greater than the given key are deleted.
  You can use the special number ""0"" to delete all values.
  E.g. ``set-config place place-uri-0 any_text`` will remove all values of the list //place-uri-//.
; ''shutdown''
: Terminate the Zettelstore itself (and closes the connection to the administrator console).
; ''start SERVICE''
: Start the given bservice and all dependent services.
; ''stat SERVICE''
: Display some statistical values for the given service.
; ''stop SERVICE''
: Stop the given service and all other that depend on this.

Changes to docs/manual/00001005000000.zettel.

18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32







-
+







Zettelstore becomes extensible by external software.
For example, a more sophisticated web interface could be build, or an application for your mobile device that allows you to send content to your Zettelstore as new zettel.

=== Where zettel are stored

Your zettel are stored as files in a specific directory.
If you have not explicitly specified the directory, a default directory will be used.
The directory has to be specified at [[start-up time|00001004010000]].
The directory has to be specified at [[startup time|00001004010000]].
Nested directories are not supported (yet).

Every file in this directory that should be monitored by Zettelstore must have a file name that begins with 14 digits (0-9), the [[zettel identifier|00001006050000]].
If you create a new zettel via the web interface or the API, the zettel identifier will be the timestamp of the current date and time (format is ''YYYYMMDDhhmmss'').
This allows zettel to be sorted naturally by creation time.

Since the only restriction on zettel identifiers are the 14 digits, you are free to use other digit sequences.

Changes to docs/manual/00001005090000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16




17
18

19
20
21
22
23
24
25
26
27
28
29
30
31

32
33
34
35
36
37
38
39

1
2
3
4
5
6
7
8
9
10
11
12
13




14
15
16
17
18

19

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

40





+







-
-
-
-
+
+
+
+

-
+
-












+







-
+
id: 00001005090000
title: List of predefined zettel
role: manual
tags: #manual #reference #zettelstore
syntax: zmk
modified: 20210511180816

The following table lists all predefined zettel with their purpose.

|= Identifier :|= Title | Purpose
| [[00000000000001]] | Zettelstore Version | Contains the version string of the running Zettelstore
| [[00000000000002]] | Zettelstore Host | Contains the name of the computer running the Zettelstore
| [[00000000000003]] | Zettelstore Operating System | Contains the operating system and CPU architecture of the computer running the Zettelstore
| [[00000000000006]] | Zettelstore Environment Values | Contains environmental data of Zettelstore executable
| [[00000000000008]] | Zettelstore Runtime Values | Contains values that reflect the inner working; see [[here|https://golang.org/pkg/runtime/]] for a technical description of these values
| [[00000000000018]] | Zettelstore Indexer | Provides some statistics about the index process
| [[00000000000020]] | Zettelstore Place Manager | Contains some statistics about zettel places
| [[00000000000004]] | Zettelstore License | Lists the license of Zettelstore
| [[00000000000005]] | Zettelstore Contributors | Lists all contributors of Zettelstore
| [[00000000000006]] | Zettelstore Dependencies | Lists all licensed content
| [[00000000000020]] | Zettelstore Place Manager | Contains some statistics about zettel places and the the index process
| [[00000000000090]] | Zettelstore Supported Metadata Keys | Contains all supported metadata keys, their [[types|00001006030000]], and more
| [[00000000000096]] | Zettelstore Start-up Configuration | Contains the effective values of the [[start-up configuration|00001004010000]]
| [[00000000000096]] | Zettelstore Startup Configuration | Contains the effective values of the [[startup configuration|00001004010000]]
| [[00000000000098]] | Zettelstore Start-up Values | Contains all values computed from the [[start-up configuration|00001004010000]]
| [[00000000000100]] | Zettelstore Runtime Configuration | Allows to [[configure Zettelstore at runtime|00001004020000]]
| [[00000000010100]] | Zettelstore Base HTML Template | Contains the general layout of the HTML view
| [[00000000010200]] | Zettelstore Login Form HTML Template | Layout of the login form, when authentication is [[enabled|00001010040100]]
| [[00000000010300]] | Zettelstore List Meta HTML Template | Used when displaying a list of zettel
| [[00000000010401]] | Zettelstore Detail HTML Template | Layout for the HTML detail view of one zettel
| [[00000000010402]] | Zettelstore Info HTML Templöate | Layout for the information view of a specific zettel
| [[00000000010403]] | Zettelstore Form HTML Template | Form that is used to create a new or to change an existing zettel that contains text
| [[00000000010404]] | Zettelstore Rename Form HTML Template | View that is displayed to change the [[zettel identifier|00001006050000]]
| [[00000000010405]] | Zettelstore Delete HTML Template | View to confirm the deletion of a zettel
| [[00000000010500]] | Zettelstore List Roles HTML Template | Layout for listing all roles
| [[00000000010600]] | Zettelstore List Tags HTML Template | Layout of tags lists
| [[00000000020001]] | Zettelstore Base CSS | CSS file that is included by the [[Base HTML Template|00000000010100]]
| [[00000000040001]] | Generic Emoji | Image that is shown if original image reference is invalid
| [[00000000090000]] | New Menu | Contains items that should contain in the zettel template menu
| [[00000000090001]] | New Zettel | Template for a new zettel with role ""[[zettel|00001006020100]]""
| [[00000000090002]] | New User | Template for a new zettel with role ""[[user|00001006020100#user]]""
| [[00010000000000]] | Home | Default home zettel, contains some welcome information

If a zettel is not linked, it is not accessible for the current user.

**Important:** The identifier may change until a stable version of the software is released.
**Important:** All identifier may change until a stable version of the software is released.

Changes to docs/manual/00001006034500.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28





+














-
+







id: 00001006034500
title: Timestamp Key Type
role: manual
tags: #manual #meta #reference #zettel #zettelstore
syntax: zmk
modified: 20210511131903

Values of this type denote a point in time.

=== Allowed values
Must be a sequence of 14 digits (""0""--""9"") (same as an [[Identifier|00001006032000]]), with the restriction that is conforms to the pattern ""YYYYMMDDhhmmss"".

* YYYY is the year,
* MM is the month,
* DD is the day,
* hh is the hour,
* mm is the minute,
* ss is the second.

=== Match operator
A value matches an timestampvalue, if the first value is the prefix of the timestamp value.
A value matches a timestamp value, if the first value is the prefix of the timestamp value.

For example, ""202102"" matches ""20210212143200"".

=== Sorting
Sorting is done by comparing the [[String|00001006033500]] values.

If both values are timestamp values, this works well because both have the same length.

Changes to docs/manual/00001007031000.zettel.

1
2

3
4
5

6
7
8
9
10
11
12
1
2
3
4
5

6
7
8
9
10
11
12
13


+


-
+







id: 00001007031000
title: Zettelmarkup: Tables
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
role: manual
modified: 20210523185812

Tables are used to show some data in a two-dimenensional fashion.
In zettelmarkup, table are not specified explicitly, but by entering //table rows//.
Therefore, a table can be seen as a sequence of table rows.
A table row is nothing as a sequence of //table cells//.
The length of a table is the number of table rows, the width of a table is the maximum length of its rows.

26
27
28
29
30
31
32

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

48
49
50
51
52
53
54
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57







+















+







will be rendered in HTML as:
:::example
| a1 | a2 | a3|
| b1 | b2 | b3
| c1 | c2
:::

=== Header row
If any cell in the first row of a table contains an equal sing character (""''=''"", ''U+003D'') as the very first character, then this first row will be interpreted as a //table header// row.

For example:
```zmk
| a1 | a2 |= a3|
| b1 | b2 | b3
| c1 | c2
```
will be rendered in HTML as:
:::example
| a1 | a2 |= a3|
| b1 | b2 | b3
| c1 | c2
:::

=== Column alignment
Inside a header row, you can specify the alignment of each header cell by a given character as the last character of a cell.
The alignment of a header cell determines the alignment of every cell in the same column.
The following characters specify the alignment:

* the colon character (""'':''"", ''U+003A'') forces a centered aligment,
* the less-than sign character (""''<''"", ''U+0060'') specifies an alignment to the left,
* the greater-than sign character (""''>''"", ''U+0062'') will produce right aligned cells.
64
65
66
67
68
69
70

71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

















67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107







+
















+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
will be rendered in HTML as:
:::example
|=Left<|Right>|Center:|Default
|123456|123456|123456|123456|
|123|123|123|123
:::

=== Cell alignment
To specify the alignment of an individual cell, you can enter these characters for alignment as the first character of that cell.

For example:
```zmk
|=Left<|Right>|Center:|Default
|>R|:C|<L
|123456|123456|123456|123456|
|123|123|123|123
```
will be rendered in HTML as:
:::example
|=Left<|Right>|Center:|Default
|>R|:C|<L
|123456|123456|123456|123456|
|123|123|123|123
:::

=== Rows to be ignored
A line that begins with the sequence ''|%'' (vertical bar character (""''|''"", ''U+007C''), followed by a percent sign character (“%”, U+0025)) will be ignored.
This allows to specify a horizontal rule that is not rendered.
Such tables are emitted by some commands of the [[administrator console|00001004100000]].
For example, the command ``get-config place`` will emit
```
|=Key        | Value  | Description
|%-----------+--------+-----------------------------
| defdirtype | notify | Default directory place type
```
This is rendered in HTML as:
:::example
|=Key        | Value  | Description
|%-----------+--------+-----------------------------
| defdirtype | notify | Default directory place type
:::

Changes to docs/manual/00001007040200.zettel.

1
2

3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29


+


-
+















-
+







id: 00001007040200
title: Zettelmarkup: Literal-like formatting
role: manual
tags: #manual #zettelmarkup #zettelstore
syntax: zmk
role: manual
modified: 20210525121114

There are some reasons to mark text that should be rendered as uninterpreted:
* Mark text as literal, sometimes as part of a program.
* Mark text as input you give into a computer via a keyboard.
* Mark text as output from some computer, e.g. shown at the command line.

=== Literal text
Literal text somehow relates to [[verbatim blocks|00001007030500]]: their content should not be interpreted further, but may be rendered special.
It is specified by two grave accent characters (""''`''"", ''U+0060''), followed by the text, followed by again two grave accent characters, optionally followed by an [[attribute|00001007050000]] specification.
Similar to the verbatim block, the literal element allows also a modifier letter grave accent (""''ˋ''"", ''U+02CB'') as an alternative to the grave accent character[^On some devices, such as an iPhone / iPad, a grave accent character is harder to enter and is often confused with a modifier letter grave accent.].
However, all four characters must be the same.

The literal element supports the default attribute: when given, all spaces in the text are rendered in HTML as open box characters (""''&#x2423;''"", ''U+2423'').
The use of a generic attribute allwos to specify a (programming) language that controls syntax colouring when rendered in HTML.

If you want to specify a grave accent character in the text, either use modifier grave accent characters as delimiters for the element, or place a backslash character before the grave accent character you want to use inside the element.
If you want to specify a grave accent character in the text, either use modifier grave accent characters as delimiters for the element, or put a backslash character before the grave accent character you want to use inside the element.
If you want to enter a backslash character, you need to enter two of these.

Examples:
* ``\`\`abc def\`\``` is rendered in HTML as ::``abc def``::{=example}.
* ``\`\`abc def\`\`{-}`` is rendered in HTML as ::``abc def``{-}::{=example}.
* ``\`\`abc\\\`def\`\``` is rendered in HTML as ::``abc\`def``::{=example}.
* ``\`\`abc\\\\def\`\``` is rendered in HTML as ::``abc\\def``::{=example}.

Changes to docs/manual/00001007040300.zettel.

55
56
57
58
59
60
61
62
63
64





65
66
67
68
69
70
71
55
56
57
58
59
60
61



62
63
64
65
66
67
68
69
70
71
72
73







-
-
-
+
+
+
+
+








If the text is given, it will be interpreted as an alternative textual representation, to help persons with some visual disabilities.

[[Attributes|00001007050000]] are supported.
They must follow the last right curly bracket character immediately.
One prominent example is to specify an explicit title attribute that is shown on certain web browsers when the zettel is rendered in HTML:

%%Example:

%%``{{External link|00000000030001}}{title=External width=30}`` is rendered as ::{{External link|00000000030001}}{title=External width=30}::{=example}.
Examples:
* ``{{Spinning Emoji|00000000040001}}{title=Emoji width=30}`` is rendered as ::{{Spinning Emoji|00000000040001}}{title=Emoji width=30}::{=example}.
* The above image is also a placeholder for a non-existent image:
** ``{{00000000000000}}`` will be rendered as ::{{00000000000000}}::{=example}.
** ``{{00000000009999}}`` will be rendered as ::{{00000000009999}}::{=example}.

=== Footnotes

A footnote begins with a left square bracket, followed by a circumflex accent (""''^''"", ''U+005E''), followed by some text, and ends with a right square bracket.

Example:

Changes to docs/manual/00001008000000.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15


16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


31
32
33
34
35
36
37
1
2
3
4
5
6
7
8
9
10
11
12


13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


30
31
32
33
34
35
36
37
38





+






-
-


+
+













-
-
+
+







id: 00001008000000
title: Other Markup Languages
role: manual
tags: #manual #zettelstore
syntax: zmk
modified: 20210523194915

[[Zettelmarkup|00001007000000]] is not the only markup language you can use to define your content.
Zettelstore is quite agnostic with respect to markup languages.
Of course, Zettelmarkup plays an important role.
However, with the exception of zettel titles, you can use any (markup) language that is supported:

* Markdown
* Images: GIF, PNG, JPEG, SVG
* CSS
* HTML template data
* Image formats: GIF, PNG, JPEG, SVG
* Markdown
* Plain text, not further interpreted

The [[metadata key|00001006020000#syntax]] ""''syntax''"" specifies which language should be used.
If it is not given, the key ""''default-syntax''"" will be used (specified in the [[configuration zettel|00001004020000#default-syntax]]).
The following syntax values are supported:

; [!css]''css''
: A [[Cascading Style Sheet|https://www.w3.org/Style/CSS/]], to be used when rendering a zettel as HTML.
; [!gif]''gif''; [!jpeg]''jpeg''; [!jpg]''jpg''; [!png]''png''
: The formats for pixel graphics.
  Typically the data is stored in a separate file and the syntax is given in the ''.meta'' file.
; [!markdown]''markdown'', [!md]''md''
: For those who desperately need [[Markdown|https://daringfireball.net/projects/markdown/]].
  Since the world of Markdown is so diverse, a [[CommonMark|https://commonmark.org]] parser is used.
  See [[Use Markdown as the main markup language of Zettelstore|00001008010000]].
  Since the world of Markdown is so diverse, a [[CommonMark|https://commonmark.org/]] parser is used.
  See [[Use Markdown within Zettelstore|00001008010000]].
; [!mustache]''mustache''
: A [[Mustache template|https://mustache.github.io/]], used when rendering a zettel as HTML.
; [!none]''none''
: Only the metadata of a zettel is ""parsed"".
  Useful for displaying the full metadata.
  The [[runtime configuration zettel|00000000000100]] uses this syntax.
  The zettel content is ignored.

Changes to docs/manual/00001008010000.zettel.

1
2


3
4
5

6
7

8


9
10
11



12
13
14
15
16
17
18
19
20

21
22
23
24


















1

2
3
4
5

6
7
8
9
10
11
12



13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

-
+
+


-
+


+

+
+
-
-
-
+
+
+









+




+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
id: 00001008010000
title: Use Markdown as the main markup language of Zettelstore
title: Use Markdown within Zettelstore
role: manual
tags: #manual #markdown #zettelstore
syntax: zmk
role: manual
modified: 20210518102155

If you are customized to use Markdown as your markup language, you can configure Zettelstore to support your decision.
Zettelstore supports [[CommonMark|https://commonmark.org/]], an [[attempt|https://xkcd.com/927/]] to unify all the different, divergent dialects of Markdown.

=== Use Markdown as the default markup language of Zettelstore

Just add the key ''default-syntax'' with a value of ''md'' or ''markdown'' to the [[configuration zettel|00000000000100]].
Whether to use ''md'' or ''markdown'' is not just a matter to taste, but also depends on the value of ''zettel-file-syntax'' and, to some degree, on the value of ''yaml-header''.
All key are described [[here|00001004020000]].
Add the key ''default-syntax'' with a value of ''md'' or ''markdown'' to the [[configuration zettel|00000000000100]].
Whether to use ''md'' or ''markdown'' is not just a matter to taste.
It also depends on the value of [[''zettel-file-syntax''|00001004020000#zettel-file-syntax]] and, to some degree, on the value of [[''yaml-header''|00001004020000#yaml-header]].

If you set ''yaml-header'' to true, then new content is always stored in a file with the extension ''.zettel''.

Otherwise ''zettel-file-syntax'' lists all syntax values, where its content should be stored in a file with the extension ''.zettel''.

If neither ''yaml-header'' nor ''zettel-file-syntax'' is set, new content is stored in a file where its file name extension is the same as the syntax value of that zettel.
In this case it makes a difference, whether you specify ''md'' or ''markdown''.
If you specify the syntax ''md'', your content will be stored in a file with the ''.md'' extension.
Similar for the syntax ''markdown''.

If you want to process the files that store the zettel content, e.g. with some other Markdown tools, this may be important.
Not every Markdown tool allows both file extensions.

BTW, metadata is stored in a file with the extension ''.meta'', if neither ''yaml-header'' nor ''zettel-file-syntax'' is set.

=== Security aspects

You should be aware that Markdown is a superset of HTML.
Any HTML code is valid Markdown code.
If you write your own zettel, this is probably not a problem.

However, if you receive zettel from others, you should be careful.
An attacker might include malicious HTML code in your zettel.
For example, HTML allows to embed JavaScript, a full-sized programming language that drives many web sites.
When a zettel is displayed, JavaScript code might be executed, sometimes with harmful results.

Zettelstore mitigates this problem by ignoring suspicious text when it encodes a zettel as HTML.
Any HTML text that might contain the ``<script>`` tag or the ``<iframe>`` tag is ignored.
This may lead to unexpected results if you depend on these.
Other encoding [[formats|00001012920500]] may still contain the full HTML text.

Any external client of Zettelstore, which does not use Zettelstore's HTML encoding, must be programmed to take care of malicious code.

Changes to docs/manual/00001010000000.zettel.

1
2

3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31


+


-


















-
+







id: 00001010000000
title: Security
role: manual
tags: #configuration #manual #security #zettelstore
syntax: zmk
role: manual

Your zettel could contain sensitive content.
You probably want to ensure that only authorized person can read and/or modify them.
Zettelstore ensures this in various ways.

=== Local first
The Zettelstore is designed to run on your local computer.
If you do not configure it in other ways, no person from another computer can connect to your Zettelstore.
You must explicitly configure it to allow access from other computers.

In the case that your own multiple computers, you do not have to access the Zettelstore remotely.
You could install Zettelstore on each computer and set-up some software to synchronize your zettel.
Since zettel are stored as ordinary files, this task could be done in various ways.

=== Read-only
You can start the Zettelstore in an read-only mode.
Nobody, not even you as the owner of the Zettelstore, can change something via its interfaces[^However, as an owner, you have access to the files that store the zettel. If you modify the files, these changes will be reflected via its interfaces.].

You enable read-only mode through the key ''readonly'' in the [[start-up configuration zettel|00001004010000]] or with the ''-r'' option of the ``zettelstore run`` sub-command.
You enable read-only mode through the key ''readonly'' in the [[startup configuration zettel|00001004010000#readonly]] or with the ''-r'' option of the ``zettelstore run`` sub-command.

=== Authentication
The Zettelstore can be configured that a user must authenticate itself to gain access to the content.

* [[How to enable authentication|00001010040100]]
* [[How to add a new user|00001010040200]]
* [[How users are authenticated|00001010040400]] (some technical background)
56
57
58
59
60
61
62
63

64
65
66
56
57
58
59
60
61
62

63
64
65
66







-
+



Otherwise, an eavesdropper could fetch sensible data, such as passwords or precious content that is not for the public.

The Zettelstore itself does not encrypt messages.
But you can put a server in front of it, which is able to handle encryption.
Most generic web server software do allow this.

To enforce encryption, [[authentication sessions|00001010040700]] are marked as secure by default.
If you still want to access the Zettelstore remotely without encryption, you must change the start-up configuration.
If you still want to access the Zettelstore remotely without encryption, you must change the startup configuration.
Otherwise, authentication will not work.

* [[Use a server for encryption|00001010090100]]

Changes to docs/manual/00001010040100.zettel.

1
2

3
4
5
6
7
8
9


1
2
3
4
5

6
7


8
9


+


-


-
-
+
+
id: 00001010040100
title: Enable authentication
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk
role: manual

To enable authentication, you must create a zettel that stores [[authentication data|00001010040200]] for the owner.
Then you must reference this zettel within the [[start-up configuration|00001004010000#owner]] under the key ''owner''.
Once the start-up configuration contains a valid [[zettel identifier|00001006050000]] under that key, authentication is enabled.
Then you must reference this zettel within the [[startup configuration|00001004010000#owner]] under the key ''owner''.
Once the startup configuration contains a valid [[zettel identifier|00001006050000]] under that key, authentication is enabled.

Changes to docs/manual/00001010040700.zettel.

1
2

3
4
5
6
7
8
9
10
11
12

13
14
15

16
17
18
19
20

21
22
23

24
25
1
2
3
4
5

6
7
8
9
10
11

12
13
14

15
16
17
18
19

20
21
22

23
24
25


+


-






-
+


-
+




-
+


-
+


id: 00001010040700
title: Access token
role: manual
tags: #authentication #configuration #manual #security #zettelstore
syntax: zmk
role: manual

If an user is authenticated, an ""access token"" is created that must be sent with every request to prove the identity of the caller.
Otherwise the user will not be recognized by Zettelstore.

If the user was authenticated via the web interface, the access token is stored in a [[""session cookie""|https://en.wikipedia.org/wiki/HTTP_cookie#Session_cookie]].
When the web browser is closed, theses cookies are not saved.
If you want web browser to store the cookie as long as lifetime of that token, the owner must set ''persistent-cookie'' of the [[start-up configuration|00001004010000]] to ''true''.
If you want web browser to store the cookie as long as lifetime of that token, the owner must set ''persistent-cookie'' of the [[startup configuration|00001004010000]] to ''true''.

If the web browser remains inactive for a period, the user will be automatically logged off, because each access token has a limited lifetime.
The maximum length of this period is specified by the ''token-lifetime-html'' value of the start-up configuration.
The maximum length of this period is specified by the ''token-lifetime-html'' value of the startup configuration.
Every time a web page is displayed, a fresh token is created and stored inside the cookie.

If the user was authenticated via the API, the access token will be returned as the content of the response.
Typically, the lifetime of this token is more short term, e.g. 10 minutes.
It is specified by the ''token-timeout-api'' value of the start-up configuration.
It is specified by the ''token-timeout-api'' value of the startup configuration.
If you need more time, you can either [[re-authenticate|00001012050200]] the user or use an API call to [[renew the access token|00001012050400]].

If you remotely access your Zettelstore via HTTP (not via HTTPS, which allows encrypted communication), your must set the ''insecure-cookie'' value of the start-up configuration to ''true''.
If you remotely access your Zettelstore via HTTP (not via HTTPS, which allows encrypted communication), your must set the ''insecure-cookie'' value of the startup configuration to ''true''.
In most cases, such a scenario is not recommended, because user name and password will be transferred as plain text.
You could make use of such scenario if you know all parties that access the local network where you access the Zettelstore.

Changes to docs/manual/00001010070200.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33


34
35



36
37
38
39
40

41
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


27





28
29
30

31
32
33
34
35
36
37

38
39





+




















-
-

-
-
-
-
-
+
+

-
+
+
+




-
+

id: 00001010070200
title: Visibility rules for zettel
role: manual
tags: #authorization #configuration #manual #security #zettelstore
syntax: zmk
modified: 20210510194652

For every zettel you can specify under which condition the zettel is visible to others.
This is controlled with the metadata key [[''visibility''|00001006020000#visibility]].
The following values are supported:

; [!public]""public""
: The zettel is visible to everybody, even if the user is not authenticated.
; [!login]""login""
: Only an authenticated user can access the zettel.

  This is the default value for [[''default-visibility''|00001004020000#default-visibility]].
; [!owner]""owner""
: Only the owner of the Zettelstore can access the zettel.

  This is for zettel with sensitive content, e.g. the [[configuration zettel|00001004020000]] or the various zettel that contains the templates for rendering zettel in HTML.
; [!expert]""expert""
: Only the owner of the Zettelstore can access the zettel, if runtime configuration [[''expert-mode''|00001004020000#expert-mode]] is set to a boolean true value.

  This is for zettel with sensitive content that might irritate the owner.
  Computed zettel with internal runtime information are examples for such a zettel.
; [!simple-expert]""simple-expert""
: The owner of the Zettelstore can access the zettel, if expert mode is enabled, or if authentication is disabled and the Zettelstore is started without any command.

  The reason for this is to show all computed zettel to an user that started the Zettestore in simple mode.
  Many computed zettel should be given in error reporting and a new user might not be able to enable expert mode.

When you install a Zettelstore, only two zettel have visibility ""public"".
The first zettel is the zettel that contains CSS for displaying the web interface.
When you install a Zettelstore, only some zettel have visibility ""public"".
One is the zettel that contains CSS for displaying the web interface.
This is to ensure that the web interface looks nice even for not authenticated users.
The other zettel is the zettel containing the [[version|00000000000001]] of the Zettelstore.
Another is the zettel containing the [[version|00000000000001]] of the Zettelstore.
Yet other zettel lists the Zettelstore [[license|00000000000004]], its [[contributors|00000000000005]], and external, licensed [[dependencies|00000000000006]], such as program code written by others or graphics designed by others.
The [[default image|00000000040001]], used if an image reference is invalid,is also public visible.

Please note: if authentication is not enabled, every user has the same rights as the owner of a Zettelstore.
This is also true, if the Zettelstore runs additionally in [[read-only mode|00001004010000#read-only-mode]].
In this case, the [[runtime configuration zettel|00001004020000]] is shown (its visibility is ""owner"").
The [[start-up configuration|00001004010000]] is not shown, because the associated computed zettel with identifier ''00000000000098'' is stored with the visibility ""expert"".
The [[startup configuration|00001004010000]] is not shown, because the associated computed zettel with identifier ''00000000000096'' is stored with the visibility ""expert"".
If you want to show such a zettel, you must set ''expert-mode'' to true.

Changes to docs/manual/00001010090100.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
1
2
3
4
5
6
7
8
9
10
11
12
13





+







id: 00001010090100
title: External server to encrypt message transport
role: manual
tags: #configuration #encryption #manual #security #zettelstore
syntax: zmk
modified: 20210511131719

Since Zettelstore does not encrypt the messages it exchanges with its clients, you may need some additional software to enable encryption.

=== Public-key encryption
To enable encryption, you probably use some kind of encryption keys.
In most cases, you need to deploy a ""public-key encryption"" process, where your side publish a public encryption key that only works with a corresponding private decryption key.
Technically, this is not trivial.
54
55
56
57
58
59
60
61
62
63
64
65
66

67
68
69
70


55
56
57
58
59
60
61

62
63
64
65

66
67
68


69
70







-




-
+


-
-
+
+
If you want to add some additional content on the server, you could change the configuration as follows:
```
zettelstore.de {
  file_server * {
    root /var/www/html
  }
  route /manual/* {
    uri strip_prefix /manual
    reverse_proxy localhost:23123
  }
}
```
This will forwards requests with the prefix ""/manual"" to the running Zettelstore.
This will forwards requests with the prefix ""/manual/"" to the running Zettelstore.
All other requests will be handled by Caddy itself.

In this case you must specify the start-up configuration key ''url-prefix'' with the value ""/manual"".
This is to allow the Zettelstore to give you the correct URLs with the given prefix.
In this case you must specify the [[startup configuration key ''url-prefix''|00001004010000#url-prefix]] with the value ""/manual/"".
This is to allow Zettelstore ignore the prefix while reading web requests and to give the correct URLs with the given prefix when sending a web response.

Changes to docs/manual/00001012050200.zettel.

1
2
3
4
5
6
7
8
9

10
11
12
13
14
15
16
1
2
3
4
5
6
7
8

9
10
11
12
13
14
15
16








-
+







id: 00001012050200
title: API: Authenticate a client
role: manual
tags: #api #manual #zettelstore
syntax: zmk

Authentication for future API calls is done by sending a [[user identification|00001010040200]] and a password to the Zettelstore to obtain an [[access token|00001010040700]].
This token has to be used for other API calls.
It is valid for a relatively short amount of time, as configured with the key ''token-timeout-api'' of the [[start-up configuration|00001004010000]] (typically 10 minutes).
It is valid for a relatively short amount of time, as configured with the key ''token-timeout-api'' of the [[startup configuration|00001004010000]] (typically 10 minutes).

The simplest way is to send user identification (''IDENT'') and password (''PASSWORD'') via [[HTTP Basic Authentication|https://tools.ietf.org/html/rfc7617]] and send them to the [[endpoint|00001012920000]] ''/a'' with a POST request:
```sh
# curl -X POST -u IDENT:PASSWORD http://127.0.0.1:23123/a
{"access_token":"eyJhbGciOiJIUzUxMiJ9.eyJfdGsiOjEsImV4cCI6MTYwMTczMTI3NSwiaWF0IjoxNjAxNzMwNjc1LCJzdWIiOiJhYmMiLCJ6aWQiOiIyMDIwMTAwMzE1MDEwMCJ9.ekhXkvn146P2bMKFQcU-bNlvgbeO6sS39hs6U5EKfjIqnSInkuHYjYAIfUqf_clYRfr6YBlX5izii8XfxV8jhg","token_type":"Bearer","expires_in":600}
```

Changes to docs/manual/00001012051800.zettel.

1
2
3
4
5

6
7
8
9
10
11
12
1
2
3
4
5
6
7
8
9
10
11
12
13





+







id: 00001012051800
title: API: Shape the list of zettel metadata with filter options
role: manual
tags: #api #manual #zettelstore
syntax: zmk
modified: 20210510150129

In most cases, it is not essential to list //all// zettel.
Typically, you are interested only in a subset of the zettel maintained by your Zettelstore.
This is done by adding some query parameters to the general ''GET /z'' request.

=== Filter
Every query parameter that does //not// begin with the low line character (""_"", ''U+005F'') is treated as the name of a [[metadata|00001006010000]] key.
20
21
22
23
24
25
26
27

28
29
30
31
32
33
34
21
22
23
24
25
26
27

28
29
30
31
32
33
34
35







-
+







{"list":[{"id":"00001012921000","url":"/z/00001012921000","meta":{"title":"API: JSON structure of an access token","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920500","url":"/z/00001012920500","meta":{"title":"Formats available by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}},{"id":"00001012920000","url":"/z/00001012920000","meta":{"title":"Endpoints used by the API","tags":"#api #manual #reference #zettelstore","syntax":"zmk","role":"manual"}}, ...
```

However, if you want all zettel that does //not// match a given value, you must prefix the value with the exclamation mark character (""!"", ''U+0021'').
For example, if you want to retrieve all zettel that do not contain the string ""API"" in their title, your request will be:
```sh
# curl 'http://127.0.0.1:23123/z?title=!API'
{"list":[{"id":"00010000000000","url":"/z/00010000000000","meta":{"back":"00001003000000 00001005090000","backward":"00001003000000 00001005090000","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00000000000001 00000000000003 00000000000096 00000000000098 00000000000100","lang":"en","license":"EUPL-1.2-or-later","role":"zettel","syntax":"zmk","title":"Home"}},{"id":"00001014000000","url":"/z/00001014000000","meta":{"back":"00001000000000 00001004020000 00001012920510","backward":"00001000000000 00001004020000 00001012000000 00001012920510","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00001012000000","lang":"en","license":"EUPL-1.2-or-later","published":"00001014000000","role":"manual","syntax":"zmk","tags":"#manual #webui #zettelstore","title":"Web user interface"}},
{"list":[{"id":"00010000000000","url":"/z/00010000000000","meta":{"back":"00001003000000 00001005090000","backward":"00001003000000 00001005090000","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00000000000001 00000000000003 00000000000096 00000000000100","lang":"en","license":"EUPL-1.2-or-later","role":"zettel","syntax":"zmk","title":"Home"}},{"id":"00001014000000","url":"/z/00001014000000","meta":{"back":"00001000000000 00001004020000 00001012920510","backward":"00001000000000 00001004020000 00001012000000 00001012920510","copyright":"(c) 2020-2021 by Detlef Stern <ds@zettelstore.de>","forward":"00001012000000","lang":"en","license":"EUPL-1.2-or-later","published":"00001014000000","role":"manual","syntax":"zmk","tags":"#manual #webui #zettelstore","title":"Web user interface"}},
...
```
The empty query parameter values matches all zettel that contain the given metadata key.
Similar, if you specify just the exclamation mark character as a query parameter value, only those zettel match that does //not// contain the given metadata key.
For example ``curl 'http://localhost:23123/z?back=!&backward='`` returns all zettel that are reachable via other zettel, but also references these zettel.

=== Output only specific parts of a zettel
60
61
62
63
64
65
66























67
68

69
70
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

92
93
94







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
+


{"list":[{"id":"00001012050200","url":"/z/00001012050200"},{"id":"00001012050400","url":"/z/00001012050400"}]}
```

=== General filter
The query parameter ""''_s''"" allows to provide a string for a full-text search of all zettel.
The search string will be normalized according to Unicode NKFD, ignoring everything except letters and numbers.

If the search string starts with the character ""''!''"", it will be removed and the query matches all zettel that **does not match** the search string.

In the next step, the first character of the search string will be inspected.
If it contains one of the characters ""'':''"", ""''=''"", ""''>''"", or ""''~''"", this will modify how the search will be performed.
The character will be removed from the start of the search string.

For example, assume the search string is ""def"":

; ""'':''"", ""''~''"" (or none of these characters)[^""'':''"" is always the character for specifying the default comparison. In this case, it is equal to ""''~''"". If you omit a comparison character, the default comparison is used.]
: The zettel must contain a word that contains the search string.
  ""def"", ""defghi"", and ""abcdefghi"" are matching the search string.
; ""''=''""
: The zettel must contain a word that is equal to the search string.
  Only the word ""def"" matches the search string.
; ""''>''""
: The zettel must contain a word with the search string as a prefix.
  A word like ""def"" or ""defghi"" matches the search string.

If you want to include an initial ""''!''"" into the search string, you must prefix that with the escape character ""''\\''"".
For example ""\\!abc"" will search for zettel that contains the string ""!abc"".
A similar rule applies to the characters that specify the way how the search will be done.
For example, ""!\\=abc"" will search for zettel that do not contains the string ""=abc"".

You are allowed to specify this query parameter more than once.
All results will be intersected, i.e. a zettel will be included into the list if both of the provided values match.
All results will be intersected, i.e. a zettel will be included into the list if all of the provided values match.

This parameter loosely resembles the search box of the web user interface.

Changes to docs/manual/00001012920000.zettel.

1
2
3
4
5

6
7

8
9

10
11
12
13
14
15
16
1
2
3
4
5
6
7

8
9

10
11
12
13
14
15
16
17





+

-
+

-
+







id: 00001012920000
title: Endpoints used by the API
role: manual
tags: #api #manual #reference #zettelstore
syntax: zmk
modified: 20210511131339

All API endpoints conform to the pattern ''[PREFIX]/LETTER[/ZETTEL-ID]'', where:
All API endpoints conform to the pattern ''[PREFIX]LETTER[/ZETTEL-ID]'', where:
; ''PREFIX''
: is an optional URL prefix, configured via the ''url-prefix'' [[start-up configuration|00001004010000]],
: is the URL prefix (default: ''/''), configured via the ''url-prefix'' [[startup configuration|00001004010000]],
; ''LETTER''
: is a single letter that specifies the ressource type,
; ''ZETTEL-ID''
: is an optional 14 digits string that uniquely [[identify a zettel|00001006050000]].

The following letters are currently in use:

Changes to domain/id/id.go.

24
25
26
27
28
29
30
31
32













33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

49
50
51




52
53
54
55
56
57
58
59
60
61
62













63
64

65
66
67
68
69

70
71
72
73
74
75
76
77
78
79
80
81
82
24
25
26
27
28
29
30


31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

59
60
61

62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

91
92
93
94
95

96
97
98
99



100
101
102
103
104
105
106







-
-
+
+
+
+
+
+
+
+
+
+
+
+
+















-
+


-
+
+
+
+











+
+
+
+
+
+
+
+
+
+
+
+
+

-
+




-
+



-
-
-







type Zid uint64

// Some important ZettelIDs.
// Note: if you change some values, ensure that you also change them in the
//       constant place. They are mentioned there literally, because these
//       constants are not available there.
const (
	Invalid          = Zid(0) // Invalid is a Zid that will never be valid
	ConfigurationZid = Zid(100)
	Invalid = Zid(0) // Invalid is a Zid that will never be valid

	// System zettel
	VersionZid              = Zid(1)
	HostZid                 = Zid(2)
	OperatingSystemZid      = Zid(3)
	LicenseZid              = Zid(4)
	AuthorsZid              = Zid(5)
	DependenciesZid         = Zid(6)
	PlaceManagerZid         = Zid(20)
	MetadataKeyZid          = Zid(90)
	StartupConfigurationZid = Zid(96)
	ConfigurationZid        = Zid(100)

	// WebUI HTML templates are in the range 10000..19999
	BaseTemplateZid    = Zid(10100)
	LoginTemplateZid   = Zid(10200)
	ListTemplateZid    = Zid(10300)
	ZettelTemplateZid  = Zid(10401)
	InfoTemplateZid    = Zid(10402)
	FormTemplateZid    = Zid(10403)
	RenameTemplateZid  = Zid(10404)
	DeleteTemplateZid  = Zid(10405)
	ContextTemplateZid = Zid(10406)
	RolesTemplateZid   = Zid(10500)
	TagsTemplateZid    = Zid(10600)
	ErrorTemplateZid   = Zid(10700)

	// WebUI CSS pages are in the range 20000..29999
	// WebUI CSS zettel are in the range 20000..29999
	BaseCSSZid = Zid(20001)

	// WebUI JS pages are in the range 30000..39999
	// WebUI JS zettel are in the range 30000..39999

	// WebUI image zettel are in the range 40000..49999
	EmojiZid = Zid(40001)

	// Range 90000...99999 is reserved for zettel templates
	TOCNewTemplateZid    = Zid(90000)
	TemplateNewZettelZid = Zid(90001)
	TemplateNewUserZid   = Zid(90002)

	DefaultHomeZid = Zid(10000000000)
)

const maxZid = 99999999999999

// ParseUint interprets a string as a possible zettel identifier
// and returns its integer value.
func ParseUint(s string) (uint64, error) {
	res, err := strconv.ParseUint(s, 10, 47)
	if err != nil {
		return 0, err
	}
	if res == 0 || res > maxZid {
		return res, strconv.ErrRange
	}
	return res, nil
}

// Parse interprets a string as a zettel identification and
// returns its integer value.
// returns its value.
func Parse(s string) (Zid, error) {
	if len(s) != 14 {
		return Invalid, strconv.ErrSyntax
	}
	res, err := strconv.ParseUint(s, 10, 47)
	res, err := ParseUint(s)
	if err != nil {
		return Invalid, err
	}
	if res == 0 {
		return Invalid, strconv.ErrRange
	}
	return Zid(res), nil
}

const digits = "0123456789"

// String converts the zettel identification to a string of 14 digits.
// Only defined for valid ids.

Changes to domain/id/id_test.go.

49
50
51
52
53
54
55

56
57
58
59
60
61
62
63



64
65
66
49
50
51
52
53
54
55
56
57

58
59
60



61
62
63
64
65
66







+

-



-
-
-
+
+
+



				"i=%d: zid=%v does not format to %q, but to %q", i, sid, zid, s)
		}
	}

	invalidIDs := []string{
		"", "0", "a",
		"00000000000000",
		"0000000000000a",
		"000000000000000",
		"99999999999999a",
		"20200310T195100",
	}

	for i, zid := range invalidIDs {
		if _, err := id.Parse(zid); err == nil {
			t.Errorf("i=%d: zid=%q is valid, but should not be", i, zid)
	for i, sid := range invalidIDs {
		if zid, err := id.Parse(sid); err == nil {
			t.Errorf("i=%d: sid=%q is valid (zid=%s), but should not be", i, sid, zid)
		}
	}
}

Changes to domain/id/set.go.

18
19
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35
36
37
38
39
40






41
42


43
44
45
46
47
48
49
50
51
52
18
19
20
21
22
23
24


25

26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


45
46
47

48
49
50
51
52
53
54
55







-
-
+
-













+
+
+
+
+
+
-
-
+
+

-








// NewSet returns a new set of identifier with the given initial values.
func NewSet(zids ...Zid) Set {
	l := len(zids)
	if l < 8 {
		l = 8
	}
	result := make(Set, l)
	for _, zid := range zids {
		result[zid] = true
	result.AddSlice(zids)
	}
	return result
}

// NewSetCap returns a new set of identifier with the given capacity and initial values.
func NewSetCap(c int, zids ...Zid) Set {
	l := len(zids)
	if c < l {
		c = l
	}
	if c < 8 {
		c = 8
	}
	result := make(Set, c)
	result.AddSlice(zids)
	return result
}

// AddSlice adds all identifier of the given slice to the set.
func (s Set) AddSlice(sl Slice) {
	for _, zid := range zids {
		result[zid] = true
	for _, zid := range sl {
		s[zid] = true
	}
	return result
}

// Sorted returns the set as a sorted slice of zettel identifier.
func (s Set) Sorted() Slice {
	if l := len(s); l > 0 {
		result := make(Slice, 0, l)
		for zid := range s {
			result = append(result, zid)
72
73
74
75
76
77
78












75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93







+
+
+
+
+
+
+
+
+
+
+
+
		otherInSet, otherOk := other[zid]
		if !otherInSet || !otherOk {
			delete(s, zid)
		}
	}
	return s
}

// Remove all zettel identifier from 's' that are in the set 'other'.
func (s Set) Remove(other Set) {
	if s == nil || other == nil {
		return
	}
	for zid, inSet := range other {
		if inSet {
			delete(s, zid)
		}
	}
}

Changes to domain/id/set_test.go.

57
58
59
60
61
62
63

























57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
		}
		got = id.NewSet(sl2...).Intersect(id.NewSet(sl1...)).Sorted()
		if !got.Equal(tc.exp) {
			t.Errorf("%d: %v.Intersect(%v) should be %v, but got %v", i, sl2, sl1, tc.exp, got)
		}
	}
}

func TestSetRemove(t *testing.T) {
	testcases := []struct {
		s1, s2 id.Set
		exp    id.Slice
	}{
		{nil, nil, nil},
		{id.NewSet(), nil, nil},
		{id.NewSet(), id.NewSet(), nil},
		{id.NewSet(1), nil, id.Slice{1}},
		{id.NewSet(1), id.NewSet(), id.Slice{1}},
		{id.NewSet(1), id.NewSet(2), id.Slice{1}},
		{id.NewSet(1), id.NewSet(1), id.Slice{}},
	}
	for i, tc := range testcases {
		sl1 := tc.s1.Sorted()
		sl2 := tc.s2.Sorted()
		newS1 := id.NewSet(sl1...)
		newS1.Remove(tc.s2)
		got := newS1.Sorted()
		if !got.Equal(tc.exp) {
			t.Errorf("%d: %v.Remove(%v) should be %v, but got %v", i, sl1, sl2, tc.exp, got)
		}
	}
}

Changes to domain/meta/meta.go.

151
152
153
154
155
156
157


158
159
160

161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170

171
172
173
174
175
176
177







+
+



+







-








// Important values for some keys.
const (
	ValueRoleConfiguration = "configuration"
	ValueRoleUser          = "user"
	ValueRoleZettel        = "zettel"
	ValueSyntaxNone        = "none"
	ValueSyntaxGif         = "gif"
	ValueSyntaxText        = "text"
	ValueSyntaxZmk         = "zmk"
	ValueTrue              = "true"
	ValueFalse             = "false"
	ValueLangEN            = "en"
	ValueUserRoleReader    = "reader"
	ValueUserRoleWriter    = "writer"
	ValueUserRoleOwner     = "owner"
	ValueVisibilityExpert  = "expert"
	ValueVisibilityOwner   = "owner"
	ValueVisibilityLogin   = "login"
	ValueVisibilityPublic  = "public"
	ValueVisibilitySimple  = "simple-expert"
)

// Meta contains all meta-data of a zettel.
type Meta struct {
	Zid     id.Zid
	pairs   map[string]string
	YamlSep bool

Changes to domain/meta/values.go.

1
2

3
4
5
6
7
8
9
1

2
3
4
5
6
7
8
9

-
+







//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
17
18
19
20
21
22
23

24
25
26
27
28
29
30

31
32
33
34
35
36
37







-







-







// Supported values for visibility.
const (
	_ Visibility = iota
	VisibilityUnknown
	VisibilityPublic
	VisibilityLogin
	VisibilityOwner
	VisibilitySimple
	VisibilityExpert
)

var visMap = map[string]Visibility{
	ValueVisibilityPublic: VisibilityPublic,
	ValueVisibilityLogin:  VisibilityLogin,
	ValueVisibilityOwner:  VisibilityOwner,
	ValueVisibilitySimple: VisibilitySimple,
	ValueVisibilityExpert: VisibilityExpert,
}

// GetVisibility returns the visibility value of the given string
func GetVisibility(val string) Visibility {
	if vis, ok := visMap[val]; ok {
		return vis

Changes to encoder/htmlenc/block.go.

52
53
54
55
56
57
58

59


60
61
62
63
64

















65
66
67
68
69
70
71
52
53
54
55
56
57
58
59

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90







+
-
+
+





+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







				v.b.WriteByte('\n')
			}
			v.b.WriteString("-->\n")
		}

	case ast.VerbatimHTML:
		for _, line := range vn.Lines {
			if !ignoreHTMLText(line) {
			v.b.WriteStrings(line, "\n")
				v.b.WriteStrings(line, "\n")
			}
		}
	default:
		panic(fmt.Sprintf("Unknown verbatim code %v", vn.Code))
	}
}

var htmlSnippetsIgnore = []string{
	"<script",
	"</script",
	"<iframe",
	"</iframe",
}

func ignoreHTMLText(s string) bool {
	lower := strings.ToLower(s)
	for _, snippet := range htmlSnippetsIgnore {
		if strings.Contains(lower, snippet) {
			return true
		}
	}
	return false
}

var specialSpanAttr = map[string]bool{
	"example":   true,
	"note":      true,
	"tip":       true,
	"important": true,
	"caution":   true,

Changes to encoder/htmlenc/inline.go.

13
14
15
16
17
18
19

20
21
22
23
24
25
26
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27







+








import (
	"fmt"
	"strconv"
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/meta"
)

// VisitText writes text content.
func (v *visitor) VisitText(tn *ast.TextNode) {
	v.writeHTMLEscaped(tn.Text)
}

249
250
251
252
253
254
255
256
257
258



259
260
261
262
263
264
265
250
251
252
253
254
255
256



257
258
259
260
261
262
263
264
265
266







-
-
-
+
+
+







	v.b.WriteByte('>')
	v.acceptInlineSlice(ins)
	v.b.WriteString("</span>")

}

var langQuotes = map[string][2]string{
	"en": {"&ldquo;", "&rdquo;"},
	"de": {"&bdquo;", "&ldquo;"},
	"fr": {"&laquo;&nbsp;", "&nbsp;&raquo;"},
	meta.ValueLangEN: {"&ldquo;", "&rdquo;"},
	"de":             {"&bdquo;", "&ldquo;"},
	"fr":             {"&laquo;&nbsp;", "&nbsp;&raquo;"},
}

func getQuotes(lang string) (string, string) {
	langFields := strings.FieldsFunc(lang, func(r rune) bool { return r == '-' || r == '_' })
	for len(langFields) > 0 {
		langSup := strings.Join(langFields, "-")
		quotes, ok := langQuotes[langSup]
297
298
299
300
301
302
303

304


305
306
307
308
309
310
311
298
299
300
301
302
303
304
305

306
307
308
309
310
311
312
313
314







+
-
+
+







	case ast.LiteralOutput:
		v.writeLiteral("<samp", "</samp>", ln.Attrs, ln.Text)
	case ast.LiteralComment:
		v.b.WriteString("<!-- ")
		v.writeHTMLEscaped(ln.Text) // writeCommentEscaped
		v.b.WriteString(" -->")
	case ast.LiteralHTML:
		if !ignoreHTMLText(ln.Text) {
		v.b.WriteString(ln.Text)
			v.b.WriteString(ln.Text)
		}
	default:
		panic(fmt.Sprintf("Unknown literal code %v", ln.Code))
	}
}

func (v *visitor) writeLiteral(codeS, codeE string, attrs *ast.Attributes, text string) {
	oldVisible := v.visibleSpace

Changes to encoder/zmkenc/zmkenc.go.

268
269
270
271
272
273
274

275
276
277
278
279
280
281
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282







+







		if i < len(tn.Text)-1 {
			s := tn.Text[i : i+2]
			if _, ok := escapeSeqs[s]; ok {
				v.b.WriteString(tn.Text[last:i])
				for j := 0; j < len(s); j++ {
					v.b.WriteBytes('\\', s[j])
				}
				i++
				last = i + 1
				continue
			}
		}
	}
	v.b.WriteString(tn.Text[last:])
}

Changes to go.mod.

1
2
3
4
5
6
7
8

9
10
11
12
1
2
3
4
5
6
7

8
9
10
11
12







-
+




module zettelstore.de/z

go 1.16

require (
	github.com/fsnotify/fsnotify v1.4.9
	github.com/pascaldekloe/jwt v1.10.0
	github.com/yuin/goldmark v1.3.5
	github.com/yuin/goldmark v1.3.7
	golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83
	golang.org/x/term v0.0.0-20201117132131-f5c789dd3221
	golang.org/x/text v0.3.6
)

Changes to go.sum.

1
2
3
4
5
6


7
8
9
10
11
12
13
1
2
3
4


5
6
7
8
9
10
11
12
13




-
-
+
+







github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/pascaldekloe/jwt v1.10.0 h1:ktcIUV4TPvh404R5dIBEnPCsSwj0sqi3/0+XafE5gJs=
github.com/pascaldekloe/jwt v1.10.0/go.mod h1:TKhllgThT7TOP5rGr2zMLKEDZRAgJfBbtKyVeRsNB9A=
github.com/yuin/goldmark v1.3.5 h1:dPmz1Snjq0kmkz159iL7S6WzdahUTHnHB5M56WFVifs=
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yuin/goldmark v1.3.7 h1:NSaHgaeJFCtWXCBkBKXw0rhgMuJ0VoE9FB5mWldcrQ4=
github.com/yuin/goldmark v1.3.7/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83 h1:/ZScEX8SfEmUGRHs0gxpqteO5nfNW6axyZbBdw9A12g=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=

Deleted index/filter.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83



















































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package index allows to search for metadata and content.
package index

import (
	"context"

	"zettelstore.de/z/domain/meta"
)

// Remover is used to remove some metadata before they are stored in a place.
type Remover interface {
	// Remove removes computed properties from the given metadata.
	// It is called by the manager place before meta data is updated.
	Remove(ctx context.Context, m *meta.Meta)
}

// MetaFilter is used by places to filter and set computed metadata value.
type MetaFilter interface {
	Enricher
	Remover
}

type metaFilter struct {
	index      Indexer
	properties map[string]bool // Set of property key names
}

// NewMetaFilter creates a new meta filter.
func NewMetaFilter(idx Indexer) MetaFilter {
	properties := make(map[string]bool)
	for _, kd := range meta.GetSortedKeyDescriptions() {
		if kd.IsProperty() {
			properties[kd.Name] = true
		}
	}
	return &metaFilter{
		index:      idx,
		properties: properties,
	}
}

func (mf *metaFilter) Enrich(ctx context.Context, m *meta.Meta) {
	computePublished(m)
	mf.index.Enrich(ctx, m)
}

func computePublished(m *meta.Meta) {
	if _, ok := m.Get(meta.KeyPublished); ok {
		return
	}
	if modified, ok := m.Get(meta.KeyModified); ok {
		if _, ok = meta.TimeValue(modified); ok {
			m.Set(meta.KeyPublished, modified)
			return
		}
	}
	zid := m.Zid.String()
	if _, ok := meta.TimeValue(zid); ok {
		m.Set(meta.KeyPublished, zid)
		return
	}

	// Neither the zettel was modified nor the zettel identifer contains a valid
	// timestamp. In this case do not set the "published" property.
}

func (mf *metaFilter) Remove(ctx context.Context, m *meta.Meta) {
	for _, p := range m.PairsRest(true) {
		if mf.properties[p.Key] {
			m.Delete(p.Key)
		}
	}
}

Deleted index/index.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135







































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package index allows to search for metadata and content.
package index

import (
	"context"
	"io"
	"time"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place/change"
)

// Enricher is used to update metadata by adding new properties.
type Enricher interface {
	// Enrich computes additional properties and updates the given metadata.
	// It is typically called by zettel reading methods.
	Enrich(ctx context.Context, m *meta.Meta)
}

// Selector is used to select zettel identifier based on selection criteria.
type Selector interface {
	// Select all zettel that contains the given exact word.
	// The word must be normalized through Unicode NKFD.
	Select(word string) id.Set

	// Select all zettel that have a word with the given prefix.
	// The prefix must be normalized through Unicode NKFD.
	SelectPrefix(prefix string) id.Set

	// Select all zettel that contains the given string.
	// The string must be normalized through Unicode NKFD.
	SelectContains(s string) id.Set
}

// NoEnrichContext will signal an enricher that nothing has to be done.
// This is useful for an Indexer, but also for some place.Place calls, when
// just the plain metadata is needed.
func NoEnrichContext(ctx context.Context) context.Context {
	return context.WithValue(ctx, ctxNoEnrichKey, &ctxNoEnrichKey)
}

type ctxNoEnrichType struct{}

var ctxNoEnrichKey ctxNoEnrichType

// DoNotEnrich determines if the context is marked to not enrich metadata.
func DoNotEnrich(ctx context.Context) bool {
	_, ok := ctx.Value(ctxNoEnrichKey).(*ctxNoEnrichType)
	return ok
}

// Port contains all the used functions to access zettel to be indexed.
type Port interface {
	RegisterObserver(change.Func)
	FetchZids(context.Context) (id.Set, error)
	GetMeta(context.Context, id.Zid) (*meta.Meta, error)
	GetZettel(context.Context, id.Zid) (domain.Zettel, error)
}

// Indexer contains all the functions of an index.
type Indexer interface {
	Enricher
	Selector

	// Start the index. It will read all zettel and store index data for later retrieval.
	Start(Port)

	// Stop the index. No zettel are read any more, but the current index data
	// can stil be retrieved.
	Stop()

	// ReadStats populates st with indexer statistics.
	ReadStats(st *IndexerStats)
}

// IndexerStats records statistics about the indexer.
type IndexerStats struct {
	// LastReload stores the timestamp when a full re-index was done.
	LastReload time.Time

	// IndexesSinceReload counts indexing a zettel since the full re-index.
	IndexesSinceReload uint64

	// DurLastIndex is the duration of the last index run. This could be a
	// full re-index or a re-index of a single zettel.
	DurLastIndex time.Duration

	// Store records statistics about the underlying index store.
	Store StoreStats
}

// Store all relevant zettel data. There may be multiple implementations, i.e.
// memory-based, file-based, based on SQLite, ...
type Store interface {
	Enricher
	Selector

	// UpdateReferences for a specific zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	UpdateReferences(context.Context, *ZettelIndex) id.Set

	// DeleteZettel removes index data for given zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	DeleteZettel(context.Context, id.Zid) id.Set

	// ReadStats populates st with store statistics.
	ReadStats(st *StoreStats)

	// Write the content to a Writer.
	Write(io.Writer)
}

// StoreStats records statistics about the store.
type StoreStats struct {
	// Zettel is the number of zettel managed by the indexer.
	Zettel int

	// Updates count the number of metadata updates.
	Updates uint64

	// Words count the different words stored in the store.
	Words uint64
}

Deleted index/indexer/anteroom.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187



























































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package indexer allows to search for metadata and content.
package indexer

import (
	"sync"

	"zettelstore.de/z/domain/id"
)

type arAction int

const (
	arNothing arAction = iota
	arReload
	arUpdate
	arDelete
)

type anteroom struct {
	next    *anteroom
	waiting map[id.Zid]arAction
	curLoad int
	reload  bool
}

type anterooms struct {
	mx      sync.Mutex
	first   *anteroom
	last    *anteroom
	maxLoad int
}

func newAnterooms(maxLoad int) *anterooms {
	return &anterooms{maxLoad: maxLoad}
}

func (ar *anterooms) Enqueue(zid id.Zid, action arAction) {
	if !zid.IsValid() || action == arNothing || action == arReload {
		return
	}
	ar.mx.Lock()
	defer ar.mx.Unlock()
	if ar.first == nil {
		ar.first = ar.makeAnteroom(zid, action)
		ar.last = ar.first
		return
	}
	for room := ar.first; room != nil; room = room.next {
		if room.reload {
			continue // Do not place zettel in reload room
		}
		a, ok := room.waiting[zid]
		if !ok {
			continue
		}
		switch action {
		case a:
			return
		case arUpdate:
			room.waiting[zid] = action
		case arDelete:
			room.waiting[zid] = action
		}
		return
	}
	if room := ar.last; !room.reload && (ar.maxLoad == 0 || room.curLoad < ar.maxLoad) {
		room.waiting[zid] = action
		room.curLoad++
		return
	}
	room := ar.makeAnteroom(zid, action)
	ar.last.next = room
	ar.last = room
}

func (ar *anterooms) makeAnteroom(zid id.Zid, action arAction) *anteroom {
	c := ar.maxLoad
	if c == 0 {
		c = 100
	}
	waiting := make(map[id.Zid]arAction, c)
	waiting[zid] = action
	return &anteroom{next: nil, waiting: waiting, curLoad: 1, reload: false}
}

func (ar *anterooms) Reset() {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	ar.first = ar.makeAnteroom(id.Invalid, arReload)
	ar.last = ar.first
}

func (ar *anterooms) Reload(delZids id.Slice, newZids id.Set) {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	delWaiting := createWaitingSlice(delZids, arDelete)
	newWaiting := createWaitingSet(newZids, arUpdate)
	ar.deleteReloadedRooms()

	if ds := len(delWaiting); ds > 0 {
		if ns := len(newWaiting); ns > 0 {
			roomNew := &anteroom{next: ar.first, waiting: newWaiting, curLoad: ns, reload: true}
			ar.first = &anteroom{next: roomNew, waiting: delWaiting, curLoad: ds, reload: true}
			if roomNew.next == nil {
				ar.last = roomNew
			}
			return
		}

		ar.first = &anteroom{next: ar.first, waiting: delWaiting, curLoad: ds}
		if ar.first.next == nil {
			ar.last = ar.first
		}
		return
	}

	if ns := len(newWaiting); ns > 0 {
		ar.first = &anteroom{next: ar.first, waiting: newWaiting, curLoad: ns}
		if ar.first.next == nil {
			ar.last = ar.first
		}
		return
	}

	ar.first = nil
	ar.last = nil
}

func createWaitingSlice(zids id.Slice, action arAction) map[id.Zid]arAction {
	waitingSet := make(map[id.Zid]arAction, len(zids))
	for _, zid := range zids {
		if zid.IsValid() {
			waitingSet[zid] = action
		}
	}
	return waitingSet
}

func createWaitingSet(zids id.Set, action arAction) map[id.Zid]arAction {
	waitingSet := make(map[id.Zid]arAction, len(zids))
	for zid := range zids {
		if zid.IsValid() {
			waitingSet[zid] = action
		}
	}
	return waitingSet
}

func (ar *anterooms) deleteReloadedRooms() {
	room := ar.first
	for room != nil && room.reload {
		room = room.next
	}
	ar.first = room
	if room == nil {
		ar.last = nil
	}
}

func (ar *anterooms) Dequeue() (arAction, id.Zid) {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	if ar.first == nil {
		return arNothing, id.Invalid
	}
	for zid, action := range ar.first.waiting {
		delete(ar.first.waiting, zid)
		if len(ar.first.waiting) == 0 {
			ar.first = ar.first.next
			if ar.first == nil {
				ar.last = nil
			}
		}
		return action, zid
	}
	return arNothing, id.Invalid
}

Deleted index/indexer/anteroom_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122


























































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package indexer allows to search for metadata and content.
package indexer

import (
	"testing"

	"zettelstore.de/z/domain/id"
)

func TestSimple(t *testing.T) {
	ar := newAnterooms(2)
	ar.Enqueue(id.Zid(1), arUpdate)
	action, zid := ar.Dequeue()
	if zid != id.Zid(1) || action != arUpdate {
		t.Errorf("Expected 1/arUpdate, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if zid != id.Invalid && action != arDelete {
		t.Errorf("Expected invalid Zid, but got %v", zid)
	}
	ar.Enqueue(id.Zid(1), arUpdate)
	ar.Enqueue(id.Zid(2), arUpdate)
	if ar.first != ar.last {
		t.Errorf("Expected one room, but got more")
	}
	ar.Enqueue(id.Zid(3), arUpdate)
	if ar.first == ar.last {
		t.Errorf("Expected more than one room, but got only one")
	}

	count := 0
	for ; count < 1000; count++ {
		action, _ := ar.Dequeue()
		if action == arNothing {
			break
		}
	}
	if count != 3 {
		t.Errorf("Expected 3 dequeues, but got %v", count)
	}
}

func TestReset(t *testing.T) {
	ar := newAnterooms(1)
	ar.Enqueue(id.Zid(1), arUpdate)
	ar.Reset()
	action, zid := ar.Dequeue()
	if action != arReload || zid != id.Invalid {
		t.Errorf("Expected reload & invalid Zid, but got %v/%v", action, zid)
	}
	ar.Reload(id.Slice{2}, id.NewSet(3, 4))
	ar.Enqueue(id.Zid(5), arUpdate)
	ar.Enqueue(id.Zid(5), arDelete)
	ar.Enqueue(id.Zid(5), arDelete)
	ar.Enqueue(id.Zid(5), arUpdate)
	if ar.first == ar.last || ar.first.next == ar.last || ar.first.next.next != ar.last {
		t.Errorf("Expected 3 rooms")
	}
	action, zid = ar.Dequeue()
	if zid != id.Zid(2) || action != arDelete {
		t.Errorf("Expected 2/arDelete, but got %v/%v", zid, action)
	}
	action, zid1 := ar.Dequeue()
	if action != arUpdate {
		t.Errorf("Expected arUpdate, but got %v", action)
	}
	action, zid2 := ar.Dequeue()
	if action != arUpdate {
		t.Errorf("Expected arUpdate, but got %v", action)
	}
	if !(zid1 == id.Zid(3) && zid2 == id.Zid(4) || zid1 == id.Zid(4) && zid2 == id.Zid(3)) {
		t.Errorf("Zids must be 3 or 4, but got %v/%v", zid1, zid2)
	}
	action, zid = ar.Dequeue()
	if zid != id.Zid(5) || action != arUpdate {
		t.Errorf("Expected 5/arUpdate, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}

	ar = newAnterooms(1)
	ar.Reload(nil, id.NewSet(id.Zid(6)))
	action, zid = ar.Dequeue()
	if zid != id.Zid(6) || action != arUpdate {
		t.Errorf("Expected 6/arUpdate, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}

	ar = newAnterooms(1)
	ar.Reload(id.Slice{7}, nil)
	action, zid = ar.Dequeue()
	if zid != id.Zid(7) || action != arDelete {
		t.Errorf("Expected 7/arDelete, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}

	ar = newAnterooms(1)
	ar.Enqueue(id.Zid(8), arUpdate)
	ar.Reload(nil, nil)
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}
}

Deleted index/indexer/collect.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126






























































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package indexer allows to search for metadata and content.
package indexer

import (
	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/index"
	"zettelstore.de/z/strfun"
)

func collectZettelIndexData(zn *ast.ZettelNode, refs id.Set, words index.WordSet) {
	ixv := ixVisitor{refs: refs, words: words}
	ast.NewTopDownTraverser(&ixv).VisitBlockSlice(zn.Ast)
}

func collectInlineIndexData(ins ast.InlineSlice, refs id.Set, words index.WordSet) {
	ixv := ixVisitor{refs: refs, words: words}
	ast.NewTopDownTraverser(&ixv).VisitInlineSlice(ins)
}

type ixVisitor struct {
	refs  id.Set
	words index.WordSet
}

// VisitVerbatim collects the verbatim text in the word set.
func (lv *ixVisitor) VisitVerbatim(vn *ast.VerbatimNode) {
	for _, line := range vn.Lines {
		lv.addText(line)
	}
}

// VisitRegion does nothing.
func (lv *ixVisitor) VisitRegion(rn *ast.RegionNode) {}

// VisitHeading does nothing.
func (lv *ixVisitor) VisitHeading(hn *ast.HeadingNode) {}

// VisitHRule does nothing.
func (lv *ixVisitor) VisitHRule(hn *ast.HRuleNode) {}

// VisitList does nothing.
func (lv *ixVisitor) VisitNestedList(ln *ast.NestedListNode) {}

// VisitDescriptionList does nothing.
func (lv *ixVisitor) VisitDescriptionList(dn *ast.DescriptionListNode) {}

// VisitPara does nothing.
func (lv *ixVisitor) VisitPara(pn *ast.ParaNode) {}

// VisitTable does nothing.
func (lv *ixVisitor) VisitTable(tn *ast.TableNode) {}

// VisitBLOB does nothing.
func (lv *ixVisitor) VisitBLOB(bn *ast.BLOBNode) {}

// VisitText collects the text in the word set.
func (lv *ixVisitor) VisitText(tn *ast.TextNode) {
	lv.addText(tn.Text)
}

// VisitTag collects the tag name in the word set.
func (lv *ixVisitor) VisitTag(tn *ast.TagNode) {
	lv.addText(tn.Tag)
}

// VisitSpace does nothing.
func (lv *ixVisitor) VisitSpace(sn *ast.SpaceNode) {}

// VisitBreak does nothing.
func (lv *ixVisitor) VisitBreak(bn *ast.BreakNode) {}

// VisitLink collects the given link as a reference.
func (lv *ixVisitor) VisitLink(ln *ast.LinkNode) {
	ref := ln.Ref
	if ref == nil || !ref.IsZettel() {
		return
	}
	if zid, err := id.Parse(ref.URL.Path); err == nil {
		lv.refs[zid] = true
	}
}

// VisitImage collects the image links as a reference.
func (lv *ixVisitor) VisitImage(in *ast.ImageNode) {
	ref := in.Ref
	if ref == nil || !ref.IsZettel() {
		return
	}
	if zid, err := id.Parse(ref.URL.Path); err == nil {
		lv.refs[zid] = true
	}
}

// VisitCite does nothing.
func (lv *ixVisitor) VisitCite(cn *ast.CiteNode) {}

// VisitFootnote does nothing.
func (lv *ixVisitor) VisitFootnote(fn *ast.FootnoteNode) {}

// VisitMark does nothing.
func (lv *ixVisitor) VisitMark(mn *ast.MarkNode) {}

// VisitFormat does nothing.
func (lv *ixVisitor) VisitFormat(fn *ast.FormatNode) {}

// VisitLiteral collects the literal words in the word set.
func (lv *ixVisitor) VisitLiteral(ln *ast.LiteralNode) {
	lv.addText(ln.Text)
}

func (lv *ixVisitor) addText(s string) {
	for _, word := range strfun.NormalizeWords(s) {
		lv.words[word] = lv.words[word] + 1
	}
}

Deleted index/indexer/indexer.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297









































































































































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package indexer allows to search for metadata and content.
package indexer

import (
	"context"
	"log"
	"runtime/debug"
	"sync"
	"time"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/index/memstore"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/strfun"
)

type indexer struct {
	store   index.Store
	ar      *anterooms
	ready   chan struct{} // Signal a non-empty anteroom to background task
	done    chan struct{} // Stop background task
	observe bool
	started bool

	// Stats data
	mx           sync.RWMutex
	lastReload   time.Time
	sinceReload  uint64
	durLastIndex time.Duration
}

// New creates a new indexer.
func New() index.Indexer {
	return &indexer{
		store: memstore.New(),
		ar:    newAnterooms(10),
		ready: make(chan struct{}, 1),
	}
}

func (idx *indexer) observer(ci change.Info) {
	switch ci.Reason {
	case change.OnReload:
		idx.ar.Reset()
	case change.OnUpdate:
		idx.ar.Enqueue(ci.Zid, arUpdate)
	case change.OnDelete:
		idx.ar.Enqueue(ci.Zid, arDelete)
	default:
		return
	}
	select {
	case idx.ready <- struct{}{}:
	default:
	}
}

func (idx *indexer) Start(p index.Port) {
	if idx.started {
		panic("Index already started")
	}
	idx.done = make(chan struct{})
	if !idx.observe {
		p.RegisterObserver(idx.observer)
		idx.observe = true
	}
	idx.ar.Reset() // Ensure an initial index run
	go idx.indexer(p)
	idx.started = true
}

func (idx *indexer) Stop() {
	if !idx.started {
		panic("Index already stopped")
	}
	close(idx.done)
	idx.started = false
}

// Enrich reads all properties in the index and updates the metadata.
func (idx *indexer) Enrich(ctx context.Context, m *meta.Meta) {
	if index.DoNotEnrich(ctx) {
		// Enrich is called indirectly via indexer or enrichment is not requested
		// because of other reasons -> ignore this call, do not update meta data
		return
	}
	idx.store.Enrich(ctx, m)
}

// Select all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD.
func (idx *indexer) Select(word string) id.Set {
	return idx.store.Select(word)
}

// Select all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD.
func (idx *indexer) SelectPrefix(prefix string) id.Set {
	return idx.store.SelectPrefix(prefix)
}

// Select all zettel that contains the given string.
// The string must be normalized through Unicode NKFD.
func (idx *indexer) SelectContains(s string) id.Set {
	return idx.store.SelectContains(s)
}

// ReadStats populates st with indexer statistics.
func (idx *indexer) ReadStats(st *index.IndexerStats) {
	idx.mx.RLock()
	st.LastReload = idx.lastReload
	st.IndexesSinceReload = idx.sinceReload
	st.DurLastIndex = idx.durLastIndex
	idx.mx.RUnlock()
	idx.store.ReadStats(&st.Store)
}

type indexerPort interface {
	getMetaPort
	FetchZids(ctx context.Context) (id.Set, error)
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)
}

// indexer runs in the background and updates the index data structures.
// This is the main service of the indexer.
func (idx *indexer) indexer(p indexerPort) {
	// Something may panic. Ensure a running indexer.
	defer func() {
		if r := recover(); r != nil {
			log.Println("recovered from:", r)
			debug.PrintStack()
			go idx.indexer(p)
		}
	}()

	timerDuration := 15 * time.Second
	timer := time.NewTimer(timerDuration)
	ctx := index.NoEnrichContext(context.Background())
	for {
		start := time.Now()
		if idx.workService(ctx, p) {
			idx.mx.Lock()
			idx.durLastIndex = time.Since(start)
			idx.mx.Unlock()
		}
		if !idx.sleepService(timer, timerDuration) {
			return
		}
	}
}

func (idx *indexer) workService(ctx context.Context, p indexerPort) bool {
	changed := false
	for {
		switch action, zid := idx.ar.Dequeue(); action {
		case arNothing:
			return changed
		case arReload:
			zids, err := p.FetchZids(ctx)
			if err == nil {
				idx.ar.Reload(nil, zids)
				idx.mx.Lock()
				idx.lastReload = time.Now()
				idx.sinceReload = 0
				idx.mx.Unlock()
			}
		case arUpdate:
			changed = true
			idx.mx.Lock()
			idx.sinceReload++
			idx.mx.Unlock()
			zettel, err := p.GetZettel(ctx, zid)
			if err != nil {
				// TODO: on some errors put the zid into a "try later" set
				continue
			}
			idx.updateZettel(ctx, zettel, p)
		case arDelete:
			changed = true
			idx.mx.Lock()
			idx.sinceReload++
			idx.mx.Unlock()
			idx.deleteZettel(zid)
		}
	}
}

func (idx *indexer) sleepService(timer *time.Timer, timerDuration time.Duration) bool {
	select {
	case _, ok := <-idx.ready:
		if !ok {
			return false
		}
	case _, ok := <-timer.C:
		if !ok {
			return false
		}
		timer.Reset(timerDuration)
	case _, ok := <-idx.done:
		if !ok {
			if !timer.Stop() {
				<-timer.C
			}
			return false
		}
	}
	return true
}

type getMetaPort interface {
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
}

func (idx *indexer) updateZettel(ctx context.Context, zettel domain.Zettel, p getMetaPort) {
	m := zettel.Meta
	if m.GetBool(meta.KeyNoIndex) {
		// Zettel maybe in index
		toCheck := idx.store.DeleteZettel(ctx, m.Zid)
		idx.checkZettel(toCheck)
		return
	}
	refs := id.NewSet()
	words := make(index.WordSet)
	collectZettelIndexData(parser.ParseZettel(zettel, ""), refs, words)
	zi := index.NewZettelIndex(m.Zid)
	for _, pair := range m.Pairs(false) {
		descr := meta.GetDescription(pair.Key)
		if descr.IsComputed() {
			continue
		}
		switch descr.Type {
		case meta.TypeID:
			updateValue(ctx, descr.Inverse, pair.Value, p, zi)
		case meta.TypeIDSet:
			for _, val := range meta.ListFromValue(pair.Value) {
				updateValue(ctx, descr.Inverse, val, p, zi)
			}
		case meta.TypeZettelmarkup:
			collectInlineIndexData(parser.ParseMetadata(pair.Value), refs, words)
		default:
			for _, word := range strfun.NormalizeWords(pair.Value) {
				words[word] = words[word] + 1
			}
		}
	}
	for ref := range refs {
		if _, err := p.GetMeta(ctx, ref); err == nil {
			zi.AddBackRef(ref)
		} else {
			zi.AddDeadRef(ref)
		}
	}
	zi.SetWords(words)
	toCheck := idx.store.UpdateReferences(ctx, zi)
	idx.checkZettel(toCheck)
}

func updateValue(ctx context.Context, inverse string, value string, p getMetaPort, zi *index.ZettelIndex) {
	zid, err := id.Parse(value)
	if err != nil {
		return
	}
	if _, err := p.GetMeta(ctx, zid); err != nil {
		zi.AddDeadRef(zid)
		return
	}
	if inverse == "" {
		zi.AddBackRef(zid)
		return
	}
	zi.AddMetaRef(inverse, zid)
}

func (idx *indexer) deleteZettel(zid id.Zid) {
	toCheck := idx.store.DeleteZettel(context.Background(), zid)
	idx.checkZettel(toCheck)
}

func (idx *indexer) checkZettel(s id.Set) {
	for zid := range s {
		idx.ar.Enqueue(zid, arUpdate)
	}
}

Deleted index/memstore/memstore.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441

























































































































































































































































































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package memstore stored the index in main memory.
package memstore

import (
	"context"
	"fmt"
	"io"
	"strings"
	"sync"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
)

type metaRefs struct {
	forward  id.Slice
	backward id.Slice
}

type zettelIndex struct {
	dead     id.Slice
	forward  id.Slice
	backward id.Slice
	meta     map[string]metaRefs
	words    []string
}

func (zi *zettelIndex) isEmpty() bool {
	if len(zi.forward) > 0 || len(zi.backward) > 0 || len(zi.dead) > 0 || len(zi.words) > 0 {
		return false
	}
	return zi.meta == nil || len(zi.meta) == 0
}

type memStore struct {
	mx    sync.RWMutex
	idx   map[id.Zid]*zettelIndex
	dead  map[id.Zid]id.Slice // map dead refs where they occur
	words map[string]id.Slice

	// Stats
	updates uint64
}

// New returns a new memory-based index store.
func New() index.Store {
	return &memStore{
		idx:   make(map[id.Zid]*zettelIndex),
		dead:  make(map[id.Zid]id.Slice),
		words: make(map[string]id.Slice),
	}
}

func (ms *memStore) Enrich(ctx context.Context, m *meta.Meta) {
	if ms.doEnrich(ctx, m) {
		ms.mx.Lock()
		ms.updates++
		ms.mx.Unlock()
	}
}

func (ms *memStore) doEnrich(ctx context.Context, m *meta.Meta) bool {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	zi, ok := ms.idx[m.Zid]
	if !ok {
		return false
	}
	var updated bool
	if len(zi.dead) > 0 {
		m.Set(meta.KeyDead, zi.dead.String())
		updated = true
	}
	back := removeOtherMetaRefs(m, zi.backward.Copy())
	if len(zi.backward) > 0 {
		m.Set(meta.KeyBackward, zi.backward.String())
		updated = true
	}
	if len(zi.forward) > 0 {
		m.Set(meta.KeyForward, zi.forward.String())
		back = remRefs(back, zi.forward)
		updated = true
	}
	if len(zi.meta) > 0 {
		for k, refs := range zi.meta {
			if len(refs.backward) > 0 {
				m.Set(k, refs.backward.String())
				back = remRefs(back, refs.backward)
				updated = true
			}
		}
	}
	if len(back) > 0 {
		m.Set(meta.KeyBack, back.String())
		updated = true
	}
	return updated
}

// Select all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD.
func (ms *memStore) Select(word string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	if refs, ok := ms.words[word]; ok {
		return id.NewSet(refs...)
	}
	return nil
}

// Select all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD.
func (ms *memStore) SelectPrefix(prefix string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	return ms.selectWithPred(prefix, strings.HasPrefix)
}

// Select all zettel that contains the given string.
// The string must be normalized through Unicode NKFD.
func (ms *memStore) SelectContains(s string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	return ms.selectWithPred(s, strings.Contains)
}

func (ms *memStore) selectWithPred(s string, pred func(string, string) bool) id.Set {
	result := id.NewSet()
	for word, refs := range ms.words {
		if !pred(word, s) {
			continue
		}
		for _, ref := range refs {
			result[ref] = true
		}
	}
	return result
}

func removeOtherMetaRefs(m *meta.Meta, back id.Slice) id.Slice {
	for _, p := range m.PairsRest(false) {
		switch meta.Type(p.Key) {
		case meta.TypeID:
			if zid, err := id.Parse(p.Value); err == nil {
				back = remRef(back, zid)
			}
		case meta.TypeIDSet:
			for _, val := range meta.ListFromValue(p.Value) {
				if zid, err := id.Parse(val); err == nil {
					back = remRef(back, zid)
				}
			}
		}
	}
	return back
}

func (ms *memStore) UpdateReferences(ctx context.Context, zidx *index.ZettelIndex) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()
	zi, ziExist := ms.idx[zidx.Zid]
	if !ziExist || zi == nil {
		zi = &zettelIndex{}
		ziExist = false
	}

	// Is this zettel an old dead reference mentioned in other zettel?
	var toCheck id.Set
	if refs, ok := ms.dead[zidx.Zid]; ok {
		// These must be checked later again
		toCheck = id.NewSet(refs...)
		delete(ms.dead, zidx.Zid)
	}

	ms.updateDeadReferences(zidx, zi)
	ms.updateForwardBackwardReferences(zidx, zi)
	ms.updateMetadataReferences(zidx, zi)
	ms.updateWords(zidx, zi)

	// Check if zi must be inserted into ms.idx
	if !ziExist && !zi.isEmpty() {
		ms.idx[zidx.Zid] = zi
	}

	return toCheck
}

func (ms *memStore) updateDeadReferences(zidx *index.ZettelIndex, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	drefs := zidx.GetDeadRefs()
	newRefs, remRefs := refsDiff(drefs, zi.dead)
	zi.dead = drefs
	for _, ref := range remRefs {
		ms.dead[ref] = remRef(ms.dead[ref], zidx.Zid)
	}
	for _, ref := range newRefs {
		ms.dead[ref] = addRef(ms.dead[ref], zidx.Zid)
	}
}

func (ms *memStore) updateForwardBackwardReferences(zidx *index.ZettelIndex, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	brefs := zidx.GetBackRefs()
	newRefs, remRefs := refsDiff(brefs, zi.forward)
	zi.forward = brefs
	for _, ref := range remRefs {
		bzi := ms.getEntry(ref)
		bzi.backward = remRef(bzi.backward, zidx.Zid)
	}
	for _, ref := range newRefs {
		bzi := ms.getEntry(ref)
		bzi.backward = addRef(bzi.backward, zidx.Zid)
	}
}

func (ms *memStore) updateMetadataReferences(zidx *index.ZettelIndex, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	metarefs := zidx.GetMetaRefs()
	for key, mr := range zi.meta {
		if _, ok := metarefs[key]; ok {
			continue
		}
		ms.removeInverseMeta(zidx.Zid, key, mr.forward)
	}
	if zi.meta == nil {
		zi.meta = make(map[string]metaRefs)
	}
	for key, mrefs := range metarefs {
		mr := zi.meta[key]
		newRefs, remRefs := refsDiff(mrefs, mr.forward)
		mr.forward = mrefs
		zi.meta[key] = mr

		for _, ref := range newRefs {
			bzi := ms.getEntry(ref)
			if bzi.meta == nil {
				bzi.meta = make(map[string]metaRefs)
			}
			bmr := bzi.meta[key]
			bmr.backward = addRef(bmr.backward, zidx.Zid)
			bzi.meta[key] = bmr
		}
		ms.removeInverseMeta(zidx.Zid, key, remRefs)
	}
}

func (ms *memStore) updateWords(zidx *index.ZettelIndex, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	words := zidx.GetWords()
	newWords, removeWords := words.Diff(zi.words)
	for _, word := range newWords {
		if refs, ok := ms.words[word]; ok {
			ms.words[word] = addRef(refs, zidx.Zid)
			continue
		}
		ms.words[word] = id.Slice{zidx.Zid}
	}
	for _, word := range removeWords {
		refs, ok := ms.words[word]
		if !ok {
			continue
		}
		refs2 := remRef(refs, zidx.Zid)
		if len(refs2) == 0 {
			delete(ms.words, word)
			continue
		}
		ms.words[word] = refs2
	}
	zi.words = words.Words()
}

func (ms *memStore) getEntry(zid id.Zid) *zettelIndex {
	// Must only be called if ms.mx is write-locked!
	if zi, ok := ms.idx[zid]; ok {
		return zi
	}
	zi := &zettelIndex{}
	ms.idx[zid] = zi
	return zi
}

func (ms *memStore) DeleteZettel(ctx context.Context, zid id.Zid) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()

	zi, ok := ms.idx[zid]
	if !ok {
		return nil
	}

	ms.deleteDeadSources(zid, zi)
	toCheck := ms.deleteForwardBackward(zid, zi)
	if len(zi.meta) > 0 {
		for key, mrefs := range zi.meta {
			ms.removeInverseMeta(zid, key, mrefs.forward)
		}
	}
	ms.deleteWords(zid, zi.words)
	delete(ms.idx, zid)
	return toCheck
}

func (ms *memStore) deleteDeadSources(zid id.Zid, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	for _, ref := range zi.dead {
		if drefs, ok := ms.dead[ref]; ok {
			drefs = remRef(drefs, zid)
			if len(drefs) > 0 {
				ms.dead[ref] = drefs
			} else {
				delete(ms.dead, ref)
			}
		}
	}
}

func (ms *memStore) deleteForwardBackward(zid id.Zid, zi *zettelIndex) id.Set {
	// Must only be called if ms.mx is write-locked!
	var toCheck id.Set
	for _, ref := range zi.forward {
		if fzi, ok := ms.idx[ref]; ok {
			fzi.backward = remRef(fzi.backward, zid)
		}
	}
	for _, ref := range zi.backward {
		if bzi, ok := ms.idx[ref]; ok {
			bzi.forward = remRef(bzi.forward, zid)
			if toCheck == nil {
				toCheck = id.NewSet()
			}
			toCheck[ref] = true
		}
	}
	return toCheck
}

func (ms *memStore) removeInverseMeta(zid id.Zid, key string, forward id.Slice) {
	// Must only be called if ms.mx is write-locked!
	for _, ref := range forward {
		bzi, ok := ms.idx[ref]
		if !ok || bzi.meta == nil {
			continue
		}
		bmr, ok := bzi.meta[key]
		if !ok {
			continue
		}
		bmr.backward = remRef(bmr.backward, zid)
		if len(bmr.backward) > 0 || len(bmr.forward) > 0 {
			bzi.meta[key] = bmr
		} else {
			delete(bzi.meta, key)
			if len(bzi.meta) == 0 {
				bzi.meta = nil
			}
		}
	}
}

func (ms *memStore) deleteWords(zid id.Zid, words []string) {
	// Must only be called if ms.mx is write-locked!
	for _, word := range words {
		refs, ok := ms.words[word]
		if !ok {
			continue
		}
		refs2 := remRef(refs, zid)
		if len(refs2) == 0 {
			delete(ms.words, word)
			continue
		}
		ms.words[word] = refs2
	}
}

func (ms *memStore) ReadStats(st *index.StoreStats) {
	ms.mx.RLock()
	st.Zettel = len(ms.idx)
	st.Updates = ms.updates
	st.Words = uint64(len(ms.words))
	ms.mx.RUnlock()
}

func (ms *memStore) Write(w io.Writer) {
	ms.mx.RLock()
	defer ms.mx.RUnlock()

	zids := make(id.Slice, 0, len(ms.idx))
	for id := range ms.idx {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, id)
		zi := ms.idx[id]
		fmt.Fprintln(w, "-", zi.dead)
		writeZidsLn(w, ">", zi.forward)
		writeZidsLn(w, "<", zi.backward)
		if zi.meta == nil {
			fmt.Fprintln(w, "*NIL")
		} else if len(zi.meta) == 0 {
			fmt.Fprintln(w, "*(0)")
		} else {
			for k, fb := range zi.meta {
				fmt.Fprintln(w, "*", k)
				writeZidsLn(w, "]", fb.forward)
				writeZidsLn(w, "[", fb.backward)
			}
		}
	}

	zids = make(id.Slice, 0, len(ms.dead))
	for id := range ms.dead {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, "~", id, ms.dead[id])
	}
}

func writeZidsLn(w io.Writer, prefix string, zids id.Slice) {
	io.WriteString(w, prefix)
	for _, zid := range zids {
		io.WriteString(w, " ")
		w.Write(zid.Bytes())
	}
	fmt.Fprintln(w)
}

Deleted index/memstore/refs.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package memstore stored the index in main memory.
package memstore

import (
	"zettelstore.de/z/domain/id"
)

func refsDiff(refsN, refsO id.Slice) (newRefs, remRefs id.Slice) {
	npos, opos := 0, 0
	for npos < len(refsN) && opos < len(refsO) {
		rn, ro := refsN[npos], refsO[opos]
		if rn == ro {
			npos++
			opos++
			continue
		}
		if rn < ro {
			newRefs = append(newRefs, rn)
			npos++
			continue
		}
		remRefs = append(remRefs, ro)
		opos++
	}
	if npos < len(refsN) {
		newRefs = append(newRefs, refsN[npos:]...)
	}
	if opos < len(refsO) {
		remRefs = append(remRefs, refsO[opos:]...)
	}
	return newRefs, remRefs
}

func addRef(refs id.Slice, ref id.Zid) id.Slice {
	if len(refs) == 0 {
		return append(refs, ref)
	}
	for i, r := range refs {
		if r == ref {
			return refs
		}
		if r > ref {
			return append(refs[:i], append(id.Slice{ref}, refs[i:]...)...)
		}
	}
	return append(refs, ref)
}

func remRefs(refs id.Slice, rem id.Slice) id.Slice {
	if len(refs) == 0 || len(rem) == 0 {
		return refs
	}
	result := make(id.Slice, 0, len(refs))
	rpos, dpos := 0, 0
	for rpos < len(refs) && dpos < len(rem) {
		rr, dr := refs[rpos], rem[dpos]
		if rr < dr {
			result = append(result, rr)
			rpos++
			continue
		}
		if dr < rr {
			dpos++
			continue
		}
		rpos++
		dpos++
	}
	if rpos < len(refs) {
		result = append(result, refs[rpos:]...)
	}
	return result
}

func remRef(refs id.Slice, ref id.Zid) id.Slice {
	for i, r := range refs {
		if r == ref {
			return append(refs[:i], refs[i+1:]...)
		}
		if r > ref {
			return refs
		}
	}
	return refs
}

Deleted index/memstore/refs_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134






































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package memstore stored the index in main memory.
package memstore

import (
	"testing"

	"zettelstore.de/z/domain/id"
)

func assertRefs(t *testing.T, i int, got, exp id.Slice) {
	t.Helper()
	if got == nil && exp != nil {
		t.Errorf("%d: got nil, but expected %v", i, exp)
		return
	}
	if got != nil && exp == nil {
		t.Errorf("%d: expected nil, but got %v", i, got)
		return
	}
	if len(got) != len(exp) {
		t.Errorf("%d: expected len(%v)==%d, but got len(%v)==%d", i, exp, len(exp), got, len(got))
		return
	}
	for p, n := range exp {
		if got := got[p]; got != id.Zid(n) {
			t.Errorf("%d: pos %d: expected %d, but got %d", i, p, n, got)
		}
	}
}

func TestRefsDiff(t *testing.T) {
	testcases := []struct {
		in1, in2   id.Slice
		exp1, exp2 id.Slice
	}{
		{nil, nil, nil, nil},
		{id.Slice{1}, nil, id.Slice{1}, nil},
		{nil, id.Slice{1}, nil, id.Slice{1}},
		{id.Slice{1}, id.Slice{1}, nil, nil},
		{id.Slice{1, 2}, id.Slice{1}, id.Slice{2}, nil},
		{id.Slice{1, 2}, id.Slice{1, 3}, id.Slice{2}, id.Slice{3}},
		{id.Slice{1, 4}, id.Slice{1, 3}, id.Slice{4}, id.Slice{3}},
	}
	for i, tc := range testcases {
		got1, got2 := refsDiff(tc.in1, tc.in2)
		assertRefs(t, i, got1, tc.exp1)
		assertRefs(t, i, got2, tc.exp2)
	}
}

func TestAddRef(t *testing.T) {
	testcases := []struct {
		ref id.Slice
		zid uint
		exp id.Slice
	}{
		{nil, 5, id.Slice{5}},
		{id.Slice{1}, 5, id.Slice{1, 5}},
		{id.Slice{10}, 5, id.Slice{5, 10}},
		{id.Slice{5}, 5, id.Slice{5}},
		{id.Slice{1, 10}, 5, id.Slice{1, 5, 10}},
		{id.Slice{1, 5, 10}, 5, id.Slice{1, 5, 10}},
	}
	for i, tc := range testcases {
		got := addRef(tc.ref, id.Zid(tc.zid))
		assertRefs(t, i, got, tc.exp)
	}
}

func TestRemRefs(t *testing.T) {
	testcases := []struct {
		in1, in2 id.Slice
		exp      id.Slice
	}{
		{nil, nil, nil},
		{nil, id.Slice{}, nil},
		{id.Slice{}, nil, id.Slice{}},
		{id.Slice{}, id.Slice{}, id.Slice{}},
		{id.Slice{1}, id.Slice{5}, id.Slice{1}},
		{id.Slice{10}, id.Slice{5}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{5}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{5}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{5}, id.Slice{1, 10}},
		{id.Slice{1}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 2, 5}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{2, 5, 10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{2, 5}, id.Slice{1, 10}},
		{id.Slice{1}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 5, 9}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{5, 9, 10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{5, 9}, id.Slice{1, 10}},
	}
	for i, tc := range testcases {
		got := remRefs(tc.in1, tc.in2)
		assertRefs(t, i, got, tc.exp)
	}
}

func TestRemRef(t *testing.T) {
	testcases := []struct {
		ref id.Slice
		zid uint
		exp id.Slice
	}{
		{nil, 5, nil},
		{id.Slice{}, 5, id.Slice{}},
		{id.Slice{5}, 5, id.Slice{}},
		{id.Slice{1}, 5, id.Slice{1}},
		{id.Slice{10}, 5, id.Slice{10}},
		{id.Slice{1, 5}, 5, id.Slice{1}},
		{id.Slice{5, 10}, 5, id.Slice{10}},
		{id.Slice{1, 5, 10}, 5, id.Slice{1, 10}},
	}
	for i, tc := range testcases {
		got := remRef(tc.ref, id.Zid(tc.zid))
		assertRefs(t, i, got, tc.exp)
	}
}

Deleted index/wordset.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53





















































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package index allows to search for metadata and content.
package index

// WordSet contains the set of all words, with the count of their occurrences.
type WordSet map[string]int

// Words gives the slice of all words in the set.
func (ws WordSet) Words() []string {
	if len(ws) == 0 {
		return nil
	}
	words := make([]string, 0, len(ws))
	for w := range ws {
		words = append(words, w)
	}
	return words
}

// Diff calculates the word slice to be added and to be removed from oldWords
// to get the given word set.
func (ws WordSet) Diff(oldWords []string) (newWords, removeWords []string) {
	if len(ws) == 0 {
		return nil, oldWords
	}
	if len(oldWords) == 0 {
		return ws.Words(), nil
	}
	oldSet := make(WordSet, len(oldWords))
	for _, ow := range oldWords {
		if _, ok := ws[ow]; ok {
			oldSet[ow] = 1
			continue
		}
		removeWords = append(removeWords, ow)
	}
	for w := range ws {
		if _, ok := oldSet[w]; ok {
			continue
		}
		newWords = append(newWords, w)
	}
	return newWords, removeWords
}

Deleted index/wordset_test.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76












































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package index allows to search for metadata and content.
package index_test

import (
	"sort"
	"testing"

	"zettelstore.de/z/index"
)

func equalWordList(exp, got []string) bool {
	if len(exp) != len(got) {
		return false
	}
	if len(got) == 0 {
		return len(exp) == 0
	}
	sort.Strings(got)
	for i, w := range exp {
		if w != got[i] {
			return false
		}
	}
	return true
}

func TestWordsWords(t *testing.T) {
	testcases := []struct {
		words index.WordSet
		exp   []string
	}{
		{nil, nil},
		{index.WordSet{}, nil},
		{index.WordSet{"a": 1, "b": 2}, []string{"a", "b"}},
	}
	for i, tc := range testcases {
		got := tc.words.Words()
		if !equalWordList(tc.exp, got) {
			t.Errorf("%d: %v.Words() == %v, but got %v", i, tc.words, tc.exp, got)
		}
	}
}

func TestWordsDiff(t *testing.T) {
	testcases := []struct {
		cur        index.WordSet
		old        []string
		expN, expR []string
	}{
		{nil, nil, nil, nil},
		{index.WordSet{}, []string{}, nil, nil},
		{index.WordSet{"a": 1}, []string{}, []string{"a"}, nil},
		{index.WordSet{"a": 1}, []string{"b"}, []string{"a"}, []string{"b"}},
		{index.WordSet{}, []string{"b"}, nil, []string{"b"}},
		{index.WordSet{"a": 1}, []string{"a"}, nil, nil},
	}
	for i, tc := range testcases {
		gotN, gotR := tc.cur.Diff(tc.old)
		if !equalWordList(tc.expN, gotN) {
			t.Errorf("%d: %v.Diff(%v)->new %v, but got %v", i, tc.cur, tc.old, tc.expN, gotN)
		}
		if !equalWordList(tc.expR, gotR) {
			t.Errorf("%d: %v.Diff(%v)->rem %v, but got %v", i, tc.cur, tc.old, tc.expR, gotR)
		}
	}
}

Deleted index/zettel.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84




















































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package index allows to search for metadata and content.
package index

import (
	"zettelstore.de/z/domain/id"
)

// ZettelIndex contains all index data of a zettel.
type ZettelIndex struct {
	Zid      id.Zid            // zid of the indexed zettel
	backrefs id.Set            // set of back references
	metarefs map[string]id.Set // references to inverse keys
	deadrefs id.Set            // set of dead references
	words    WordSet
}

// NewZettelIndex creates a new zettel index.
func NewZettelIndex(zid id.Zid) *ZettelIndex {
	return &ZettelIndex{
		Zid:      zid,
		backrefs: id.NewSet(),
		metarefs: make(map[string]id.Set),
		deadrefs: id.NewSet(),
	}
}

// AddBackRef adds a reference to a zettel where the current zettel links to
// without any more information.
func (zi *ZettelIndex) AddBackRef(zid id.Zid) {
	zi.backrefs[zid] = true
}

// AddMetaRef adds a named reference to a zettel. On that zettel, the given
// metadata key should point back to the current zettel.
func (zi *ZettelIndex) AddMetaRef(key string, zid id.Zid) {
	if zids, ok := zi.metarefs[key]; ok {
		zids[zid] = true
		return
	}
	zi.metarefs[key] = id.NewSet(zid)
}

// AddDeadRef adds a dead reference to a zettel.
func (zi *ZettelIndex) AddDeadRef(zid id.Zid) {
	zi.deadrefs[zid] = true
}

// SetWords sets the words to the given value.
func (zi *ZettelIndex) SetWords(words WordSet) { zi.words = words }

// GetDeadRefs returns all dead references as a sorted list.
func (zi *ZettelIndex) GetDeadRefs() id.Slice {
	return zi.deadrefs.Sorted()
}

// GetBackRefs returns all back references as a sorted list.
func (zi *ZettelIndex) GetBackRefs() id.Slice {
	return zi.backrefs.Sorted()
}

// GetMetaRefs returns all meta references as a map of strings to a sorted list of references
func (zi *ZettelIndex) GetMetaRefs() map[string]id.Slice {
	if len(zi.metarefs) == 0 {
		return nil
	}
	result := make(map[string]id.Slice, len(zi.metarefs))
	for key, refs := range zi.metarefs {
		result[key] = refs.Sorted()
	}
	return result
}

// GetWords returns a reference to the WordSet. It must not be modified.
func (zi *ZettelIndex) GetWords() WordSet { return zi.words }

Added kernel/impl/auth.go.


























































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"sync"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/kernel"
)

type authService struct {
	srvConfig
	mxService     sync.RWMutex
	manager       auth.Manager
	createManager kernel.CreateAuthManagerFunc
}

func (as *authService) Initialize() {
	as.descr = descriptionMap{
		kernel.AuthOwner: {
			"Owner's zettel id",
			func(val string) interface{} {
				if owner := as.cur[kernel.AuthOwner]; owner != nil && owner != id.Invalid {
					return nil
				}
				return parseZid(val)
			},
			false,
		},
		kernel.AuthReadonly: {
			"Read-only mode",
			func(val string) interface{} {
				if ro := as.cur[kernel.AuthReadonly]; ro == true {
					return nil
				}
				return parseBool(val)
			},
			true,
		},
	}
	as.next = interfaceMap{
		kernel.AuthOwner:    id.Invalid,
		kernel.AuthReadonly: false,
	}
}

func (as *authService) Start(kern *myKernel) error {
	as.mxService.Lock()
	defer as.mxService.Unlock()
	readonlyMode := as.GetNextConfig(kernel.AuthReadonly).(bool)
	owner := as.GetNextConfig(kernel.AuthOwner).(id.Zid)
	authMgr, err := as.createManager(readonlyMode, owner)
	if err != nil {
		kern.doLog("Unable to create auth manager:", err)
		return err
	}
	kern.doLog("Start Auth Manager")
	as.manager = authMgr
	return nil
}

func (as *authService) IsStarted() bool {
	as.mxService.RLock()
	defer as.mxService.RUnlock()
	return as.manager != nil
}

func (as *authService) Stop(kern *myKernel) error {
	kern.doLog("Stop Auth Manager")
	as.mxService.Lock()
	defer as.mxService.Unlock()
	as.manager = nil
	return nil
}

func (as *authService) GetStatistics() []kernel.KeyValue {
	return nil
}

Added kernel/impl/cfg.go.









































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"context"
	"fmt"
	"strconv"
	"strings"
	"sync"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/place"
)

type configService struct {
	srvConfig
	mxService sync.RWMutex
	rtConfig  *myConfig
}

func (cs *configService) Initialize() {
	cs.descr = descriptionMap{
		meta.KeyDefaultCopyright: {"Default copyright", parseString, true},
		meta.KeyDefaultLang:      {"Default language", parseString, true},
		meta.KeyDefaultRole:      {"Default role", parseString, true},
		meta.KeyDefaultSyntax:    {"Default syntax", parseString, true},
		meta.KeyDefaultTitle:     {"Default title", parseString, true},
		meta.KeyDefaultVisibility: {
			"Default zettel visibility",
			func(val string) interface{} {
				vis := meta.GetVisibility(val)
				if vis == meta.VisibilityUnknown {
					return nil
				}
				return vis
			},
			true,
		},
		meta.KeyExpertMode: {"Expert mode", parseBool, true},
		meta.KeyFooterHTML: {"Footer HTML", parseString, true},
		meta.KeyHomeZettel: {"Home zettel", parseZid, true},
		meta.KeyListPageSize: {
			"List page size",
			func(val string) interface{} {
				iVal, err := strconv.Atoi(val)
				if err != nil {
					return nil
				}
				return iVal
			},
			true,
		},
		meta.KeyMarkerExternal: {"Marker external URL", parseString, true},
		meta.KeySiteName:       {"Site name", parseString, true},
		meta.KeyYAMLHeader:     {"YAML header", parseBool, true},
		meta.KeyZettelFileSyntax: {
			"Zettel file syntax",
			func(val string) interface{} { return strings.Fields(val) },
			true,
		},
	}
	cs.next = interfaceMap{
		meta.KeyDefaultCopyright:  "",
		meta.KeyDefaultLang:       meta.ValueLangEN,
		meta.KeyDefaultRole:       meta.ValueRoleZettel,
		meta.KeyDefaultSyntax:     meta.ValueSyntaxZmk,
		meta.KeyDefaultTitle:      "Untitled",
		meta.KeyDefaultVisibility: meta.VisibilityLogin,
		meta.KeyExpertMode:        false,
		meta.KeyFooterHTML:        "",
		meta.KeyHomeZettel:        id.DefaultHomeZid,
		meta.KeyListPageSize:      0,
		meta.KeyMarkerExternal:    "&#10138;",
		meta.KeySiteName:          "Zettelstore",
		meta.KeyYAMLHeader:        false,
		meta.KeyZettelFileSyntax:  nil,
	}
}

func (cs *configService) Start(kern *myKernel) error {
	kern.doLog("Start Config Service")
	data := meta.New(id.ConfigurationZid)
	for _, kv := range cs.GetNextConfigList() {
		data.Set(kv.Key, fmt.Sprintf("%v", kv.Value))
	}
	cs.mxService.Lock()
	cs.rtConfig = newConfig(data)
	cs.mxService.Unlock()
	return nil
}

func (cs *configService) IsStarted() bool {
	cs.mxService.RLock()
	defer cs.mxService.RUnlock()
	return cs.rtConfig != nil
}

func (cs *configService) Stop(kern *myKernel) error {
	kern.doLog("Stop Config Service")
	cs.mxService.Lock()
	cs.rtConfig = nil
	cs.mxService.Unlock()
	return nil
}

func (cs *configService) GetStatistics() []kernel.KeyValue {
	return nil
}

func (cs *configService) setPlace(mgr place.Manager) {
	cs.rtConfig.setPlace(mgr)
}

// myConfig contains all runtime configuration data relevant for the software.
type myConfig struct {
	mx   sync.RWMutex
	orig *meta.Meta
	data *meta.Meta
}

// New creates a new Config value.
func newConfig(orig *meta.Meta) *myConfig {
	cfg := myConfig{
		orig: orig,
		data: orig.Clone(),
	}
	return &cfg
}
func (cfg *myConfig) setPlace(mgr place.Manager) {
	mgr.RegisterObserver(cfg.observe)
	cfg.doUpdate(mgr)
}

func (cfg *myConfig) doUpdate(p place.Place) error {
	m, err := p.GetMeta(context.Background(), cfg.data.Zid)
	if err != nil {
		return err
	}
	cfg.mx.Lock()
	for _, pair := range cfg.data.Pairs(false) {
		if val, ok := m.Get(pair.Key); ok {
			cfg.data.Set(pair.Key, val)
		}
	}
	cfg.mx.Unlock()
	return nil
}

func (cfg *myConfig) observe(ci place.UpdateInfo) {
	if ci.Reason == place.OnReload || ci.Zid == id.ConfigurationZid {
		go func() { cfg.doUpdate(ci.Place) }()
	}
}

var defaultKeys = map[string]string{
	meta.KeyCopyright: meta.KeyDefaultCopyright,
	meta.KeyLang:      meta.KeyDefaultLang,
	meta.KeyLicense:   meta.KeyDefaultLicense,
	meta.KeyRole:      meta.KeyDefaultRole,
	meta.KeySyntax:    meta.KeyDefaultSyntax,
	meta.KeyTitle:     meta.KeyDefaultTitle,
}

// AddDefaultValues enriches the given meta data with its default values.
func (cfg *myConfig) AddDefaultValues(m *meta.Meta) *meta.Meta {
	if cfg == nil {
		return m
	}
	result := m
	cfg.mx.RLock()
	for k, d := range defaultKeys {
		if _, ok := result.Get(k); !ok {
			if result == m {
				result = m.Clone()
			}
			if val, ok := cfg.data.Get(d); ok {
				result.Set(k, val)
			}
		}
	}
	cfg.mx.RUnlock()
	return result
}

func (cfg *myConfig) getString(key string) string {
	cfg.mx.RLock()
	val, _ := cfg.data.Get(key)
	cfg.mx.RUnlock()
	return val
}
func (cfg *myConfig) getBool(key string) bool {
	cfg.mx.RLock()
	val := cfg.data.GetBool(key)
	cfg.mx.RUnlock()
	return val
}

// GetDefaultTitle returns the current value of the "default-title" key.
func (cfg *myConfig) GetDefaultTitle() string { return cfg.getString(meta.KeyDefaultTitle) }

// GetDefaultRole returns the current value of the "default-role" key.
func (cfg *myConfig) GetDefaultRole() string { return cfg.getString(meta.KeyDefaultRole) }

// GetDefaultSyntax returns the current value of the "default-syntax" key.
func (cfg *myConfig) GetDefaultSyntax() string { return cfg.getString(meta.KeyDefaultSyntax) }

// GetDefaultLang returns the current value of the "default-lang" key.
func (cfg *myConfig) GetDefaultLang() string { return cfg.getString(meta.KeyDefaultLang) }

// GetSiteName returns the current value of the "site-name" key.
func (cfg *myConfig) GetSiteName() string { return cfg.getString(meta.KeySiteName) }

// GetHomeZettel returns the value of the "home-zettel" key.
func (cfg *myConfig) GetHomeZettel() id.Zid {
	val := cfg.getString(meta.KeyHomeZettel)
	if homeZid, err := id.Parse(val); err == nil {
		return homeZid
	}
	cfg.mx.RLock()
	val, _ = cfg.orig.Get(meta.KeyHomeZettel)
	homeZid, _ := id.Parse(val)
	cfg.mx.RUnlock()
	return homeZid
}

// GetDefaultVisibility returns the default value for zettel visibility.
func (cfg *myConfig) GetDefaultVisibility() meta.Visibility {
	val := cfg.getString(meta.KeyDefaultVisibility)
	if vis := meta.GetVisibility(val); vis != meta.VisibilityUnknown {
		return vis
	}
	cfg.mx.RLock()
	val, _ = cfg.orig.Get(meta.KeyDefaultVisibility)
	vis := meta.GetVisibility(val)
	cfg.mx.RUnlock()
	return vis
}

// GetYAMLHeader returns the current value of the "yaml-header" key.
func (cfg *myConfig) GetYAMLHeader() bool { return cfg.getBool(meta.KeyYAMLHeader) }

// GetMarkerExternal returns the current value of the "marker-external" key.
func (cfg *myConfig) GetMarkerExternal() string {
	return cfg.getString(meta.KeyMarkerExternal)
}

// GetFooterHTML returns HTML code that should be embedded into the footer
// of each WebUI page.
func (cfg *myConfig) GetFooterHTML() string { return cfg.getString(meta.KeyFooterHTML) }

// GetListPageSize returns the maximum length of a list to be returned in WebUI.
// A value less or equal to zero signals no limit.
func (cfg *myConfig) GetListPageSize() int {
	cfg.mx.RLock()
	defer cfg.mx.RUnlock()

	if value, ok := cfg.data.GetNumber(meta.KeyListPageSize); ok {
		return value
	}
	value, _ := cfg.orig.GetNumber(meta.KeyListPageSize)
	return value
}

// GetZettelFileSyntax returns the current value of the "zettel-file-syntax" key.
func (cfg *myConfig) GetZettelFileSyntax() []string {
	cfg.mx.RLock()
	defer cfg.mx.RUnlock()
	return cfg.data.GetListOrNil(meta.KeyZettelFileSyntax)
}

// --- AuthConfig

// GetExpertMode returns the current value of the "expert-mode" key
func (cfg *myConfig) GetExpertMode() bool { return cfg.getBool(meta.KeyExpertMode) }

// GetVisibility returns the visibility value, or "login" if none is given.
func (cfg *myConfig) GetVisibility(m *meta.Meta) meta.Visibility {
	if val, ok := m.Get(meta.KeyVisibility); ok {
		if vis := meta.GetVisibility(val); vis != meta.VisibilityUnknown {
			return vis
		}
	}
	return cfg.GetDefaultVisibility()
}

Added kernel/impl/cmd.go.




























































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"fmt"
	"io"
	"os"
	"runtime/metrics"
	"sort"
	"strings"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/strfun"
)

type cmdSession struct {
	w        io.Writer
	kern     *myKernel
	echo     bool
	header   bool
	colwidth int
	eol      []byte
}

func (sess *cmdSession) initialize(w io.Writer, kern *myKernel) {
	sess.w = w
	sess.kern = kern
	sess.header = true
	sess.colwidth = 80
	sess.eol = []byte{'\n'}
}

func (sess *cmdSession) executeLine(line string) bool {
	if sess.echo {
		sess.println(line)
	}
	cmd, args := splitLine(line)
	if c, ok := commands[cmd]; ok {
		return c.Func(sess, cmd, args)
	}
	if cmd == "help" {
		return cmdHelp(sess, cmd, args)
	}
	sess.println("Unknown command:", cmd, strings.Join(args, " "))
	sess.println("-- Enter 'help' go get a list of valid commands.")
	return true
}

func (sess *cmdSession) println(args ...string) {
	if len(args) > 0 {
		io.WriteString(sess.w, args[0])
		for _, arg := range args[1:] {
			io.WriteString(sess.w, " ")
			io.WriteString(sess.w, arg)
		}
	}
	sess.w.Write(sess.eol)
}

func (sess *cmdSession) printTable(table [][]string) {
	maxLen := sess.calcMaxLen(table)
	if len(maxLen) == 0 {
		return
	}
	if sess.header {
		sess.printRow(table[0], maxLen, "|=", " | ", ' ')
		hLine := make([]string, len(table[0]))
		sess.printRow(hLine, maxLen, "|%", "-+-", '-')
	}

	for _, row := range table[1:] {
		sess.printRow(row, maxLen, "| ", " | ", ' ')
	}
}

func (sess *cmdSession) calcMaxLen(table [][]string) []int {
	maxLen := make([]int, 0)
	for _, row := range table {
		for colno, column := range row {
			if colno >= len(maxLen) {
				maxLen = append(maxLen, 0)
			}
			colLen := strfun.Length(column)
			if colLen <= maxLen[colno] {
				continue
			}
			if colLen < sess.colwidth {
				maxLen[colno] = colLen
			} else {
				maxLen[colno] = sess.colwidth
			}
		}
	}
	return maxLen
}

func (sess *cmdSession) printRow(row []string, maxLen []int, prefix, delim string, pad rune) {
	for colno, column := range row {
		io.WriteString(sess.w, prefix)
		prefix = delim
		io.WriteString(sess.w, strfun.JustifyLeft(column, maxLen[colno], pad))
	}
	sess.w.Write(sess.eol)
}

func splitLine(line string) (string, []string) {
	s := strings.Fields(line)
	if len(s) == 0 {
		return "", nil
	}
	return strings.ToLower(s[0]), s[1:]
}

type command struct {
	Text string
	Func func(sess *cmdSession, cmd string, args []string) bool
}

var commands = map[string]command{
	"": {"", func(*cmdSession, string, []string) bool { return true }},
	"bye": {
		"end this session",
		func(*cmdSession, string, []string) bool { return false },
	},
	"config": {"show configuration keys", cmdConfig},
	"crlf": {
		"toggle crlf mode",
		func(sess *cmdSession, cmd string, args []string) bool {
			if len(sess.eol) == 1 {
				sess.eol = []byte{'\r', '\n'}
				sess.println("crlf is on")
			} else {
				sess.eol = []byte{'\n'}
				sess.println("crlf is off")
			}
			return true
		},
	},
	"dump-index":   {"writes the content of the index", cmdDumpIndex},
	"dump-recover": {"show data of last recovery", cmdDumpRecover},
	"echo": {
		"toggle echo mode",
		func(sess *cmdSession, cmd string, args []string) bool {
			sess.echo = !sess.echo
			if sess.echo {
				sess.println("echo is on")
			} else {
				sess.println("echo is off")
			}
			return true
		},
	},
	"env":        {"show environment values", cmdEnvironment},
	"get-config": {"show current configuration data", cmdGetConfig},
	"header": {
		"toggle table header",
		func(sess *cmdSession, cmd string, args []string) bool {
			sess.header = !sess.header
			if sess.header {
				sess.println("header are on")
			} else {
				sess.println("header are off")
			}
			return true
		},
	},
	"metrics":     {"show Go runtime metrics", cmdMetrics},
	"next-config": {"show next configuration data", cmdNextConfig},
	"restart":     {"restart service", cmdRestart},
	"services":    {"show available services", cmdServices},
	"set-config":  {"set next configuration data", cmdSetConfig},
	"shutdown": {
		"shutdown Zettelstore",
		func(sess *cmdSession, cmd string, args []string) bool { sess.kern.Shutdown(false); return false },
	},
	"start": {"start service", cmdStart},
	"stat":  {"show service statistics", cmdStat},
	"stop":  {"stop service", cmdStop},
}

func cmdHelp(sess *cmdSession, cmd string, args []string) bool {
	cmds := make([]string, 0, len(commands))
	for key := range commands {
		if key == "" {
			continue
		}
		cmds = append(cmds, key)
	}
	sort.Strings(cmds)
	table := [][]string{{"Command", "Description"}}
	for _, cmd := range cmds {
		table = append(table, []string{cmd, commands[cmd].Text})
	}
	sess.printTable(table)
	return true
}

func cmdConfig(sess *cmdSession, cmd string, args []string) bool {
	srvnum, ok := lookupService(sess, cmd, args)
	if !ok {
		return true
	}
	srv := sess.kern.srvs[srvnum].srv
	table := [][]string{{"Key", "Description"}}
	for _, kd := range srv.ConfigDescriptions() {
		table = append(table, []string{kd.Key, kd.Descr})
	}
	sess.printTable(table)
	return true
}
func cmdGetConfig(sess *cmdSession, cmd string, args []string) bool {
	showConfig(sess, args,
		listCurConfig, func(srv service, key string) interface{} { return srv.GetConfig(key) })
	return true
}
func cmdNextConfig(sess *cmdSession, cmd string, args []string) bool {
	showConfig(sess, args,
		listNextConfig, func(srv service, key string) interface{} { return srv.GetNextConfig(key) })
	return true
}
func showConfig(sess *cmdSession, args []string,
	listConfig func(*cmdSession, service), getConfig func(service, string) interface{}) {

	if len(args) == 0 {
		keys := make([]int, 0, len(sess.kern.srvs))
		for k := range sess.kern.srvs {
			keys = append(keys, int(k))
		}
		sort.Ints(keys)
		for i, k := range keys {
			if i > 0 {
				sess.println()
			}
			srvD := sess.kern.srvs[kernel.Service(k)]
			sess.println("%% Service", srvD.name)
			listConfig(sess, srvD.srv)

		}
		return
	}
	srvD, ok := sess.kern.srvNames[args[0]]
	if !ok {
		sess.println("Unknown service:", args[0])
		return
	}
	if len(args) == 1 {
		listConfig(sess, srvD.srv)
		return
	}
	val := getConfig(srvD.srv, args[1])
	if val == nil {
		sess.println("Unknown key", args[1], "for service", args[0])
		return
	}
	sess.println(fmt.Sprintf("%v", val))
}
func listCurConfig(sess *cmdSession, srv service) {
	listConfig(sess, func() []kernel.KeyDescrValue { return srv.GetConfigList(true) })
}
func listNextConfig(sess *cmdSession, srv service) {
	listConfig(sess, srv.GetNextConfigList)
}
func listConfig(sess *cmdSession, getConfigList func() []kernel.KeyDescrValue) {
	l := getConfigList()
	table := [][]string{{"Key", "Value", "Description"}}
	for _, kdv := range l {
		table = append(table, []string{kdv.Key, kdv.Value, kdv.Descr})
	}
	sess.printTable(table)
}

func cmdSetConfig(sess *cmdSession, cmd string, args []string) bool {
	if len(args) < 3 {
		sess.println("Usage:", cmd, "SERIVCE KEY VALUE")
		return true
	}
	srvD, ok := sess.kern.srvNames[args[0]]
	if !ok {
		sess.println("Unknown service:", args[0])
		return true
	}
	newValue := strings.Join(args[2:], " ")
	if !srvD.srv.SetConfig(args[1], newValue) {
		sess.println("Unable to set key", args[1], "to value", newValue)
	}
	return true
}

func cmdServices(sess *cmdSession, cmd string, args []string) bool {
	names := make([]string, 0, len(sess.kern.srvNames))
	for name := range sess.kern.srvNames {
		names = append(names, name)
	}
	sort.Strings(names)

	table := [][]string{{"Service", "Status"}}
	for _, name := range names {
		if sess.kern.srvNames[name].srv.IsStarted() {
			table = append(table, []string{name, "started"})
		} else {
			table = append(table, []string{name, "stopped"})
		}
	}
	sess.printTable(table)
	return true
}

func cmdStart(sess *cmdSession, cmd string, args []string) bool {
	srvnum, ok := lookupService(sess, cmd, args)
	if !ok {
		return true
	}
	err := sess.kern.doStartService(srvnum)
	if err != nil {
		sess.println(err.Error())
	}
	return true
}

func cmdRestart(sess *cmdSession, cmd string, args []string) bool {
	srvnum, ok := lookupService(sess, cmd, args)
	if !ok {
		return true
	}
	err := sess.kern.doRestartService(srvnum)
	if err != nil {
		sess.println(err.Error())
	}
	return true
}

func cmdStop(sess *cmdSession, cmd string, args []string) bool {
	srvnum, ok := lookupService(sess, cmd, args)
	if !ok {
		return true
	}
	err := sess.kern.doStopService(srvnum)
	if err != nil {
		sess.println(err.Error())
	}
	return true
}

func cmdStat(sess *cmdSession, cmd string, args []string) bool {
	if len(args) == 0 {
		sess.println("Usage:", cmd, "SERVICE")
		return true
	}
	srvD, ok := sess.kern.srvNames[args[0]]
	if !ok {
		sess.println("Unknown service", args[0])
		return true
	}
	kvl := srvD.srv.GetStatistics()
	if len(kvl) == 0 {
		return true
	}
	table := [][]string{{"Key", "Value"}}
	for _, kv := range kvl {
		table = append(table, []string{kv.Key, kv.Value})
	}
	sess.printTable(table)
	return true
}

func lookupService(sess *cmdSession, cmd string, args []string) (kernel.Service, bool) {
	if len(args) == 0 {
		sess.println("Usage:", cmd, "SERVICE")
		return 0, false
	}
	srvD, ok := sess.kern.srvNames[args[0]]
	if !ok {
		sess.println("Unknown service", args[0])
		return 0, false
	}
	return srvD.srvnum, true
}

func cmdMetrics(sess *cmdSession, cmd string, args []string) bool {
	var samples []metrics.Sample
	all := metrics.All()
	for _, d := range all {
		if d.Kind == metrics.KindFloat64Histogram {
			continue
		}
		samples = append(samples, metrics.Sample{Name: d.Name})
	}
	metrics.Read(samples)

	table := [][]string{{"Value", "Description"}}
	i := 0
	for _, d := range all {
		if d.Kind == metrics.KindFloat64Histogram {
			continue
		}
		descr := d.Description
		if pos := strings.IndexByte(descr, '.'); pos > 0 {
			descr = descr[:pos]
		}
		value := samples[i].Value
		i++
		var sVal string
		switch value.Kind() {
		case metrics.KindUint64:
			sVal = fmt.Sprintf("%v", value.Uint64())
		case metrics.KindFloat64:
			sVal = fmt.Sprintf("%v", value.Float64())
		case metrics.KindFloat64Histogram:
			sVal = "(Histogramm)"
		case metrics.KindBad:
			sVal = "BAD"
		default:
			sVal = fmt.Sprintf("(unexpected metric kind: %v)", value.Kind())
		}
		table = append(table, []string{sVal, descr})
	}
	sess.printTable(table)
	return true
}

func cmdDumpIndex(sess *cmdSession, cmd string, args []string) bool {
	sess.kern.DumpIndex(sess.w)
	return true
}
func cmdDumpRecover(sess *cmdSession, cmd string, args []string) bool {
	if len(args) == 0 {
		sess.println("Usage:", cmd, "RECOVER")
		sess.println("-- A valid value for RECOVER can be obtained via 'stat core'.")
		return true
	}
	lines := sess.kern.core.RecoverLines(args[0])
	if len(lines) == 0 {
		return true
	}
	for _, line := range lines {
		sess.println(line)
	}
	return true
}

func cmdEnvironment(sess *cmdSession, cmd string, args []string) bool {
	workDir, err := os.Getwd()
	if err != nil {
		workDir = err.Error()
	}
	execName, err := os.Executable()
	if err != nil {
		execName = err.Error()
	}
	envs := os.Environ()
	sort.Strings(envs)

	table := [][]string{
		{"Key", "Value"},
		{"workdir", workDir},
		{"executable", execName},
	}
	for _, env := range envs {
		if pos := strings.IndexByte(env, '='); pos >= 0 && pos < len(env) {
			table = append(table, []string{env[:pos], env[pos+1:]})
		}
	}
	sess.printTable(table)
	return true
}

Added kernel/impl/config.go.






































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"fmt"
	"sort"
	"strconv"
	"strings"
	"sync"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/kernel"
)

type parseFunc func(string) interface{}
type configDescription struct {
	text    string
	parse   parseFunc
	canList bool
}
type descriptionMap map[string]configDescription
type interfaceMap map[string]interface{}

func (m interfaceMap) Clone() interfaceMap {
	if m == nil {
		return nil
	}
	result := make(interfaceMap, len(m))
	for k, v := range m {
		result[k] = v
	}
	return result
}

type srvConfig struct {
	mxConfig sync.RWMutex
	frozen   bool
	descr    descriptionMap
	cur      interfaceMap
	next     interfaceMap
}

func (cfg *srvConfig) ConfigDescriptions() []serviceConfigDescription {
	cfg.mxConfig.RLock()
	defer cfg.mxConfig.RUnlock()
	keys := make([]string, 0, len(cfg.descr))
	for k := range cfg.descr {
		keys = append(keys, k)
	}
	sort.Strings(keys)
	result := make([]serviceConfigDescription, 0, len(keys))
	for _, k := range keys {
		text := cfg.descr[k].text
		if strings.HasSuffix(k, "-") {
			text = text + " (list)"
		}
		result = append(result, serviceConfigDescription{Key: k, Descr: text})
	}
	return result
}

func (cfg *srvConfig) noFrozen(parse parseFunc) parseFunc {
	return func(val string) interface{} {
		if cfg.frozen {
			return nil
		}
		return parse(val)
	}
}

func (cfg *srvConfig) SetConfig(key, value string) bool {
	cfg.mxConfig.Lock()
	defer cfg.mxConfig.Unlock()
	descr, ok := cfg.descr[key]
	if !ok {
		d, baseKey, num := cfg.getListDescription(key)
		if num < 0 {
			return false
		}
		format := baseKey + "%d"
		for i := num + 1; ; i++ {
			k := fmt.Sprintf(format, i)
			if _, ok = cfg.next[k]; !ok {
				break
			}
			delete(cfg.next, k)
		}
		if num == 0 {
			return true
		}
		descr = d
	}
	parse := descr.parse
	if parse == nil {
		if cfg.frozen {
			return false
		}
		cfg.next[key] = value
		return true
	}
	iVal := parse(value)
	if iVal == nil {
		return false
	}
	cfg.next[key] = iVal
	return true
}

func (cfg *srvConfig) getListDescription(key string) (configDescription, string, int) {
	for k, d := range cfg.descr {
		if !strings.HasSuffix(k, "-") {
			continue
		}
		if !strings.HasPrefix(key, k) {
			continue
		}
		num, err := strconv.Atoi(key[len(k):])
		if err != nil || num < 0 {
			continue
		}
		return d, k, num
	}
	return configDescription{}, "", -1
}

func (cfg *srvConfig) GetConfig(key string) interface{} {
	cfg.mxConfig.RLock()
	defer cfg.mxConfig.RUnlock()
	if cfg.cur == nil {
		return cfg.next[key]
	}
	return cfg.cur[key]
}

func (cfg *srvConfig) GetNextConfig(key string) interface{} {
	cfg.mxConfig.RLock()
	defer cfg.mxConfig.RUnlock()
	return cfg.next[key]
}

func (cfg *srvConfig) GetConfigList(all bool) []kernel.KeyDescrValue {
	return cfg.getConfigList(all, cfg.GetConfig)
}
func (cfg *srvConfig) GetNextConfigList() []kernel.KeyDescrValue {
	return cfg.getConfigList(true, cfg.GetNextConfig)
}
func (cfg *srvConfig) getConfigList(all bool, getConfig func(string) interface{}) []kernel.KeyDescrValue {
	if len(cfg.descr) == 0 {
		return nil
	}
	keys := make([]string, 0, len(cfg.descr))
	for k, descr := range cfg.descr {
		if all || descr.canList {
			if !strings.HasSuffix(k, "-") {
				keys = append(keys, k)
				continue
			}
			format := k + "%d"
			for i := 1; ; i++ {
				key := fmt.Sprintf(format, i)
				val := getConfig(key)
				if val == nil {
					break
				}
				keys = append(keys, key)
			}
		}
	}
	sort.Strings(keys)
	result := make([]kernel.KeyDescrValue, 0, len(keys))
	for _, k := range keys {
		val := getConfig(k)
		if val == nil {
			continue
		}
		descr, ok := cfg.descr[k]
		if !ok {
			descr, _, _ = cfg.getListDescription(k)
		}
		result = append(result, kernel.KeyDescrValue{
			Key:   k,
			Descr: descr.text,
			Value: fmt.Sprintf("%v", val),
		})
	}
	return result
}

func (cfg *srvConfig) Freeze() {
	cfg.mxConfig.Lock()
	cfg.frozen = true
	cfg.mxConfig.Unlock()
}

func (cfg *srvConfig) SwitchNextToCur() {
	cfg.mxConfig.Lock()
	defer cfg.mxConfig.Unlock()
	cfg.cur = cfg.next.Clone()
}

func parseString(val string) interface{} { return val }

func parseBool(val string) interface{} {
	if val == "" {
		return false
	}
	switch val[0] {
	case '0', 'f', 'F', 'n', 'N':
		return false
	}
	return true
}

func parseZid(val string) interface{} {
	if zid, err := id.Parse(val); err == nil {
		return zid
	}
	return id.Invalid
}

Added kernel/impl/core.go.
























































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"fmt"
	"net"
	"os"
	"runtime"
	"sort"
	"sync"
	"time"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/strfun"
)

type coreService struct {
	srvConfig
	started bool

	mxRecover  sync.RWMutex
	mapRecover map[string]recoverInfo
}
type recoverInfo struct {
	count uint64
	ts    time.Time
	info  interface{}
	stack []byte
}

func (cs *coreService) Initialize() {
	cs.mapRecover = make(map[string]recoverInfo)
	cs.descr = descriptionMap{
		kernel.CoreGoArch:    {"Go processor architecture", nil, false},
		kernel.CoreGoOS:      {"Go Operating System", nil, false},
		kernel.CoreGoVersion: {"Go Version", nil, false},
		kernel.CoreHostname:  {"Host name", nil, false},
		kernel.CorePort: {
			"Port of command line server",
			cs.noFrozen(func(val string) interface{} {
				port, err := net.LookupPort("tcp", val)
				if err != nil {
					return nil
				}
				return port
			}),
			true,
		},
		kernel.CoreProgname: {"Program name", nil, false},
		kernel.CoreVerbose:  {"Verbose output", parseBool, true},
		kernel.CoreVersion: {
			"Version",
			cs.noFrozen(func(val string) interface{} {
				if val == "" {
					return "unknown"
				}
				return val
			}),
			false,
		},
	}
	cs.next = interfaceMap{
		kernel.CoreGoArch:    runtime.GOARCH,
		kernel.CoreGoOS:      runtime.GOOS,
		kernel.CoreGoVersion: runtime.Version(),
		kernel.CoreHostname:  "*unknown host*",
		kernel.CorePort:      0,
		kernel.CoreVerbose:   false,
	}
	if hn, err := os.Hostname(); err == nil {
		cs.next[kernel.CoreHostname] = hn
	}
}

func (cs *coreService) Start(kern *myKernel) error {
	cs.started = true
	return nil
}
func (cs *coreService) IsStarted() bool { return cs.started }
func (cs *coreService) Stop(*myKernel) error {
	cs.started = false
	return nil
}

func (cs *coreService) GetStatistics() []kernel.KeyValue {
	cs.mxRecover.RLock()
	defer cs.mxRecover.RUnlock()
	names := make([]string, 0, len(cs.mapRecover))
	for n := range cs.mapRecover {
		names = append(names, n)
	}
	sort.Strings(names)
	result := make([]kernel.KeyValue, 0, 3*len(names))
	for _, n := range names {
		ri := cs.mapRecover[n]
		result = append(
			result,
			kernel.KeyValue{
				Key:   fmt.Sprintf("Recover %q / Count", n),
				Value: fmt.Sprintf("%d", ri.count),
			},
			kernel.KeyValue{
				Key:   fmt.Sprintf("Recover %q / Last ", n),
				Value: fmt.Sprintf("%v", ri.ts),
			},
			kernel.KeyValue{
				Key:   fmt.Sprintf("Recover %q / Info ", n),
				Value: fmt.Sprintf("%v", ri.info),
			},
		)
	}
	return result
}

func (cs *coreService) RecoverLines(name string) []string {
	cs.mxRecover.RLock()
	ri := cs.mapRecover[name]
	cs.mxRecover.RUnlock()
	if ri.stack == nil {
		return nil
	}
	return append(
		[]string{
			fmt.Sprintf("Count: %d", ri.count),
			fmt.Sprintf("Time: %v", ri.ts),
			fmt.Sprintf("Reason: %v", ri.info),
		},
		strfun.SplitLines(string(ri.stack))...,
	)
}

func (cs *coreService) updateRecoverInfo(name string, recoverInfo interface{}, stack []byte) {
	cs.mxRecover.Lock()
	ri := cs.mapRecover[name]
	ri.count++
	ri.ts = time.Now()
	ri.info = recoverInfo
	ri.stack = stack
	cs.mapRecover[name] = ri
	cs.mxRecover.Unlock()
}

Added kernel/impl/impl.go.













































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"fmt"
	"io"
	"log"
	"net"
	"os"
	"os/signal"
	"runtime/debug"
	"strconv"
	"sync"
	"syscall"

	"zettelstore.de/z/kernel"
)

// myKernel is the main internal kernel.
type myKernel struct {
	// started   bool
	wg        sync.WaitGroup
	mx        sync.RWMutex
	interrupt chan os.Signal
	debug     bool

	core  coreService
	cfg   configService
	auth  authService
	place placeService
	web   webService

	srvs     map[kernel.Service]serviceDescr
	srvNames map[string]serviceData
	depStart serviceDependency
	depStop  serviceDependency // reverse of depStart
}

type serviceDescr struct {
	srv  service
	name string
}
type serviceData struct {
	srv    service
	srvnum kernel.Service
}
type serviceDependency map[kernel.Service][]kernel.Service

// create and start a new kernel.
func init() {
	kernel.Main = createAndStart()
}

// create and start a new kernel.
func createAndStart() kernel.Kernel {
	kern := &myKernel{
		interrupt: make(chan os.Signal, 5),
	}
	kern.srvs = map[kernel.Service]serviceDescr{
		kernel.CoreService:   {&kern.core, "core"},
		kernel.ConfigService: {&kern.cfg, "config"},
		kernel.AuthService:   {&kern.auth, "auth"},
		kernel.PlaceService:  {&kern.place, "place"},
		kernel.WebService:    {&kern.web, "web"},
	}
	kern.srvNames = make(map[string]serviceData, len(kern.srvs))
	for key, srvD := range kern.srvs {
		if _, ok := kern.srvNames[srvD.name]; ok {
			panic(fmt.Sprintf("Key %q already given for service %v", key, srvD.name))
		}
		kern.srvNames[srvD.name] = serviceData{srvD.srv, key}
		srvD.srv.Initialize()
	}
	kern.depStart = serviceDependency{
		kernel.CoreService:   nil,
		kernel.ConfigService: {kernel.CoreService},
		kernel.AuthService:   {kernel.CoreService},
		kernel.PlaceService:  {kernel.CoreService, kernel.ConfigService, kernel.AuthService},
		kernel.WebService:    {kernel.ConfigService, kernel.AuthService, kernel.PlaceService},
	}
	kern.depStop = make(serviceDependency, len(kern.depStart))
	for srv, deps := range kern.depStart {
		for _, dep := range deps {
			kern.depStop[dep] = append(kern.depStop[dep], srv)
		}
	}
	return kern
}

func (kern *myKernel) Start(headline bool) {
	for _, srvD := range kern.srvs {
		srvD.srv.Freeze()
	}
	kern.wg.Add(1)
	signal.Notify(kern.interrupt, os.Interrupt, syscall.SIGTERM)
	go func() {
		// Wait for interrupt.
		sig := <-kern.interrupt
		if strSig := sig.String(); strSig != "" {
			kern.doLog("Shut down Zettelstore:", strSig)
		}
		kern.shutdown()
		kern.wg.Done()
	}()

	kern.StartService(kernel.CoreService)
	if headline {
		kern.doLog(fmt.Sprintf(
			"%v %v (%v@%v/%v)",
			kern.core.GetConfig(kernel.CoreProgname),
			kern.core.GetConfig(kernel.CoreVersion),
			kern.core.GetConfig(kernel.CoreGoVersion),
			kern.core.GetConfig(kernel.CoreGoOS),
			kern.core.GetConfig(kernel.CoreGoArch),
		))
		kern.doLog("Licensed under the latest version of the EUPL (European Union Public License)")
		if kern.auth.GetConfig(kernel.AuthReadonly).(bool) {
			kern.doLog("Read-only mode")
		}
	}
	port := kern.core.GetNextConfig(kernel.CorePort).(int)
	if port > 0 {
		listenAddr := net.JoinHostPort("127.0.0.1", strconv.Itoa(port))
		startLineServer(kern, listenAddr)
	}
}

func (kern *myKernel) shutdown() {
	kern.StopService(kernel.CoreService) // Will stop all other services.
}

func (kern *myKernel) WaitForShutdown() {
	kern.wg.Wait()
}

func (kern *myKernel) SetDebug(enable bool) bool {
	kern.mx.Lock()
	prevDebug := kern.debug
	kern.debug = enable
	kern.mx.Unlock()
	return prevDebug
}

// --- Shutdown operation ----------------------------------------------------

// Shutdown the service. Waits for all concurrent activity to stop.
func (kern *myKernel) Shutdown(silent bool) {
	kern.interrupt <- &shutdownSignal{silent: silent}
}

type shutdownSignal struct{ silent bool }

func (s *shutdownSignal) String() string {
	if s.silent {
		return ""
	}
	return "shutdown"
}
func (s *shutdownSignal) Signal() { /* Just a signal */ }

// --- Log operation ---------------------------------------------------------

// Log some activity.
func (kern *myKernel) Log(args ...interface{}) {
	kern.mx.Lock()
	defer kern.mx.Unlock()
	kern.doLog(args...)
}
func (kern *myKernel) doLog(args ...interface{}) {
	log.Println(args...)
}

// LogRecover outputs some information about the previous panic.
func (kern *myKernel) LogRecover(name string, recoverInfo interface{}) bool {
	return kern.doLogRecover(name, recoverInfo)
}
func (kern *myKernel) doLogRecover(name string, recoverInfo interface{}) bool {
	kern.Log(name, "recovered from:", recoverInfo)
	stack := debug.Stack()
	os.Stderr.Write(stack)
	kern.core.updateRecoverInfo(name, recoverInfo, stack)
	return true
}

// --- Service handling --------------------------------------------------

func (kern *myKernel) SetConfig(srvnum kernel.Service, key, value string) bool {
	kern.mx.Lock()
	defer kern.mx.Unlock()
	if srvD, ok := kern.srvs[srvnum]; ok {
		return srvD.srv.SetConfig(key, value)
	}
	return false
}

func (kern *myKernel) GetConfig(srvnum kernel.Service, key string) interface{} {
	kern.mx.RLock()
	defer kern.mx.RUnlock()
	if srvD, ok := kern.srvs[srvnum]; ok {
		return srvD.srv.GetConfig(key)
	}
	return nil
}

func (kern *myKernel) GetConfigList(srvnum kernel.Service) []kernel.KeyDescrValue {
	kern.mx.RLock()
	defer kern.mx.RUnlock()
	if srvD, ok := kern.srvs[srvnum]; ok {
		return srvD.srv.GetConfigList(false)
	}
	return nil
}
func (kern *myKernel) GetServiceStatistics(srvnum kernel.Service) []kernel.KeyValue {
	kern.mx.RLock()
	defer kern.mx.RUnlock()
	if srvD, ok := kern.srvs[srvnum]; ok {
		return srvD.srv.GetStatistics()
	}
	return nil
}

func (kern *myKernel) StartService(srvnum kernel.Service) error {
	kern.mx.RLock()
	defer kern.mx.RUnlock()
	return kern.doStartService(srvnum)
}
func (kern *myKernel) doStartService(srvnum kernel.Service) error {
	for _, srv := range kern.sortDependency(srvnum, kern.depStart, true) {
		if err := srv.Start(kern); err != nil {
			return err
		}
		srv.SwitchNextToCur()
	}
	return nil
}

func (kern *myKernel) RestartService(srvnum kernel.Service) error {
	kern.mx.RLock()
	defer kern.mx.RUnlock()
	return kern.doRestartService(srvnum)
}
func (kern *myKernel) doRestartService(srvnum kernel.Service) error {
	deps := kern.sortDependency(srvnum, kern.depStop, false)
	for _, srv := range deps {
		if err := srv.Stop(kern); err != nil {
			return err
		}
	}
	for i := len(deps) - 1; i >= 0; i-- {
		srv := deps[i]
		if err := srv.Start(kern); err != nil {
			return err
		}
		srv.SwitchNextToCur()
	}
	return nil
}

func (kern *myKernel) StopService(srvnum kernel.Service) error {
	kern.mx.RLock()
	defer kern.mx.RUnlock()
	return kern.doStopService(srvnum)
}
func (kern *myKernel) doStopService(srvnum kernel.Service) error {
	for _, srv := range kern.sortDependency(srvnum, kern.depStop, false) {
		if err := srv.Stop(kern); err != nil {
			return err
		}
	}
	return nil
}

func (kern *myKernel) sortDependency(
	srvnum kernel.Service,
	srvdeps serviceDependency,
	isStarted bool,
) []service {
	srvD, ok := kern.srvs[srvnum]
	if !ok {
		return nil
	}
	if srvD.srv.IsStarted() == isStarted {
		return nil
	}
	deps := srvdeps[srvnum]
	found := make(map[service]bool, 4)
	result := make([]service, 0, 4)
	for _, dep := range deps {
		srvDeps := kern.sortDependency(dep, srvdeps, isStarted)
		for _, depSrv := range srvDeps {
			if !found[depSrv] {
				result = append(result, depSrv)
				found[depSrv] = true
			}
		}
	}
	return append(result, srvD.srv)
}
func (kern *myKernel) DumpIndex(w io.Writer) {
	kern.place.DumpIndex(w)
}

type service interface {
	// Initialize the data for the service.
	Initialize()

	// ConfigDescriptions returns a sorted list of configuration descriptions.
	ConfigDescriptions() []serviceConfigDescription

	// SetConfig stores a configuration value.
	SetConfig(key, value string) bool

	// GetConfig returns the current configuration value.
	GetConfig(key string) interface{}

	// GetNextConfig returns the next configuration value.
	GetNextConfig(key string) interface{}

	// GetConfigList returns a sorted list of current configuration data.
	GetConfigList(all bool) []kernel.KeyDescrValue

	// GetNextConfigList returns a sorted list of next configuration data.
	GetNextConfigList() []kernel.KeyDescrValue

	// GetStatistics returns a key/value list of statistical data.
	GetStatistics() []kernel.KeyValue

	// Freeze disallows to change some fixed configuration values.
	Freeze()

	// Start the service.
	Start(*myKernel) error

	// SwitchNextToCur moves next config data to current.
	SwitchNextToCur()

	// IsStarted returns true if the service was started successfully.
	IsStarted() bool

	// Stop the service.
	Stop(*myKernel) error
}

type serviceConfigDescription struct{ Key, Descr string }

func (kern *myKernel) SetCreators(
	createAuthManager kernel.CreateAuthManagerFunc,
	createPlaceManager kernel.CreatePlaceManagerFunc,
	setupWebServer kernel.SetupWebServerFunc,
) {
	kern.auth.createManager = createAuthManager
	kern.place.createManager = createPlaceManager
	kern.web.setupServer = setupWebServer
}

Added kernel/impl/place.go.



































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"context"
	"fmt"
	"io"
	"net/url"
	"sync"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/place"
)

type placeService struct {
	srvConfig
	mxService     sync.RWMutex
	manager       place.Manager
	createManager kernel.CreatePlaceManagerFunc
}

func (ps *placeService) Initialize() {
	ps.descr = descriptionMap{
		kernel.PlaceDefaultDirType: {
			"Default directory place type",
			ps.noFrozen(func(val string) interface{} {
				switch val {
				case kernel.PlaceDirTypeNotify, kernel.PlaceDirTypeSimple:
					return val
				}
				return nil
			}),
			true,
		},
		kernel.PlaceURIs: {
			"Place URI",
			func(val string) interface{} {
				uVal, err := url.Parse(val)
				if err != nil {
					return nil
				}
				if uVal.Scheme == "" {
					uVal.Scheme = "dir"
				}
				return uVal
			},
			true,
		},
	}
	ps.next = interfaceMap{
		kernel.PlaceDefaultDirType: kernel.PlaceDirTypeNotify,
	}
}

func (ps *placeService) Start(kern *myKernel) error {
	placeURIs := make([]*url.URL, 0, 4)
	format := kernel.PlaceURIs + "%d"
	for i := 1; ; i++ {
		u := ps.GetNextConfig(fmt.Sprintf(format, i))
		if u == nil {
			break
		}
		placeURIs = append(placeURIs, u.(*url.URL))
	}
	ps.mxService.Lock()
	defer ps.mxService.Unlock()
	mgr, err := ps.createManager(placeURIs, kern.auth.manager, kern.cfg.rtConfig)
	if err != nil {
		kern.doLog("Unable to create place manager:", err)
		return err
	}
	kern.doLog("Start Place Manager:", mgr.Location())
	if err := mgr.Start(context.Background()); err != nil {
		kern.doLog("Unable to start place manager:", err)
	}
	kern.cfg.setPlace(mgr)
	ps.manager = mgr
	return nil
}

func (ps *placeService) IsStarted() bool {
	ps.mxService.RLock()
	defer ps.mxService.RUnlock()
	return ps.manager != nil
}

func (ps *placeService) Stop(kern *myKernel) error {
	kern.doLog("Stop Place Manager")
	ps.mxService.RLock()
	mgr := ps.manager
	ps.mxService.RUnlock()
	err := mgr.Stop(context.Background())
	ps.mxService.Lock()
	ps.manager = nil
	ps.mxService.Unlock()
	return err
}

func (ps *placeService) GetStatistics() []kernel.KeyValue {
	var st place.Stats
	ps.mxService.RLock()
	ps.manager.ReadStats(&st)
	ps.mxService.RUnlock()
	return []kernel.KeyValue{
		{Key: "Read-only", Value: fmt.Sprintf("%v", st.ReadOnly)},
		{Key: "Sub-places", Value: fmt.Sprintf("%v", st.NumManagedPlaces)},
		{Key: "Zettel (total)", Value: fmt.Sprintf("%v", st.ZettelTotal)},
		{Key: "Zettel (indexed)", Value: fmt.Sprintf("%v", st.ZettelIndexed)},
		{Key: "Last re-index", Value: st.LastReload.Format("2006-01-02 15:04:05 -0700 MST")},
		{Key: "Indexes since last re-index", Value: fmt.Sprintf("%v", st.IndexesSinceReload)},
		{Key: "Duration last index", Value: fmt.Sprintf("%vms", st.DurLastIndex.Milliseconds())},
		{Key: "Indexed words", Value: fmt.Sprintf("%v", st.IndexedWords)},
		{Key: "Indexed URLs", Value: fmt.Sprintf("%v", st.IndexedUrls)},
		{Key: "Zettel enrichments", Value: fmt.Sprintf("%v", st.IndexUpdates)},
	}
}

func (ps *placeService) DumpIndex(w io.Writer) {
	ps.manager.Dump(w)
}

Added kernel/impl/server.go.







































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"bufio"
	"net"
)

func startLineServer(kern *myKernel, listenAddr string) error {
	ln, err := net.Listen("tcp", listenAddr)
	if err != nil {
		kern.doLog("Unable to start Line Command Server:", err)
		return err
	}
	kern.doLog("Start Line Command Server:", listenAddr)
	go func() { lineServer(ln, kern) }()
	return nil
}

func lineServer(ln net.Listener, kern *myKernel) {
	// Something may panic. Ensure a running line service.
	defer func() {
		if r := recover(); r != nil {
			kern.doLogRecover("Line", r)
			go lineServer(ln, kern)
		}
	}()

	for {
		conn, err := ln.Accept()
		if err != nil {
			// handle error
			kern.doLog("Unable to accept connection:", err)
			break
		}
		go handleLineConnection(conn, kern)
	}
	ln.Close()
}

func handleLineConnection(conn net.Conn, kern *myKernel) {
	// Something may panic. Ensure a running connection.
	defer func() {
		if r := recover(); r != nil {
			kern.doLogRecover("LineConn", r)
			go handleLineConnection(conn, kern)
		}
	}()

	cmds := cmdSession{}
	cmds.initialize(conn, kern)
	s := bufio.NewScanner(conn)
	for s.Scan() {
		line := s.Text()
		if !cmds.executeLine(line) {
			break
		}
	}
	conn.Close()
}

Added kernel/impl/web.go.






















































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the kernel implementation.
package impl

import (
	"net"
	"strconv"
	"sync"
	"time"

	"zettelstore.de/z/kernel"
	"zettelstore.de/z/web/server"
	"zettelstore.de/z/web/server/impl"
)

type webService struct {
	srvConfig
	mxService   sync.RWMutex
	srvw        server.Server
	setupServer kernel.SetupWebServerFunc
}

// Constants for web service keys.
const (
	WebSecureCookie      = "secure"
	WebListenAddress     = "listen"
	WebPersistentCookie  = "persistent"
	WebTokenLifetimeAPI  = "api-lifetime"
	WebTokenLifetimeHTML = "html-lifetime"
	WebURLPrefix         = "prefix"
)

func (ws *webService) Initialize() {
	ws.descr = descriptionMap{
		kernel.WebListenAddress: {
			"Listen address",
			func(val string) interface{} {
				host, port, err := net.SplitHostPort(val)
				if err != nil {
					return nil
				}
				if _, err := net.LookupPort("tcp", port); err != nil {
					return nil
				}
				return net.JoinHostPort(host, port)
			},
			true},
		kernel.WebPersistentCookie: {"Persistent cookie", parseBool, true},
		kernel.WebSecureCookie:     {"Secure cookie", parseBool, true},
		kernel.WebTokenLifetimeAPI: {
			"Token lifetime API",
			makeDurationParser(10*time.Minute, 0, 1*time.Hour),
			true,
		},
		kernel.WebTokenLifetimeHTML: {
			"Token lifetime HTML",
			makeDurationParser(1*time.Hour, 1*time.Minute, 30*24*time.Hour),
			true,
		},
		kernel.WebURLPrefix: {
			"URL prefix under which the web server runs",
			func(val string) interface{} {
				if val != "" && val[0] == '/' && val[len(val)-1] == '/' {
					return val
				}
				return nil
			},
			true,
		},
	}
	ws.next = interfaceMap{
		kernel.WebListenAddress:     "127.0.0.1:23123",
		kernel.WebPersistentCookie:  false,
		kernel.WebSecureCookie:      true,
		kernel.WebTokenLifetimeAPI:  1 * time.Hour,
		kernel.WebTokenLifetimeHTML: 10 * time.Minute,
		kernel.WebURLPrefix:         "/",
	}
}

func makeDurationParser(defDur, minDur, maxDur time.Duration) parseFunc {
	return func(val string) interface{} {
		if d, err := strconv.ParseUint(val, 10, 64); err == nil {
			secs := time.Duration(d) * time.Minute
			if secs < minDur {
				return minDur
			}
			if secs > maxDur {
				return maxDur
			}
			return secs
		}
		return defDur
	}
}

func (ws *webService) Start(kern *myKernel) error {
	listenAddr := ws.GetNextConfig(kernel.WebListenAddress).(string)
	urlPrefix := ws.GetNextConfig(kernel.WebURLPrefix).(string)
	persistentCookie := ws.GetNextConfig(kernel.WebPersistentCookie).(bool)
	secureCookie := ws.GetNextConfig(kernel.WebSecureCookie).(bool)

	srvw := impl.New(listenAddr, urlPrefix, persistentCookie, secureCookie, kern.auth.manager)
	err := kern.web.setupServer(srvw, kern.place.manager, kern.auth.manager, kern.cfg.rtConfig)
	if err != nil {
		kern.doLog("Unable to create Web Server:", err)
		return err
	}
	if kern.debug {
		srvw.SetDebug()
	}
	if err := srvw.Run(); err != nil {
		kern.doLog("Unable to start Web Service:", err)
		return err
	}
	kern.doLog("Start Web Service:", listenAddr)
	ws.mxService.Lock()
	ws.srvw = srvw
	ws.mxService.Unlock()
	return nil
}

func (ws *webService) IsStarted() bool {
	ws.mxService.RLock()
	defer ws.mxService.RUnlock()
	return ws.srvw != nil
}

func (ws *webService) Stop(kern *myKernel) error {
	kern.doLog("Stop Web Service")
	err := ws.srvw.Stop()
	ws.mxService.Lock()
	ws.srvw = nil
	ws.mxService.Unlock()
	return err
}

func (ws *webService) GetStatistics() []kernel.KeyValue {
	return nil
}

Added kernel/kernel.go.






























































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package kernel provides the main kernel service.
package kernel

import (
	"io"
	"net/url"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/place"
	"zettelstore.de/z/web/server"
)

// Kernel is the main internal service.
type Kernel interface {
	// Start the service.
	Start(headline bool)

	// WaitForShutdown blocks the call until Shutdown is called.
	WaitForShutdown()

	// SetDebug to enable/disable debug mode
	SetDebug(enable bool) bool

	// Shutdown the service. Waits for all concurrent activities to stop.
	Shutdown(silent bool)

	// Log some activity.
	Log(args ...interface{})

	// LogRecover outputs some information about the previous panic.
	LogRecover(name string, recoverInfo interface{}) bool

	// SetConfig stores a configuration value.
	SetConfig(srv Service, key, value string) bool

	// GetConfig returns a configuration value.
	GetConfig(srv Service, key string) interface{}

	// GetConfigList returns a sorted list of configuration data.
	GetConfigList(Service) []KeyDescrValue

	// StartService start the given service.
	StartService(Service) error

	// RestartService stops and restarts the given service, while maintaining service dependencies.
	RestartService(Service) error

	// StopService stop the given service.
	StopService(Service) error

	// GetServiceStatistics returns a key/value list with statistical data.
	GetServiceStatistics(Service) []KeyValue

	// DumpIndex writes some data about the internal index into a writer.
	DumpIndex(io.Writer)

	// SetCreators store functions to be called when a service has to be created.
	SetCreators(CreateAuthManagerFunc, CreatePlaceManagerFunc, SetupWebServerFunc)
}

// Main references the main kernel.
var Main Kernel

// Unit is a type with just one value.
type Unit struct{}

// ShutdownChan is a channel used to signal a system shutdown.
type ShutdownChan <-chan Unit

// Service specifies a service, e.g. web, ...
type Service uint8

// Constants for type Service.
const (
	_ Service = iota
	CoreService
	ConfigService
	AuthService
	PlaceService
	WebService
)

// Constants for core service system keys.
const (
	CoreGoArch    = "go-arch"
	CoreGoOS      = "go-os"
	CoreGoVersion = "go-version"
	CoreHostname  = "hostname"
	CorePort      = "port"
	CoreProgname  = "progname"
	CoreVerbose   = "verbose"
	CoreVersion   = "version"
)

// Constants for authentication service keys.
const (
	AuthOwner    = "owner"
	AuthReadonly = "readonly"
)

// Constants for place service keys.
const (
	PlaceDefaultDirType = "defdirtype"
	PlaceURIs           = "place-uri-"
)

// Allowed values for PlaceDefaultDirType
const (
	PlaceDirTypeNotify = "notify"
	PlaceDirTypeSimple = "simple"
)

// Constants for web service keys.
const (
	WebListenAddress     = "listen"
	WebPersistentCookie  = "persistent"
	WebSecureCookie      = "secure"
	WebTokenLifetimeAPI  = "api-lifetime"
	WebTokenLifetimeHTML = "html-lifetime"
	WebURLPrefix         = "prefix"
)

// KeyDescrValue is a triple of config data.
type KeyDescrValue struct{ Key, Descr, Value string }

// KeyValue is a pair of key and value.
type KeyValue struct{ Key, Value string }

// CreateAuthManagerFunc is called to create a new auth manager.
type CreateAuthManagerFunc func(readonly bool, owner id.Zid) (auth.Manager, error)

// CreatePlaceManagerFunc is called to create a new place manager.
type CreatePlaceManagerFunc func(
	placeURIs []*url.URL,
	authManager auth.Manager,
	rtConfig config.Config,
) (place.Manager, error)

// SetupWebServerFunc is called to create a new web service handler.
type SetupWebServerFunc func(
	webServer server.Server,
	placeManager place.Manager,
	authManager auth.Manager,
	rtConfig config.Config,
) error

Changes to parser/markdown/markdown.go.

410
411
412
413
414
415
416

417
418
419
420
421
422
423
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424







+







	if title := string(node.Title); len(title) > 0 {
		attrs = attrs.Set("title", cleanText(title, true))
	}
	return ast.InlineSlice{
		&ast.LinkNode{
			Ref:     ref,
			Inlines: p.acceptInlineSlice(node),
			OnlyRef: false,
			Attrs:   attrs,
		},
	}
}

func (p *mdP) acceptImage(node *gmAst.Image) ast.InlineSlice {
	ref := ast.ParseReference(cleanText(string(node.Destination), true))
436
437
438
439
440
441
442



443

444
445
446
447

448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466

467
468
469
470
471
472
473
437
438
439
440
441
442
443
444
445
446

447
448
449
450

451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478







+
+
+
-
+



-
+



















+








func (p *mdP) flattenInlineSlice(node gmAst.Node) ast.InlineSlice {
	ins := p.acceptInlineSlice(node)
	var sb strings.Builder
	_, err := p.textEnc.WriteInlines(&sb, ins)
	if err != nil {
		panic(err)
	}
	text := sb.String()
	if text == "" {
		//return ins
		return nil
	}
	return ast.InlineSlice{
		&ast.TextNode{
			Text: sb.String(),
			Text: text,
		},
	}
}

func (p *mdP) acceptAutoLink(node *gmAst.AutoLink) ast.InlineSlice {
	url := node.URL(p.source)
	if node.AutoLinkType == gmAst.AutoLinkEmail &&
		!bytes.HasPrefix(bytes.ToLower(url), []byte("mailto:")) {
		url = append([]byte("mailto:"), url...)
	}
	ref := ast.ParseReference(cleanText(string(url), false))
	label := node.Label(p.source)
	if len(label) == 0 {
		label = url
	}
	return ast.InlineSlice{
		&ast.LinkNode{
			Ref:     ref,
			Inlines: ast.InlineSlice{&ast.TextNode{Text: string(label)}},
			OnlyRef: true,
			Attrs:   nil, //TODO
		},
	}
}

func (p *mdP) acceptRawHTML(node *gmAst.RawHTML) ast.InlineSlice {
	segs := make([]string, 0, node.Segments.Len())

Changes to parser/parser.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25







-
+







// Package parser provides a generic interface to a range of different parsers.
package parser

import (
	"log"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser/cleaner"
)

// Info describes a single parser.
78
79
80
81
82
83
84
85

86
87

88
89
90
91
92
93
94
78
79
80
81
82
83
84

85
86

87
88
89
90
91
92
93
94







-
+

-
+







// ParseMetadata parses a string as Zettelmarkup, resulting in an inline slice.
// Typically used to parse the title or other metadata of type Zettelmarkup.
func ParseMetadata(title string) ast.InlineSlice {
	return ParseInlines(input.NewInput(title), meta.ValueSyntaxZmk)
}

// ParseZettel parses the zettel based on the syntax.
func ParseZettel(zettel domain.Zettel, syntax string) *ast.ZettelNode {
func ParseZettel(zettel domain.Zettel, syntax string, rtConfig config.Config) *ast.ZettelNode {
	m := zettel.Meta
	inhMeta := runtime.AddDefaultValues(m)
	inhMeta := rtConfig.AddDefaultValues(m)
	if syntax == "" {
		syntax, _ = inhMeta.Get(meta.KeySyntax)
	}
	parseMeta := inhMeta
	if syntax == meta.ValueSyntaxNone {
		parseMeta = m
	}

Changes to parser/zettelmark/block.go.

545
546
547
548
549
550
551




552
553
554
555
556
557
558
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562







+
+
+
+







		}
	}
}

// parseRow parse one table row.
func (cp *zmkP) parseRow() (res ast.BlockNode, success bool) {
	inp := cp.inp
	if inp.Peek() == '%' {
		inp.SkipToEOL()
		return nil, true
	}
	row := ast.TableRow{}
	for {
		inp.Next()
		cell := cp.parseCell()
		if cell != nil {
			row = append(row, cell)
		}

Changes to parser/zettelmark/inline.go.

185
186
187
188
189
190
191

192

193


194
195
196
197
198
199
200
185
186
187
188
189
190
191
192
193
194

195
196
197
198
199
200
201
202
203







+

+
-
+
+







	if !ok {
		return "", nil, false
	}
	if inp.Ch == '|' { // First part must be inline text
		if pos == inp.Pos { // [[| or {{|
			return "", nil, false
		}
		sepPos := inp.Pos
		inp.SetPos(pos)
		for inp.Pos < sepPos {
		ins = cp.parseReferenceInline()
			ins = append(ins, cp.parseInline())
		}
		inp.Next()
		pos = inp.Pos
	} else if hasSpace {
		return "", nil, false
	}

	inp.SetPos(pos)
217
218
219
220
221
222
223
224

225






226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
220
221
222
223
224
225
226

227
228
229
230
231
232
233
234
235
236
237
238
239











240
241
242
243
244
245
246







-
+

+
+
+
+
+
+





-
-
-
-
-
-
-
-
-
-
-







	inp := cp.inp
	for {
		switch inp.Ch {
		case input.EOS:
			return false, false
		case '\n', '\r', ' ':
			hasSpace = true
		case '|', closeCh:
		case '|':
			return hasSpace, true
		case closeCh:
			inp.Next()
			if inp.Ch == closeCh {
				return hasSpace, true
			}
			continue
		}
		inp.Next()
	}
}

func (cp *zmkP) parseReferenceInline() (ins ast.InlineSlice) {
	for {
		switch cp.inp.Ch {
		case input.EOS, '|':
			return ins
		}
		in := cp.parseInline()
		ins = append(ins, in)
	}
}

func (cp *zmkP) readReferenceToClose(closeCh rune) bool {
	inp := cp.inp
	for {
		switch inp.Ch {
		case input.EOS, '\n', '\r', ' ':
			return false
		case closeCh:

Changes to parser/zettelmark/zettelmark_test.go.

149
150
151
152
153
154
155




156
157
158
159
160
161
162
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166







+
+
+
+







		{"[[b%%c|a]]", "(PARA [[b {% c|a]]})"},
		{"[[b|a]", "(PARA [[b|a])"},
		{"[[b\nc|a]]", "(PARA (LINK a b SB c))"},
		{"[[b c|a#n]]", "(PARA (LINK a#n b SP c))"},
		{"[[a]]go", "(PARA (LINK a a) go)"},
		{"[[a]]{go}", "(PARA (LINK a a)[ATTR go])"},
		{"[[[[a]]|b]]", "(PARA (LINK [[a [[a) |b]])"},
		{"[[a[b]c|d]]", "(PARA (LINK d a[b]c))"},
		{"[[[b]c|d]]", "(PARA (LINK d [b]c))"},
		{"[[a[]c|d]]", "(PARA (LINK d a[]c))"},
		{"[[a[b]|d]]", "(PARA (LINK d a[b]))"},
	})
}

func TestCite(t *testing.T) {
	checkTcs(t, TestCases{
		{"[@", "(PARA [@)"},
		{"[@]", "(PARA [@])"},
522
523
524
525
526
527
528


529
530
531
532
533
534
535
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541







+
+







	checkTcs(t, TestCases{
		{"|", "(TAB (TR))"},
		{"|a", "(TAB (TR (TD a)))"},
		{"|a|", "(TAB (TR (TD a)))"},
		{"|a| ", "(TAB (TR (TD a)(TD)))"},
		{"|a|b", "(TAB (TR (TD a)(TD b)))"},
		{"|a|b\n|c|d", "(TAB (TR (TD a)(TD b))(TR (TD c)(TD d)))"},
		{"|%", ""},
		{"|a|b\n|%---\n|c|d", "(TAB (TR (TD a)(TD b))(TR (TD c)(TD d)))"},
	})
}

func TestBlockAttr(t *testing.T) {
	checkTcs(t, TestCases{
		{":::go\n:::", "(SPAN)[ATTR =go]"},
		{":::go=\n:::", "(SPAN)[ATTR =go]"},

Deleted place/change/change.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43











































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package change provides definition for place changes.
package change

import (
	"zettelstore.de/z/domain/id"
)

// Reason gives an indication, why the ObserverFunc was called.
type Reason int

// Values for Reason
const (
	_        Reason = iota
	OnReload        // Place was reloaded
	OnUpdate        // A zettel was created or changed
	OnDelete        // A zettel was removed
)

// Info contains all the data about a changed zettel.
type Info struct {
	Reason Reason
	Zid    id.Zid
}

// Func is a function to be called when a change is detected.
type Func func(Info)

// Subject is a place that notifies observers about changes.
type Subject interface {
	// RegisterObserver registers an observer that will be notified
	// if one or all zettel are found to be changed.
	RegisterObserver(Func)
}

Changes to place/constplace/base.mustache.

1
2
3
4
5
6
7

8
9
10
11
12
13
14
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15







+







<!DOCTYPE html>
<html{{#Lang}} lang="{{Lang}}"{{/Lang}}>
<head>
<meta charset="utf-8">
<meta name="referrer" content="no-referrer">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Zettelstore">
<meta name="format-detection" content="telephone=no">
{{{MetaHeader}}}
<link rel="stylesheet" href="{{{StylesheetURL}}}">
<title>{{Title}}</title>
</head>
<body>
<nav class="zs-menu">
<a href="{{{HomeURL}}}">Home</a>

Changes to place/constplace/constplace.go.

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53


54
55
56
57
58
59
60
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50


51
52
53
54
55
56
57
58
59







-









-
+



















-
-
+
+







	"context"
	_ "embed" // Allow to embed file content
	"net/url"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/manager"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register(
		" const",
		func(u *url.URL, cdata *manager.ConnectData) (place.ManagedPlace, error) {
			return &constPlace{zettel: constZettelMap, filter: cdata.Filter}, nil
			return &constPlace{zettel: constZettelMap, enricher: cdata.Enricher}, nil
		})
}

type constHeader map[string]string

func makeMeta(zid id.Zid, h constHeader) *meta.Meta {
	m := meta.New(zid)
	for k, v := range h {
		m.Set(k, v)
	}
	return m
}

type constZettel struct {
	header  constHeader
	content domain.Content
}

type constPlace struct {
	zettel map[id.Zid]constZettel
	filter index.MetaFilter
	zettel   map[id.Zid]constZettel
	enricher place.Enricher
}

func (cp *constPlace) Location() string {
	return "const:"
}

func (cp *constPlace) CanCreateZettel(ctx context.Context) bool { return false }
84
85
86
87
88
89
90
91

92
93
94
95
96
97
98
83
84
85
86
87
88
89

90
91
92
93
94
95
96
97







-
+







	}
	return result, nil
}

func (cp *constPlace) SelectMeta(ctx context.Context, match search.MetaMatchFunc) (res []*meta.Meta, err error) {
	for zid, zettel := range cp.zettel {
		m := makeMeta(zid, zettel.header)
		cp.filter.Enrich(ctx, m)
		cp.enricher.Enrich(ctx, m)
		if match(m) {
			res = append(res, m)
		}
	}
	return res, nil
}

120
121
122
123
124
125
126
127

128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143






























144
145
146
147
148
149
150
119
120
121
122
123
124
125

126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179







-
+
















+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







func (cp *constPlace) DeleteZettel(ctx context.Context, zid id.Zid) error {
	if _, ok := cp.zettel[zid]; ok {
		return place.ErrReadOnly
	}
	return place.ErrNotFound
}

func (cp *constPlace) ReadStats(st *place.Stats) {
func (cp *constPlace) ReadStats(st *place.ManagedPlaceStats) {
	st.ReadOnly = true
	st.Zettel = len(cp.zettel)
}

const syntaxTemplate = "mustache"

var constZettelMap = map[id.Zid]constZettel{
	id.ConfigurationZid: {
		constHeader{
			meta.KeyTitle:      "Zettelstore Runtime Configuration",
			meta.KeyRole:       meta.ValueRoleConfiguration,
			meta.KeyVisibility: meta.ValueVisibilityOwner,
			meta.KeySyntax:     meta.ValueSyntaxNone,
			meta.KeyNoIndex:    meta.ValueTrue,
		},
		""},
	id.LicenseZid: {
		constHeader{
			meta.KeyTitle:      "Zettelstore License",
			meta.KeyRole:       meta.ValueRoleConfiguration,
			meta.KeyVisibility: meta.ValueVisibilityPublic,
			meta.KeySyntax:     meta.ValueSyntaxText,
			meta.KeyLang:       meta.ValueLangEN,
			meta.KeyReadOnly:   meta.ValueTrue,
		},
		domain.NewContent(contentLicense)},
	id.AuthorsZid: {
		constHeader{
			meta.KeyTitle:      "Zettelstore Contributors",
			meta.KeyRole:       meta.ValueRoleConfiguration,
			meta.KeyVisibility: meta.ValueVisibilityPublic,
			meta.KeySyntax:     meta.ValueSyntaxZmk,
			meta.KeyLang:       meta.ValueLangEN,
			meta.KeyReadOnly:   meta.ValueTrue,
		},
		domain.NewContent(contentContributors)},
	id.DependenciesZid: {
		constHeader{
			meta.KeyTitle:      "Zettelstore Dependencies",
			meta.KeyRole:       meta.ValueRoleConfiguration,
			meta.KeyVisibility: meta.ValueVisibilityPublic,
			meta.KeySyntax:     meta.ValueSyntaxZmk,
			meta.KeyLang:       meta.ValueLangEN,
			meta.KeyReadOnly:   meta.ValueTrue,
		},
		domain.NewContent(contentDependencies)},
	id.BaseTemplateZid: {
		constHeader{
			meta.KeyTitle:      "Zettelstore Base HTML Template",
			meta.KeyRole:       meta.ValueRoleConfiguration,
			meta.KeyVisibility: meta.ValueVisibilityExpert,
			meta.KeySyntax:     syntaxTemplate,
			meta.KeyNoIndex:    meta.ValueTrue,
254
255
256
257
258
259
260









261
262
263
264
265

266
267
268
269
270
271
272
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311







+
+
+
+
+
+
+
+
+





+







			meta.KeyTitle:      "Zettelstore Base CSS",
			meta.KeyRole:       meta.ValueRoleConfiguration,
			meta.KeyVisibility: meta.ValueVisibilityPublic,
			meta.KeySyntax:     "css",
			meta.KeyNoIndex:    meta.ValueTrue,
		},
		domain.NewContent(contentBaseCSS)},
	id.EmojiZid: {
		constHeader{
			meta.KeyTitle:      "Generic Emoji",
			meta.KeyRole:       meta.ValueRoleConfiguration,
			meta.KeyVisibility: meta.ValueVisibilityPublic,
			meta.KeySyntax:     meta.ValueSyntaxGif,
			meta.KeyReadOnly:   meta.ValueTrue,
		},
		domain.NewContent(contentEmoji)},
	id.TOCNewTemplateZid: {
		constHeader{
			meta.KeyTitle:  "New Menu",
			meta.KeyRole:   meta.ValueRoleConfiguration,
			meta.KeySyntax: meta.ValueSyntaxZmk,
			meta.KeyLang:   meta.ValueLangEN,
		},
		domain.NewContent(contentNewTOCZettel)},
	id.TemplateNewZettelZid: {
		constHeader{
			meta.KeyTitle:  "New Zettel",
			meta.KeyRole:   meta.ValueRoleZettel,
			meta.KeySyntax: meta.ValueSyntaxZmk,
283
284
285
286
287
288
289

290
291
292
293









294
295
296
297
298
299
300
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349







+




+
+
+
+
+
+
+
+
+







		},
		""},
	id.DefaultHomeZid: {
		constHeader{
			meta.KeyTitle:  "Home",
			meta.KeyRole:   meta.ValueRoleZettel,
			meta.KeySyntax: meta.ValueSyntaxZmk,
			meta.KeyLang:   meta.ValueLangEN,
		},
		domain.NewContent(contentHomeZettel)},
}

//go:embed license.txt
var contentLicense string

//go:embed contributors.zettel
var contentContributors string

//go:embed dependencies.zettel
var contentDependencies string

//go:embed base.mustache
var contentBaseMustache string

//go:embed login.mustache
var contentLoginMustache string

//go:embed zettel.mustache
325
326
327
328
329
330
331



332
333
334
335
336
337
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389







+
+
+






var contentListTagsMustache string

//go:embed error.mustache
var contentErrorMustache string

//go:embed base.css
var contentBaseCSS string

//go:embed emoji_spin.gif
var contentEmoji string

//go:embed newtoc.zettel
var contentNewTOCZettel string

//go:embed home.zettel
var contentHomeZettel string

Added place/constplace/contributors.zettel.









1
2
3
4
5
6
7
8
+
+
+
+
+
+
+
+
Zettelstore is a software for humans made from humans.

=== Licensor(s)
* Detlef Stern [[mailto:ds@zettelstore.de]]
** Main author
** Maintainer

=== Contributors

Added place/constplace/dependencies.zettel.






















































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Zettelstore is made with the help of other software and other artifacts.
Thank you very much!

This zettel lists all of them, together with their license.

=== Go runtime and associated libraries
; License
: BSD 3-Clause "New" or "Revised" License
```
Copyright (c) 2009 The Go Authors. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

   * Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
   * Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
   * Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```

=== Fsnotify
; URL
: [[https://fsnotify.org/]]
; License
: BSD 3-Clause "New" or "Revised" License
; Source
: [[https://github.com/fsnotify/fsnotify]]
```
Copyright (c) 2012 The Go Authors. All rights reserved.
Copyright (c) 2012-2019 fsnotify Authors. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

   * Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
   * Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
   * Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
```

=== hoisie/mustache / cbroglie/mustache
; URL & Source
: [[https://github.com/hoisie/mustache]] / [[https://github.com/cbroglie/mustache]]
; License
: MIT License
; Remarks
: cbroglie/mustache is a fork from hoisie/mustache (starting with commit [f9b4cbf]).
  cbroglie/mustache does not claim a copyright and includes just the license file from hoisie/mustache.
  cbroglie/mustache obviously continues with the original license.

```
Copyright (c) 2009 Michael Hoisie

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
```

===  pascaldekloe/jwt
; URL & Source
: [[https://github.com/pascaldekloe/jwt]]
; License
: [[CC0 1.0 Universal|https://creativecommons.org/publicdomain/zero/1.0/legalcode]]
```
To the extent possible under law, Pascal S. de Kloe has waived all
copyright and related or neighboring rights to JWT. This work is
published from The Netherlands.

https://creativecommons.org/publicdomain/zero/1.0/legalcode
```

=== yuin/goldmark
; URL & Source
: [[https://github.com/yuin/goldmark]]
; License
: MIT License
```
MIT License

Copyright (c) 2019 Yusuke Inuzuka

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```

Added place/constplace/emoji_spin.gif.

cannot compute difference between binary files

Changes to place/constplace/home.zettel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
1
2
3
4
5
6
7
8
9
10
11
12
13

14
15
16
17

18
19
20
21
22
23
24













-
+



-







=== Thank you for using Zettelstore!

You will find the lastest information about Zettelstore at [[https://zettelstore.de]].
Check that website regulary for [[upgrades|https://zettelstore.de/home/doc/trunk/www/download.wiki]] to the latest version.
You should consult the [[change log|https://zettelstore.de/home/doc/trunk/www/changes.wiki]] before upgrading.
Sometimes, you have to edit some of your Zettelstore-related zettel before upgrading.
Since Zettelstore is currently in a development state, every upgrade might fix some of your problems.
To check for versions, there is a zettel with the [[current version|00000000000001]] of your Zettelstore.

If you have problems concerning Zettelstore,
do not hesitate to get in [[contact with the main developer|mailto:ds@zettelstore.de]].

=== Reporting errors
If you have encountered an error, please include the content of the following zettel in your mail:
If you have encountered an error, please include the content of the following zettel in your mail (if possible):
* [[Zettelstore Version|00000000000001]]
* [[Zettelstore Operating System|00000000000003]]
* [[Zettelstore Startup Configuration|00000000000096]]
* [[Zettelstore Startup Values|00000000000098]]
* [[Zettelstore Runtime Configuration|00000000000100]]

Additionally, you have to describe, what you have done before that error occurs
and what you have expected instead.
Please do not forget to include the error message, if there is one.

Some of above Zettelstore zettel can only be retrieved if you enabled ""expert mode"".

Changes to place/constplace/info.mustache.

13
14
15
16
17
18
19
20


21
22
23
24
25
26
27
13
14
15
16
17
18
19

20
21
22
23
24
25
26
27
28







-
+
+







<table>{{#MetaData}}<tr><td>{{Key}}</td><td>{{{Value}}}</td></tr>{{/MetaData}}</table>
{{#HasLinks}}
<h2>References</h2>
{{#HasLocLinks}}
<h3>Local</h3>
<ul>
{{#LocLinks}}
<li><a href="{{{.}}}">{{.}}</a></li>
{{#Valid}}<li><a href="{{{Zid}}}">{{Zid}}</a></li>{{/Valid}}
{{^Valid}}<li>{{Zid}}</li>{{/Valid}}
{{/LocLinks}}
</ul>
{{/HasLocLinks}}
{{#HasExtLinks}}
<h3>External</h3>
<ul>
{{#ExtLinks}}

Added place/constplace/license.txt.








































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Copyright (c) 2020-2021 Detlef Stern

                          Licensed under the EUPL

Zettelstore is licensed under the European Union Public License, version 1.2 or
later (EUPL v. 1.2). The license is available in the official languages of the
EU. The English version is included here. Please see
https://joinup.ec.europa.eu/community/eupl/og_page/eupl for official
translations of the other languages.


-------------------------------------------------------------------------------


EUROPEAN UNION PUBLIC LICENCE v. 1.2
EUPL © the European Union 2007, 2016

This European Union Public Licence (the ‘EUPL’) applies to the Work (as defined
below) which is provided under the terms of this Licence. Any use of the Work,
other than as authorised under this Licence is prohibited (to the extent such
use is covered by a right of the copyright holder of the Work).

The Work is provided under the terms of this Licence when the Licensor (as
defined below) has placed the following notice immediately following the
copyright notice for the Work:

                          Licensed under the EUPL

or has expressed by any other means his willingness to license under the EUPL.

1. Definitions

In this Licence, the following terms have the following meaning:

— ‘The Licence’: this Licence.
— ‘The Original Work’: the work or software distributed or communicated by the
  Licensor under this Licence, available as Source Code and also as Executable
  Code as the case may be.
— ‘Derivative Works’: the works or software that could be created by the
  Licensee, based upon the Original Work or modifications thereof. This Licence
  does not define the extent of modification or dependence on the Original Work
  required in order to classify a work as a Derivative Work; this extent is
  determined by copyright law applicable in the country mentioned in Article
  15.
— ‘The Work’: the Original Work or its Derivative Works.
— ‘The Source Code’: the human-readable form of the Work which is the most
  convenient for people to study and modify.
— ‘The Executable Code’: any code which has generally been compiled and which
  is meant to be interpreted by a computer as a program.
— ‘The Licensor’: the natural or legal person that distributes or communicates
  the Work under the Licence.
— ‘Contributor(s)’: any natural or legal person who modifies the Work under the
  Licence, or otherwise contributes to the creation of a Derivative Work.
— ‘The Licensee’ or ‘You’: any natural or legal person who makes any usage of
  the Work under the terms of the Licence.
— ‘Distribution’ or ‘Communication’: any act of selling, giving, lending,
  renting, distributing, communicating, transmitting, or otherwise making
  available, online or offline, copies of the Work or providing access to its
  essential functionalities at the disposal of any other natural or legal
  person.

2. Scope of the rights granted by the Licence

The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
sublicensable licence to do the following, for the duration of copyright vested
in the Original Work:

— use the Work in any circumstance and for all usage,
— reproduce the Work,
— modify the Work, and make Derivative Works based upon the Work,
— communicate to the public, including the right to make available or display
  the Work or copies thereof to the public and perform publicly, as the case
  may be, the Work,
— distribute the Work or copies thereof,
— lend and rent the Work or copies thereof,
— sublicense rights in the Work or copies thereof.

Those rights can be exercised on any media, supports and formats, whether now
known or later invented, as far as the applicable law permits so.

In the countries where moral rights apply, the Licensor waives his right to
exercise his moral right to the extent allowed by law in order to make
effective the licence of the economic rights here above listed.

The Licensor grants to the Licensee royalty-free, non-exclusive usage rights to
any patents held by the Licensor, to the extent necessary to make use of the
rights granted on the Work under this Licence.

3. Communication of the Source Code

The Licensor may provide the Work either in its Source Code form, or as
Executable Code. If the Work is provided as Executable Code, the Licensor
provides in addition a machine-readable copy of the Source Code of the Work
along with each copy of the Work that the Licensor distributes or indicates, in
a notice following the copyright notice attached to the Work, a repository
where the Source Code is easily and freely accessible for as long as the
Licensor continues to distribute or communicate the Work.

4. Limitations on copyright

Nothing in this Licence is intended to deprive the Licensee of the benefits
from any exception or limitation to the exclusive rights of the rights owners
in the Work, of the exhaustion of those rights or of other applicable
limitations thereto.

5. Obligations of the Licensee

The grant of the rights mentioned above is subject to some restrictions and
obligations imposed on the Licensee. Those obligations are the following:

Attribution right: The Licensee shall keep intact all copyright, patent or
trademarks notices and all notices that refer to the Licence and to the
disclaimer of warranties. The Licensee must include a copy of such notices and
a copy of the Licence with every copy of the Work he/she distributes or
communicates. The Licensee must cause any Derivative Work to carry prominent
notices stating that the Work has been modified and the date of modification.

Copyleft clause: If the Licensee distributes or communicates copies of the
Original Works or Derivative Works, this Distribution or Communication will be
done under the terms of this Licence or of a later version of this Licence
unless the Original Work is expressly distributed only under this version of
the Licence — for example by communicating ‘EUPL v. 1.2 only’. The Licensee
(becoming Licensor) cannot offer or impose any additional terms or conditions
on the Work or Derivative Work that alter or restrict the terms of the Licence.

Compatibility clause: If the Licensee Distributes or Communicates Derivative
Works or copies thereof based upon both the Work and another work licensed
under a Compatible Licence, this Distribution or Communication can be done
under the terms of this Compatible Licence. For the sake of this clause,
‘Compatible Licence’ refers to the licences listed in the appendix attached to
this Licence. Should the Licensee's obligations under the Compatible Licence
conflict with his/her obligations under this Licence, the obligations of the
Compatible Licence shall prevail.

Provision of Source Code: When distributing or communicating copies of the
Work, the Licensee will provide a machine-readable copy of the Source Code or
indicate a repository where this Source will be easily and freely available for
as long as the Licensee continues to distribute or communicate the Work.

Legal Protection: This Licence does not grant permission to use the trade
names, trademarks, service marks, or names of the Licensor, except as required
for reasonable and customary use in describing the origin of the Work and
reproducing the content of the copyright notice.

6. Chain of Authorship

The original Licensor warrants that the copyright in the Original Work granted
hereunder is owned by him/her or licensed to him/her and that he/she has the
power and authority to grant the Licence.

Each Contributor warrants that the copyright in the modifications he/she brings
to the Work are owned by him/her or licensed to him/her and that he/she has the
power and authority to grant the Licence.

Each time You accept the Licence, the original Licensor and subsequent
Contributors grant You a licence to their contributions to the Work, under the
terms of this Licence.

7. Disclaimer of Warranty

The Work is a work in progress, which is continuously improved by numerous
Contributors. It is not a finished work and may therefore contain defects or
‘bugs’ inherent to this type of development.

For the above reason, the Work is provided under the Licence on an ‘as is’
basis and without warranties of any kind concerning the Work, including without
limitation merchantability, fitness for a particular purpose, absence of
defects or errors, accuracy, non-infringement of intellectual property rights
other than copyright as stated in Article 6 of this Licence.

This disclaimer of warranty is an essential part of the Licence and a condition
for the grant of any rights to the Work.

8. Disclaimer of Liability

Except in the cases of wilful misconduct or damages directly caused to natural
persons, the Licensor will in no event be liable for any direct or indirect,
material or moral, damages of any kind, arising out of the Licence or of the
use of the Work, including without limitation, damages for loss of goodwill,
work stoppage, computer failure or malfunction, loss of data or any commercial
damage, even if the Licensor has been advised of the possibility of such
damage. However, the Licensor will be liable under statutory product liability
laws as far such laws apply to the Work.

9. Additional agreements

While distributing the Work, You may choose to conclude an additional
agreement, defining obligations or services consistent with this Licence.
However, if accepting obligations, You may act only on your own behalf and on
your sole responsibility, not on behalf of the original Licensor or any other
Contributor, and only if You agree to indemnify, defend, and hold each
Contributor harmless for any liability incurred by, or claims asserted against
such Contributor by the fact You have accepted any warranty or additional
liability.

10. Acceptance of the Licence

The provisions of this Licence can be accepted by clicking on an icon ‘I agree’
placed under the bottom of a window displaying the text of this Licence or by
affirming consent in any other similar way, in accordance with the rules of
applicable law. Clicking on that icon indicates your clear and irrevocable
acceptance of this Licence and all of its terms and conditions.

Similarly, you irrevocably accept this Licence and all of its terms and
conditions by exercising any rights granted to You by Article 2 of this
Licence, such as the use of the Work, the creation by You of a Derivative Work
or the Distribution or Communication by You of the Work or copies thereof.

11. Information to the public

In case of any Distribution or Communication of the Work by means of electronic
communication by You (for example, by offering to download the Work from
a remote location) the distribution channel or media (for example, a website)
must at least provide to the public the information requested by the applicable
law regarding the Licensor, the Licence and the way it may be accessible,
concluded, stored and reproduced by the Licensee.

12. Termination of the Licence

The Licence and the rights granted hereunder will terminate automatically upon
any breach by the Licensee of the terms of the Licence.

Such a termination will not terminate the licences of any person who has
received the Work from the Licensee under the Licence, provided such persons
remain in full compliance with the Licence.

13. Miscellaneous

Without prejudice of Article 9 above, the Licence represents the complete
agreement between the Parties as to the Work.

If any provision of the Licence is invalid or unenforceable under applicable
law, this will not affect the validity or enforceability of the Licence as
a whole. Such provision will be construed or reformed so as necessary to make
it valid and enforceable.

The European Commission may publish other linguistic versions or new versions
of this Licence or updated versions of the Appendix, so far this is required
and reasonable, without reducing the scope of the rights granted by the
Licence. New versions of the Licence will be published with a unique version
number.

All linguistic versions of this Licence, approved by the European Commission,
have identical value. Parties can take advantage of the linguistic version of
their choice.

14. Jurisdiction

Without prejudice to specific agreement between parties,

— any litigation resulting from the interpretation of this License, arising
  between the European Union institutions, bodies, offices or agencies, as
  a Licensor, and any Licensee, will be subject to the jurisdiction of the
  Court of Justice of the European Union, as laid down in article 272 of the
  Treaty on the Functioning of the European Union,
— any litigation arising between other parties and resulting from the
  interpretation of this License, will be subject to the exclusive jurisdiction
  of the competent court where the Licensor resides or conducts its primary
  business.

15. Applicable Law

Without prejudice to specific agreement between parties,

— this Licence shall be governed by the law of the European Union Member State
  where the Licensor has his seat, resides or has his registered office,
— this licence shall be governed by Belgian law if the Licensor has no seat,
  residence or registered office inside a European Union Member State.


                                  Appendix


‘Compatible Licences’ according to Article 5 EUPL are:

— GNU General Public License (GPL) v. 2, v. 3
— GNU Affero General Public License (AGPL) v. 3
— Open Software License (OSL) v. 2.1, v. 3.0
— Eclipse Public License (EPL) v. 1.0
— CeCILL v. 2.0, v. 2.1
— Mozilla Public Licence (MPL) v. 2
— GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
— Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
  works other than software
— European Union Public Licence (EUPL) v. 1.1, v. 1.2
— Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
  Reciprocity (LiLiQ-R+)

The European Commission may update this Appendix to later versions of the above
licences without producing a new version of the EUPL, as long as they provide
the rights granted in Article 2 of this Licence and protect the covered Source
Code from exclusive appropriation.

All other changes or additions to this Appendix require the production of a new
EUPL version.

Changes to place/dirplace/dirplace.go.

9
10
11
12
13
14
15

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28

29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45







+








-




-









-
+







//-----------------------------------------------------------------------------

// Package dirplace provides a directory-based zettel place.
package dirplace

import (
	"context"
	"errors"
	"net/url"
	"os"
	"path/filepath"
	"strconv"
	"strings"
	"sync"
	"time"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/place/dirplace/directory"
	"zettelstore.de/z/place/fileplace"
	"zettelstore.de/z/place/manager"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register("dir", func(u *url.URL, cdata *manager.ConnectData) (place.ManagedPlace, error) {
		path := getDirPath(u)
		if _, err := os.Stat(path); os.IsNotExist(err) {
		if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {
			return nil, err
		}
		dirSrvSpec, defWorker, maxWorker := getDirSrvInfo(u.Query().Get("type"))
		dp := dirPlace{
			location:   u.String(),
			readonly:   getQueryBool(u, "readonly"),
			cdata:      *cdata,
133
134
135
136
137
138
139
140

141
142
143

144
145
146
147
148
149
150
132
133
134
135
136
137
138

139
140
141

142
143
144
145
146
147
148
149







-
+


-
+







	err := dirSrv.Stop()
	for _, c := range dp.fCmds {
		close(c)
	}
	return err
}

func (dp *dirPlace) notifyChanged(reason change.Reason, zid id.Zid) {
func (dp *dirPlace) notifyChanged(reason place.UpdateReason, zid id.Zid) {
	if dp.mustNotify {
		if chci := dp.cdata.Notify; chci != nil {
			chci <- change.Info{Reason: reason, Zid: zid}
			chci <- place.UpdateInfo{Reason: reason, Zid: zid}
		}
	}
}

func (dp *dirPlace) getFileChan(zid id.Zid) chan fileCmd {
	// Based on https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function
	var sum uint32 = 2166136261 ^ uint32(zid)
174
175
176
177
178
179
180
181

182
183
184
185
186
187
188
173
174
175
176
177
178
179

180
181
182
183
184
185
186
187







-
+







	meta.Zid = entry.Zid
	dp.updateEntryFromMeta(entry, meta)

	err = setZettel(dp, entry, zettel)
	if err == nil {
		dp.dirSrv.UpdateEntry(entry)
	}
	dp.notifyChanged(change.OnUpdate, meta.Zid)
	dp.notifyChanged(place.OnUpdate, meta.Zid)
	return meta.Zid, err
}

func (dp *dirPlace) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	entry, err := dp.dirSrv.GetEntry(zid)
	if err != nil || !entry.IsValid() {
		return domain.Zettel{}, place.ErrNotFound
231
232
233
234
235
236
237
238

239
240
241
242
243
244
245
230
231
232
233
234
235
236

237
238
239
240
241
242
243
244







-
+







	for _, entry := range entries {
		m, err1 := getMeta(dp, entry, entry.Zid)
		err = err1
		if err != nil {
			continue
		}
		dp.cleanupMeta(ctx, m)
		dp.cdata.Filter.Enrich(ctx, m)
		dp.cdata.Enricher.Enrich(ctx, m)

		if match(m) {
			res = append(res, m)
		}
	}
	if err != nil {
		return nil, err
273
274
275
276
277
278
279
280

281
282
283
284
285
286

287
288
289
290
291
292
293
294
295

296
297
298
299
300
301
302
303
304

305
306
307
308
309
310
311
272
273
274
275
276
277
278

279
280
281
282
283
284

285
286
287
288
289
290
291
292
293

294
295
296
297
298
299
300
301
302

303
304
305
306
307
308
309
310







-
+





-
+








-
+








-
+







		if !meta.Equal(defaultMeta, true) {
			dp.updateEntryFromMeta(entry, meta)
			dp.dirSrv.UpdateEntry(entry)
		}
	}
	err = setZettel(dp, entry, zettel)
	if err == nil {
		dp.notifyChanged(change.OnUpdate, meta.Zid)
		dp.notifyChanged(place.OnUpdate, meta.Zid)
	}
	return err
}

func (dp *dirPlace) updateEntryFromMeta(entry *directory.Entry, meta *meta.Meta) {
	entry.MetaSpec, entry.ContentExt = calcSpecExt(meta)
	entry.MetaSpec, entry.ContentExt = dp.calcSpecExt(meta)
	basePath := filepath.Join(dp.dir, entry.Zid.String())
	if entry.MetaSpec == directory.MetaSpecFile {
		entry.MetaPath = basePath + ".meta"
	}
	entry.ContentPath = basePath + "." + entry.ContentExt
	entry.Duplicates = false
}

func calcSpecExt(m *meta.Meta) (directory.MetaSpec, string) {
func (dp *dirPlace) calcSpecExt(m *meta.Meta) (directory.MetaSpec, string) {
	if m.YamlSep {
		return directory.MetaSpecHeader, "zettel"
	}
	syntax := m.GetDefault(meta.KeySyntax, "bin")
	switch syntax {
	case meta.ValueSyntaxNone, meta.ValueSyntaxZmk:
		return directory.MetaSpecHeader, "zettel"
	}
	for _, s := range runtime.GetZettelFileSyntax() {
	for _, s := range dp.cdata.Config.GetZettelFileSyntax() {
		if s == syntax {
			return directory.MetaSpecHeader, "zettel"
		}
	}
	return directory.MetaSpecFile, syntax
}

351
352
353
354
355
356
357
358
359


360
361
362
363
364
365
366
350
351
352
353
354
355
356


357
358
359
360
361
362
363
364
365







-
-
+
+







	if err = setZettel(dp, &newEntry, newZettel); err != nil {
		// "Rollback" rename. No error checking...
		dp.dirSrv.RenameEntry(&newEntry, curEntry)
		return err
	}
	err = deleteZettel(dp, curEntry, curZid)
	if err == nil {
		dp.notifyChanged(change.OnDelete, curZid)
		dp.notifyChanged(change.OnUpdate, newZid)
		dp.notifyChanged(place.OnDelete, curZid)
		dp.notifyChanged(place.OnUpdate, newZid)
	}
	return err
}

func (dp *dirPlace) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	if dp.readonly {
		return false
377
378
379
380
381
382
383
384

385
386
387
388
389

390
391
392
393
394
395
396

397
398
399

400
401
402
403
404
405
406
407
408
409
410
376
377
378
379
380
381
382

383
384
385
386
387

388
389
390
391
392
393
394

395
396
397

398
399
400
401
402
403
404
405
406
407
408
409







-
+




-
+






-
+


-
+











	entry, err := dp.dirSrv.GetEntry(zid)
	if err != nil || !entry.IsValid() {
		return nil
	}
	dp.dirSrv.DeleteEntry(zid)
	err = deleteZettel(dp, entry, zid)
	if err == nil {
		dp.notifyChanged(change.OnDelete, zid)
		dp.notifyChanged(place.OnDelete, zid)
	}
	return err
}

func (dp *dirPlace) ReadStats(st *place.Stats) {
func (dp *dirPlace) ReadStats(st *place.ManagedPlaceStats) {
	st.ReadOnly = dp.readonly
	st.Zettel, _ = dp.dirSrv.NumEntries()
}

func (dp *dirPlace) cleanupMeta(ctx context.Context, m *meta.Meta) {
	if role, ok := m.Get(meta.KeyRole); !ok || role == "" {
		m.Set(meta.KeyRole, runtime.GetDefaultRole())
		m.Set(meta.KeyRole, dp.cdata.Config.GetDefaultRole())
	}
	if syntax, ok := m.Get(meta.KeySyntax); !ok || syntax == "" {
		m.Set(meta.KeySyntax, runtime.GetDefaultSyntax())
		m.Set(meta.KeySyntax, dp.cdata.Config.GetDefaultSyntax())
	}
}

func renamePath(path string, curID, newID id.Zid) string {
	dir, file := filepath.Split(path)
	if cur := curID.String(); strings.HasPrefix(file, cur) {
		file = newID.String() + file[len(cur):]
		return filepath.Join(dir, file)
	}
	return path
}

Changes to place/dirplace/makedir.go.

8
9
10
11
12
13
14
15

16
17
18
19
20
21
22
23

24
25

26
27
28

29
30
31
32
33
34
35
8
9
10
11
12
13
14

15
16
17
18
19
20
21
22

23
24

25
26
27

28
29
30
31
32
33
34
35







-
+







-
+

-
+


-
+







// under this license.
//-----------------------------------------------------------------------------

// Package dirplace provides a directory-based zettel place.
package dirplace

import (
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/place/dirplace/notifydir"
	"zettelstore.de/z/place/dirplace/simpledir"
)

func getDirSrvInfo(dirType string) (directoryServiceSpec, int, int) {
	for count := 0; count < 2; count++ {
		switch dirType {
		case startup.ValueDirPlaceTypeNotify:
		case kernel.PlaceDirTypeNotify:
			return dirSrvNotify, 7, 1499
		case startup.ValueDirPlaceTypeSimple:
		case kernel.PlaceDirTypeSimple:
			return dirSrvSimple, 1, 1
		default:
			dirType = startup.DefaultDirPlaceType()
			dirType = kernel.Main.GetConfig(kernel.PlaceService, kernel.PlaceDefaultDirType).(string)
		}
	}
	panic("unable to set default dir place type: " + dirType)
}

func (dp *dirPlace) setupDirService() {
	switch dp.dirSrvSpec {

Changes to place/dirplace/notifydir/notifydir.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28

29
30
31
32

33
34
35
36
37
38
39
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27

28
29
30
31

32
33
34
35
36
37
38
39







-
+









-
+



-
+







// Package notifydir manages the notified directory part of a dirstore.
package notifydir

import (
	"time"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/dirplace/directory"
)

// notifyService specifies a directory scan service.
type notifyService struct {
	dirPath    string
	rescanTime time.Duration
	done       chan struct{}
	cmds       chan dirCmd
	infos      chan<- change.Info
	infos      chan<- place.UpdateInfo
}

// NewService creates a new directory service.
func NewService(directoryPath string, rescanTime time.Duration, chci chan<- change.Info) directory.Service {
func NewService(directoryPath string, rescanTime time.Duration, chci chan<- place.UpdateInfo) directory.Service {
	srv := &notifyService{
		dirPath:    directoryPath,
		rescanTime: rescanTime,
		cmds:       make(chan dirCmd),
		infos:      chci,
	}
	return srv
62
63
64
65
66
67
68
69

70
71

72
73
74
75
76
77
78
62
63
64
65
66
67
68

69
70

71
72
73
74
75
76
77
78







-
+

-
+







// Stop stops the directory service.
func (srv *notifyService) Stop() error {
	close(srv.done)
	srv.done = nil
	return nil
}

func (srv *notifyService) notifyChange(reason change.Reason, zid id.Zid) {
func (srv *notifyService) notifyChange(reason place.UpdateReason, zid id.Zid) {
	if chci := srv.infos; chci != nil {
		chci <- change.Info{Reason: reason, Zid: zid}
		chci <- place.UpdateInfo{Reason: reason, Zid: zid}
	}
}

// NumEntries returns the number of managed zettel.
func (srv *notifyService) NumEntries() (int, error) {
	resChan := make(chan resNumEntries)
	srv.cmds <- &cmdNumEntries{resChan}

Changes to place/dirplace/notifydir/service.go.

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
13
14
15
16
17
18
19

20
21
22
23
24
25
26







-








import (
	"log"
	"time"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/place/dirplace/directory"
)

// ping sends every tick a signal to reload the directory list
func ping(tick chan<- struct{}, rescanTime time.Duration, done <-chan struct{}) {
	ticker := time.NewTicker(rescanTime)
	defer close(tick)
109
110
111
112
113
114
115
116

117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137

138
139
140
141
142
143
144
145
146

147
148
149
150
151
152
153
108
109
110
111
112
113
114

115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135

136
137
138
139
140
141
142
143
144

145
146
147
148
149
150
151
152







-
+




















-
+








-
+







				curMap = newMap
				newMap = nil
				if ready != nil {
					ready <- len(curMap)
					close(ready)
					ready = nil
				}
				srv.notifyChange(change.OnReload, id.Invalid)
				srv.notifyChange(place.OnReload, id.Invalid)
			case fileStatusError:
				log.Println("DIRPLACE", "ERROR", ev.err)
			case fileStatusUpdate:
				srv.processFileUpdateEvent(ev, curMap, newMap)
			case fileStatusDelete:
				srv.processFileDeleteEvent(ev, curMap, newMap)
			}
		case cmd, ok := <-srv.cmds:
			if ok {
				cmd.run(curMap)
			}
		}
	}
}

func (srv *notifyService) processFileUpdateEvent(ev *fileEvent, curMap, newMap dirMap) {
	if newMap != nil {
		dirMapUpdate(newMap, ev)
	} else {
		dirMapUpdate(curMap, ev)
		srv.notifyChange(change.OnUpdate, ev.zid)
		srv.notifyChange(place.OnUpdate, ev.zid)
	}
}

func (srv *notifyService) processFileDeleteEvent(ev *fileEvent, curMap, newMap dirMap) {
	if newMap != nil {
		deleteFromMap(newMap, ev)
	} else {
		deleteFromMap(curMap, ev)
		srv.notifyChange(change.OnDelete, ev.zid)
		srv.notifyChange(place.OnDelete, ev.zid)
	}
}

type dirCmd interface {
	run(m dirMap)
}

Changes to place/fileplace/fileplace.go.

26
27
28
29
30
31
32
33

34
35
36
37
38
39
40
26
27
28
29
30
31
32

33
34
35
36
37
38
39
40







-
+







func init() {
	manager.Register("file", func(u *url.URL, cdata *manager.ConnectData) (place.ManagedPlace, error) {
		path := getFilepathFromURL(u)
		ext := strings.ToLower(filepath.Ext(path))
		if ext != ".zip" {
			return nil, errors.New("unknown extension '" + ext + "' in place URL: " + u.String())
		}
		return &zipPlace{name: path, filter: cdata.Filter}, nil
		return &zipPlace{name: path, enricher: cdata.Enricher}, nil
	})
}

func getFilepathFromURL(u *url.URL) string {
	name := u.Opaque
	if name == "" {
		name = u.Path

Changes to place/fileplace/zipplace.go.

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46



47
48
49
50
51
52
53
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42



43
44
45
46
47
48
49
50
51
52







-



















-
-
-
+
+
+







	"io"
	"regexp"
	"strings"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/input"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
)

var validFileName = regexp.MustCompile(`^(\d{14}).*(\.(.+))$`)

func matchValidFileName(name string) []string {
	return validFileName.FindStringSubmatch(name)
}

type zipEntry struct {
	metaName     string
	contentName  string
	contentExt   string // (normalized) file extension of zettel content
	metaInHeader bool
}

type zipPlace struct {
	name   string
	filter index.MetaFilter
	zettel map[id.Zid]*zipEntry // no lock needed, because read-only after creation
	name     string
	enricher place.Enricher
	zettel   map[id.Zid]*zipEntry // no lock needed, because read-only after creation
}

func (zp *zipPlace) Location() string {
	if strings.HasPrefix(zp.name, "/") {
		return "file://" + zp.name
	}
	return "file:" + zp.name
176
177
178
179
180
181
182
183

184
185
186
187
188
189
190
175
176
177
178
179
180
181

182
183
184
185
186
187
188
189







-
+







	}
	defer reader.Close()
	for zid, entry := range zp.zettel {
		m, err := readZipMeta(reader, zid, entry)
		if err != nil {
			continue
		}
		zp.filter.Enrich(ctx, m)
		zp.enricher.Enrich(ctx, m)
		if match(m) {
			res = append(res, m)
		}
	}
	return res, nil
}

213
214
215
216
217
218
219
220

221
222
223
224
225
226
227
212
213
214
215
216
217
218

219
220
221
222
223
224
225
226







-
+







func (zp *zipPlace) DeleteZettel(ctx context.Context, zid id.Zid) error {
	if _, ok := zp.zettel[zid]; ok {
		return place.ErrReadOnly
	}
	return place.ErrNotFound
}

func (zp *zipPlace) ReadStats(st *place.Stats) {
func (zp *zipPlace) ReadStats(st *place.ManagedPlaceStats) {
	st.ReadOnly = true
	st.Zettel = len(zp.zettel)
}

func readZipMeta(reader *zip.ReadCloser, zid id.Zid, entry *zipEntry) (m *meta.Meta, err error) {
	var inMeta bool
	if entry.metaInHeader {

Changes to place/helper.go.

29
30
31
32
33
34
35
36

37
29
30
31
32
33
34
35

36
37







-
+

		if found {
			return zid, nil
		}
		// TODO: do not wait here unconditionally.
		time.Sleep(100 * time.Millisecond)
		withSeconds = true
	}
	return id.Invalid, ErrTimeout
	return id.Invalid, ErrConflict
}

Added place/manager/anteroom.go.




























































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package manager coordinates the various places and indexes of a Zettelstore.
package manager

import (
	"sync"

	"zettelstore.de/z/domain/id"
)

type arAction int

const (
	arNothing arAction = iota
	arReload
	arUpdate
	arDelete
)

type anteroom struct {
	next    *anteroom
	waiting map[id.Zid]arAction
	curLoad int
	reload  bool
}

type anterooms struct {
	mx      sync.Mutex
	first   *anteroom
	last    *anteroom
	maxLoad int
}

func newAnterooms(maxLoad int) *anterooms {
	return &anterooms{maxLoad: maxLoad}
}

func (ar *anterooms) Enqueue(zid id.Zid, action arAction) {
	if !zid.IsValid() || action == arNothing || action == arReload {
		return
	}
	ar.mx.Lock()
	defer ar.mx.Unlock()
	if ar.first == nil {
		ar.first = ar.makeAnteroom(zid, action)
		ar.last = ar.first
		return
	}
	for room := ar.first; room != nil; room = room.next {
		if room.reload {
			continue // Do not place zettel in reload room
		}
		a, ok := room.waiting[zid]
		if !ok {
			continue
		}
		switch action {
		case a:
			return
		case arUpdate:
			room.waiting[zid] = action
		case arDelete:
			room.waiting[zid] = action
		}
		return
	}
	if room := ar.last; !room.reload && (ar.maxLoad == 0 || room.curLoad < ar.maxLoad) {
		room.waiting[zid] = action
		room.curLoad++
		return
	}
	room := ar.makeAnteroom(zid, action)
	ar.last.next = room
	ar.last = room
}

func (ar *anterooms) makeAnteroom(zid id.Zid, action arAction) *anteroom {
	c := ar.maxLoad
	if c == 0 {
		c = 100
	}
	waiting := make(map[id.Zid]arAction, c)
	waiting[zid] = action
	return &anteroom{next: nil, waiting: waiting, curLoad: 1, reload: false}
}

func (ar *anterooms) Reset() {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	ar.first = ar.makeAnteroom(id.Invalid, arReload)
	ar.last = ar.first
}

func (ar *anterooms) Reload(delZids id.Slice, newZids id.Set) {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	delWaiting := createWaitingSlice(delZids, arDelete)
	newWaiting := createWaitingSet(newZids, arUpdate)
	ar.deleteReloadedRooms()

	if ds := len(delWaiting); ds > 0 {
		if ns := len(newWaiting); ns > 0 {
			roomNew := &anteroom{next: ar.first, waiting: newWaiting, curLoad: ns, reload: true}
			ar.first = &anteroom{next: roomNew, waiting: delWaiting, curLoad: ds, reload: true}
			if roomNew.next == nil {
				ar.last = roomNew
			}
			return
		}

		ar.first = &anteroom{next: ar.first, waiting: delWaiting, curLoad: ds}
		if ar.first.next == nil {
			ar.last = ar.first
		}
		return
	}

	if ns := len(newWaiting); ns > 0 {
		ar.first = &anteroom{next: ar.first, waiting: newWaiting, curLoad: ns}
		if ar.first.next == nil {
			ar.last = ar.first
		}
		return
	}

	ar.first = nil
	ar.last = nil
}

func createWaitingSlice(zids id.Slice, action arAction) map[id.Zid]arAction {
	waitingSet := make(map[id.Zid]arAction, len(zids))
	for _, zid := range zids {
		if zid.IsValid() {
			waitingSet[zid] = action
		}
	}
	return waitingSet
}

func createWaitingSet(zids id.Set, action arAction) map[id.Zid]arAction {
	waitingSet := make(map[id.Zid]arAction, len(zids))
	for zid := range zids {
		if zid.IsValid() {
			waitingSet[zid] = action
		}
	}
	return waitingSet
}

func (ar *anterooms) deleteReloadedRooms() {
	room := ar.first
	for room != nil && room.reload {
		room = room.next
	}
	ar.first = room
	if room == nil {
		ar.last = nil
	}
}

func (ar *anterooms) Dequeue() (arAction, id.Zid) {
	ar.mx.Lock()
	defer ar.mx.Unlock()
	if ar.first == nil {
		return arNothing, id.Invalid
	}
	for zid, action := range ar.first.waiting {
		delete(ar.first.waiting, zid)
		if len(ar.first.waiting) == 0 {
			ar.first = ar.first.next
			if ar.first == nil {
				ar.last = nil
			}
		}
		return action, zid
	}
	return arNothing, id.Invalid
}

Added place/manager/anteroom_test.go.



























































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package manager coordinates the various places and indexes of a Zettelstore.
package manager

import (
	"testing"

	"zettelstore.de/z/domain/id"
)

func TestSimple(t *testing.T) {
	ar := newAnterooms(2)
	ar.Enqueue(id.Zid(1), arUpdate)
	action, zid := ar.Dequeue()
	if zid != id.Zid(1) || action != arUpdate {
		t.Errorf("Expected 1/arUpdate, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if zid != id.Invalid && action != arDelete {
		t.Errorf("Expected invalid Zid, but got %v", zid)
	}
	ar.Enqueue(id.Zid(1), arUpdate)
	ar.Enqueue(id.Zid(2), arUpdate)
	if ar.first != ar.last {
		t.Errorf("Expected one room, but got more")
	}
	ar.Enqueue(id.Zid(3), arUpdate)
	if ar.first == ar.last {
		t.Errorf("Expected more than one room, but got only one")
	}

	count := 0
	for ; count < 1000; count++ {
		action, _ := ar.Dequeue()
		if action == arNothing {
			break
		}
	}
	if count != 3 {
		t.Errorf("Expected 3 dequeues, but got %v", count)
	}
}

func TestReset(t *testing.T) {
	ar := newAnterooms(1)
	ar.Enqueue(id.Zid(1), arUpdate)
	ar.Reset()
	action, zid := ar.Dequeue()
	if action != arReload || zid != id.Invalid {
		t.Errorf("Expected reload & invalid Zid, but got %v/%v", action, zid)
	}
	ar.Reload(id.Slice{2}, id.NewSet(3, 4))
	ar.Enqueue(id.Zid(5), arUpdate)
	ar.Enqueue(id.Zid(5), arDelete)
	ar.Enqueue(id.Zid(5), arDelete)
	ar.Enqueue(id.Zid(5), arUpdate)
	if ar.first == ar.last || ar.first.next == ar.last || ar.first.next.next != ar.last {
		t.Errorf("Expected 3 rooms")
	}
	action, zid = ar.Dequeue()
	if zid != id.Zid(2) || action != arDelete {
		t.Errorf("Expected 2/arDelete, but got %v/%v", zid, action)
	}
	action, zid1 := ar.Dequeue()
	if action != arUpdate {
		t.Errorf("Expected arUpdate, but got %v", action)
	}
	action, zid2 := ar.Dequeue()
	if action != arUpdate {
		t.Errorf("Expected arUpdate, but got %v", action)
	}
	if !(zid1 == id.Zid(3) && zid2 == id.Zid(4) || zid1 == id.Zid(4) && zid2 == id.Zid(3)) {
		t.Errorf("Zids must be 3 or 4, but got %v/%v", zid1, zid2)
	}
	action, zid = ar.Dequeue()
	if zid != id.Zid(5) || action != arUpdate {
		t.Errorf("Expected 5/arUpdate, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}

	ar = newAnterooms(1)
	ar.Reload(nil, id.NewSet(id.Zid(6)))
	action, zid = ar.Dequeue()
	if zid != id.Zid(6) || action != arUpdate {
		t.Errorf("Expected 6/arUpdate, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}

	ar = newAnterooms(1)
	ar.Reload(id.Slice{7}, nil)
	action, zid = ar.Dequeue()
	if zid != id.Zid(7) || action != arDelete {
		t.Errorf("Expected 7/arDelete, but got %v/%v", zid, action)
	}
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}

	ar = newAnterooms(1)
	ar.Enqueue(id.Zid(8), arUpdate)
	ar.Reload(nil, nil)
	action, zid = ar.Dequeue()
	if action != arNothing || zid != id.Invalid {
		t.Errorf("Expected nothing & invalid Zid, but got %v/%v", action, zid)
	}
}

Added place/manager/collect.go.


















































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package manager coordinates the various places and indexes of a Zettelstore.
package manager

import (
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/place/manager/store"
	"zettelstore.de/z/strfun"
)

type collectData struct {
	refs  id.Set
	words store.WordSet
	urls  store.WordSet
}

func (data *collectData) initialize() {
	data.refs = id.NewSet()
	data.words = store.NewWordSet()
	data.urls = store.NewWordSet()
}

func collectZettelIndexData(zn *ast.ZettelNode, data *collectData) {
	ast.NewTopDownTraverser(data).VisitBlockSlice(zn.Ast)
}

func collectInlineIndexData(ins ast.InlineSlice, data *collectData) {
	ast.NewTopDownTraverser(data).VisitInlineSlice(ins)
}

// VisitVerbatim collects the verbatim text in the word set.
func (data *collectData) VisitVerbatim(vn *ast.VerbatimNode) {
	for _, line := range vn.Lines {
		data.addText(line)
	}
}

// VisitRegion does nothing.
func (data *collectData) VisitRegion(rn *ast.RegionNode) {}

// VisitHeading does nothing.
func (data *collectData) VisitHeading(hn *ast.HeadingNode) {}

// VisitHRule does nothing.
func (data *collectData) VisitHRule(hn *ast.HRuleNode) {}

// VisitList does nothing.
func (data *collectData) VisitNestedList(ln *ast.NestedListNode) {}

// VisitDescriptionList does nothing.
func (data *collectData) VisitDescriptionList(dn *ast.DescriptionListNode) {}

// VisitPara does nothing.
func (data *collectData) VisitPara(pn *ast.ParaNode) {}

// VisitTable does nothing.
func (data *collectData) VisitTable(tn *ast.TableNode) {}

// VisitBLOB does nothing.
func (data *collectData) VisitBLOB(bn *ast.BLOBNode) {}

// VisitText collects the text in the word set.
func (data *collectData) VisitText(tn *ast.TextNode) {
	data.addText(tn.Text)
}

// VisitTag collects the tag name in the word set.
func (data *collectData) VisitTag(tn *ast.TagNode) {
	data.addText(tn.Tag)
}

// VisitSpace does nothing.
func (data *collectData) VisitSpace(sn *ast.SpaceNode) {}

// VisitBreak does nothing.
func (data *collectData) VisitBreak(bn *ast.BreakNode) {}

// VisitLink collects the given link as a reference.
func (data *collectData) VisitLink(ln *ast.LinkNode) {
	ref := ln.Ref
	if ref == nil {
		return
	}
	if ref.IsExternal() {
		data.urls.Add(strings.ToLower(ref.Value))
	}
	if !ref.IsZettel() {
		return
	}
	if zid, err := id.Parse(ref.URL.Path); err == nil {
		data.refs[zid] = true
	}
}

// VisitImage collects the image links as a reference.
func (data *collectData) VisitImage(in *ast.ImageNode) {
	ref := in.Ref
	if ref == nil {
		return
	}
	if ref.IsExternal() {
		data.urls.Add(strings.ToLower(ref.Value))
	}
	if !ref.IsZettel() {
		return
	}
	if zid, err := id.Parse(ref.URL.Path); err == nil {
		data.refs[zid] = true
	}
}

// VisitCite does nothing.
func (data *collectData) VisitCite(cn *ast.CiteNode) {}

// VisitFootnote does nothing.
func (data *collectData) VisitFootnote(fn *ast.FootnoteNode) {}

// VisitMark does nothing.
func (data *collectData) VisitMark(mn *ast.MarkNode) {}

// VisitFormat does nothing.
func (data *collectData) VisitFormat(fn *ast.FormatNode) {}

// VisitLiteral collects the literal words in the word set.
func (data *collectData) VisitLiteral(ln *ast.LiteralNode) {
	data.addText(ln.Text)
}

func (data *collectData) addText(s string) {
	for _, word := range strfun.NormalizeWords(s) {
		data.words.Add(word)
	}
}

Added place/manager/enrich.go.



















































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package manager coordinates the various places and indexes of a Zettelstore.
package manager

import (
	"context"

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
)

// Enrich computes additional properties and updates the given metadata.
func (mgr *Manager) Enrich(ctx context.Context, m *meta.Meta) {
	if place.DoNotEnrich(ctx) {
		// Enrich is called indirectly via indexer or enrichment is not requested
		// because of other reasons -> ignore this call, do not update meta data
		return
	}
	computePublished(m)
	mgr.idxStore.Enrich(ctx, m)
}

func computePublished(m *meta.Meta) {
	if _, ok := m.Get(meta.KeyPublished); ok {
		return
	}
	if modified, ok := m.Get(meta.KeyModified); ok {
		if _, ok = meta.TimeValue(modified); ok {
			m.Set(meta.KeyPublished, modified)
			return
		}
	}
	zid := m.Zid.String()
	if _, ok := meta.TimeValue(zid); ok {
		m.Set(meta.KeyPublished, zid)
		return
	}

	// Neither the zettel was modified nor the zettel identifer contains a valid
	// timestamp. In this case do not set the "published" property.
}

Added place/manager/indexer.go.




























































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package manager coordinates the various places and indexes of a Zettelstore.
package manager

import (
	"context"
	"net/url"
	"time"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/manager/store"
	"zettelstore.de/z/strfun"
)

// SelectEqual all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SelectEqual(word string) id.Set {
	return mgr.idxStore.SelectEqual(word)
}

// SelectPrefix all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SelectPrefix(prefix string) id.Set {
	return mgr.idxStore.SelectPrefix(prefix)
}

// SelectSuffix all zettel that have a word with the given suffix.
// The suffix must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SelectSuffix(suffix string) id.Set {
	return mgr.idxStore.SelectSuffix(suffix)
}

// SelectContains all zettel that contains the given string.
// The string must be normalized through Unicode NKFD, trimmed and not empty.
func (mgr *Manager) SelectContains(s string) id.Set {
	return mgr.idxStore.SelectContains(s)
}

// idxIndexer runs in the background and updates the index data structures.
// This is the main service of the idxIndexer.
func (mgr *Manager) idxIndexer() {
	// Something may panic. Ensure a running indexer.
	defer func() {
		if r := recover(); r != nil {
			kernel.Main.LogRecover("Indexer", r)
			go mgr.idxIndexer()
		}
	}()

	timerDuration := 15 * time.Second
	timer := time.NewTimer(timerDuration)
	ctx := place.NoEnrichContext(context.Background())
	for {
		start := time.Now()
		if mgr.idxWorkService(ctx) {
			mgr.idxMx.Lock()
			mgr.idxDurLastIndex = time.Since(start)
			mgr.idxMx.Unlock()
		}
		if !mgr.idxSleepService(timer, timerDuration) {
			return
		}
	}
}

func (mgr *Manager) idxWorkService(ctx context.Context) bool {
	changed := false
	for {
		switch action, zid := mgr.idxAr.Dequeue(); action {
		case arNothing:
			return changed
		case arReload:
			zids, err := mgr.FetchZids(ctx)
			if err == nil {
				mgr.idxAr.Reload(nil, zids)
				mgr.idxMx.Lock()
				mgr.idxLastReload = time.Now()
				mgr.idxSinceReload = 0
				mgr.idxMx.Unlock()
			}
		case arUpdate:
			changed = true
			mgr.idxMx.Lock()
			mgr.idxSinceReload++
			mgr.idxMx.Unlock()
			zettel, err := mgr.GetZettel(ctx, zid)
			if err != nil {
				// TODO: on some errors put the zid into a "try later" set
				continue
			}
			mgr.idxUpdateZettel(ctx, zettel)
		case arDelete:
			changed = true
			mgr.idxMx.Lock()
			mgr.idxSinceReload++
			mgr.idxMx.Unlock()
			mgr.idxDeleteZettel(zid)
		}
	}
}

func (mgr *Manager) idxSleepService(timer *time.Timer, timerDuration time.Duration) bool {
	select {
	case _, ok := <-mgr.idxReady:
		if !ok {
			return false
		}
	case _, ok := <-timer.C:
		if !ok {
			return false
		}
		timer.Reset(timerDuration)
	case <-mgr.done:
		if !timer.Stop() {
			<-timer.C
		}
		return false
	}
	return true
}

func (mgr *Manager) idxUpdateZettel(ctx context.Context, zettel domain.Zettel) {
	m := zettel.Meta
	if m.GetBool(meta.KeyNoIndex) {
		// Zettel maybe in index
		toCheck := mgr.idxStore.DeleteZettel(ctx, m.Zid)
		mgr.idxCheckZettel(toCheck)
		return
	}

	var cData collectData
	cData.initialize()
	collectZettelIndexData(parser.ParseZettel(zettel, "", mgr.rtConfig), &cData)
	zi := store.NewZettelIndex(m.Zid)
	mgr.idxCollectFromMeta(ctx, m, zi, &cData)
	mgr.idxProcessData(ctx, zi, &cData)
	toCheck := mgr.idxStore.UpdateReferences(ctx, zi)
	mgr.idxCheckZettel(toCheck)
}

func (mgr *Manager) idxCollectFromMeta(ctx context.Context, m *meta.Meta, zi *store.ZettelIndex, cData *collectData) {
	for _, pair := range m.Pairs(false) {
		descr := meta.GetDescription(pair.Key)
		if descr.IsComputed() {
			continue
		}
		switch descr.Type {
		case meta.TypeID:
			mgr.idxUpdateValue(ctx, descr.Inverse, pair.Value, zi)
		case meta.TypeIDSet:
			for _, val := range meta.ListFromValue(pair.Value) {
				mgr.idxUpdateValue(ctx, descr.Inverse, val, zi)
			}
		case meta.TypeZettelmarkup:
			collectInlineIndexData(parser.ParseMetadata(pair.Value), cData)
		case meta.TypeURL:
			if _, err := url.Parse(pair.Value); err == nil {
				cData.urls.Add(pair.Value)
			}
		default:
			for _, word := range strfun.NormalizeWords(pair.Value) {
				cData.words.Add(word)
			}
		}
	}
}

func (mgr *Manager) idxProcessData(ctx context.Context, zi *store.ZettelIndex, cData *collectData) {
	for ref := range cData.refs {
		if _, err := mgr.GetMeta(ctx, ref); err == nil {
			zi.AddBackRef(ref)
		} else {
			zi.AddDeadRef(ref)
		}
	}
	zi.SetWords(cData.words)
	zi.SetUrls(cData.urls)
}

func (mgr *Manager) idxUpdateValue(ctx context.Context, inverse string, value string, zi *store.ZettelIndex) {
	zid, err := id.Parse(value)
	if err != nil {
		return
	}
	if _, err := mgr.GetMeta(ctx, zid); err != nil {
		zi.AddDeadRef(zid)
		return
	}
	if inverse == "" {
		zi.AddBackRef(zid)
		return
	}
	zi.AddMetaRef(inverse, zid)
}

func (mgr *Manager) idxDeleteZettel(zid id.Zid) {
	toCheck := mgr.idxStore.DeleteZettel(context.Background(), zid)
	mgr.idxCheckZettel(toCheck)
}

func (mgr *Manager) idxCheckZettel(s id.Set) {
	for zid := range s {
		mgr.idxAr.Enqueue(zid, arUpdate)
	}
}

Changes to place/manager/manager.go.

1
2
3
4
5
6
7
8
9
10
11

12
13
14
15

16
17
18
19
20
21

22
23
24


25
26
27

28
29
30


31
32
33
34

35
36


37
38
39
40

41
42

43
44
45

46
47
48
49
50
51
52
53
54
55

56
57
58
59
60
61
62
1
2
3
4
5
6
7
8
9
10

11
12
13
14
15
16
17
18

19

20
21
22


23
24
25
26

27
28


29
30
31
32
33
34
35


36
37
38
39
40

41


42



43



44
45
46
47
48
49
50
51
52
53
54
55
56
57
58










-
+




+


-

-

+

-
-
+
+


-
+

-
-
+
+




+
-
-
+
+



-
+
-
-
+
-
-
-
+
-
-
-







+







//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package manager coordinates the various places of a Zettelstore.
// Package manager coordinates the various places and indexes of a Zettelstore.
package manager

import (
	"context"
	"io"
	"log"
	"net/url"
	"runtime/debug"
	"sort"
	"strings"
	"sync"
	"time"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/search"
	"zettelstore.de/z/place/manager/memstore"
	"zettelstore.de/z/place/manager/store"
)

// ConnectData contains all administration related values.
type ConnectData struct {
	Config   config.Config
	Filter index.MetaFilter
	Notify chan<- change.Info
	Enricher place.Enricher
	Notify   chan<- place.UpdateInfo
}

// Connect returns a handle to the specified place
func Connect(rawURL string, readonlyMode bool, cdata *ConnectData) (place.ManagedPlace, error) {
func Connect(u *url.URL, authManager auth.BaseManager, cdata *ConnectData) (place.ManagedPlace, error) {
	u, err := url.Parse(rawURL)
	if err != nil {
	if authManager.IsReadonly() {
		return nil, err
	}
	if u.Scheme == "" {
		rawURL := u.String()
		u.Scheme = "dir"
	}
	if readonlyMode {
		// TODO: the following is wrong under some circumstances:
		// 1. fragment is set
		if q := u.Query(); len(q) == 0 {
			rawURL += "?readonly"
		} else if _, ok := q["readonly"]; !ok {
			rawURL += "&readonly"
		}
		var err error
		if u, err = url.Parse(rawURL); err != nil {
			return nil, err
		}
	}

	if create, ok := registry[u.Scheme]; ok {
		return create(u, cdata)
89
90
91
92
93
94
95
96
97
98




99
100
101
102
103
104










105







106
107







108



109
110




111
112

113
114
115

116
117
118
119
120
121
122
85
86
87
88
89
90
91



92
93
94
95






96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

115
116
117
118
119
120
121
122
123
124
125


126
127
128
129
130

131
132
133

134
135
136
137
138
139
140
141







-
-
-
+
+
+
+
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+

+
+
+
+
+
+
+

-
+
+
+
+
+
+
+

+
+
+
-
-
+
+
+
+

-
+


-
+







	}
	sort.Strings(result)
	return result
}

// Manager is a coordinating place.
type Manager struct {
	mx         sync.RWMutex
	started    bool
	subplaces  []place.ManagedPlace
	mgrMx        sync.RWMutex
	started      bool
	rtConfig     config.Config
	subplaces    []place.ManagedPlace
	filter     index.MetaFilter
	observers  []change.Func
	mxObserver sync.RWMutex
	done       chan struct{}
	infos      chan change.Info
}
	observers    []place.UpdateFunc
	mxObserver   sync.RWMutex
	done         chan struct{}
	infos        chan place.UpdateInfo
	propertyKeys map[string]bool // Set of property key names

	// Indexer data
	idxStore store.Store
	idxAr    *anterooms
	idxReady chan struct{} // Signal a non-empty anteroom to background task

	// Indexer stats data
	idxMx           sync.RWMutex
	idxLastReload   time.Time
	idxSinceReload  uint64
	idxDurLastIndex time.Duration
}

// New creates a new managing place.
func New(placeURIs []string, readonlyMode bool, filter index.MetaFilter) (*Manager, error) {
func New(placeURIs []*url.URL, authManager auth.BaseManager, rtConfig config.Config) (*Manager, error) {
	propertyKeys := make(map[string]bool)
	for _, kd := range meta.GetSortedKeyDescriptions() {
		if kd.IsProperty() {
			propertyKeys[kd.Name] = true
		}
	}
	mgr := &Manager{
		rtConfig:     rtConfig,
		infos:        make(chan place.UpdateInfo, len(placeURIs)*10),
		propertyKeys: propertyKeys,
		filter: filter,
		infos:  make(chan change.Info, len(placeURIs)*10),

		idxStore: memstore.New(),
		idxAr:    newAnterooms(10),
		idxReady: make(chan struct{}, 1),
	}
	cdata := ConnectData{Filter: filter, Notify: mgr.infos}
	cdata := ConnectData{Config: rtConfig, Enricher: mgr, Notify: mgr.infos}
	subplaces := make([]place.ManagedPlace, 0, len(placeURIs)+2)
	for _, uri := range placeURIs {
		p, err := Connect(uri, readonlyMode, &cdata)
		p, err := Connect(uri, authManager, &cdata)
		if err != nil {
			return nil, err
		}
		if p != nil {
			subplaces = append(subplaces, p)
		}
	}
131
132
133
134
135
136
137
138

139
140
141
142
143
144
145
146

147
148
149
150
151

152
153
154
155

156
157
158
159

160
161

162
163
164
165
166
167

168




169

170
171

172
173

174
175
176
177
178
179
180
181
182










183
184
185
186



187
188

189
190
191
192
193
194
195
196
197

198
199

200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216

217
218

219


220


221
222
223


224
225
226
227
228
229
230


231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440


441

442
443
444
445
446
447
448
449
450
451
452
453

454
455






456








457
458





150
151
152
153
154
155
156

157
158
159
160
161
162
163
164

165
166
167
168
169

170
171
172
173

174
175
176
177

178


179
180
181
182
183
184

185
186
187
188
189
190

191
192

193


194

195
196
197
198




199
200
201
202
203
204
205
206
207
208
209



210
211
212


213



214
215
216
217
218

219
220

221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237

238
239
240
241
242
243
244

245
246
247


248
249
250
251
252
253
254


255
256
257
258
259
260

261
262
263
264
265
266
267
268
269
270
271
272































































































































































































273
274
275
276

277
278
279
280
281
282
283
284
285
286
287
288
289
290


291
292
293
294
295
296
297
298
299
300
301
302
303
304
305


306
307
308
309
310







-
+







-
+




-
+



-
+



-
+
-
-
+





-
+

+
+
+
+
-
+

-
+
-
-
+
-




-
-
-
-
+
+
+
+
+
+
+
+
+
+

-
-
-
+
+
+
-
-
+
-
-
-





-
+

-
+
















-
+


+

+
+
-
+
+

-
-
+
+





-
-
+
+




-












-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-


+
+
-
+












+
-
-
+
+
+
+
+
+

+
+
+
+
+
+
+
+
-
-
+
+
+
+
+
	subplaces = append(subplaces, constplace, progplace)
	mgr.subplaces = subplaces
	return mgr, nil
}

// RegisterObserver registers an observer that will be notified
// if a zettel was found to be changed.
func (mgr *Manager) RegisterObserver(f change.Func) {
func (mgr *Manager) RegisterObserver(f place.UpdateFunc) {
	if f != nil {
		mgr.mxObserver.Lock()
		mgr.observers = append(mgr.observers, f)
		mgr.mxObserver.Unlock()
	}
}

func (mgr *Manager) notifyObserver(ci change.Info) {
func (mgr *Manager) notifyObserver(ci *place.UpdateInfo) {
	mgr.mxObserver.RLock()
	observers := mgr.observers
	mgr.mxObserver.RUnlock()
	for _, ob := range observers {
		ob(ci)
		ob(*ci)
	}
}

func notifier(notify change.Func, infos <-chan change.Info, done <-chan struct{}) {
func (mgr *Manager) notifier() {
	// The call to notify may panic. Ensure a running notifier.
	defer func() {
		if r := recover(); r != nil {
			log.Println("recovered from:", r)
			kernel.Main.LogRecover("Notifier", r)
			debug.PrintStack()
			go notifier(notify, infos, done)
			go mgr.notifier()
		}
	}()

	for {
		select {
		case ci, ok := <-infos:
		case ci, ok := <-mgr.infos:
			if ok {
				mgr.idxEnqueue(ci.Reason, ci.Zid)
				if ci.Place == nil {
					ci.Place = mgr
				}
				notify(ci)
				mgr.notifyObserver(&ci)
			}
		case _, ok := <-done:
		case <-mgr.done:
			if !ok {
				return
			return
			}
		}
	}
}

// Location returns some information where the place is located.
func (mgr *Manager) Location() string {
	if len(mgr.subplaces) < 2 {
		return mgr.subplaces[0].Location()
func (mgr *Manager) idxEnqueue(reason place.UpdateReason, zid id.Zid) {
	switch reason {
	case place.OnReload:
		mgr.idxAr.Reset()
	case place.OnUpdate:
		mgr.idxAr.Enqueue(zid, arUpdate)
	case place.OnDelete:
		mgr.idxAr.Enqueue(zid, arDelete)
	default:
		return
	}
	var sb strings.Builder
	for i := 0; i < len(mgr.subplaces)-2; i++ {
		if i > 0 {
	select {
	case mgr.idxReady <- struct{}{}:
	default:
			sb.WriteString(", ")
		}
	}
		sb.WriteString(mgr.subplaces[i].Location())
	}
	return sb.String()
}

// Start the place. Now all other functions of the place are allowed.
// Starting an already started place is not allowed.
func (mgr *Manager) Start(ctx context.Context) error {
	mgr.mx.Lock()
	mgr.mgrMx.Lock()
	if mgr.started {
		mgr.mx.Unlock()
		mgr.mgrMx.Unlock()
		return place.ErrStarted
	}
	for i := len(mgr.subplaces) - 1; i >= 0; i-- {
		ssi, ok := mgr.subplaces[i].(place.StartStopper)
		if !ok {
			continue
		}
		err := ssi.Start(ctx)
		if err == nil {
			continue
		}
		for j := i + 1; j < len(mgr.subplaces); j++ {
			if ssj, ok := mgr.subplaces[j].(place.StartStopper); ok {
				ssj.Stop(ctx)
			}
		}
		mgr.mx.Unlock()
		mgr.mgrMx.Unlock()
		return err
	}
	mgr.idxAr.Reset() // Ensure an initial index run
	mgr.done = make(chan struct{})
	go mgr.notifier()
	go mgr.idxIndexer()
	go notifier(mgr.notifyObserver, mgr.infos, mgr.done)

	// mgr.startIndexer(mgr)
	mgr.started = true
	mgr.mx.Unlock()
	mgr.infos <- change.Info{Reason: change.OnReload, Zid: id.Invalid}
	mgr.mgrMx.Unlock()
	mgr.infos <- place.UpdateInfo{Reason: place.OnReload, Zid: id.Invalid}
	return nil
}

// Stop the started place. Now only the Start() function is allowed.
func (mgr *Manager) Stop(ctx context.Context) error {
	mgr.mx.Lock()
	defer mgr.mx.Unlock()
	mgr.mgrMx.Lock()
	defer mgr.mgrMx.Unlock()
	if !mgr.started {
		return place.ErrStopped
	}
	close(mgr.done)
	mgr.done = nil
	var err error
	for _, p := range mgr.subplaces {
		if ss, ok := p.(place.StartStopper); ok {
			if err1 := ss.Stop(ctx); err1 != nil && err == nil {
				err = err1
			}
		}
	}
	mgr.started = false
	return err
}

// CanCreateZettel returns true, if place could possibly create a new zettel.
func (mgr *Manager) CanCreateZettel(ctx context.Context) bool {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	return mgr.started && mgr.subplaces[0].CanCreateZettel(ctx)
}

// CreateZettel creates a new zettel.
func (mgr *Manager) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return id.Invalid, place.ErrStopped
	}
	return mgr.subplaces[0].CreateZettel(ctx, zettel)
}

// GetZettel retrieves a specific zettel.
func (mgr *Manager) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return domain.Zettel{}, place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		if z, err := p.GetZettel(ctx, zid); err != place.ErrNotFound {
			if err == nil {
				mgr.filter.Enrich(ctx, z.Meta)
			}
			return z, err
		}
	}
	return domain.Zettel{}, place.ErrNotFound
}

// GetMeta retrieves just the meta data of a specific zettel.
func (mgr *Manager) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return nil, place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		if m, err := p.GetMeta(ctx, zid); err != place.ErrNotFound {
			if err == nil {
				mgr.filter.Enrich(ctx, m)
			}
			return m, err
		}
	}
	return nil, place.ErrNotFound
}

// FetchZids returns the set of all zettel identifer managed by the place.
func (mgr *Manager) FetchZids(ctx context.Context) (result id.Set, err error) {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return nil, place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		zids, err := p.FetchZids(ctx)
		if err != nil {
			return nil, err
		}
		if result == nil {
			result = zids
		} else if len(result) <= len(zids) {
			for zid := range result {
				zids[zid] = true
			}
			result = zids
		} else {
			for zid := range zids {
				result[zid] = true
			}
		}
	}
	return result, nil
}

// SelectMeta returns all zettel meta data that match the selection
// criteria. The result is ordered by descending zettel id.
func (mgr *Manager) SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return nil, place.ErrStopped
	}
	var result []*meta.Meta
	match := s.CompileMatch(startup.Indexer())
	for _, p := range mgr.subplaces {
		selected, err := p.SelectMeta(ctx, match)
		if err != nil {
			return nil, err
		}
		sort.Slice(selected, func(i, j int) bool { return selected[i].Zid > selected[j].Zid })
		if len(result) == 0 {
			result = selected
		} else {
			result = place.MergeSorted(result, selected)
		}
	}
	if s == nil {
		return result, nil
	}
	return s.Sort(result), nil
}

// CanUpdateZettel returns true, if place could possibly update the given zettel.
func (mgr *Manager) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	return mgr.started && mgr.subplaces[0].CanUpdateZettel(ctx, zettel)
}

// UpdateZettel updates an existing zettel.
func (mgr *Manager) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return place.ErrStopped
	}
	zettel.Meta = zettel.Meta.Clone()
	mgr.filter.Remove(ctx, zettel.Meta)
	return mgr.subplaces[0].UpdateZettel(ctx, zettel)
}

// AllowRenameZettel returns true, if place will not disallow renaming the zettel.
func (mgr *Manager) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return false
	}
	for _, p := range mgr.subplaces {
		if !p.AllowRenameZettel(ctx, zid) {
			return false
		}
	}
	return true
}

// RenameZettel changes the current zid to a new zid.
func (mgr *Manager) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return place.ErrStopped
	}
	for i, p := range mgr.subplaces {
		if err := p.RenameZettel(ctx, curZid, newZid); err != nil && err != place.ErrNotFound {
			for j := 0; j < i; j++ {
				mgr.subplaces[j].RenameZettel(ctx, newZid, curZid)
			}
			return err
		}
	}
	return nil
}

// CanDeleteZettel returns true, if place could possibly delete the given zettel.
func (mgr *Manager) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return false
	}
	for _, p := range mgr.subplaces {
		if p.CanDeleteZettel(ctx, zid) {
			return true
		}
	}
	return false
}

// DeleteZettel removes the zettel from the place.
func (mgr *Manager) DeleteZettel(ctx context.Context, zid id.Zid) error {
	mgr.mx.RLock()
	defer mgr.mx.RUnlock()
	if !mgr.started {
		return place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		if err := p.DeleteZettel(ctx, zid); err != place.ErrNotFound && err != place.ErrReadOnly {
			return err
		}
	}
	return place.ErrNotFound
}

// ReadStats populates st with place statistics
func (mgr *Manager) ReadStats(st *place.Stats) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	subStats := make([]place.Stats, len(mgr.subplaces))
	subStats := make([]place.ManagedPlaceStats, len(mgr.subplaces))
	for i, p := range mgr.subplaces {
		p.ReadStats(&subStats[i])
	}

	st.ReadOnly = true
	sumZettel := 0
	for _, sst := range subStats {
		if !sst.ReadOnly {
			st.ReadOnly = false
		}
		sumZettel += sst.Zettel
	}
	st.NumManagedPlaces = len(mgr.subplaces)
	st.Zettel = sumZettel
}
	st.ZettelTotal = sumZettel

	var storeSt store.Stats
	mgr.idxMx.RLock()
	defer mgr.idxMx.RUnlock()
	mgr.idxStore.ReadStats(&storeSt)

	st.LastReload = mgr.idxLastReload
	st.IndexesSinceReload = mgr.idxSinceReload
	st.DurLastIndex = mgr.idxDurLastIndex
	st.ZettelIndexed = storeSt.Zettel
	st.IndexUpdates = storeSt.Updates
	st.IndexedWords = storeSt.Words
	st.IndexedUrls = storeSt.Urls
}
// NumPlaces returns the number of managed places.
func (mgr *Manager) NumPlaces() int { return len(mgr.subplaces) }

// Dump internal data structures to a Writer.
func (mgr *Manager) Dump(w io.Writer) {
	mgr.idxStore.Dump(w)
}

Added place/manager/memstore/memstore.go.





































































































































































































































































































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package memstore stored the index in main memory.
package memstore

import (
	"context"
	"fmt"
	"io"
	"sort"
	"strings"
	"sync"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place/manager/store"
)

type metaRefs struct {
	forward  id.Slice
	backward id.Slice
}

type zettelIndex struct {
	dead     id.Slice
	forward  id.Slice
	backward id.Slice
	meta     map[string]metaRefs
	words    []string
	urls     []string
}

func (zi *zettelIndex) isEmpty() bool {
	if len(zi.forward) > 0 || len(zi.backward) > 0 || len(zi.dead) > 0 || len(zi.words) > 0 {
		return false
	}
	return zi.meta == nil || len(zi.meta) == 0
}

type stringRefs map[string]id.Slice

type memStore struct {
	mx    sync.RWMutex
	idx   map[id.Zid]*zettelIndex
	dead  map[id.Zid]id.Slice // map dead refs where they occur
	words stringRefs
	urls  stringRefs

	// Stats
	updates uint64
}

// New returns a new memory-based index store.
func New() store.Store {
	return &memStore{
		idx:   make(map[id.Zid]*zettelIndex),
		dead:  make(map[id.Zid]id.Slice),
		words: make(stringRefs),
		urls:  make(stringRefs),
	}
}

func (ms *memStore) Enrich(ctx context.Context, m *meta.Meta) {
	if ms.doEnrich(ctx, m) {
		ms.mx.Lock()
		ms.updates++
		ms.mx.Unlock()
	}
}

func (ms *memStore) doEnrich(ctx context.Context, m *meta.Meta) bool {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	zi, ok := ms.idx[m.Zid]
	if !ok {
		return false
	}
	var updated bool
	if len(zi.dead) > 0 {
		m.Set(meta.KeyDead, zi.dead.String())
		updated = true
	}
	back := removeOtherMetaRefs(m, zi.backward.Copy())
	if len(zi.backward) > 0 {
		m.Set(meta.KeyBackward, zi.backward.String())
		updated = true
	}
	if len(zi.forward) > 0 {
		m.Set(meta.KeyForward, zi.forward.String())
		back = remRefs(back, zi.forward)
		updated = true
	}
	if len(zi.meta) > 0 {
		for k, refs := range zi.meta {
			if len(refs.backward) > 0 {
				m.Set(k, refs.backward.String())
				back = remRefs(back, refs.backward)
				updated = true
			}
		}
	}
	if len(back) > 0 {
		m.Set(meta.KeyBack, back.String())
		updated = true
	}
	return updated
}

// SelectEqual all zettel that contains the given exact word.
// The word must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SelectEqual(word string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := id.NewSet()
	if refs, ok := ms.words[word]; ok {
		result.AddSlice(refs)
	}
	if refs, ok := ms.urls[word]; ok {
		result.AddSlice(refs)
	}
	zid, err := id.Parse(word)
	if err != nil {
		return result
	}
	zi, ok := ms.idx[zid]
	if !ok {
		return result
	}

	addBackwardZids(result, zid, zi)
	return result
}

// Select all zettel that have a word with the given prefix.
// The prefix must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SelectPrefix(prefix string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(prefix, strings.HasPrefix)
	l := len(prefix)
	if l > 14 {
		return result
	}
	minZid, err := id.Parse(prefix + "00000000000000"[:14-l])
	if err != nil {
		return result
	}
	maxZid, err := id.Parse(prefix + "99999999999999"[:14-l])
	if err != nil {
		return result
	}
	for zid, zi := range ms.idx {
		if minZid <= zid && zid <= maxZid {
			addBackwardZids(result, zid, zi)
		}
	}
	return result
}

// Select all zettel that have a word with the given suffix.
// The suffix must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SelectSuffix(suffix string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(suffix, strings.HasSuffix)
	l := len(suffix)
	if l > 14 {
		return result
	}
	val, err := id.ParseUint(suffix)
	if err != nil {
		return result
	}
	modulo := uint64(1)
	for i := 0; i < l; i++ {
		modulo *= 10
	}
	for zid, zi := range ms.idx {
		if uint64(zid)%modulo == val {
			addBackwardZids(result, zid, zi)
		}
	}
	return result
}

// Select all zettel that contains the given string.
// The string must be normalized through Unicode NKFD, trimmed and not empty.
func (ms *memStore) SelectContains(s string) id.Set {
	ms.mx.RLock()
	defer ms.mx.RUnlock()
	result := ms.selectWithPred(s, strings.Contains)
	if len(s) > 14 {
		return result
	}
	if _, err := id.ParseUint(s); err != nil {
		return result
	}
	for zid, zi := range ms.idx {
		if strings.Contains(zid.String(), s) {
			addBackwardZids(result, zid, zi)
		}
	}
	return result
}

func (ms *memStore) selectWithPred(s string, pred func(string, string) bool) id.Set {
	// Must only be called if ms.mx is read-locked!
	result := id.NewSet()
	for word, refs := range ms.words {
		if !pred(word, s) {
			continue
		}
		result.AddSlice(refs)
	}
	for u, refs := range ms.urls {
		if !pred(u, s) {
			continue
		}
		result.AddSlice(refs)
	}
	return result
}

func addBackwardZids(result id.Set, zid id.Zid, zi *zettelIndex) {
	// Must only be called if ms.mx is read-locked!
	result[zid] = true
	result.AddSlice(zi.backward)
	for _, mref := range zi.meta {
		result.AddSlice(mref.backward)
	}
}

func removeOtherMetaRefs(m *meta.Meta, back id.Slice) id.Slice {
	for _, p := range m.PairsRest(false) {
		switch meta.Type(p.Key) {
		case meta.TypeID:
			if zid, err := id.Parse(p.Value); err == nil {
				back = remRef(back, zid)
			}
		case meta.TypeIDSet:
			for _, val := range meta.ListFromValue(p.Value) {
				if zid, err := id.Parse(val); err == nil {
					back = remRef(back, zid)
				}
			}
		}
	}
	return back
}

func (ms *memStore) UpdateReferences(ctx context.Context, zidx *store.ZettelIndex) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()
	zi, ziExist := ms.idx[zidx.Zid]
	if !ziExist || zi == nil {
		zi = &zettelIndex{}
		ziExist = false
	}

	// Is this zettel an old dead reference mentioned in other zettel?
	var toCheck id.Set
	if refs, ok := ms.dead[zidx.Zid]; ok {
		// These must be checked later again
		toCheck = id.NewSet(refs...)
		delete(ms.dead, zidx.Zid)
	}

	ms.updateDeadReferences(zidx, zi)
	ms.updateForwardBackwardReferences(zidx, zi)
	ms.updateMetadataReferences(zidx, zi)
	zi.words = updateWordSet(zidx.Zid, ms.words, zi.words, zidx.GetWords())
	zi.urls = updateWordSet(zidx.Zid, ms.urls, zi.urls, zidx.GetUrls())

	// Check if zi must be inserted into ms.idx
	if !ziExist && !zi.isEmpty() {
		ms.idx[zidx.Zid] = zi
	}

	return toCheck
}

func (ms *memStore) updateDeadReferences(zidx *store.ZettelIndex, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	drefs := zidx.GetDeadRefs()
	newRefs, remRefs := refsDiff(drefs, zi.dead)
	zi.dead = drefs
	for _, ref := range remRefs {
		ms.dead[ref] = remRef(ms.dead[ref], zidx.Zid)
	}
	for _, ref := range newRefs {
		ms.dead[ref] = addRef(ms.dead[ref], zidx.Zid)
	}
}

func (ms *memStore) updateForwardBackwardReferences(zidx *store.ZettelIndex, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	brefs := zidx.GetBackRefs()
	newRefs, remRefs := refsDiff(brefs, zi.forward)
	zi.forward = brefs
	for _, ref := range remRefs {
		bzi := ms.getEntry(ref)
		bzi.backward = remRef(bzi.backward, zidx.Zid)
	}
	for _, ref := range newRefs {
		bzi := ms.getEntry(ref)
		bzi.backward = addRef(bzi.backward, zidx.Zid)
	}
}

func (ms *memStore) updateMetadataReferences(zidx *store.ZettelIndex, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	metarefs := zidx.GetMetaRefs()
	for key, mr := range zi.meta {
		if _, ok := metarefs[key]; ok {
			continue
		}
		ms.removeInverseMeta(zidx.Zid, key, mr.forward)
	}
	if zi.meta == nil {
		zi.meta = make(map[string]metaRefs)
	}
	for key, mrefs := range metarefs {
		mr := zi.meta[key]
		newRefs, remRefs := refsDiff(mrefs, mr.forward)
		mr.forward = mrefs
		zi.meta[key] = mr

		for _, ref := range newRefs {
			bzi := ms.getEntry(ref)
			if bzi.meta == nil {
				bzi.meta = make(map[string]metaRefs)
			}
			bmr := bzi.meta[key]
			bmr.backward = addRef(bmr.backward, zidx.Zid)
			bzi.meta[key] = bmr
		}
		ms.removeInverseMeta(zidx.Zid, key, remRefs)
	}
}

func updateWordSet(zid id.Zid, srefs stringRefs, prev []string, next store.WordSet) []string {
	// Must only be called if ms.mx is write-locked!
	newWords, removeWords := next.Diff(prev)
	for _, word := range newWords {
		if refs, ok := srefs[word]; ok {
			srefs[word] = addRef(refs, zid)
			continue
		}
		srefs[word] = id.Slice{zid}
	}
	for _, word := range removeWords {
		refs, ok := srefs[word]
		if !ok {
			continue
		}
		refs2 := remRef(refs, zid)
		if len(refs2) == 0 {
			delete(srefs, word)
			continue
		}
		srefs[word] = refs2
	}
	return next.Words()
}

func (ms *memStore) getEntry(zid id.Zid) *zettelIndex {
	// Must only be called if ms.mx is write-locked!
	if zi, ok := ms.idx[zid]; ok {
		return zi
	}
	zi := &zettelIndex{}
	ms.idx[zid] = zi
	return zi
}

func (ms *memStore) DeleteZettel(ctx context.Context, zid id.Zid) id.Set {
	ms.mx.Lock()
	defer ms.mx.Unlock()

	zi, ok := ms.idx[zid]
	if !ok {
		return nil
	}

	ms.deleteDeadSources(zid, zi)
	toCheck := ms.deleteForwardBackward(zid, zi)
	if len(zi.meta) > 0 {
		for key, mrefs := range zi.meta {
			ms.removeInverseMeta(zid, key, mrefs.forward)
		}
	}
	ms.deleteWords(zid, zi.words)
	delete(ms.idx, zid)
	return toCheck
}

func (ms *memStore) deleteDeadSources(zid id.Zid, zi *zettelIndex) {
	// Must only be called if ms.mx is write-locked!
	for _, ref := range zi.dead {
		if drefs, ok := ms.dead[ref]; ok {
			drefs = remRef(drefs, zid)
			if len(drefs) > 0 {
				ms.dead[ref] = drefs
			} else {
				delete(ms.dead, ref)
			}
		}
	}
}

func (ms *memStore) deleteForwardBackward(zid id.Zid, zi *zettelIndex) id.Set {
	// Must only be called if ms.mx is write-locked!
	var toCheck id.Set
	for _, ref := range zi.forward {
		if fzi, ok := ms.idx[ref]; ok {
			fzi.backward = remRef(fzi.backward, zid)
		}
	}
	for _, ref := range zi.backward {
		if bzi, ok := ms.idx[ref]; ok {
			bzi.forward = remRef(bzi.forward, zid)
			if toCheck == nil {
				toCheck = id.NewSet()
			}
			toCheck[ref] = true
		}
	}
	return toCheck
}

func (ms *memStore) removeInverseMeta(zid id.Zid, key string, forward id.Slice) {
	// Must only be called if ms.mx is write-locked!
	for _, ref := range forward {
		bzi, ok := ms.idx[ref]
		if !ok || bzi.meta == nil {
			continue
		}
		bmr, ok := bzi.meta[key]
		if !ok {
			continue
		}
		bmr.backward = remRef(bmr.backward, zid)
		if len(bmr.backward) > 0 || len(bmr.forward) > 0 {
			bzi.meta[key] = bmr
		} else {
			delete(bzi.meta, key)
			if len(bzi.meta) == 0 {
				bzi.meta = nil
			}
		}
	}
}

func (ms *memStore) deleteWords(zid id.Zid, words []string) {
	// Must only be called if ms.mx is write-locked!
	for _, word := range words {
		refs, ok := ms.words[word]
		if !ok {
			continue
		}
		refs2 := remRef(refs, zid)
		if len(refs2) == 0 {
			delete(ms.words, word)
			continue
		}
		ms.words[word] = refs2
	}
}

func (ms *memStore) ReadStats(st *store.Stats) {
	ms.mx.RLock()
	st.Zettel = len(ms.idx)
	st.Updates = ms.updates
	st.Words = uint64(len(ms.words))
	st.Urls = uint64(len(ms.urls))
	ms.mx.RUnlock()
}

func (ms *memStore) Dump(w io.Writer) {
	ms.mx.RLock()
	defer ms.mx.RUnlock()

	io.WriteString(w, "=== Dump\n")
	ms.dumpIndex(w)
	ms.dumpDead(w)
	dumpStringRefs(w, "Words", "", "", ms.words)
	dumpStringRefs(w, "URLs", "[[", "]]", ms.urls)
}

func (ms *memStore) dumpIndex(w io.Writer) {
	if len(ms.idx) == 0 {
		return
	}
	io.WriteString(w, "==== Zettel Index\n")
	zids := make(id.Slice, 0, len(ms.idx))
	for id := range ms.idx {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, "=====", id)
		zi := ms.idx[id]
		if len(zi.dead) > 0 {
			fmt.Fprintln(w, "* Dead:", zi.dead)
		}
		dumpZids(w, "* Forward:", zi.forward)
		dumpZids(w, "* Backward:", zi.backward)
		for k, fb := range zi.meta {
			fmt.Fprintln(w, "* Meta", k)
			dumpZids(w, "** Forward:", fb.forward)
			dumpZids(w, "** Backward:", fb.backward)
		}
		dumpStrings(w, "* Words", "", "", zi.words)
		dumpStrings(w, "* URLs", "[[", "]]", zi.urls)
	}
}

func (ms *memStore) dumpDead(w io.Writer) {
	if len(ms.dead) == 0 {
		return
	}
	fmt.Fprintf(w, "==== Dead References\n")
	zids := make(id.Slice, 0, len(ms.dead))
	for id := range ms.dead {
		zids = append(zids, id)
	}
	zids.Sort()
	for _, id := range zids {
		fmt.Fprintln(w, ";", id)
		fmt.Fprintln(w, ":", ms.dead[id])
	}
}

func dumpZids(w io.Writer, prefix string, zids id.Slice) {
	if len(zids) > 0 {
		io.WriteString(w, prefix)
		for _, zid := range zids {
			io.WriteString(w, " ")
			w.Write(zid.Bytes())
		}
		fmt.Fprintln(w)
	}
}

func dumpStrings(w io.Writer, title, preString, postString string, slice []string) {
	if len(slice) > 0 {
		sl := make([]string, len(slice))
		copy(sl, slice)
		sort.Strings(sl)
		fmt.Fprintln(w, title)
		for _, s := range sl {
			fmt.Fprintf(w, "** %s%s%s\n", preString, s, postString)
		}
	}

}

func dumpStringRefs(w io.Writer, title, preString, postString string, srefs stringRefs) {
	if len(srefs) == 0 {
		return
	}
	fmt.Fprintln(w, "====", title)
	slice := make([]string, 0, len(srefs))
	for s := range srefs {
		slice = append(slice, s)
	}
	sort.Strings(slice)
	for _, s := range slice {
		fmt.Fprintf(w, "; %s%s%s\n", preString, s, postString)
		fmt.Fprintln(w, ":", srefs[s])
	}
}

Added place/manager/memstore/refs.go.






































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package memstore stored the index in main memory.
package memstore

import "zettelstore.de/z/domain/id"

func refsDiff(refsN, refsO id.Slice) (newRefs, remRefs id.Slice) {
	npos, opos := 0, 0
	for npos < len(refsN) && opos < len(refsO) {
		rn, ro := refsN[npos], refsO[opos]
		if rn == ro {
			npos++
			opos++
			continue
		}
		if rn < ro {
			newRefs = append(newRefs, rn)
			npos++
			continue
		}
		remRefs = append(remRefs, ro)
		opos++
	}
	if npos < len(refsN) {
		newRefs = append(newRefs, refsN[npos:]...)
	}
	if opos < len(refsO) {
		remRefs = append(remRefs, refsO[opos:]...)
	}
	return newRefs, remRefs
}

func addRef(refs id.Slice, ref id.Zid) id.Slice {
	hi := len(refs)
	for lo := 0; lo < hi; {
		m := lo + (hi-lo)/2
		if r := refs[m]; r == ref {
			return refs
		} else if r < ref {
			lo = m + 1
		} else {
			hi = m
		}
	}
	refs = append(refs, id.Invalid)
	copy(refs[hi+1:], refs[hi:])
	refs[hi] = ref
	return refs
}

func remRefs(refs, rem id.Slice) id.Slice {
	if len(refs) == 0 || len(rem) == 0 {
		return refs
	}
	result := make(id.Slice, 0, len(refs))
	rpos, dpos := 0, 0
	for rpos < len(refs) && dpos < len(rem) {
		rr, dr := refs[rpos], rem[dpos]
		if rr < dr {
			result = append(result, rr)
			rpos++
			continue
		}
		if dr < rr {
			dpos++
			continue
		}
		rpos++
		dpos++
	}
	if rpos < len(refs) {
		result = append(result, refs[rpos:]...)
	}
	return result
}

func remRef(refs id.Slice, ref id.Zid) id.Slice {
	hi := len(refs)
	for lo := 0; lo < hi; {
		m := lo + (hi-lo)/2
		if r := refs[m]; r == ref {
			copy(refs[m:], refs[m+1:])
			refs = refs[:len(refs)-1]
			return refs
		} else if r < ref {
			lo = m + 1
		} else {
			hi = m
		}
	}
	return refs
}

Added place/manager/memstore/refs_test.go.







































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package memstore stored the index in main memory.
package memstore

import (
	"testing"

	"zettelstore.de/z/domain/id"
)

func assertRefs(t *testing.T, i int, got, exp id.Slice) {
	t.Helper()
	if got == nil && exp != nil {
		t.Errorf("%d: got nil, but expected %v", i, exp)
		return
	}
	if got != nil && exp == nil {
		t.Errorf("%d: expected nil, but got %v", i, got)
		return
	}
	if len(got) != len(exp) {
		t.Errorf("%d: expected len(%v)==%d, but got len(%v)==%d", i, exp, len(exp), got, len(got))
		return
	}
	for p, n := range exp {
		if got := got[p]; got != id.Zid(n) {
			t.Errorf("%d: pos %d: expected %d, but got %d", i, p, n, got)
		}
	}
}

func TestRefsDiff(t *testing.T) {
	testcases := []struct {
		in1, in2   id.Slice
		exp1, exp2 id.Slice
	}{
		{nil, nil, nil, nil},
		{id.Slice{1}, nil, id.Slice{1}, nil},
		{nil, id.Slice{1}, nil, id.Slice{1}},
		{id.Slice{1}, id.Slice{1}, nil, nil},
		{id.Slice{1, 2}, id.Slice{1}, id.Slice{2}, nil},
		{id.Slice{1, 2}, id.Slice{1, 3}, id.Slice{2}, id.Slice{3}},
		{id.Slice{1, 4}, id.Slice{1, 3}, id.Slice{4}, id.Slice{3}},
	}
	for i, tc := range testcases {
		got1, got2 := refsDiff(tc.in1, tc.in2)
		assertRefs(t, i, got1, tc.exp1)
		assertRefs(t, i, got2, tc.exp2)
	}
}

func TestAddRef(t *testing.T) {
	testcases := []struct {
		ref id.Slice
		zid uint
		exp id.Slice
	}{
		{nil, 5, id.Slice{5}},
		{id.Slice{1}, 5, id.Slice{1, 5}},
		{id.Slice{10}, 5, id.Slice{5, 10}},
		{id.Slice{5}, 5, id.Slice{5}},
		{id.Slice{1, 10}, 5, id.Slice{1, 5, 10}},
		{id.Slice{1, 5, 10}, 5, id.Slice{1, 5, 10}},
	}
	for i, tc := range testcases {
		got := addRef(tc.ref, id.Zid(tc.zid))
		assertRefs(t, i, got, tc.exp)
	}
}

func TestRemRefs(t *testing.T) {
	testcases := []struct {
		in1, in2 id.Slice
		exp      id.Slice
	}{
		{nil, nil, nil},
		{nil, id.Slice{}, nil},
		{id.Slice{}, nil, id.Slice{}},
		{id.Slice{}, id.Slice{}, id.Slice{}},
		{id.Slice{1}, id.Slice{5}, id.Slice{1}},
		{id.Slice{10}, id.Slice{5}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{5}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{5}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{5}, id.Slice{1, 10}},
		{id.Slice{1}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 2, 5}, id.Slice{2, 5}, id.Slice{1}},
		{id.Slice{2, 5, 10}, id.Slice{2, 5}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{2, 5}, id.Slice{1, 10}},
		{id.Slice{1}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 5}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{5, 10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 5, 9}, id.Slice{5, 9}, id.Slice{1}},
		{id.Slice{5, 9, 10}, id.Slice{5, 9}, id.Slice{10}},
		{id.Slice{1, 10}, id.Slice{5, 9}, id.Slice{1, 10}},
	}
	for i, tc := range testcases {
		got := remRefs(tc.in1, tc.in2)
		assertRefs(t, i, got, tc.exp)
	}
}

func TestRemRef(t *testing.T) {
	testcases := []struct {
		ref id.Slice
		zid uint
		exp id.Slice
	}{
		{nil, 5, nil},
		{id.Slice{}, 5, id.Slice{}},
		{id.Slice{5}, 5, id.Slice{}},
		{id.Slice{1}, 5, id.Slice{1}},
		{id.Slice{10}, 5, id.Slice{10}},
		{id.Slice{1, 5}, 5, id.Slice{1}},
		{id.Slice{5, 10}, 5, id.Slice{10}},
		{id.Slice{1, 5, 10}, 5, id.Slice{1, 10}},
	}
	for i, tc := range testcases {
		got := remRef(tc.ref, id.Zid(tc.zid))
		assertRefs(t, i, got, tc.exp)
	}
}

Added place/manager/place.go.




















































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package manager coordinates the various places and indexes of a Zettelstore.
package manager

import (
	"context"
	"errors"
	"sort"
	"strings"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
)

// Conatains all place.Place related functions

// Location returns some information where the place is located.
func (mgr *Manager) Location() string {
	if len(mgr.subplaces) <= 2 {
		return "NONE"
	}
	var sb strings.Builder
	for i := 0; i < len(mgr.subplaces)-2; i++ {
		if i > 0 {
			sb.WriteString(", ")
		}
		sb.WriteString(mgr.subplaces[i].Location())
	}
	return sb.String()
}

// CanCreateZettel returns true, if place could possibly create a new zettel.
func (mgr *Manager) CanCreateZettel(ctx context.Context) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.started && mgr.subplaces[0].CanCreateZettel(ctx)
}

// CreateZettel creates a new zettel.
func (mgr *Manager) CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return id.Invalid, place.ErrStopped
	}
	return mgr.subplaces[0].CreateZettel(ctx, zettel)
}

// GetZettel retrieves a specific zettel.
func (mgr *Manager) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return domain.Zettel{}, place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		if z, err := p.GetZettel(ctx, zid); err != place.ErrNotFound {
			if err == nil {
				mgr.Enrich(ctx, z.Meta)
			}
			return z, err
		}
	}
	return domain.Zettel{}, place.ErrNotFound
}

// GetMeta retrieves just the meta data of a specific zettel.
func (mgr *Manager) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return nil, place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		if m, err := p.GetMeta(ctx, zid); err != place.ErrNotFound {
			if err == nil {
				mgr.Enrich(ctx, m)
			}
			return m, err
		}
	}
	return nil, place.ErrNotFound
}

// FetchZids returns the set of all zettel identifer managed by the place.
func (mgr *Manager) FetchZids(ctx context.Context) (result id.Set, err error) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return nil, place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		zids, err := p.FetchZids(ctx)
		if err != nil {
			return nil, err
		}
		if result == nil {
			result = zids
		} else if len(result) <= len(zids) {
			for zid := range result {
				zids[zid] = true
			}
			result = zids
		} else {
			for zid := range zids {
				result[zid] = true
			}
		}
	}
	return result, nil
}

// SelectMeta returns all zettel meta data that match the selection
// criteria. The result is ordered by descending zettel id.
func (mgr *Manager) SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error) {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return nil, place.ErrStopped
	}
	var result []*meta.Meta
	match := s.CompileMatch(mgr)
	for _, p := range mgr.subplaces {
		selected, err := p.SelectMeta(ctx, match)
		if err != nil {
			return nil, err
		}
		sort.Slice(selected, func(i, j int) bool { return selected[i].Zid > selected[j].Zid })
		if len(result) == 0 {
			result = selected
		} else {
			result = place.MergeSorted(result, selected)
		}
	}
	if s == nil {
		return result, nil
	}
	return s.Sort(result), nil
}

// CanUpdateZettel returns true, if place could possibly update the given zettel.
func (mgr *Manager) CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	return mgr.started && mgr.subplaces[0].CanUpdateZettel(ctx, zettel)
}

// UpdateZettel updates an existing zettel.
func (mgr *Manager) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return place.ErrStopped
	}
	// Remove all (computed) properties from metadata before storing the zettel.
	zettel.Meta = zettel.Meta.Clone()
	for _, p := range zettel.Meta.PairsRest(true) {
		if mgr.propertyKeys[p.Key] {
			zettel.Meta.Delete(p.Key)
		}
	}
	return mgr.subplaces[0].UpdateZettel(ctx, zettel)
}

// AllowRenameZettel returns true, if place will not disallow renaming the zettel.
func (mgr *Manager) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return false
	}
	for _, p := range mgr.subplaces {
		if !p.AllowRenameZettel(ctx, zid) {
			return false
		}
	}
	return true
}

// RenameZettel changes the current zid to a new zid.
func (mgr *Manager) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return place.ErrStopped
	}
	for i, p := range mgr.subplaces {
		err := p.RenameZettel(ctx, curZid, newZid)
		if err != nil && !errors.Is(err, place.ErrNotFound) {
			for j := 0; j < i; j++ {
				mgr.subplaces[j].RenameZettel(ctx, newZid, curZid)
			}
			return err
		}
	}
	return nil
}

// CanDeleteZettel returns true, if place could possibly delete the given zettel.
func (mgr *Manager) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return false
	}
	for _, p := range mgr.subplaces {
		if p.CanDeleteZettel(ctx, zid) {
			return true
		}
	}
	return false
}

// DeleteZettel removes the zettel from the place.
func (mgr *Manager) DeleteZettel(ctx context.Context, zid id.Zid) error {
	mgr.mgrMx.RLock()
	defer mgr.mgrMx.RUnlock()
	if !mgr.started {
		return place.ErrStopped
	}
	for _, p := range mgr.subplaces {
		err := p.DeleteZettel(ctx, zid)
		if err == nil {
			return nil
		}
		if !errors.Is(err, place.ErrNotFound) && !errors.Is(err, place.ErrReadOnly) {
			return err
		}
	}
	return place.ErrNotFound
}

Added place/manager/store/store.go.


























































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package store contains general index data for storing a zettel index.
package store

import (
	"context"
	"io"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
)

// Stats records statistics about the store.
type Stats struct {
	// Zettel is the number of zettel managed by the indexer.
	Zettel int

	// Updates count the number of metadata updates.
	Updates uint64

	// Words count the different words stored in the store.
	Words uint64

	// Urls count the different URLs stored in the store.
	Urls uint64
}

// Store all relevant zettel data. There may be multiple implementations, i.e.
// memory-based, file-based, based on SQLite, ...
type Store interface {
	place.Enricher
	search.Selector

	// UpdateReferences for a specific zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	UpdateReferences(context.Context, *ZettelIndex) id.Set

	// DeleteZettel removes index data for given zettel.
	// Returns set of zettel identifier that must also be checked for changes.
	DeleteZettel(context.Context, id.Zid) id.Set

	// ReadStats populates st with store statistics.
	ReadStats(st *Stats)

	// Dump the content to a Writer.
	Dump(io.Writer)
}

Added place/manager/store/wordset.go.






























































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package store contains general index data for storing a zettel index.
package store

// WordSet contains the set of all words, with the count of their occurrences.
type WordSet map[string]int

// NewWordSet returns a new WordSet.
func NewWordSet() WordSet { return make(WordSet) }

// Add one word to the set
func (ws WordSet) Add(s string) {
	ws[s] = ws[s] + 1
}

// Words gives the slice of all words in the set.
func (ws WordSet) Words() []string {
	if len(ws) == 0 {
		return nil
	}
	words := make([]string, 0, len(ws))
	for w := range ws {
		words = append(words, w)
	}
	return words
}

// Diff calculates the word slice to be added and to be removed from oldWords
// to get the given word set.
func (ws WordSet) Diff(oldWords []string) (newWords, removeWords []string) {
	if len(ws) == 0 {
		return nil, oldWords
	}
	if len(oldWords) == 0 {
		return ws.Words(), nil
	}
	oldSet := make(WordSet, len(oldWords))
	for _, ow := range oldWords {
		if _, ok := ws[ow]; ok {
			oldSet[ow] = 1
			continue
		}
		removeWords = append(removeWords, ow)
	}
	for w := range ws {
		if _, ok := oldSet[w]; ok {
			continue
		}
		newWords = append(newWords, w)
	}
	return newWords, removeWords
}

Added place/manager/store/wordset_test.go.













































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package store contains general index data for storing a zettel index.
package store_test

import (
	"sort"
	"testing"

	"zettelstore.de/z/place/manager/store"
)

func equalWordList(exp, got []string) bool {
	if len(exp) != len(got) {
		return false
	}
	if len(got) == 0 {
		return len(exp) == 0
	}
	sort.Strings(got)
	for i, w := range exp {
		if w != got[i] {
			return false
		}
	}
	return true
}

func TestWordsWords(t *testing.T) {
	testcases := []struct {
		words store.WordSet
		exp   []string
	}{
		{nil, nil},
		{store.WordSet{}, nil},
		{store.WordSet{"a": 1, "b": 2}, []string{"a", "b"}},
	}
	for i, tc := range testcases {
		got := tc.words.Words()
		if !equalWordList(tc.exp, got) {
			t.Errorf("%d: %v.Words() == %v, but got %v", i, tc.words, tc.exp, got)
		}
	}
}

func TestWordsDiff(t *testing.T) {
	testcases := []struct {
		cur        store.WordSet
		old        []string
		expN, expR []string
	}{
		{nil, nil, nil, nil},
		{store.WordSet{}, []string{}, nil, nil},
		{store.WordSet{"a": 1}, []string{}, []string{"a"}, nil},
		{store.WordSet{"a": 1}, []string{"b"}, []string{"a"}, []string{"b"}},
		{store.WordSet{}, []string{"b"}, nil, []string{"b"}},
		{store.WordSet{"a": 1}, []string{"a"}, nil, nil},
	}
	for i, tc := range testcases {
		gotN, gotR := tc.cur.Diff(tc.old)
		if !equalWordList(tc.expN, gotN) {
			t.Errorf("%d: %v.Diff(%v)->new %v, but got %v", i, tc.cur, tc.old, tc.expN, gotN)
		}
		if !equalWordList(tc.expR, gotR) {
			t.Errorf("%d: %v.Diff(%v)->rem %v, but got %v", i, tc.cur, tc.old, tc.expR, gotR)
		}
	}
}

Added place/manager/store/zettel.go.


























































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package store contains general index data for storing a zettel index.
package store

import "zettelstore.de/z/domain/id"

// ZettelIndex contains all index data of a zettel.
type ZettelIndex struct {
	Zid      id.Zid            // zid of the indexed zettel
	backrefs id.Set            // set of back references
	metarefs map[string]id.Set // references to inverse keys
	deadrefs id.Set            // set of dead references
	words    WordSet
	urls     WordSet
}

// NewZettelIndex creates a new zettel index.
func NewZettelIndex(zid id.Zid) *ZettelIndex {
	return &ZettelIndex{
		Zid:      zid,
		backrefs: id.NewSet(),
		metarefs: make(map[string]id.Set),
		deadrefs: id.NewSet(),
	}
}

// AddBackRef adds a reference to a zettel where the current zettel links to
// without any more information.
func (zi *ZettelIndex) AddBackRef(zid id.Zid) {
	zi.backrefs[zid] = true
}

// AddMetaRef adds a named reference to a zettel. On that zettel, the given
// metadata key should point back to the current zettel.
func (zi *ZettelIndex) AddMetaRef(key string, zid id.Zid) {
	if zids, ok := zi.metarefs[key]; ok {
		zids[zid] = true
		return
	}
	zi.metarefs[key] = id.NewSet(zid)
}

// AddDeadRef adds a dead reference to a zettel.
func (zi *ZettelIndex) AddDeadRef(zid id.Zid) {
	zi.deadrefs[zid] = true
}

// SetWords sets the words to the given value.
func (zi *ZettelIndex) SetWords(words WordSet) { zi.words = words }

// SetUrls sets the words to the given value.
func (zi *ZettelIndex) SetUrls(urls WordSet) { zi.urls = urls }

// GetDeadRefs returns all dead references as a sorted list.
func (zi *ZettelIndex) GetDeadRefs() id.Slice {
	return zi.deadrefs.Sorted()
}

// GetBackRefs returns all back references as a sorted list.
func (zi *ZettelIndex) GetBackRefs() id.Slice {
	return zi.backrefs.Sorted()
}

// GetMetaRefs returns all meta references as a map of strings to a sorted list of references
func (zi *ZettelIndex) GetMetaRefs() map[string]id.Slice {
	if len(zi.metarefs) == 0 {
		return nil
	}
	result := make(map[string]id.Slice, len(zi.metarefs))
	for key, refs := range zi.metarefs {
		result[key] = refs.Sorted()
	}
	return result
}

// GetWords returns a reference to the set of words. It must not be modified.
func (zi *ZettelIndex) GetWords() WordSet { return zi.words }

// GetUrls returns a reference to the set of URLs. It must not be modified.
func (zi *ZettelIndex) GetUrls() WordSet { return zi.urls }

Changes to place/memplace/memplace.go.

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45

46
47
48
49
50
51
52
16
17
18
19
20
21
22

23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

42
43

44
45
46
47
48
49
50
51







-



















-
+

-
+







	"net/url"
	"sync"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/place/manager"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register(
		"mem",
		func(u *url.URL, cdata *manager.ConnectData) (place.ManagedPlace, error) {
			return &memPlace{u: u, cdata: *cdata}, nil
		})
}

type memPlace struct {
	u      *url.URL
	cdata  manager.ConnectData
	zettel map[id.Zid]domain.Zettel
	mx     sync.RWMutex
}

func (mp *memPlace) notifyChanged(reason change.Reason, zid id.Zid) {
func (mp *memPlace) notifyChanged(reason place.UpdateReason, zid id.Zid) {
	if chci := mp.cdata.Notify; chci != nil {
		chci <- change.Info{Reason: reason, Zid: zid}
		chci <- place.UpdateInfo{Reason: reason, Zid: zid}
	}
}

func (mp *memPlace) Location() string {
	return mp.u.String()
}

77
78
79
80
81
82
83
84

85
86
87
88
89
90
91
76
77
78
79
80
81
82

83
84
85
86
87
88
89
90







-
+







		return id.Invalid, err
	}
	meta := zettel.Meta.Clone()
	meta.Zid = zid
	zettel.Meta = meta
	mp.zettel[zid] = zettel
	mp.mx.Unlock()
	mp.notifyChanged(change.OnUpdate, zid)
	mp.notifyChanged(place.OnUpdate, zid)
	return zid, nil
}

func (mp *memPlace) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	mp.mx.RLock()
	zettel, ok := mp.zettel[zid]
	mp.mx.RUnlock()
117
118
119
120
121
122
123
124

125
126
127
128
129
130
131
116
117
118
119
120
121
122

123
124
125
126
127
128
129
130







-
+







}

func (mp *memPlace) SelectMeta(ctx context.Context, match search.MetaMatchFunc) ([]*meta.Meta, error) {
	result := make([]*meta.Meta, 0, len(mp.zettel))
	mp.mx.RLock()
	for _, zettel := range mp.zettel {
		m := zettel.Meta.Clone()
		mp.cdata.Filter.Enrich(ctx, m)
		mp.cdata.Enricher.Enrich(ctx, m)
		if match(m) {
			result = append(result, m)
		}
	}
	mp.mx.RUnlock()
	return result, nil
}
139
140
141
142
143
144
145
146

147
148
149
150
151
152
153
138
139
140
141
142
143
144

145
146
147
148
149
150
151
152







-
+







	meta := zettel.Meta.Clone()
	if !meta.Zid.IsValid() {
		return &place.ErrInvalidID{Zid: meta.Zid}
	}
	zettel.Meta = meta
	mp.zettel[meta.Zid] = zettel
	mp.mx.Unlock()
	mp.notifyChanged(change.OnUpdate, meta.Zid)
	mp.notifyChanged(place.OnUpdate, meta.Zid)
	return nil
}

func (mp *memPlace) AllowRenameZettel(ctx context.Context, zid id.Zid) bool { return true }

func (mp *memPlace) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	mp.mx.Lock()
165
166
167
168
169
170
171
172
173


174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192

193
194
195
196

197
198
199
200
201
164
165
166
167
168
169
170


171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190

191
192
193
194

195
196
197
198
199
200







-
-
+
+


















-
+



-
+






	meta := zettel.Meta.Clone()
	meta.Zid = newZid
	zettel.Meta = meta
	mp.zettel[newZid] = zettel
	delete(mp.zettel, curZid)
	mp.mx.Unlock()
	mp.notifyChanged(change.OnDelete, curZid)
	mp.notifyChanged(change.OnUpdate, newZid)
	mp.notifyChanged(place.OnDelete, curZid)
	mp.notifyChanged(place.OnUpdate, newZid)
	return nil
}

func (mp *memPlace) CanDeleteZettel(ctx context.Context, zid id.Zid) bool {
	mp.mx.RLock()
	_, ok := mp.zettel[zid]
	mp.mx.RUnlock()
	return ok
}

func (mp *memPlace) DeleteZettel(ctx context.Context, zid id.Zid) error {
	mp.mx.Lock()
	if _, ok := mp.zettel[zid]; !ok {
		mp.mx.Unlock()
		return place.ErrNotFound
	}
	delete(mp.zettel, zid)
	mp.mx.Unlock()
	mp.notifyChanged(change.OnDelete, zid)
	mp.notifyChanged(place.OnDelete, zid)
	return nil
}

func (mp *memPlace) ReadStats(st *place.Stats) {
func (mp *memPlace) ReadStats(st *place.ManagedPlaceStats) {
	st.ReadOnly = false
	mp.mx.RLock()
	st.Zettel = len(mp.zettel)
	mp.mx.RUnlock()
}

Changes to place/place.go.

11
12
13
14
15
16
17


18
19
20
21
22
23
24
25
26
27
28
29
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30







+
+




-







// Package place provides a generic interface to zettel places.
package place

import (
	"context"
	"errors"
	"fmt"
	"io"
	"time"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/search"
)

// BasePlace is implemented by all Zettel places.
type BasePlace interface {
	// Location returns some information where the place is located.
	// Format is dependent of the place.
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79







80
81
82
83
84
85
86
59
60
61
62
63
64
65



66
67
68
69
70
71
72
73




74
75
76
77
78
79
80
81
82
83
84
85
86
87







-
-
-








-
-
-
-
+
+
+
+
+
+
+







	RenameZettel(ctx context.Context, curZid, newZid id.Zid) error

	// CanDeleteZettel returns true, if place could possibly delete the given zettel.
	CanDeleteZettel(ctx context.Context, zid id.Zid) bool

	// DeleteZettel removes the zettel from the place.
	DeleteZettel(ctx context.Context, zid id.Zid) error

	// ReadStats populates st with place statistics
	ReadStats(st *Stats)
}

// ManagedPlace is the interface of managed places.
type ManagedPlace interface {
	BasePlace

	// SelectMeta returns all zettel meta data that match the selection criteria.
	SelectMeta(ctx context.Context, match search.MetaMatchFunc) ([]*meta.Meta, error)
}

// Stats records statistics about the place.
type Stats struct {

	// ReadStats populates st with place statistics
	ReadStats(st *ManagedPlaceStats)
}

// ManagedPlaceStats records statistics about the place.
type ManagedPlaceStats struct {
	// ReadOnly indicates that the places cannot be changed
	ReadOnly bool

	// Zettel is the number of zettel managed by the place.
	Zettel int
}

96
97
98
99
100
101
102
103



104







































105
106
107
108
109
110
111
112





















































113
114
115
116
117
118
119
97
98
99
100
101
102
103

104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150




151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210







-
+
+
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+




-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+








// Place is a place to be used outside the place package and its descendants.
type Place interface {
	BasePlace

	// SelectMeta returns a list of metadata that comply to the given selection criteria.
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)
}

	// ReadStats populates st with place statistics
	ReadStats(st *Stats)

	// Dump internal data to a Writer.
	Dump(w io.Writer)
}

// Stats record stattistics about a full place.
type Stats struct {
	// ReadOnly indicates that the places cannot be changed
	ReadOnly bool

	// NumManagedPlaces is the number of places managed.
	NumManagedPlaces int

	// Zettel is the number of zettel managed by the place, including
	// duplicates across managed places.
	ZettelTotal int

	// LastReload stores the timestamp when a full re-index was done.
	LastReload time.Time

	// IndexesSinceReload counts indexing a zettel since the full re-index.
	IndexesSinceReload uint64

	// DurLastIndex is the duration of the last index run. This could be a
	// full re-index or a re-index of a single zettel.
	DurLastIndex time.Duration

	// ZettelIndexed is the number of zettel managed by the indexer.
	ZettelIndexed int

	// IndexUpdates count the number of metadata updates.
	IndexUpdates uint64

	// IndexedWords count the different words indexed.
	IndexedWords uint64

	// IndexedUrls count the different URLs indexed.
	IndexedUrls uint64
}

// Manager is a place-managing place.
type Manager interface {
	Place
	StartStopper
	change.Subject

	// NumPlaces returns the number of managed places.
	NumPlaces() int
	Subject
}

// UpdateReason gives an indication, why the ObserverFunc was called.
type UpdateReason uint8

// Values for Reason
const (
	_        UpdateReason = iota
	OnReload              // Place was reloaded
	OnUpdate              // A zettel was created or changed
	OnDelete              // A zettel was removed
)

// UpdateInfo contains all the data about a changed zettel.
type UpdateInfo struct {
	Place  Place
	Reason UpdateReason
	Zid    id.Zid
}

// UpdateFunc is a function to be called when a change is detected.
type UpdateFunc func(UpdateInfo)

// Subject is a place that notifies observers about changes.
type Subject interface {
	// RegisterObserver registers an observer that will be notified
	// if one or all zettel are found to be changed.
	RegisterObserver(UpdateFunc)
}

// Enricher is used to update metadata by adding new properties.
type Enricher interface {
	// Enrich computes additional properties and updates the given metadata.
	// It is typically called by zettel reading methods.
	Enrich(ctx context.Context, m *meta.Meta)
}

// NoEnrichContext will signal an enricher that nothing has to be done.
// This is useful for an Indexer, but also for some place.Place calls, when
// just the plain metadata is needed.
func NoEnrichContext(ctx context.Context) context.Context {
	return context.WithValue(ctx, ctxNoEnrichKey, &ctxNoEnrichKey)
}

type ctxNoEnrichType struct{}

var ctxNoEnrichKey ctxNoEnrichType

// DoNotEnrich determines if the context is marked to not enrich metadata.
func DoNotEnrich(ctx context.Context) bool {
	_, ok := ctx.Value(ctxNoEnrichKey).(*ctxNoEnrichType)
	return ok
}

// ErrNotAllowed is returned if the caller is not allowed to perform the operation.
type ErrNotAllowed struct {
	Op   string
	User *meta.Meta
	Zid  id.Zid
149
150
151
152
153
154
155
156

157
158

159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174

175
176

177
178
179
180
181
240
241
242
243
244
245
246

247


248


249
250
251
252
253
254
255
256
257
258
259
260
261

262
263

264
265
266
267
268
269







-
+
-
-
+
-
-













-
+

-
+





	return fmt.Sprintf(
		"operation %q not allowed for user %v/%v",
		err.Op,
		err.User.GetDefault(meta.KeyUserID, "?"),
		err.User.Zid.String())
}

// IsErrNotAllowed return true, if the error is of type ErrNotAllowed.
// Is return true, if the error is of type ErrNotAllowed.
func IsErrNotAllowed(err error) bool {
	_, ok := err.(*ErrNotAllowed)
func (err *ErrNotAllowed) Is(target error) bool { return true }
	return ok
}

// ErrStarted is returned when trying to start an already started place.
var ErrStarted = errors.New("place is already started")

// ErrStopped is returned if calling methods on a place that was not started.
var ErrStopped = errors.New("place is stopped")

// ErrReadOnly is returned if there is an attepmt to write to a read-only place.
var ErrReadOnly = errors.New("read-only place")

// ErrNotFound is returned if a zettel was not found in the place.
var ErrNotFound = errors.New("zettel not found")

// ErrTimeout is returned if a place operation takes too long.
// ErrConflict is returned if a place operation detected a conflict..
// One example: if calculating a new zettel identifier takes too long.
var ErrTimeout = errors.New("timeout")
var ErrConflict = errors.New("conflict")

// ErrInvalidID is returned if the zettel id is not appropriate for the place operation.
type ErrInvalidID struct{ Zid id.Zid }

func (err *ErrInvalidID) Error() string { return "invalid Zettel id: " + err.Zid.String() }

Changes to place/progplace/config.go.

8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29

30
31
32
33
34
35

36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
8
9
10
11
12
13
14

15
16

17
18
19
20
21

22
23
24
25
26

27
28
29
30
31
32

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52







































-


-





-
+




-
+





-
+



















-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
// under this license.
//-----------------------------------------------------------------------------

// Package progplace provides zettel that inform the user about the internal Zettelstore state.
package progplace

import (
	"fmt"
	"strings"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

func genConfigZettelM(zid id.Zid) *meta.Meta {
	if myPlace.startConfig == nil {
	if myConfig == nil {
		return nil
	}
	m := meta.New(zid)
	m.Set(meta.KeyTitle, "Zettelstore Startup Configuration")
	m.Set(meta.KeyVisibility, meta.ValueVisibilitySimple)
	m.Set(meta.KeyVisibility, meta.ValueVisibilityExpert)
	return m
}

func genConfigZettelC(m *meta.Meta) string {
	var sb strings.Builder
	for i, p := range myPlace.startConfig.Pairs(false) {
	for i, p := range myConfig.Pairs(false) {
		if i > 0 {
			sb.WriteByte('\n')
		}
		sb.WriteString("; ''")
		sb.WriteString(p.Key)
		sb.WriteString("''")
		if p.Value != "" {
			sb.WriteString("\n: ``")
			for _, r := range p.Value {
				if r == '`' {
					sb.WriteByte('\\')
				}
				sb.WriteRune(r)
			}
			sb.WriteString("``")
		}
	}
	return sb.String()
}

func genConfigM(zid id.Zid) *meta.Meta {
	if myPlace.startConfig == nil {
		return nil
	}
	m := meta.New(zid)
	m.Set(meta.KeyTitle, "Zettelstore Startup Values")
	m.Set(meta.KeyRole, meta.ValueRoleConfiguration)
	m.Set(meta.KeySyntax, meta.ValueSyntaxZmk)
	m.Set(meta.KeyVisibility, meta.ValueVisibilitySimple)
	m.Set(meta.KeyReadOnly, meta.ValueTrue)
	return m
}

func genConfigC(m *meta.Meta) string {
	var sb strings.Builder
	sb.WriteString("|=Name|=Value>\n")
	fmt.Fprintf(&sb, "|Simple|%v\n", startup.IsSimple())
	fmt.Fprintf(&sb, "|Verbose|%v\n", startup.IsVerbose())
	fmt.Fprintf(&sb, "|Read-only|%v\n", startup.IsReadOnlyMode())
	fmt.Fprintf(&sb, "|URL prefix|%v\n", startup.URLPrefix())
	// There must be a space before the next "%v". Listen address may start with a ":"
	fmt.Fprintf(&sb, "|Listen address| %v\n", startup.ListenAddress())
	fmt.Fprintf(&sb, "|Authentication enabled|%v\n", startup.WithAuth())
	fmt.Fprintf(&sb, "|Secure cookie|%v\n", startup.SecureCookie())
	fmt.Fprintf(&sb, "|Persistent Cookie|%v\n", startup.PersistentCookie())
	html, api := startup.TokenLifetime()
	fmt.Fprintf(&sb, "|API Token lifetime|%v\n", api)
	fmt.Fprintf(&sb, "|HTML Token lifetime|%v\n", html)
	fmt.Fprintf(&sb, "|Default directory place type|%v", startup.DefaultDirPlaceType())
	return sb.String()
}

Deleted place/progplace/env.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62






























































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package progplace provides zettel that inform the user about the internal Zettelstore state.
package progplace

import (
	"fmt"
	"os"
	"sort"
	"strings"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

func genEnvironmentM(zid id.Zid) *meta.Meta {
	if myPlace.startConfig == nil {
		return nil
	}
	m := meta.New(zid)
	m.Set(meta.KeyTitle, "Zettelstore Environment Values")
	return m
}

func genEnvironmentC(*meta.Meta) string {
	workDir, err := os.Getwd()
	if err != nil {
		workDir = err.Error()
	}
	execName, err := os.Executable()
	if err != nil {
		execName = err.Error()
	}
	envs := os.Environ()
	sort.Strings(envs)

	var sb strings.Builder
	sb.WriteString("|=Name|=Value>\n")
	fmt.Fprintf(&sb, "|Working directory| %v\n", workDir)
	fmt.Fprintf(&sb, "|Executable| %v\n", execName)
	fmt.Fprintf(&sb, "|Build with| %v\n", startup.GetVersion().GoVersion)

	sb.WriteString("=== Environment\n")
	sb.WriteString("|=Key>|=Value<\n")
	for _, env := range envs {
		if pos := strings.IndexByte(env, '='); pos >= 0 && pos < len(env) {
			fmt.Fprintf(&sb, "| %v| %v\n", env[:pos], env[pos+1:])
		} else {
			fmt.Fprintf(&sb, "| %v\n", env)
		}
	}
	return sb.String()
}

Deleted place/progplace/indexer.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
















































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package progplace provides zettel that inform the user about the internal
// Zettelstore state.
package progplace

import (
	"fmt"
	"strings"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
)

func genIndexerM(zid id.Zid) *meta.Meta {
	if myPlace.indexer == nil {
		return nil
	}
	m := meta.New(zid)
	m.Set(meta.KeyTitle, "Zettelstore Indexer")
	return m
}

func genIndexerC(*meta.Meta) string {
	ixer := myPlace.indexer

	var stats index.IndexerStats
	ixer.ReadStats(&stats)

	var sb strings.Builder
	sb.WriteString("|=Name|=Value>\n")
	fmt.Fprintf(&sb, "|Zettel| %v\n", stats.Store.Zettel)
	fmt.Fprintf(&sb, "|Last re-index| %v\n", stats.LastReload.Format("2006-01-02 15:04:05 -0700 MST"))
	fmt.Fprintf(&sb, "|Indexes since last re-index| %v\n", stats.IndexesSinceReload)
	fmt.Fprintf(&sb, "|Duration last index| %vms\n", stats.DurLastIndex.Milliseconds())
	fmt.Fprintf(&sb, "|Zettel enrichments| %v\n", stats.Store.Updates)
	fmt.Fprintf(&sb, "|Indexed words| %v\n", stats.Store.Words)
	return sb.String()
}

Changes to place/progplace/manager.go.

14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30
31
32
33
34
35
36



37
38

39
40
41
42


43

44
45
14
15
16
17
18
19
20

21
22
23
24



25
26
27
28
29
30



31
32
33


34
35
36


37
38

39
40
41







-
+



-
-
-






-
-
-
+
+
+
-
-
+


-
-
+
+
-
+



import (
	"fmt"
	"strings"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/kernel"
)

func genManagerM(zid id.Zid) *meta.Meta {
	if myPlace.manager == nil {
		return nil
	}
	m := meta.New(zid)
	m.Set(meta.KeyTitle, "Zettelstore Place Manager")
	return m
}

func genManagerC(*meta.Meta) string {
	mgr := myPlace.manager

	var stats place.Stats
	kvl := kernel.Main.GetServiceStatistics(kernel.PlaceService)
	if len(kvl) == 0 {
		return "No statistics available"
	mgr.ReadStats(&stats)

	}
	var sb strings.Builder
	sb.WriteString("|=Name|=Value>\n")
	fmt.Fprintf(&sb, "|Read-only| %v\n", stats.ReadOnly)
	fmt.Fprintf(&sb, "|Zettel| %v\n", stats.Zettel)
	for _, kv := range kvl {
		fmt.Fprintf(&sb, "| %v | %v\n", kv.Key, kv.Value)
	fmt.Fprintf(&sb, "|Sub-places| %v\n", mgr.NumPlaces())
	}
	return sb.String()
}

Changes to place/progplace/progplace.go.

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

33
34
35
36
37


38
39
40

41
42
43
44
45
46



47
48


49
50
51
52
53
54
55
56
57
58
59
60



61
62
63
64
65
66



67
68
69
70
71
72





73
74
75
76
77
78
79
80
81
82
83

84
85
86
87
88
89
90
91
92
93
94
95
96
97
98

99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

115
116
117
118
119
120
121
122
123
124
125
126
127


128
129
130
131
132
133
134
135
136
137
138

139
140
141
142
143
144
145
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30

31
32
33
34


35
36



37
38





39
40
41


42
43












44
45
46






47
48
49






50
51
52
53
54
55
56
57








58



59
60
61
62
63
64
65
66
67
68
69

70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

86
87
88
89
90
91
92
93
94
95
96
97


98
99
100
101
102
103
104
105
106
107
108
109

110
111
112
113
114
115
116
117







-









-
+



-
-
+
+
-
-
-
+

-
-
-
-
-
+
+
+
-
-
+
+
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
-
-
-
-
-
-
+
+
+
-
-
-
-
-
-
+
+
+
+
+



-
-
-
-
-
-
-
-
+
-
-
-











-
+















-
+











-
-
+
+










-
+







import (
	"context"
	"net/url"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/manager"
	"zettelstore.de/z/search"
)

func init() {
	manager.Register(
		" prog",
		func(u *url.URL, cdata *manager.ConnectData) (place.ManagedPlace, error) {
			return getPlace(cdata.Filter), nil
			return getPlace(cdata.Enricher), nil
		})
}

type (
	zettelGen struct {
type progPlace struct {
	filter place.Enricher
		meta    func(id.Zid) *meta.Meta
		content func(*meta.Meta) string
	}
}

	progPlace struct {
		zettel      map[id.Zid]zettelGen
		filter      index.MetaFilter
		startConfig *meta.Meta
		manager     place.Manager
var myConfig *meta.Meta
var myZettel = map[id.Zid]struct {
	meta    func(id.Zid) *meta.Meta
		indexer     index.Indexer
	}
	content func(*meta.Meta) string
}{
)

var myPlace *progPlace

// Get returns the one program place.
func getPlace(mf index.MetaFilter) place.ManagedPlace {
	if myPlace == nil {
		myPlace = &progPlace{
			zettel: map[id.Zid]zettelGen{
				id.Zid(1):  {genVersionBuildM, genVersionBuildC},
				id.Zid(2):  {genVersionHostM, genVersionHostC},
				id.Zid(3):  {genVersionOSM, genVersionOSC},
	id.VersionZid:              {genVersionBuildM, genVersionBuildC},
	id.HostZid:                 {genVersionHostM, genVersionHostC},
	id.OperatingSystemZid:      {genVersionOSM, genVersionOSC},
				id.Zid(6):  {genEnvironmentM, genEnvironmentC},
				id.Zid(8):  {genRuntimeM, genRuntimeC},
				id.Zid(18): {genIndexerM, genIndexerC},
				id.Zid(20): {genManagerM, genManagerC},
				id.Zid(90): {genKeysM, genKeysC},
				id.Zid(96): {genConfigZettelM, genConfigZettelC},
	id.PlaceManagerZid:         {genManagerM, genManagerC},
	id.MetadataKeyZid:          {genKeysM, genKeysC},
	id.StartupConfigurationZid: {genConfigZettelM, genConfigZettelC},
				id.Zid(98): {genConfigM, genConfigC},
			},
			filter: mf,
		}
	}
	return myPlace
}

// Get returns the one program place.
func getPlace(mf place.Enricher) place.ManagedPlace {
	return &progPlace{filter: mf}
}

// Setup remembers important values.
func Setup(startConfig *meta.Meta, manager place.Manager, idx index.Indexer) {
	if myPlace == nil {
		panic("progplace.getPlace not called")
	}
	if myPlace.startConfig != nil || myPlace.manager != nil {
		panic("progplace.Setup already called")
	}
	myPlace.startConfig = startConfig.Clone()
func Setup(cfg *meta.Meta) { myConfig = cfg.Clone() }
	myPlace.manager = manager
	myPlace.indexer = idx
}

func (pp *progPlace) Location() string { return "" }

func (pp *progPlace) CanCreateZettel(ctx context.Context) bool { return false }

func (pp *progPlace) CreateZettel(
	ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	return id.Invalid, place.ErrReadOnly
}

func (pp *progPlace) GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error) {
	if gen, ok := pp.zettel[zid]; ok && gen.meta != nil {
	if gen, ok := myZettel[zid]; ok && gen.meta != nil {
		if m := gen.meta(zid); m != nil {
			updateMeta(m)
			if genContent := gen.content; genContent != nil {
				return domain.Zettel{
					Meta:    m,
					Content: domain.NewContent(genContent(m)),
				}, nil
			}
			return domain.Zettel{Meta: m}, nil
		}
	}
	return domain.Zettel{}, place.ErrNotFound
}

func (pp *progPlace) GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error) {
	if gen, ok := pp.zettel[zid]; ok {
	if gen, ok := myZettel[zid]; ok {
		if genMeta := gen.meta; genMeta != nil {
			if m := genMeta(zid); m != nil {
				updateMeta(m)
				return m, nil
			}
		}
	}
	return nil, place.ErrNotFound
}

func (pp *progPlace) FetchZids(ctx context.Context) (id.Set, error) {
	result := id.NewSetCap(len(pp.zettel))
	for zid, gen := range pp.zettel {
	result := id.NewSetCap(len(myZettel))
	for zid, gen := range myZettel {
		if genMeta := gen.meta; genMeta != nil {
			if genMeta(zid) != nil {
				result[zid] = true
			}
		}
	}
	return result, nil
}

func (pp *progPlace) SelectMeta(ctx context.Context, match search.MetaMatchFunc) (res []*meta.Meta, err error) {
	for zid, gen := range pp.zettel {
	for zid, gen := range myZettel {
		if genMeta := gen.meta; genMeta != nil {
			if m := genMeta(zid); m != nil {
				updateMeta(m)
				pp.filter.Enrich(ctx, m)
				if match(m) {
					res = append(res, m)
				}
154
155
156
157
158
159
160
161

162
163
164
165
166

167
168
169
170
171
172
173
174
175

176
177
178
179
180
181

182
183

184
185
186
187
188
189

190
191
192
193
194
126
127
128
129
130
131
132

133
134
135
136
137

138
139
140
141
142
143
144
145
146

147
148
149
150
151
152

153
154

155
156
157
158
159
160
161
162
163
164
165
166
167







-
+




-
+








-
+





-
+

-
+






+





}

func (pp *progPlace) UpdateZettel(ctx context.Context, zettel domain.Zettel) error {
	return place.ErrReadOnly
}

func (pp *progPlace) AllowRenameZettel(ctx context.Context, zid id.Zid) bool {
	_, ok := pp.zettel[zid]
	_, ok := myZettel[zid]
	return !ok
}

func (pp *progPlace) RenameZettel(ctx context.Context, curZid, newZid id.Zid) error {
	if _, ok := pp.zettel[curZid]; ok {
	if _, ok := myZettel[curZid]; ok {
		return place.ErrReadOnly
	}
	return place.ErrNotFound
}

func (pp *progPlace) CanDeleteZettel(ctx context.Context, zid id.Zid) bool { return false }

func (pp *progPlace) DeleteZettel(ctx context.Context, zid id.Zid) error {
	if _, ok := pp.zettel[zid]; ok {
	if _, ok := myZettel[zid]; ok {
		return place.ErrReadOnly
	}
	return place.ErrNotFound
}

func (pp *progPlace) ReadStats(st *place.Stats) {
func (pp *progPlace) ReadStats(st *place.ManagedPlaceStats) {
	st.ReadOnly = true
	st.Zettel = len(pp.zettel)
	st.Zettel = len(myZettel)
}

func updateMeta(m *meta.Meta) {
	m.Set(meta.KeyNoIndex, meta.ValueTrue)
	m.Set(meta.KeySyntax, meta.ValueSyntaxZmk)
	m.Set(meta.KeyRole, meta.ValueRoleConfiguration)
	m.Set(meta.KeyLang, meta.ValueLangEN)
	m.Set(meta.KeyReadOnly, meta.ValueTrue)
	if _, ok := m.Get(meta.KeyVisibility); !ok {
		m.Set(meta.KeyVisibility, meta.ValueVisibilityExpert)
	}
}

Deleted place/progplace/runtime.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72








































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package progplace provides zettel that inform the user about the internal Zettelstore state.
package progplace

import (
	"fmt"
	"runtime/metrics"
	"strings"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

func genRuntimeM(zid id.Zid) *meta.Meta {
	if myPlace.startConfig == nil {
		return nil
	}
	m := meta.New(zid)
	m.Set(meta.KeyTitle, "Zettelstore Runtime Metrics")
	return m
}

func genRuntimeC(*meta.Meta) string {
	var samples []metrics.Sample
	all := metrics.All()
	for _, d := range all {
		if d.Kind == metrics.KindFloat64Histogram {
			continue
		}
		samples = append(samples, metrics.Sample{Name: d.Name})
	}
	metrics.Read(samples)

	var sb strings.Builder
	sb.WriteString("|=Name|=Value>\n")
	i := 0
	for _, d := range all {
		if d.Kind == metrics.KindFloat64Histogram {
			continue
		}
		descr := d.Description
		if pos := strings.IndexByte(descr, '.'); pos > 0 {
			descr = descr[:pos]
		}
		fmt.Fprintf(&sb, "|%s|", descr)
		value := samples[i].Value
		i++
		switch value.Kind() {
		case metrics.KindUint64:
			fmt.Fprintf(&sb, "%v", value.Uint64())
		case metrics.KindFloat64:
			fmt.Fprintf(&sb, "%v", value.Float64())
		case metrics.KindFloat64Histogram:
			sb.WriteString("???")
		case metrics.KindBad:
			sb.WriteString("BAD")
		default:
			fmt.Fprintf(&sb, "(unexpected metric kind: %v)", value.Kind())
		}
		sb.WriteByte('\n')
	}
	return sb.String()
}

Changes to place/progplace/version.go.

10
11
12
13
14
15
16
17
18
19



20
21
22
23
24
25

26
27
28
29
30
31
32
33
34



35
36
37
38
39



40
41
42
43
44
45
46





47
10
11
12
13
14
15
16



17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32
33

34
35
36
37
38
39
40

41
42
43
44
45
46
47
48


49
50
51
52
53
54







-
-
-
+
+
+





-
+








-
+
+
+




-
+
+
+





-
-
+
+
+
+
+


// Package progplace provides zettel that inform the user about the internal Zettelstore state.
package progplace

import (
	"fmt"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
)

func getVersionMeta(zid id.Zid, title string) *meta.Meta {
	m := meta.New(zid)
	m.Set(meta.KeyTitle, title)
	m.Set(meta.KeyVisibility, meta.ValueVisibilitySimple)
	m.Set(meta.KeyVisibility, meta.ValueVisibilityExpert)
	return m
}

func genVersionBuildM(zid id.Zid) *meta.Meta {
	m := getVersionMeta(zid, "Zettelstore Version")
	m.Set(meta.KeyVisibility, meta.ValueVisibilityPublic)
	return m
}
func genVersionBuildC(*meta.Meta) string { return startup.GetVersion().Build }
func genVersionBuildC(*meta.Meta) string {
	return kernel.Main.GetConfig(kernel.CoreService, kernel.CoreVersion).(string)
}

func genVersionHostM(zid id.Zid) *meta.Meta {
	return getVersionMeta(zid, "Zettelstore Host")
}
func genVersionHostC(*meta.Meta) string { return startup.GetVersion().Hostname }
func genVersionHostC(*meta.Meta) string {
	return kernel.Main.GetConfig(kernel.CoreService, kernel.CoreHostname).(string)
}

func genVersionOSM(zid id.Zid) *meta.Meta {
	return getVersionMeta(zid, "Zettelstore Operating System")
}
func genVersionOSC(*meta.Meta) string {
	v := startup.GetVersion()
	return fmt.Sprintf("%v/%v", v.Os, v.Arch)
	return fmt.Sprintf(
		"%v/%v",
		kernel.Main.GetConfig(kernel.CoreService, kernel.CoreGoOS).(string),
		kernel.Main.GetConfig(kernel.CoreService, kernel.CoreGoArch).(string),
	)
}

Deleted place/stock/stock.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117





















































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package stock allows to get zettel without reading it from a place.
package stock

import (
	"context"
	"sync"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place/change"
)

// Place is a place that is used by a stock.
type Place interface {
	change.Subject

	// GetZettel retrieves a specific zettel.
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)
}

// Stock allow to get subscribed zettel without reading it from a place.
type Stock interface {
	Subscribe(zid id.Zid) error
	GetZettel(zid id.Zid) domain.Zettel
	GetMeta(zid id.Zid) *meta.Meta
}

// NewStock creates a new stock that operates on the given place.
func NewStock(place Place) Stock {
	stock := &defaultStock{
		place: place,
		subs:  make(map[id.Zid]domain.Zettel),
	}
	place.RegisterObserver(stock.observe)
	return stock
}

type defaultStock struct {
	place  Place
	subs   map[id.Zid]domain.Zettel
	mxSubs sync.RWMutex
}

// observe tracks all changes the place signals.
func (s *defaultStock) observe(ci change.Info) {
	if ci.Reason == change.OnReload {
		go func() {
			s.mxSubs.Lock()
			defer s.mxSubs.Unlock()
			for zid := range s.subs {
				s.update(zid)
			}
		}()
		return
	}

	s.mxSubs.RLock()
	defer s.mxSubs.RUnlock()
	if _, found := s.subs[ci.Zid]; found {
		go func() {
			s.mxSubs.Lock()
			defer s.mxSubs.Unlock()
			s.update(ci.Zid)
		}()
	}
}

func (s *defaultStock) update(zid id.Zid) {
	if zettel, err := s.place.GetZettel(context.Background(), zid); err == nil {
		s.subs[zid] = zettel
		return
	}
}

// Subscribe adds a zettel to the stock.
func (s *defaultStock) Subscribe(zid id.Zid) error {
	s.mxSubs.Lock()
	defer s.mxSubs.Unlock()
	if _, found := s.subs[zid]; found {
		return nil
	}
	zettel, err := s.place.GetZettel(context.Background(), zid)
	if err != nil {
		return err
	}
	s.subs[zid] = zettel
	return nil
}

// GetZettel returns the zettel with the given zid, if in stock, else an empty zettel
func (s *defaultStock) GetZettel(zid id.Zid) domain.Zettel {
	s.mxSubs.RLock()
	defer s.mxSubs.RUnlock()
	return s.subs[zid]
}

// GetZettel returns the zettel Meta with the given zid, if in stock, else nil.
func (s *defaultStock) GetMeta(zid id.Zid) *meta.Meta {
	s.mxSubs.RLock()
	zettel, ok := s.subs[zid]
	s.mxSubs.RUnlock()
	if ok {
		return zettel.Meta
	}
	return nil
}

Changes to search/print.go.

85
86
87
88
89
90
91


92












93
94
95
96
97
98
99
85
86
87
88
89
90
91
92
93

94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112







+
+
-
+
+
+
+
+
+
+
+
+
+
+
+







	for j, val := range values {
		if j > 0 {
			io.WriteString(w, " AND")
		}
		if val.negate {
			io.WriteString(w, " NOT")
		}
		switch val.op {
		case cmpDefault:
		io.WriteString(w, " MATCH ")
			io.WriteString(w, " MATCH ")
		case cmpEqual:
			io.WriteString(w, " HAS ")
		case cmpPrefix:
			io.WriteString(w, " PREFIX ")
		case cmpSuffix:
			io.WriteString(w, " SUFFIX ")
		case cmpContains:
			io.WriteString(w, " CONTAINS ")
		default:
			io.WriteString(w, " MaTcH ")
		}
		if val.value == "" {
			io.WriteString(w, "ANY")
		} else {
			io.WriteString(w, val.value)
		}
	}
}

Changes to search/search.go.

10
11
12
13
14
15
16

17
18
19
20


21



















22
23
24
25
26
27
28
10
11
12
13
14
15
16
17
18
19


20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48







+


-
-
+
+

+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+








// Package search provides a zettel search.
package search

import (
	"math/rand"
	"sort"
	"strings"
	"sync"

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// Selector is used to select zettel identifier based on selection criteria.
type Selector interface {
	// Select all zettel that contains the given exact word.
	// The word must be normalized through Unicode NKFD, trimmed and not empty.
	SelectEqual(word string) id.Set

	// Select all zettel that have a word with the given prefix.
	// The prefix must be normalized through Unicode NKFD, trimmed and not empty.
	SelectPrefix(prefix string) id.Set

	// Select all zettel that have a word with the given suffix.
	// The suffix must be normalized through Unicode NKFD, trimmed and not empty.
	SelectSuffix(suffix string) id.Set

	// Select all zettel that contains the given string.
	// The string must be normalized through Unicode NKFD, trimmed and not empty.
	SelectContains(s string) id.Set
}

// MetaMatchFunc is a function determine whethe some metadata should be filtered or not.
type MetaMatchFunc func(*meta.Meta) bool

// Search specifies a mechanism for selecting zettel.
type Search struct {
	mx sync.RWMutex // Protects other attributes
40
41
42
43
44
45
46










47
48
49

50
51
52
53
54


55
56
57
58
59
60
61

62
63

64
65

66
67
68
































69
70
71
72
73
74
75
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

85
86
87
88
89
90
91
92

93
94

95
96

97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139







+
+
+
+
+
+
+
+
+
+



+




-
+
+






-
+

-
+

-
+



+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







	limit      int    // <= 0: no limit
}

type expTagValues map[string][]expValue

// RandomOrder is a pseudo metadata key that selects a random order.
const RandomOrder = "_random"

type compareOp uint8

const (
	cmpDefault compareOp = iota
	cmpEqual
	cmpPrefix
	cmpSuffix
	cmpContains
)

type expValue struct {
	value  string
	op     compareOp
	negate bool
}

// AddExpr adds a match expression to the filter.
func (s *Search) AddExpr(key, val string, negate bool) *Search {
func (s *Search) AddExpr(key, val string) *Search {
	val, negate, op := parseOp(strings.TrimSpace(val))
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
	defer s.mx.Unlock()
	if key == "" {
		s.search = append(s.search, expValue{value: val, negate: negate})
		s.search = append(s.search, expValue{value: val, op: op, negate: negate})
	} else if s.tags == nil {
		s.tags = expTagValues{key: {{value: val, negate: negate}}}
		s.tags = expTagValues{key: {{value: val, op: op, negate: negate}}}
	} else {
		s.tags[key] = append(s.tags[key], expValue{value: val, negate: negate})
		s.tags[key] = append(s.tags[key], expValue{value: val, op: op, negate: negate})
	}
	return s
}

func parseOp(s string) (r string, negate bool, op compareOp) {
	if s == "" {
		return s, false, cmpDefault
	}
	if s[0] == '\\' {
		return s[1:], false, cmpDefault
	}
	if s[0] == '!' {
		negate = true
		s = s[1:]
	}
	if s == "" {
		return s, negate, cmpDefault
	}
	if s[0] == '\\' {
		return s[1:], negate, cmpDefault
	}
	switch s[0] {
	case ':':
		return s[1:], negate, cmpDefault
	case '=':
		return s[1:], negate, cmpEqual
	case '>':
		return s[1:], negate, cmpPrefix
	case '<':
		return s[1:], negate, cmpSuffix
	case '~':
		return s[1:], negate, cmpContains
	}
	return s, negate, cmpDefault
}

// SetNegate changes the filter to reverse its selection.
func (s *Search) SetNegate() *Search {
	if s == nil {
		s = new(Search)
	}
	s.mx.Lock()
174
175
176
177
178
179
180
181

182
183
184
185
186
187
188
189
190

191
192
193
194
195
196
197
198
238
239
240
241
242
243
244

245
246
247
248
249
250
251
252


253

254
255
256
257
258
259
260







-
+







-
-
+
-







	if order := s.order; order != "" && meta.IsComputed(order) {
		return true
	}
	return false
}

// CompileMatch returns a function to match meta data based on filter specification.
func (s *Search) CompileMatch(selector index.Selector) MetaMatchFunc {
func (s *Search) CompileMatch(selector Selector) MetaMatchFunc {
	if s == nil {
		return filterNone
	}
	s.mx.Lock()
	defer s.mx.Unlock()

	compMeta := compileFilter(s.tags)
	//compSearch := createSearchAllFunc(s.search)
	compSearch := compileSearch(selector, s.search)
	compSearch := compileFullSearch(selector, s.search)

	if preMatch := s.preMatch; preMatch != nil {
		return compilePreMatch(preMatch, compMeta, compSearch, s.negate)
	}
	return compileNoPreMatch(compMeta, compSearch, s.negate)
}

func filterNone(m *meta.Meta) bool { return true }

Changes to search/selector.go.

10
11
12
13
14
15
16



17
18
19
20
21
22
23
24






































































25
26
27
28
29
30
31
32
33
34
35
36
37
38


39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

58
59

60


61
62
63
64
65
66



67

68


69
70
71
72
73
74




75

76
77




78
79
80

81
82
83
84

85
86
87
88
89










90
91
92
93
94
95



























10
11
12
13
14
15
16
17
18
19
20
21






22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103


104
105



















106

107
108

109
110
111
112
113
114
115

116
117
118
119
120

121
122
123
124
125
126
127

128
129
130
131
132
133


134
135
136
137



138
139
140
141

142





143
144
145
146
147
148
149
150
151
152






153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179







+
+
+


-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+












-
-
+
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-

+
-
+
+





-
+
+
+

+
-
+
+





-
+
+
+
+

+
-
-
+
+
+
+
-
-
-
+



-
+
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

// Package search provides a zettel search.
package search

// This file is about "compiling" a search expression into a function.

import (
	"fmt"
	"strings"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/strfun"
)

func compileSearch(selector index.Selector, search []expValue) MetaMatchFunc {
	poss, negs := normalizeSearchValues(search)
	"zettelstore.de/z/strfun"
)

func compileFullSearch(selector Selector, search []expValue) MetaMatchFunc {
	normSearch := compileNormalizedSearch(selector, search)
	plainSearch := compilePlainSearch(selector, search)
	if normSearch == nil {
		if plainSearch == nil {
			return nil
		}
		return plainSearch
	}
	if plainSearch == nil {
		return normSearch
	}
	return func(m *meta.Meta) bool {
		return normSearch(m) || plainSearch(m)
	}
}

func compileNormalizedSearch(selector Selector, search []expValue) MetaMatchFunc {
	var positives, negatives []expValue
	posSet := make(map[string]bool)
	negSet := make(map[string]bool)
	for _, val := range search {
		for _, word := range strfun.NormalizeWords(val.value) {
			if val.negate {
				if _, ok := negSet[word]; !ok {
					negSet[word] = true
					negatives = append(negatives, expValue{
						value:  word,
						op:     val.op,
						negate: true,
					})
				}
			} else {
				if _, ok := posSet[word]; !ok {
					posSet[word] = true
					positives = append(positives, expValue{
						value:  word,
						op:     val.op,
						negate: false,
					})
				}
			}
		}
	}
	return compileSearch(selector, positives, negatives)
}
func compilePlainSearch(selector Selector, search []expValue) MetaMatchFunc {
	var positives, negatives []expValue
	for _, val := range search {
		if val.negate {
			negatives = append(negatives, expValue{
				value:  strings.ToLower(strings.TrimSpace(val.value)),
				op:     val.op,
				negate: true,
			})
		} else {
			positives = append(positives, expValue{
				value:  strings.ToLower(strings.TrimSpace(val.value)),
				op:     val.op,
				negate: false,
			})
		}
	}
	return compileSearch(selector, positives, negatives)
}

func compileSearch(selector Selector, poss, negs []expValue) MetaMatchFunc {
	if len(poss) == 0 {
		if len(negs) == 0 {
			return nil
		}
		return makeNegOnlySearch(selector, negs)
	}
	if len(negs) == 0 {
		return makePosOnlySearch(selector, poss)
	}
	return makePosNegSearch(selector, poss, negs)
}

func normalizeSearchValues(search []expValue) (positives, negatives []string) {
	posSet := make(map[string]bool)
func makePosOnlySearch(selector Selector, poss []expValue) MetaMatchFunc {
	retrievePos := compileRetrieveZids(selector, poss)
	negSet := make(map[string]bool)
	for _, val := range search {
		for _, word := range strfun.NormalizeWords(val.value) {
			if val.negate {
				if _, ok := negSet[word]; !ok {
					negSet[word] = true
					negatives = append(negatives, word)
				}
			} else {
				if _, ok := posSet[word]; !ok {
					posSet[word] = true
					positives = append(positives, word)
				}
			}
		}
	}
	return positives, negatives
}

	var ids id.Set
func makePosOnlySearch(selector index.Selector, poss []string) MetaMatchFunc {
	return func(m *meta.Meta) bool {
		if ids == nil {
		ids := retrieveZids(selector, poss)
			ids = retrievePos()
		}
		_, ok := ids[m.Zid]
		return ok
	}
}

func makeNegOnlySearch(selector index.Selector, negs []string) MetaMatchFunc {
func makeNegOnlySearch(selector Selector, negs []expValue) MetaMatchFunc {
	retrieveNeg := compileRetrieveZids(selector, negs)
	var ids id.Set
	return func(m *meta.Meta) bool {
		if ids == nil {
		ids := retrieveZids(selector, negs)
			ids = retrieveNeg()
		}
		_, ok := ids[m.Zid]
		return !ok
	}
}

func makePosNegSearch(selector index.Selector, poss, negs []string) MetaMatchFunc {
func makePosNegSearch(selector Selector, poss, negs []expValue) MetaMatchFunc {
	retrievePos := compileRetrieveZids(selector, poss)
	retrieveNeg := compileRetrieveZids(selector, negs)
	var ids id.Set
	return func(m *meta.Meta) bool {
		if ids == nil {
		idsPos := retrieveZids(selector, poss)
		_, okPos := idsPos[m.Zid]
			ids = retrievePos()
			ids.Remove(retrieveNeg())
		}
		_, okPos := ids[m.Zid]
		idsNeg := retrieveZids(selector, negs)
		_, okNeg := idsNeg[m.Zid]
		return okPos && !okNeg
		return okPos
	}
}

func retrieveZids(selector index.Selector, words []string) id.Set {
func compileRetrieveZids(selector Selector, values []expValue) func() id.Set {
	var result id.Set
	for i, word := range words {
		ids := selector.SelectContains(word)
		if i == 0 {
			result = ids
	selFuncs := make([]selectorFunc, 0, len(values))
	stringVals := make([]string, 0, len(values))
	for _, val := range values {
		selFuncs = append(selFuncs, compileSelectOp(selector, val.op))
		stringVals = append(stringVals, val.value)
	}
	if len(selFuncs) == 0 {
		return func() id.Set { return id.NewSet() }
	}
	if len(selFuncs) == 1 {
			continue
		}
		result = result.Intersect(ids)
	}
	return result
}
		return func() id.Set { return selFuncs[0](stringVals[0]) }
	}
	return func() id.Set {
		result := selFuncs[0](stringVals[0])
		for i, f := range selFuncs[1:] {
			result = result.Intersect(f(stringVals[i+1]))
		}
		return result
	}
}

type selectorFunc func(string) id.Set

func compileSelectOp(selector Selector, op compareOp) selectorFunc {
	switch op {
	case cmpDefault, cmpContains:
		return selector.SelectContains
	case cmpEqual:
		return selector.SelectEqual
	case cmpPrefix:
		return selector.SelectPrefix
	case cmpSuffix:
		return selector.SelectSuffix
	default:
		panic(fmt.Sprintf("Unexpected value of comparison operation: %v", op))
	}
}

Changes to strfun/slugify.go.

8
9
10
11
12
13
14

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56




57
58

59
60
61
62
63
64
65
66
67






8
9
10
11
12
13
14
15
16
17
18
19
20































21
22
23
24
25

26
27
28
29
30

31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46







+





-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-





-
+
+
+
+

-
+









+
+
+
+
+
+
// under this license.
//-----------------------------------------------------------------------------

// Package strfun provides some string functions.
package strfun

import (
	"strings"
	"unicode"

	"golang.org/x/text/unicode/norm"
)

var (
	useUnicode = []*unicode.RangeTable{
		unicode.Letter,
		unicode.Number,
	}
	ignoreUnicode = []*unicode.RangeTable{
		unicode.Mark,
		unicode.Sk,
		unicode.Lm,
	}
)

// Slugify returns a string that can be used as part of an URL
func Slugify(s string) string {
	result := make([]rune, 0, len(s))
	addDash := false
	for _, r := range norm.NFKD.String(s) {
		if unicode.IsOneOf(useUnicode, r) {
			result = append(result, unicode.ToLower(r))
			addDash = true
		} else if !unicode.IsOneOf(ignoreUnicode, r) && addDash {
			result = append(result, '-')
			addDash = false
		}
	}
	if i := len(result) - 1; i >= 0 && result[i] == '-' {
		result = result[:i]
	}
	return string(result)
}

// NormalizeWords produces a word list that is normalized for better searching.
func NormalizeWords(s string) []string {
	result := make([]string, 0, 1)
	word := make([]rune, 0, len(s))
	for _, r := range norm.NFKD.String(s) {
		if unicode.IsOneOf(useUnicode, r) {
		if unicode.Is(unicode.Diacritic, r) {
			continue
		}
		if unicode.In(r, unicode.Letter, unicode.Number) {
			word = append(word, unicode.ToLower(r))
		} else if !unicode.IsOneOf(ignoreUnicode, r) && len(word) > 0 {
		} else if !unicode.In(r, unicode.Mark, unicode.Sk, unicode.Lm) && len(word) > 0 {
			result = append(result, string(word))
			word = word[:0]
		}
	}
	if len(word) > 0 {
		result = append(result, string(word))
	}
	return result
}

// Slugify returns a string that can be used as part of an URL
func Slugify(s string) string {
	words := NormalizeWords(s)
	return strings.Join(words, "-")
}

Changes to strfun/slugify_test.go.

47
48
49
50
51
52
53



54
55
56
57
58
59
60
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63







+
+
+







}

func TestNormalizeWord(t *testing.T) {
	tests := []struct {
		in  string
		exp []string
	}{
		{"", []string{}},
		{" ", []string{}},
		{"ˋ", []string{}}, // No single diacritic char, such as U+02CB
		{"simple test", []string{"simple", "test"}},
		{"I'm a go developer", []string{"i", "m", "a", "go", "developer"}},
		{"-!->simple   test<-!-", []string{"simple", "test"}},
		{"äöüÄÖÜß", []string{"aouaouß"}},
		{"\"aèf", []string{"aef"}},
		{"a#b", []string{"a", "b"}},
		{"*", []string{}},

Changes to strfun/strfun.go.

10
11
12
13
14
15
16

17
18
19
20
21
22
23




































10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60







+







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

// Package strfun provides some string functions.
package strfun

import (
	"strings"
	"unicode"
	"unicode/utf8"
)

// TrimSpaceRight returns a slice of the string s, with all trailing white space removed,
// as defined by Unicode.
func TrimSpaceRight(s string) string {
	return strings.TrimRightFunc(s, unicode.IsSpace)
}

// Length returns the number of runes in the given string.
func Length(s string) int {
	return utf8.RuneCountInString(s)
}

// JustifyLeft ensures that the string has a defined length.
func JustifyLeft(s string, maxLen int, pad rune) string {
	if maxLen < 1 {
		return ""
	}
	runes := make([]rune, 0, len(s))
	for _, r := range s {
		runes = append(runes, r)
	}
	if len(runes) > maxLen {
		runes = runes[:maxLen]
		runes[maxLen-1] = '\u2025'
	}

	var sb strings.Builder
	for _, r := range runes {
		sb.WriteRune(r)
	}
	for i := 0; i < maxLen-len(runes); i++ {
		sb.WriteRune(pad)
	}
	return sb.String()
}

// SplitLines splits the given string into a list of lines.
func SplitLines(s string) []string {
	return strings.FieldsFunc(s, func(r rune) bool {
		return r == '\n' || r == '\r'
	})
}

Changes to strfun/strfun_test.go.

44
45
46
47
48
49
50





































































44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
	for i, tc := range testcases {
		got := strfun.TrimSpaceRight(tc.in)
		if got != tc.exp {
			t.Errorf("%d/%q: expected %q, got %q", i, tc.in, tc.exp, got)
		}
	}
}

func TestLength(t *testing.T) {
	testcases := []struct {
		in  string
		exp int
	}{
		{"", 0},
		{"äbc", 3},
	}
	for i, tc := range testcases {
		got := strfun.Length(tc.in)
		if got != tc.exp {
			t.Errorf("%d/%q: expected %v, got %v", i, tc.in, tc.exp, got)
		}
	}
}

func TestJustifyLeft(t *testing.T) {
	testcases := []struct {
		in  string
		ml  int
		exp string
	}{
		{"", 0, ""},
		{"äbc", 0, ""},
		{"äbc", 1, "\u2025"},
		{"äbc", 2, "ä\u2025"},
		{"äbc", 3, "äbc"},
		{"äbc", 4, "äbc:"},
	}
	for i, tc := range testcases {
		got := strfun.JustifyLeft(tc.in, tc.ml, ':')
		if got != tc.exp {
			t.Errorf("%d/%q/%d: expected %q, got %q", i, tc.in, tc.ml, tc.exp, got)
		}
	}
}

func TestSplitLines(t *testing.T) {
	testcases := []struct {
		in  string
		exp []string
	}{
		{"", nil},
		{"\n", nil},
		{"a", []string{"a"}},
		{"a\n", []string{"a"}},
		{"a\n\n", []string{"a"}},
		{"a\n\nb", []string{"a", "b"}},
	}
	for i, tc := range testcases {
		got := strfun.SplitLines(tc.in)
		if !compareStringslice(tc.exp, got) {
			t.Errorf("%d/%q: expected %q, got %q", i, tc.in, tc.exp, got)
		}
	}
}

func compareStringslice(s1, s2 []string) bool {
	if len(s1) != len(s2) {
		return false
	}
	for i, s := range s1 {
		if s != s2[i] {
			return false
		}
	}
	return true
}

Changes to template/spec_test.go.

18
19
20
21
22
23
24

25
26
27
28
29
30
31
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32







+







// located. See file LICENSE.
//-----------------------------------------------------------------------------

package template_test

import (
	"encoding/json"
	"errors"
	"os"
	"path/filepath"
	"sort"
	"testing"

	"zettelstore.de/z/template"
)
179
180
181
182
183
184
185
186

187
188
189
190
191
192
193
180
181
182
183
184
185
186

187
188
189
190
191
192
193
194







-
+







	}
	return filepath.Join(curDir, "..", "testdata", "mustache")
}

func TestSpec(t *testing.T) {
	root := getRoot()
	if _, err := os.Stat(root); err != nil {
		if os.IsNotExist(err) {
		if errors.Is(err, os.ErrNotExist) {
			t.Fatalf("Could not find the mustache testdata folder at %s'", root)
		}
		t.Fatal(err)
	}

	paths, err := filepath.Glob(filepath.Join(root, "*.json"))
	if err != nil {

Changes to testdata/content/table/20200618140700.zettel.

1
2
3

4
5
1
2
3
4
5
6



+


title: Testtable

|h1>|=h2|h3:|
|%--+---+---+
|<c1|c2|:c3|
|f1|f2|=f3

Changes to tests/markdown_test.go.

39
40
41
42
43
44
45
46



47
48
49
50
51
52
53
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53
54
55







-
+
+
+







	StartLine int    `json:"start_line"`
	EndLine   int    `json:"end_line"`
	Section   string `json:"section"`
}

// exceptions lists all CommonMark tests that should not be tested for identical HTML output
var exceptions = []string{
	" - foo\n   - bar\n\t - baz\n",                             // 9
	" - foo\n   - bar\n\t - baz\n", // 9
	"<script type=\"text/javascript\">\n// JavaScript example\n\ndocument.getElementById(\"demo\").innerHTML = \"Hello JavaScript!\";\n</script>\nokay\n", // 140
	"<script>\nfoo\n</script>1. *bar*\n",                       // 147
	"- foo\n  - bar\n    - baz\n      - boo\n",                 // 264
	"10) foo\n    - bar\n",                                     // 266
	"- # Foo\n- Bar\n  ---\n  baz\n",                           // 270
	"- foo\n\n- bar\n\n\n- baz\n",                              // 276
	"- foo\n  - bar\n    - baz\n\n\n      bim\n",               // 277
	"1. a\n\n  2. b\n\n   3. c\n",                              // 281
	"1. a\n\n  2. b\n\n    3. c\n",                             // 283

Changes to tests/regression_test.go.

18
19
20
21
22
23
24
25
26


27
28

29
30
31
32
33
34
35
18
19
20
21
22
23
24


25
26
27
28
29
30
31
32
33
34
35
36







-
-
+
+


+







	"net/url"
	"os"
	"path/filepath"
	"strings"
	"testing"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/manager"

	_ "zettelstore.de/z/encoder/htmlenc"
	_ "zettelstore.de/z/encoder/jsonenc"
	_ "zettelstore.de/z/encoder/nativeenc"
45
46
47
48
49
50
51
52

53
54




55

56
57
58
59
60
61
62
63
64
65
66
67
68
69

70
71
72






73
74
75
76
77
78
79
46
47
48
49
50
51
52

53
54
55
56
57
58
59

60




61
62
63
64
65
66
67
68
69

70
71


72
73
74
75
76
77
78
79
80
81
82
83
84







-
+


+
+
+
+
-
+
-
-
-
-









-
+

-
-
+
+
+
+
+
+







func getFilePlaces(wd string, kind string) (root string, places []place.ManagedPlace) {
	root = filepath.Clean(filepath.Join(wd, "..", "testdata", kind))
	entries, err := os.ReadDir(root)
	if err != nil {
		panic(err)
	}

	cdata := manager.ConnectData{Filter: &noFilter{}, Notify: nil}
	cdata := manager.ConnectData{Config: testConfig, Enricher: &noEnrich{}, Notify: nil}
	for _, entry := range entries {
		if entry.IsDir() {
			u, err := url.Parse("dir://" + filepath.Join(root, entry.Name()) + "?type=" + kernel.PlaceDirTypeSimple)
			if err != nil {
				panic(err)
			}
			place, err := manager.Connect(
			place, err := manager.Connect(u, &noAuth{}, &cdata)
				"dir://"+filepath.Join(root, entry.Name())+"?type="+startup.ValueDirPlaceTypeSimple,
				false,
				&cdata,
			)
			if err != nil {
				panic(err)
			}
			places = append(places, place)
		}
	}
	return root, places
}

type noFilter struct{}
type noEnrich struct{}

func (nf *noFilter) Enrich(ctx context.Context, m *meta.Meta) {}
func (nf *noFilter) Remove(ctx context.Context, m *meta.Meta) {}
func (nf *noEnrich) Enrich(ctx context.Context, m *meta.Meta) {}
func (nf *noEnrich) Remove(ctx context.Context, m *meta.Meta) {}

type noAuth struct{}

func (na *noAuth) IsReadonly() bool { return false }

func trimLastEOL(s string) string {
	if lastPos := len(s) - 1; lastPos >= 0 && s[lastPos] == '\n' {
		return s[:lastPos]
	}
	return s
}
118
119
120
121
122
123
124
125

126
127
128
129
130
131
132
123
124
125
126
127
128
129

130
131
132
133
134
135
136
137







-
+







	zmkEncoder := encoder.Create("zmk", nil)
	var sb strings.Builder
	zmkEncoder.WriteBlocks(&sb, zn.Ast)
	gotFirst := sb.String()
	sb.Reset()

	newZettel := parser.ParseZettel(domain.Zettel{
		Meta: zn.Meta, Content: domain.NewContent("\n" + gotFirst)}, "")
		Meta: zn.Meta, Content: domain.NewContent("\n" + gotFirst)}, "", testConfig)
	zmkEncoder.WriteBlocks(&sb, newZettel.Ast)
	gotSecond := sb.String()
	sb.Reset()

	if gotFirst != gotSecond {
		t.Errorf("\n1st: %q\n2nd: %q", gotFirst, gotSecond)
	}
152
153
154
155
156
157
158
159

160
161
162
163
164
165
166
157
158
159
160
161
162
163

164
165
166
167
168
169
170
171







-
+







		panic(err)
	}
	for _, meta := range metaList {
		zettel, err := p.GetZettel(context.Background(), meta.Zid)
		if err != nil {
			panic(err)
		}
		z := parser.ParseZettel(zettel, "")
		z := parser.ParseZettel(zettel, "", testConfig)
		for _, format := range formats {
			t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, format), func(st *testing.T) {
				resultName := filepath.Join(wd, "result", "content", placeName, z.Zid.String()+"."+format)
				checkBlocksFile(st, resultName, z, format)
			})
		}
		t.Run(fmt.Sprintf("%s::%d", p.Location(), meta.Zid), func(st *testing.T) {
205
206
207
208
209
210
211
212

213
214
215
216
217
218
219
220
221
222
223
224





















225
226
227
228
229
230
231
232
233
234
210
211
212
213
214
215
216

217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260







-
+












+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+










		panic(err)
	}
	for _, meta := range metaList {
		zettel, err := p.GetZettel(context.Background(), meta.Zid)
		if err != nil {
			panic(err)
		}
		z := parser.ParseZettel(zettel, "")
		z := parser.ParseZettel(zettel, "", testConfig)
		for _, format := range formats {
			t.Run(fmt.Sprintf("%s::%d(%s)", p.Location(), meta.Zid, format), func(st *testing.T) {
				resultName := filepath.Join(wd, "result", "meta", placeName, z.Zid.String()+"."+format)
				checkMetaFile(st, resultName, z, format)
			})
		}
	}
	if err := ss.Stop(context.Background()); err != nil {
		panic(err)
	}
}

type myConfig struct{}

func (cfg *myConfig) AddDefaultValues(m *meta.Meta) *meta.Meta { return m }
func (cfg *myConfig) GetDefaultTitle() string                  { return "" }
func (cfg *myConfig) GetDefaultRole() string                   { return meta.ValueRoleZettel }
func (cfg *myConfig) GetDefaultSyntax() string                 { return meta.ValueSyntaxZmk }
func (cfg *myConfig) GetDefaultLang() string                   { return "" }
func (cfg *myConfig) GetDefaultVisibility() meta.Visibility    { return meta.VisibilityPublic }
func (cfg *myConfig) GetFooterHTML() string                    { return "" }
func (cfg *myConfig) GetHomeZettel() id.Zid                    { return id.Invalid }
func (cfg *myConfig) GetListPageSize() int                     { return 0 }
func (cfg *myConfig) GetMarkerExternal() string                { return "" }
func (cfg *myConfig) GetSiteName() string                      { return "" }
func (cfg *myConfig) GetYAMLHeader() bool                      { return false }
func (cfg *myConfig) GetZettelFileSyntax() []string            { return nil }

func (cfg *myConfig) GetExpertMode() bool                      { return false }
func (cfg *myConfig) GetVisibility(*meta.Meta) meta.Visibility { return cfg.GetDefaultVisibility() }

var testConfig = &myConfig{}

func TestMetaRegression(t *testing.T) {
	wd, err := os.Getwd()
	if err != nil {
		panic(err)
	}
	root, places := getFilePlaces(wd, "meta")
	for _, p := range places {
		checkMetaPlace(t, p, wd, getPlaceName(p, root))
	}
}

Changes to tools/build.go.

10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


28
29
30
31
32
33
34
10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35







-










+
+








// Package main provides a command to build and run the software.
package main

import (
	"archive/zip"
	"bytes"
	"errors"
	"flag"
	"fmt"
	"io"
	"io/fs"
	"os"
	"os/exec"
	"path/filepath"
	"regexp"
	"strings"
	"time"

	"zettelstore.de/z/strfun"
)

func executeCommand(env []string, name string, arg ...string) (string, error) {
	if verbose {
		if len(env) > 0 {
			for i, e := range env {
				fmt.Fprintf(os.Stderr, "ENV%d %v\n", i+1, e)
55
56
57
58
59
60
61
62
63



64
65
66
67
68

69
70
71



72
73
74
75





76
77


78
79
80
81

82
83
84
85









86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
56
57
58
59
60
61
62


63
64
65
66
67
68
69

70
71
72
73
74
75
76




77
78
79
80
81


82
83




84




85
86
87
88
89
90
91
92
93
94
95
96
97
98
99






100
101
102
103
104
105
106







-
-
+
+
+




-
+



+
+
+
-
-
-
-
+
+
+
+
+
-
-
+
+
-
-
-
-
+
-
-
-
-
+
+
+
+
+
+
+
+
+






-
-
-
-
-
-







		return "", err
	}
	return strings.TrimFunc(string(content), func(r rune) bool {
		return r <= ' '
	}), nil
}

var fossilHash = regexp.MustCompile(`\[[0-9a-fA-F]+\]`)
var dirtyPrefixes = []string{"DELETED", "ADDED", "UPDATED", "CONFLICT", "EDITED", "RENAMED"}
var fossilCheckout = regexp.MustCompile(`^checkout:\s+([0-9a-f]+)\s`)
var dirtyPrefixes = []string{
	"DELETED ", "ADDED ", "UPDATED ", "CONFLICT ", "EDITED ", "RENAMED ", "EXTRA "}

const dirtySuffix = "-dirty"

func readFossilVersion() (string, error) {
	s, err := executeCommand(nil, "fossil", "timeline", "--limit", "1")
	s, err := executeCommand(nil, "fossil", "status", "--differ")
	if err != nil {
		return "", err
	}
	var hash, suffix string
	for _, line := range strfun.SplitLines(s) {
		if hash == "" {
	hash := fossilHash.FindString(s)
	if len(hash) < 3 {
		return "", errors.New("no fossil hash found")
	}
			if m := fossilCheckout.FindStringSubmatch(line); len(m) > 0 {
				hash = m[1][:10]
				if suffix != "" {
					return hash + suffix, nil
				}
	hash = hash[1 : len(hash)-1]

				continue
			}
	s, err = executeCommand(nil, "fossil", "status")
	if err != nil {
		return "", err
	}
		}
	for _, line := range splitLines(s) {
		for _, prefix := range dirtyPrefixes {
			if strings.HasPrefix(line, prefix) {
				return hash + dirtySuffix, nil
		if suffix == "" {
			for _, prefix := range dirtyPrefixes {
				if strings.HasPrefix(line, prefix) {
					suffix = dirtySuffix
					if hash != "" {
						return hash + suffix, nil
					}
					break
				}
			}
		}
	}
	return hash, nil
}

func splitLines(s string) []string {
	return strings.FieldsFunc(s, func(r rune) bool {
		return r == '\n' || r == '\r'
	})
}

func getVersionData() (string, string) {
	base, err := readVersionFile()
	if err != nil {
		base = "dev"
	}
	fossil, err := readFossilVersion()
	if err != nil {
139
140
141
142
143
144
145
146

147
148
149
150
151
152
153
141
142
143
144
145
146
147

148
149
150
151
152
153
154
155







-
+







	}
	return checkFossilExtra()
}

func checkGoTest() error {
	out, err := executeCommand(nil, "go", "test", "./...")
	if err != nil {
		for _, line := range splitLines(out) {
		for _, line := range strfun.SplitLines(out) {
			if strings.HasPrefix(line, "ok") || strings.HasPrefix(line, "?") {
				continue
			}
			fmt.Fprintln(os.Stderr, line)
		}
	}
	return err
202
203
204
205
206
207
208
209

210
211
212
213
214
215
216
204
205
206
207
208
209
210

211
212
213
214
215
216
217
218







-
+







	out, err := executeCommand(nil, "fossil", "extra")
	if err != nil {
		fmt.Fprintln(os.Stderr, "Unable to execute 'fossil extra'")
		return err
	}
	if len(out) > 0 {
		fmt.Fprint(os.Stderr, "Warning: unversioned file(s):")
		for i, extra := range splitLines(out) {
		for i, extra := range strfun.SplitLines(out) {
			if i > 0 {
				fmt.Fprint(os.Stderr, ",")
			}
			fmt.Fprintf(os.Stderr, " %q", extra)
		}
		fmt.Fprintln(os.Stderr)
	}

Changes to usecase/authenticate.go.

12
13
14
15
16
17
18
19
20


21
22
23
24
25
26
27
28
29
30
31
32
33

34
35
36
37
38
39

40

41
42

43
44
45
46
47

48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

63
64
65
66
67
68
69
12
13
14
15
16
17
18


19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
42
43

44
45
46
47
48

49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

64
65
66
67
68
69
70
71







-
-
+
+













+





-
+

+

-
+




-
+














-
+







package usecase

import (
	"context"
	"math/rand"
	"time"

	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/auth/token"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/auth/cred"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/search"
)

// AuthenticatePort is the interface used by this use case.
type AuthenticatePort interface {
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)
}

// Authenticate is the data for this use case.
type Authenticate struct {
	token     auth.TokenManager
	port      AuthenticatePort
	ucGetUser GetUser
}

// NewAuthenticate creates a new use case.
func NewAuthenticate(port AuthenticatePort) Authenticate {
func NewAuthenticate(token auth.TokenManager, authz auth.AuthzManager, port AuthenticatePort) Authenticate {
	return Authenticate{
		token:     token,
		port:      port,
		ucGetUser: NewGetUser(port),
		ucGetUser: NewGetUser(authz, port),
	}
}

// Run executes the use case.
func (uc Authenticate) Run(ctx context.Context, ident, credential string, d time.Duration, k token.Kind) ([]byte, error) {
func (uc Authenticate) Run(ctx context.Context, ident, credential string, d time.Duration, k auth.TokenKind) ([]byte, error) {
	identMeta, err := uc.ucGetUser.Run(ctx, ident)
	defer addDelay(time.Now(), 500*time.Millisecond, 100*time.Millisecond)

	if identMeta == nil || err != nil {
		compensateCompare()
		return nil, err
	}

	if hashCred, ok := identMeta.Get(meta.KeyCredential); ok {
		ok, err := cred.CompareHashAndCredential(hashCred, identMeta.Zid, ident, credential)
		if err != nil {
			return nil, err
		}
		if ok {
			token, err := token.GetToken(identMeta, d, k)
			token, err := uc.token.GetToken(identMeta, d, k)
			if err != nil {
				return nil, err
			}
			return token, nil
		}
		return nil, nil
	}

Changes to usecase/create_zettel.go.

10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25
26
27
28
29
30
31

32

33
34
35
36
37





38
39
40
41
42
43
44
45
46
47
48

49
50
51

52
53
54

55
56

57
58
59
60
10
11
12
13
14
15
16

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

33
34
35
36


37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

52
53
54

55
56
57

58
59

60
61
62
63
64







-
+














+
-
+



-
-
+
+
+
+
+










-
+


-
+


-
+

-
+





// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"context"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/strfun"
)

// CreateZettelPort is the interface used by this use case.
type CreateZettelPort interface {
	// CreateZettel creates a new zettel.
	CreateZettel(ctx context.Context, zettel domain.Zettel) (id.Zid, error)
}

// CreateZettel is the data for this use case.
type CreateZettel struct {
	rtConfig config.Config
	port CreateZettelPort
	port     CreateZettelPort
}

// NewCreateZettel creates a new use case.
func NewCreateZettel(port CreateZettelPort) CreateZettel {
	return CreateZettel{port: port}
func NewCreateZettel(rtConfig config.Config, port CreateZettelPort) CreateZettel {
	return CreateZettel{
		rtConfig: rtConfig,
		port:     port,
	}
}

// Run executes the use case.
func (uc CreateZettel) Run(ctx context.Context, zettel domain.Zettel) (id.Zid, error) {
	m := zettel.Meta
	if m.Zid.IsValid() {
		return m.Zid, nil // TODO: new error: already exists
	}

	if title, ok := m.Get(meta.KeyTitle); !ok || title == "" {
		m.Set(meta.KeyTitle, runtime.GetDefaultTitle())
		m.Set(meta.KeyTitle, uc.rtConfig.GetDefaultTitle())
	}
	if role, ok := m.Get(meta.KeyRole); !ok || role == "" {
		m.Set(meta.KeyRole, runtime.GetDefaultRole())
		m.Set(meta.KeyRole, uc.rtConfig.GetDefaultRole())
	}
	if syntax, ok := m.Get(meta.KeySyntax); !ok || syntax == "" {
		m.Set(meta.KeySyntax, runtime.GetDefaultSyntax())
		m.Set(meta.KeySyntax, uc.rtConfig.GetDefaultSyntax())
	}
	m.YamlSep = runtime.GetYAMLHeader()
	m.YamlSep = uc.rtConfig.GetYAMLHeader()

	zettel.Content = domain.Content(strfun.TrimSpaceRight(zettel.Content.AsString()))
	return uc.port.CreateZettel(ctx, zettel)
}

Changes to usecase/folge_zettel.go.

8
9
10
11
12
13
14
15

16
17
18
19
20
21
22
23




24
25
26


27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

42
43

44
45
46
8
9
10
11
12
13
14

15
16
17
18
19
20
21


22
23
24
25
26


27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

43
44

45
46
47
48







-
+






-
-
+
+
+
+

-
-
+
+














-
+

-
+



// under this license.
//-----------------------------------------------------------------------------

// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// FolgeZettel is the data for this use case.
type FolgeZettel struct{}

type FolgeZettel struct {
	rtConfig config.Config
}

// NewFolgeZettel creates a new use case.
func NewFolgeZettel() FolgeZettel {
	return FolgeZettel{}
func NewFolgeZettel(rtConfig config.Config) FolgeZettel {
	return FolgeZettel{rtConfig}
}

// Run executes the use case.
func (uc FolgeZettel) Run(origZettel domain.Zettel) domain.Zettel {
	origMeta := origZettel.Meta
	m := meta.New(id.Invalid)
	if title, ok := origMeta.Get(meta.KeyTitle); ok {
		if len(title) > 0 {
			title = "Folge of " + title
		} else {
			title = "Folge"
		}
		m.Set(meta.KeyTitle, title)
	}
	m.Set(meta.KeyRole, runtime.GetRole(origMeta))
	m.Set(meta.KeyRole, config.GetRole(origMeta, uc.rtConfig))
	m.Set(meta.KeyTags, origMeta.GetDefault(meta.KeyTags, ""))
	m.Set(meta.KeySyntax, runtime.GetSyntax(origMeta))
	m.Set(meta.KeySyntax, uc.rtConfig.GetDefaultSyntax())
	m.Set(meta.KeyPrecursor, origMeta.Zid.String())
	return domain.Zettel{Meta: m, Content: ""}
}

Changes to usecase/get_user.go.

10
11
12
13
14
15
16
17

18
19
20

21
22
23
24
25
26
27
28
29
30
31
32
33
34

35

36
37
38
39
40


41
42
43
44
45
46
47
48

49
50
51
52
53

54
55
56
57
58
59
60
61
62
63
64


65
66
67
68
69
70
71
10
11
12
13
14
15
16

17
18
19

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

36
37
38
39


40
41
42
43
44
45




46
47
48
49
50

51
52
53
54
55
56
57
58
59
60


61
62
63
64
65
66
67
68
69







-
+


-
+














+
-
+



-
-
+
+




-
-
-
-
+




-
+









-
-
+
+








// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"context"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
)

// Use case: return user identified by meta key ident.
// ---------------------------------------------------

// GetUserPort is the interface used by this use case.
type GetUserPort interface {
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)
}

// GetUser is the data for this use case.
type GetUser struct {
	authz auth.AuthzManager
	port GetUserPort
	port  GetUserPort
}

// NewGetUser creates a new use case.
func NewGetUser(port GetUserPort) GetUser {
	return GetUser{port: port}
func NewGetUser(authz auth.AuthzManager, port GetUserPort) GetUser {
	return GetUser{authz: authz, port: port}
}

// Run executes the use case.
func (uc GetUser) Run(ctx context.Context, ident string) (*meta.Meta, error) {
	if !startup.WithAuth() {
		return nil, nil
	}
	ctx = index.NoEnrichContext(ctx)
	ctx = place.NoEnrichContext(ctx)

	// It is important to try first with the owner. First, because another user
	// could give herself the same ''ident''. Second, in most cases the owner
	// will authenticate.
	identMeta, err := uc.port.GetMeta(ctx, startup.Owner())
	identMeta, err := uc.port.GetMeta(ctx, uc.authz.Owner())
	if err == nil && identMeta.GetDefault(meta.KeyUserID, "") == ident {
		if role, ok := identMeta.Get(meta.KeyRole); !ok ||
			role != meta.ValueRoleUser {
			return nil, nil
		}
		return identMeta, nil
	}
	// Owner was not found or has another ident. Try via list search.
	var s *search.Search
	s = s.AddExpr(meta.KeyRole, meta.ValueRoleUser, false)
	s = s.AddExpr(meta.KeyUserID, ident, false)
	s = s.AddExpr(meta.KeyRole, meta.ValueRoleUser)
	s = s.AddExpr(meta.KeyUserID, ident)
	metaList, err := uc.port.SelectMeta(ctx, s)
	if err != nil {
		return nil, err
	}
	if len(metaList) < 1 {
		return nil, nil
	}
86
87
88
89
90
91
92
93
94
95



96
97
98
99
100
101
102
103
104
84
85
86
87
88
89
90



91
92
93
94
95
96
97
98
99
100
101
102







-
-
-
+
+
+









}

// NewGetUserByZid creates a new use case.
func NewGetUserByZid(port GetUserByZidPort) GetUserByZid {
	return GetUserByZid{port: port}
}

// Run executes the use case.
func (uc GetUserByZid) Run(ctx context.Context, zid id.Zid, ident string) (*meta.Meta, error) {
	userMeta, err := uc.port.GetMeta(index.NoEnrichContext(ctx), zid)
// GetUser executes the use case.
func (uc GetUserByZid) GetUser(ctx context.Context, zid id.Zid, ident string) (*meta.Meta, error) {
	userMeta, err := uc.port.GetMeta(place.NoEnrichContext(ctx), zid)
	if err != nil {
		return nil, err
	}

	if val, ok := userMeta.Get(meta.KeyUserID); !ok || val != ident {
		return nil, nil
	}
	return userMeta, nil
}

Changes to usecase/list_role.go.

12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26







-
+







package usecase

import (
	"context"
	"sort"

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
)

// ListRolePort is the interface used by this use case.
type ListRolePort interface {
	// SelectMeta returns all zettel meta data that match the selection criteria.
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)
34
35
36
37
38
39
40
41

42
43
44
45
46
47
48
34
35
36
37
38
39
40

41
42
43
44
45
46
47
48







-
+







// NewListRole creates a new use case.
func NewListRole(port ListRolePort) ListRole {
	return ListRole{port: port}
}

// Run executes the use case.
func (uc ListRole) Run(ctx context.Context) ([]string, error) {
	metas, err := uc.port.SelectMeta(index.NoEnrichContext(ctx), nil)
	metas, err := uc.port.SelectMeta(place.NoEnrichContext(ctx), nil)
	if err != nil {
		return nil, err
	}
	roles := make(map[string]bool, 8)
	for _, m := range metas {
		if role, ok := m.Get(meta.KeyRole); ok && role != "" {
			roles[role] = true

Changes to usecase/list_tags.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25







-
+







// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"context"

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
)

// ListTagsPort is the interface used by this use case.
type ListTagsPort interface {
	// SelectMeta returns all zettel meta data that match the selection criteria.
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
36
37
38
39
40
41
42

43
44
45
46
47
48
49
50







-
+







}

// TagData associates tags with a list of all zettel meta that use this tag
type TagData map[string][]*meta.Meta

// Run executes the use case.
func (uc ListTags) Run(ctx context.Context, minCount int) (TagData, error) {
	metas, err := uc.port.SelectMeta(index.NoEnrichContext(ctx), nil)
	metas, err := uc.port.SelectMeta(place.NoEnrichContext(ctx), nil)
	if err != nil {
		return nil, err
	}
	result := make(TagData)
	for _, m := range metas {
		if tl, ok := m.GetList(meta.KeyTags); ok && len(tl) > 0 {
			for _, t := range tl {

Changes to usecase/parse_zettel.go.

11
12
13
14
15
16
17

18
19
20
21
22
23

24
25
26
27
28
29


30
31
32
33
34
35
36
37
38
39
40

41
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


30
31
32
33
34
35
36
37
38
39
40
41

42
43







+






+




-
-
+
+










-
+

// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"context"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/parser"
)

// ParseZettel is the data for this use case.
type ParseZettel struct {
	rtConfig  config.Config
	getZettel GetZettel
}

// NewParseZettel creates a new use case.
func NewParseZettel(getZettel GetZettel) ParseZettel {
	return ParseZettel{getZettel: getZettel}
func NewParseZettel(rtConfig config.Config, getZettel GetZettel) ParseZettel {
	return ParseZettel{rtConfig: rtConfig, getZettel: getZettel}
}

// Run executes the use case.
func (uc ParseZettel) Run(
	ctx context.Context, zid id.Zid, syntax string) (*ast.ZettelNode, error) {
	zettel, err := uc.getZettel.Run(ctx, zid)
	if err != nil {
		return nil, err
	}

	return parser.ParseZettel(zettel, syntax), nil
	return parser.ParseZettel(zettel, syntax, uc.rtConfig), nil
}

Changes to usecase/rename_zettel.go.

12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26







-
+







package usecase

import (
	"context"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
)

// RenameZettelPort is the interface used by this use case.
type RenameZettelPort interface {
	// GetMeta retrieves just the meta data of a specific zettel.
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)

43
44
45
46
47
48
49
50

51
52
53
54
55
56
57
58
59
60
61
62
43
44
45
46
47
48
49

50
51
52
53
54
55
56
57
58
59
60
61
62







-
+












// NewRenameZettel creates a new use case.
func NewRenameZettel(port RenameZettelPort) RenameZettel {
	return RenameZettel{port: port}
}

// Run executes the use case.
func (uc RenameZettel) Run(ctx context.Context, curZid, newZid id.Zid) error {
	noEnrichCtx := index.NoEnrichContext(ctx)
	noEnrichCtx := place.NoEnrichContext(ctx)
	if _, err := uc.port.GetMeta(noEnrichCtx, curZid); err != nil {
		return err
	}
	if newZid == curZid {
		// Nothing to do
		return nil
	}
	if _, err := uc.port.GetMeta(noEnrichCtx, newZid); err == nil {
		return &ErrZidInUse{Zid: newZid}
	}
	return uc.port.RenameZettel(ctx, curZid, newZid)
}

Changes to usecase/search.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
11
12
13
14
15
16
17

18
19
20
21
22
23
24
25







-
+







// Package usecase provides (business) use cases for the zettelstore.
package usecase

import (
	"context"

	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
)

// SearchPort is the interface used by this use case.
type SearchPort interface {
	// SelectMeta returns all zettel meta data that match the selection criteria.
	SelectMeta(ctx context.Context, s *search.Search) ([]*meta.Meta, error)
34
35
36
37
38
39
40
41

42
43
44
34
35
36
37
38
39
40

41
42
43
44







-
+



func NewSearch(port SearchPort) Search {
	return Search{port: port}
}

// Run executes the use case.
func (uc Search) Run(ctx context.Context, s *search.Search) ([]*meta.Meta, error) {
	if !s.HasComputedMetaKey() {
		ctx = index.NoEnrichContext(ctx)
		ctx = place.NoEnrichContext(ctx)
	}
	return uc.port.SelectMeta(ctx, s)
}

Changes to usecase/update_zettel.go.

13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
13
14
15
16
17
18
19

20
21
22
23
24
25
26
27







-
+








import (
	"context"

	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/strfun"
)

// UpdateZettelPort is the interface used by this use case.
type UpdateZettelPort interface {
	// GetZettel retrieves a specific zettel.
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)
39
40
41
42
43
44
45
46

47
48
49
50
51
52
53
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53







-
+







func NewUpdateZettel(port UpdateZettelPort) UpdateZettel {
	return UpdateZettel{port: port}
}

// Run executes the use case.
func (uc UpdateZettel) Run(ctx context.Context, zettel domain.Zettel, hasContent bool) error {
	m := zettel.Meta
	oldZettel, err := uc.port.GetZettel(index.NoEnrichContext(ctx), m.Zid)
	oldZettel, err := uc.port.GetZettel(place.NoEnrichContext(ctx), m.Zid)
	if err != nil {
		return err
	}
	if zettel.Equal(oldZettel, false) {
		return nil
	}
	m.SetNow(meta.KeyModified)

Added web/adapter/api/api.go.































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"context"
	"time"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/web/server"
)

// API holds all data and methods for delivering API call results.
type API struct {
	b        server.Builder
	rtConfig config.Config
	authz    auth.AuthzManager
	token    auth.TokenManager
	auth     server.Auth

	tokenLifetime time.Duration
}

// New creates a new API object.
func New(b server.Builder, authz auth.AuthzManager, token auth.TokenManager, auth server.Auth, rtConfig config.Config) *API {
	api := &API{
		b:        b,
		authz:    authz,
		token:    token,
		auth:     auth,
		rtConfig: rtConfig,

		tokenLifetime: kernel.Main.GetConfig(kernel.WebService, kernel.WebTokenLifetimeAPI).(time.Duration),
	}
	return api
}

// GetURLPrefix returns the configured URL prefix of the web server.
func (api *API) GetURLPrefix() string { return api.b.GetURLPrefix() }

// NewURLBuilder creates a new URL builder object with the given key.
func (api *API) NewURLBuilder(key byte) server.URLBuilder { return api.b.NewURLBuilder(key) }

func (api *API) getAuthData(ctx context.Context) *server.AuthData {
	return api.auth.GetAuthData(ctx)
}
func (api *API) withAuth() bool { return api.authz.WithAuth() }
func (api *API) getToken(ident *meta.Meta) ([]byte, error) {
	return api.token.GetToken(ident, api.tokenLifetime, auth.KindJSON)
}

Changes to web/adapter/api/get_links.go.

37
38
39
40
41
42
43
44

45
46
47
48
49
50

51
52

53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

69
70
71

72
73
74

75
76
77
78
79
80
81
82
83
84
85
86
87

88
89
90
91

92
93

94
95
96
97
98
99
100
101
102
103
104


105
106

107
108
109
110
111
112
113
114
115
116

117
118
119
120

121
122
123
124
125
126
127
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52

53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

69
70
71

72
73
74

75
76
77
78
79
80
81
82
83
84
85
86
87

88
89
90
91

92
93

94
95
96
97
98
99
100
101
102
103


104
105
106

107
108
109
110
111
112
113
114
115
116

117
118
119
120

121
122
123
124
125
126
127
128







-
+






+

-
+















-
+


-
+


-
+












-
+



-
+

-
+









-
-
+
+

-
+









-
+



-
+







		Local    []string    `json:"local"`
		External []string    `json:"external"`
	} `json:"images"`
	Cites []string `json:"cites"`
}

// MakeGetLinksHandler creates a new API handler to return links to other material.
func MakeGetLinksHandler(parseZettel usecase.ParseZettel) http.HandlerFunc {
func (api *API) MakeGetLinksHandler(parseZettel usecase.ParseZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		ctx := r.Context()
		q := r.URL.Query()
		zn, err := parseZettel.Run(r.Context(), zid, q.Get("syntax"))
		zn, err := parseZettel.Run(ctx, zid, q.Get("syntax"))
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		summary := collect.References(zn)

		kind := getKindFromValue(q.Get("kind"))
		matter := getMatterFromValue(q.Get("matter"))
		if !validKindMatter(kind, matter) {
			adapter.BadRequest(w, "Invalid kind/matter")
			return
		}

		outData := jsonGetLinks{
			ID:  zid.String(),
			URL: adapter.NewURLBuilder('z').SetZid(zid).String(),
			URL: api.NewURLBuilder('z').SetZid(zid).String(),
		}
		if kind&kindLink != 0 {
			setupLinkJSONRefs(summary, matter, &outData)
			api.setupLinkJSONRefs(summary, matter, &outData)
		}
		if kind&kindImage != 0 {
			setupImageJSONRefs(summary, matter, &outData)
			api.setupImageJSONRefs(summary, matter, &outData)
		}
		if kind&kindCite != 0 {
			outData.Cites = stringCites(summary.Cites)
		}

		w.Header().Set(adapter.ContentType, format2ContentType("json"))
		enc := json.NewEncoder(w)
		enc.SetEscapeHTML(false)
		enc.Encode(&outData)
	}
}

func setupLinkJSONRefs(summary collect.Summary, matter matterType, outData *jsonGetLinks) {
func (api *API) setupLinkJSONRefs(summary collect.Summary, matter matterType, outData *jsonGetLinks) {
	if matter&matterIncoming != 0 {
		outData.Links.Incoming = []jsonIDURL{}
	}
	zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Links, false)
	zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Links)
	if matter&matterOutgoing != 0 {
		outData.Links.Outgoing = idURLRefs(zetRefs)
		outData.Links.Outgoing = api.idURLRefs(zetRefs)
	}
	if matter&matterLocal != 0 {
		outData.Links.Local = stringRefs(locRefs)
	}
	if matter&matterExternal != 0 {
		outData.Links.External = stringRefs(extRefs)
	}
}

func setupImageJSONRefs(summary collect.Summary, matter matterType, outData *jsonGetLinks) {
	zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Images, false)
func (api *API) setupImageJSONRefs(summary collect.Summary, matter matterType, outData *jsonGetLinks) {
	zetRefs, locRefs, extRefs := collect.DivideReferences(summary.Images)
	if matter&matterOutgoing != 0 {
		outData.Images.Outgoing = idURLRefs(zetRefs)
		outData.Images.Outgoing = api.idURLRefs(zetRefs)
	}
	if matter&matterLocal != 0 {
		outData.Images.Local = stringRefs(locRefs)
	}
	if matter&matterExternal != 0 {
		outData.Images.External = stringRefs(extRefs)
	}
}

func idURLRefs(refs []*ast.Reference) []jsonIDURL {
func (api *API) idURLRefs(refs []*ast.Reference) []jsonIDURL {
	result := make([]jsonIDURL, 0, len(refs))
	for _, ref := range refs {
		path := ref.URL.Path
		ub := adapter.NewURLBuilder('z').AppendPath(path)
		ub := api.NewURLBuilder('z').AppendPath(path)
		if fragment := ref.URL.Fragment; len(fragment) > 0 {
			ub.SetFragment(fragment)
		}
		result = append(result, jsonIDURL{ID: path, URL: ub.String()})
	}
	return result
}

Changes to web/adapter/api/get_order.go.

17
18
19
20
21
22
23
24

25
26
27
28
29
30

31
32

33
34
35
36
37

38
39
17
18
19
20
21
22
23

24
25
26
27
28
29
30
31
32

33
34
35
36
37

38
39
40







-
+






+

-
+




-
+


	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetOrderHandler creates a new API handler to return zettel references
// of a given zettel.
func MakeGetOrderHandler(zettelOrder usecase.ZettelOrder) http.HandlerFunc {
func (api *API) MakeGetOrderHandler(zettelOrder usecase.ZettelOrder) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		ctx := r.Context()
		q := r.URL.Query()
		start, metas, err := zettelOrder.Run(r.Context(), zid, q.Get("syntax"))
		start, metas, err := zettelOrder.Run(ctx, zid, q.Get("syntax"))
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		writeMetaList(w, start, metas)
		api.writeMetaList(w, start, metas)
	}
}

Changes to web/adapter/api/get_role_list.go.

18
19
20
21
22
23
24
25

26
27
28
29
30
31
32
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32







-
+







	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/jsonenc"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeListRoleHandler creates a new HTTP handler for the use case "list some zettel".
func MakeListRoleHandler(listRole usecase.ListRole) http.HandlerFunc {
func (api *API) MakeListRoleHandler(listRole usecase.ListRole) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		roleList, err := listRole.Run(r.Context())
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}

Changes to web/adapter/api/get_tags_list.go.

20
21
22
23
24
25
26
27

28
29
30
31
32
33
34
20
21
22
23
24
25
26

27
28
29
30
31
32
33
34







-
+







	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/jsonenc"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeListTagsHandler creates a new HTTP handler for the use case "list some zettel".
func MakeListTagsHandler(listTags usecase.ListTags) http.HandlerFunc {
func (api *API) MakeListTagsHandler(listTags usecase.ListTags) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		iMinCount, _ := strconv.Atoi(r.URL.Query().Get("min"))
		tagData, err := listTags.Run(r.Context(), iMinCount)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}

Changes to web/adapter/api/get_zettel.go.

8
9
10
11
12
13
14

15
16
17

18

19
20
21
22

23
24
25
26
27
28
29

30
31
32
33
34
35
36
37
38
39
40
41

42
43
44
45
46
47
48
49
50
51
52
53
54
55






56
57
58
59
60

61
62
63
64
65
66
67
68
69
70


71
72

73
74
75
76
77
78
79
80
81

82
83
84
85
86
87
88
89
90
91
92
93
94
95

96
97
98
99
100
101
102
103
104
105
106
107

108
109
110
111
112
113

114
115
116
117
118
119
120






































8
9
10
11
12
13
14
15
16
17
18
19

20
21
22
23

24
25
26
27
28
29


30
31
32
33
34
35
36
37
38
39
40
41

42
43
44
45
46
47
48
49
50






51
52
53
54
55
56
57




58

59
60
61
62
63
64
65


66
67
68

69
70
71
72
73
74
75
76


77









78




79



80








81



82
83

84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129







+



+
-
+



-
+





-
-
+











-
+








-
-
-
-
-
-
+
+
+
+
+
+

-
-
-
-
+
-







-
-
+
+

-
+







-
-
+
-
-
-
-
-
-
-
-
-

-
-
-
-
+
-
-
-

-
-
-
-
-
-
-
-
+
-
-
-


-
+







+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
// under this license.
//-----------------------------------------------------------------------------

// Package api provides api handlers for web requests.
package api

import (
	"errors"
	"fmt"
	"net/http"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeGetZettelHandler creates a new HTTP handler to return a rendered zettel.
func MakeGetZettelHandler(
	parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc {
func (api *API) MakeGetZettelHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}

		ctx := r.Context()
		q := r.URL.Query()
		format := adapter.GetFormat(r, q, encoder.GetDefaultFormat())
		if format == "raw" {
			ctx = index.NoEnrichContext(ctx)
			ctx = place.NoEnrichContext(ctx)
		}
		zn, err := parseZettel.Run(ctx, zid, q.Get("syntax"))
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}

		part := getPart(q, partZettel)
		switch format {
		case "json", "djson":
			if part == partUnknown {
				adapter.BadRequest(w, "Unknown _part parameter")
				return
			}
		if part == partUnknown {
			adapter.BadRequest(w, "Unknown _part parameter")
			return
		}
		switch format {
		case "json", "djson":
			w.Header().Set(adapter.ContentType, format2ContentType(format))
			if format != "djson" {
				err = writeJSONZettel(w, zn, part)
			} else {
				err = writeDJSONZettel(ctx, w, zn, part, partZettel, getMeta)
			err = api.getWriteMetaZettelFunc(ctx, format, part, partZettel, getMeta)(w, zn)
			}
			if err != nil {
				adapter.InternalServerError(w, "Write D/JSON", err)
			}
			return
		}

		env := encoder.Environment{
			LinkAdapter:    adapter.MakeLinkAdapter(ctx, 'z', getMeta, part.DefString(partZettel), format),
			ImageAdapter:   adapter.MakeImageAdapter(),
			LinkAdapter:    adapter.MakeLinkAdapter(ctx, api, 'z', getMeta, part.DefString(partZettel), format),
			ImageAdapter:   adapter.MakeImageAdapter(ctx, api, getMeta),
			CiteAdapter:    nil,
			Lang:           runtime.GetLang(zn.InhMeta),
			Lang:           config.GetLang(zn.InhMeta, api.rtConfig),
			Xhtml:          false,
			MarkerExternal: "",
			NewWindow:      false,
			IgnoreMeta:     map[string]bool{meta.KeyLang: true},
		}
		switch part {
		case partZettel:
			inhMeta := false
			if format != "raw" {
			err = writeZettelPartZettel(w, zn, format, env)
				w.Header().Set(adapter.ContentType, format2ContentType(format))
				inhMeta = true
			}
			enc := encoder.Create(format, &env)
			if enc == nil {
				err = adapter.ErrNoSuchFormat
			} else {
				_, err = enc.WriteZettel(w, zn, inhMeta)
			}
		case partMeta:
			w.Header().Set(adapter.ContentType, format2ContentType(format))
			if format == "raw" {
				// Don't write inherited meta data, just the raw
				err = writeMeta(w, zn.Meta, format, nil)
			err = writeZettelPartMeta(w, zn, format)
			} else {
				err = writeMeta(w, zn.InhMeta, format, nil)
			}
		case partContent:
			if format == "raw" {
				if ct, ok := syntax2contentType(runtime.GetSyntax(zn.Meta)); ok {
					w.Header().Add(adapter.ContentType, ct)
				}
			} else {
				w.Header().Set(adapter.ContentType, format2ContentType(format))
			}
			err = writeContent(w, zn, format, &env)
			err = api.writeZettelPartContent(w, zn, format, env)
		default:
			adapter.BadRequest(w, "Unknown _part parameter")
			return
		}
		if err != nil {
			if err == adapter.ErrNoSuchFormat {
			if errors.Is(err, adapter.ErrNoSuchFormat) {
				adapter.BadRequest(w, fmt.Sprintf("Zettel %q not available in format %q", zid.String(), format))
				return
			}
			adapter.InternalServerError(w, "Get zettel", err)
		}
	}
}

func writeZettelPartZettel(w http.ResponseWriter, zn *ast.ZettelNode, format string, env encoder.Environment) error {
	enc := encoder.Create(format, &env)
	if enc == nil {
		return adapter.ErrNoSuchFormat
	}
	inhMeta := false
	if format != "raw" {
		w.Header().Set(adapter.ContentType, format2ContentType(format))
		inhMeta = true
	}
	_, err := enc.WriteZettel(w, zn, inhMeta)
	return err
}

func writeZettelPartMeta(w http.ResponseWriter, zn *ast.ZettelNode, format string) error {
	w.Header().Set(adapter.ContentType, format2ContentType(format))
	if enc := encoder.Create(format, nil); enc != nil {
		if format == "raw" {
			_, err := enc.WriteMeta(w, zn.Meta)
			return err
		}
		_, err := enc.WriteMeta(w, zn.InhMeta)
		return err
	}
	return adapter.ErrNoSuchFormat
}

func (api *API) writeZettelPartContent(w http.ResponseWriter, zn *ast.ZettelNode, format string, env encoder.Environment) error {
	if format == "raw" {
		if ct, ok := syntax2contentType(config.GetSyntax(zn.Meta, api.rtConfig)); ok {
			w.Header().Add(adapter.ContentType, ct)
		}
	} else {
		w.Header().Set(adapter.ContentType, format2ContentType(format))
	}
	return writeContent(w, zn, format, &env)
}

Changes to web/adapter/api/get_zettel_context.go.

16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
16
17
18
19
20
21
22

23
24
25
26
27
28
29
30







-
+








	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeZettelContextHandler creates a new HTTP handler for the use case "zettel context".
func MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc {
func (api *API) MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			http.NotFound(w, r)
			return
		}
		q := r.URL.Query()
39
40
41
42
43
44
45
46

47
48
39
40
41
42
43
44
45

46
47
48







-
+


		}
		ctx := r.Context()
		metaList, err := getContext.Run(ctx, zid, dir, depth, limit)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		writeMetaList(w, metaList[0], metaList[1:])
		api.writeMetaList(w, metaList[0], metaList[1:])
	}
}

Changes to web/adapter/api/get_zettel_list.go.

11
12
13
14
15
16
17
18
19
20
21
22


23
24
25
26
27
28

29
30
31
32
33
34
35
36
37
38




39
40
41

42
43
44
45
46
47
48
49
50
51
52

53
54

55
56
57
58
59
60
61
62
63

64
65
66

67
68
69
70
71
72
73
74
75
76

77
78
79
80
81
82
83
11
12
13
14
15
16
17

18
19


20
21
22
23
24
25
26

27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51
52
53
54

55
56

57
58
59
60
61
62
63
64
65

66
67
68

69
70
71
72
73
74
75
76
77
78

79
80
81
82
83
84
85
86







-


-
-
+
+





-
+










+
+
+
+


-
+










-
+

-
+








-
+


-
+









-
+







// Package api provides api handlers for web requests.
package api

import (
	"fmt"
	"net/http"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/index"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

// MakeListMetaHandler creates a new HTTP handler for the use case "list some zettel".
func MakeListMetaHandler(
func (api *API) MakeListMetaHandler(
	listMeta usecase.ListMeta,
	getMeta usecase.GetMeta,
	parseZettel usecase.ParseZettel,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		q := r.URL.Query()
		s := adapter.GetSearch(q, false)
		format := adapter.GetFormat(r, q, encoder.GetDefaultFormat())
		part := getPart(q, partMeta)
		if part == partUnknown {
			adapter.BadRequest(w, "Unknown _part parameter")
			return
		}
		ctx1 := ctx
		if format == "html" || (!s.HasComputedMetaKey() && (part == partID || part == partContent)) {
			ctx1 = index.NoEnrichContext(ctx1)
			ctx1 = place.NoEnrichContext(ctx1)
		}
		metaList, err := listMeta.Run(ctx1, s)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}

		w.Header().Set(adapter.ContentType, format2ContentType(format))
		switch format {
		case "html":
			renderListMetaHTML(w, metaList)
			api.renderListMetaHTML(w, metaList)
		case "json", "djson":
			renderListMetaXJSON(ctx, w, metaList, format, part, partMeta, getMeta, parseZettel)
			api.renderListMetaXJSON(ctx, w, metaList, format, part, partMeta, getMeta, parseZettel)
		case "native", "raw", "text", "zmk":
			adapter.NotImplemented(w, fmt.Sprintf("Zettel list in format %q not yet implemented", format))
		default:
			adapter.BadRequest(w, fmt.Sprintf("Zettel list not available in format %q", format))
		}
	}
}

func renderListMetaHTML(w http.ResponseWriter, metaList []*meta.Meta) {
func (api *API) renderListMetaHTML(w http.ResponseWriter, metaList []*meta.Meta) {
	env := encoder.Environment{Interactive: true}
	buf := encoder.NewBufWriter(w)
	buf.WriteStrings("<html lang=\"", runtime.GetDefaultLang(), "\">\n<body>\n<ul>\n")
	buf.WriteStrings("<html lang=\"", api.rtConfig.GetDefaultLang(), "\">\n<body>\n<ul>\n")
	for _, m := range metaList {
		title := m.GetDefault(meta.KeyTitle, "")
		htmlTitle, err := adapter.FormatInlines(parser.ParseMetadata(title), "html", &env)
		if err != nil {
			adapter.InternalServerError(w, "Format HTML inlines", err)
			return
		}
		buf.WriteStrings(
			"<li><a href=\"",
			adapter.NewURLBuilder('z').SetZid(m.Zid).AppendQuery("_format", "html").String(),
			api.NewURLBuilder('z').SetZid(m.Zid).AppendQuery("_format", "html").String(),
			"\">",
			htmlTitle,
			"</a></li>\n")
	}
	buf.WriteString("</ul>\n</body>\n</html>")
	buf.Flush()
}

Changes to web/adapter/api/json.go.

14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
14
15
16
17
18
19
20

21
22
23
24
25
26
27







-







import (
	"context"
	"encoding/json"
	"io"
	"net/http"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152

153
154
155
156
157
158
159
160
161

162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207

208
209
210
211
212
213
214
215
216
217
218

219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258

259
260
261

262
263
264
265
266
267
268












269
270


























































































































































































271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291

292
293
294
295
296
297
298

299
300
301

302
303
304
305
306
307

308
309

310

311
51
52
53
54
55
56
57







































58
59
60
61
62
63
64





































65
66
67
68
69
70
71
72
73
74

75
76
77
78
79
80
81
82
83

84
85
86
87
88
89
90
91
92
93
94
95
96
97

































98
99
100
101
102
103
104
105
106



107








































108



109


110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126


127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321












322

323
324
325
326
327

328
329
330

331
332
333
334
335
336

337
338
339
340

341
342







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-







-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-










-
+








-
+













-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+








-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
-
-
-
+
-
-





+
+
+
+
+
+
+
+
+
+
+
+
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+









-
-
-
-
-
-
-
-
-
-
-
-
+
-





-
+


-
+





-
+


+
-
+

type jsonContent struct {
	ID       string      `json:"id"`
	URL      string      `json:"url"`
	Encoding string      `json:"encoding"`
	Content  interface{} `json:"content"`
}

func writeJSONZettel(w http.ResponseWriter, z *ast.ZettelNode, part partType) error {
	var outData interface{}
	idData := jsonIDURL{
		ID:  z.Zid.String(),
		URL: adapter.NewURLBuilder('z').SetZid(z.Zid).String(),
	}

	switch part {
	case partZettel:
		encoding, content := encodedContent(z.Content)
		outData = jsonZettel{
			ID:       idData.ID,
			URL:      idData.URL,
			Meta:     z.InhMeta.Map(),
			Encoding: encoding,
			Content:  content,
		}
	case partMeta:
		outData = jsonMeta{
			ID:   idData.ID,
			URL:  idData.URL,
			Meta: z.InhMeta.Map(),
		}
	case partContent:
		encoding, content := encodedContent(z.Content)
		outData = jsonContent{
			ID:       idData.ID,
			URL:      idData.URL,
			Encoding: encoding,
			Content:  content,
		}
	case partID:
		outData = idData
	default:
		panic(part)
	}
	return encodeJSONData(w, outData, false)
}

func encodedContent(content domain.Content) (string, interface{}) {
	if content.IsBinary() {
		return "base64", content.AsBytes()
	}
	return "", content.AsString()
}

func writeDJSONZettel(
	ctx context.Context,
	w http.ResponseWriter,
	z *ast.ZettelNode,
	part, defPart partType,
	getMeta usecase.GetMeta,
) (err error) {
	switch part {
	case partZettel:
		err = writeDJSONHeader(w, z.Zid)
		if err == nil {
			err = writeDJSONMeta(w, z)
		}
		if err == nil {
			err = writeDJSONContent(ctx, w, z, part, defPart, getMeta)
		}
	case partMeta:
		err = writeDJSONHeader(w, z.Zid)
		if err == nil {
			err = writeDJSONMeta(w, z)
		}
	case partContent:
		err = writeDJSONHeader(w, z.Zid)
		if err == nil {
			err = writeDJSONContent(ctx, w, z, part, defPart, getMeta)
		}
	case partID:
		writeDJSONHeader(w, z.Zid)
	default:
		panic(part)
	}
	if err == nil {
		_, err = w.Write(djsonFooter)
	}
	return err
}

var (
	djsonMetaHeader    = []byte(",\"meta\":")
	djsonContentHeader = []byte(",\"content\":")
	djsonHeader1       = []byte("{\"id\":\"")
	djsonHeader2       = []byte("\",\"url\":\"")
	djsonHeader3       = []byte("?_format=")
	djsonHeader4       = []byte("\"")
	djsonFooter        = []byte("}")
)

func writeDJSONHeader(w http.ResponseWriter, zid id.Zid) error {
func (api *API) writeDJSONHeader(w io.Writer, zid id.Zid) error {
	_, err := w.Write(djsonHeader1)
	if err == nil {
		_, err = w.Write(zid.Bytes())
	}
	if err == nil {
		_, err = w.Write(djsonHeader2)
	}
	if err == nil {
		_, err = io.WriteString(w, adapter.NewURLBuilder('z').SetZid(zid).String())
		_, err = io.WriteString(w, api.NewURLBuilder('z').SetZid(zid).String())
	}
	if err == nil {
		_, err = w.Write(djsonHeader3)
		if err == nil {
			_, err = io.WriteString(w, "djson")
		}
	}
	if err == nil {
		_, err = w.Write(djsonHeader4)
	}
	return err
}

func writeDJSONMeta(w io.Writer, z *ast.ZettelNode) error {
	_, err := w.Write(djsonMetaHeader)
	if err == nil {
		err = writeMeta(w, z.InhMeta, "djson", nil)
	}
	return err
}

func writeDJSONContent(
	ctx context.Context,
	w io.Writer,
	z *ast.ZettelNode,
	part, defPart partType,
	getMeta usecase.GetMeta,
) (err error) {
	_, err = w.Write(djsonContentHeader)
	if err == nil {
		err = writeContent(w, z, "djson", &encoder.Environment{
			LinkAdapter:  adapter.MakeLinkAdapter(ctx, 'z', getMeta, part.DefString(defPart), "djson"),
			ImageAdapter: adapter.MakeImageAdapter()})
	}
	return err
}

var (
	jsonListHeader = []byte("{\"list\":[")
	jsonListSep    = []byte{','}
	jsonListFooter = []byte("]}")
)

var setJSON = map[string]bool{"json": true}

func renderListMetaXJSON(
func (api *API) renderListMetaXJSON(
	ctx context.Context,
	w http.ResponseWriter,
	metaList []*meta.Meta,
	format string,
	part, defPart partType,
	getMeta usecase.GetMeta,
	parseZettel usecase.ParseZettel,
) {
	var readZettel bool
	switch part {
	case partZettel, partContent:
	prepareZettel := api.getPrepareZettelFunc(ctx, parseZettel, part)
		readZettel = true
	case partMeta, partID:
		readZettel = false
	default:
		adapter.BadRequest(w, "Unknown _part parameter")
		return
	}
	isJSON := setJSON[format]
	_, err := w.Write(jsonListHeader)
	for i, m := range metaList {
		if err != nil {
			break
		}
		if i > 0 {
			_, err = w.Write(jsonListSep)
		}
		if err != nil {
			break
		}
		var zn *ast.ZettelNode
		if readZettel {
			z, err1 := parseZettel.Run(ctx, m.Zid, "")
			if err1 != nil {
				err = err1
				break
			}
			zn = z
		} else {
			zn = &ast.ZettelNode{
				Meta:    m,
				Content: "",
				Zid:     m.Zid,
				InhMeta: runtime.AddDefaultValues(m),
				Ast:     nil,
			}
		}
		if isJSON {
			err = writeJSONZettel(w, zn, part)
		} else {
			err = writeDJSONZettel(ctx, w, zn, part, defPart, getMeta)
	writeZettel := api.getWriteMetaZettelFunc(ctx, format, part, defPart, getMeta)
		}
	}
	if err == nil {
	err := writeListXJSON(w, metaList, prepareZettel, writeZettel)
		_, err = w.Write(jsonListFooter)
	}
	if err != nil {
		adapter.InternalServerError(w, "Get list", err)
	}
}

type prepareZettelFunc func(m *meta.Meta) (*ast.ZettelNode, error)

func (api *API) getPrepareZettelFunc(ctx context.Context, parseZettel usecase.ParseZettel, part partType) prepareZettelFunc {
	switch part {
	case partZettel, partContent:
		return func(m *meta.Meta) (*ast.ZettelNode, error) {
			return parseZettel.Run(ctx, m.Zid, "")
		}
	case partMeta, partID:
		return func(m *meta.Meta) (*ast.ZettelNode, error) {
			return &ast.ZettelNode{
				Meta:    m,
func writeContent(
	w io.Writer, zn *ast.ZettelNode, format string, env *encoder.Environment) error {
				Content: "",
				Zid:     m.Zid,
				InhMeta: api.rtConfig.AddDefaultValues(m),
				Ast:     nil,
			}, nil
		}
	}
	return nil
}

type writeZettelFunc func(io.Writer, *ast.ZettelNode) error

func (api *API) getWriteMetaZettelFunc(ctx context.Context, format string,
	part, defPart partType, getMeta usecase.GetMeta) writeZettelFunc {
	switch part {
	case partZettel:
		return api.getWriteZettelFunc(ctx, format, defPart, getMeta)
	case partMeta:
		return api.getWriteMetaFunc(ctx, format)
	case partContent:
		return api.getWriteContentFunc(ctx, format, defPart, getMeta)
	case partID:
		return api.getWriteIDFunc(ctx, format)
	default:
		panic(part)
	}
}

func (api *API) getWriteZettelFunc(ctx context.Context, format string,
	defPart partType, getMeta usecase.GetMeta) writeZettelFunc {
	if format == "json" {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			encoding, content := encodedContent(zn.Content)
			return encodeJSONData(w, jsonZettel{
				ID:       zn.Zid.String(),
				URL:      api.NewURLBuilder('z').SetZid(zn.Zid).String(),
				Meta:     zn.InhMeta.Map(),
				Encoding: encoding,
				Content:  content,
			})
		}
	}
	enc := encoder.Create("djson", nil)
	if enc == nil {
		panic("no DJSON encoder found")
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonMetaHeader)
		if err != nil {
			return err
		}
		_, err = enc.WriteMeta(w, zn.InhMeta)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonContentHeader)
		if err != nil {
			return err
		}
		err = writeContent(w, zn, "djson", &encoder.Environment{
			LinkAdapter:  adapter.MakeLinkAdapter(ctx, api, 'z', getMeta, partZettel.DefString(defPart), "djson"),
			ImageAdapter: adapter.MakeImageAdapter(ctx, api, getMeta)})
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}
func (api *API) getWriteMetaFunc(ctx context.Context, format string) writeZettelFunc {
	if format == "json" {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			return encodeJSONData(w, jsonMeta{
				ID:   zn.Zid.String(),
				URL:  api.NewURLBuilder('z').SetZid(zn.Zid).String(),
				Meta: zn.InhMeta.Map(),
			})
		}
	}
	enc := encoder.Create("djson", nil)
	if enc == nil {
		panic("no DJSON encoder found")
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonMetaHeader)
		if err != nil {
			return err
		}
		_, err = enc.WriteMeta(w, zn.InhMeta)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}
func (api *API) getWriteContentFunc(ctx context.Context, format string,
	defPart partType, getMeta usecase.GetMeta) writeZettelFunc {
	if format == "json" {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			encoding, content := encodedContent(zn.Content)
			return encodeJSONData(w, jsonContent{
				ID:       zn.Zid.String(),
				URL:      api.NewURLBuilder('z').SetZid(zn.Zid).String(),
				Encoding: encoding,
				Content:  content,
			})
		}
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonContentHeader)
		if err != nil {
			return err
		}
		err = writeContent(w, zn, "djson", &encoder.Environment{
			LinkAdapter:  adapter.MakeLinkAdapter(ctx, api, 'z', getMeta, partContent.DefString(defPart), "djson"),
			ImageAdapter: adapter.MakeImageAdapter(ctx, api, getMeta)})
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}
func (api *API) getWriteIDFunc(ctx context.Context, format string) writeZettelFunc {
	if format == "json" {
		return func(w io.Writer, zn *ast.ZettelNode) error {
			return encodeJSONData(w, jsonIDURL{
				ID:  zn.Zid.String(),
				URL: api.NewURLBuilder('z').SetZid(zn.Zid).String(),
			})
		}
	}
	return func(w io.Writer, zn *ast.ZettelNode) error {
		err := api.writeDJSONHeader(w, zn.Zid)
		if err != nil {
			return err
		}
		_, err = w.Write(djsonFooter)
		return err
	}
}

var (
	jsonListHeader = []byte("{\"list\":[")
	jsonListSep    = []byte{','}
	jsonListFooter = []byte("]}")
)

func writeListXJSON(w http.ResponseWriter, metaList []*meta.Meta, prepareZettel prepareZettelFunc, writeZettel writeZettelFunc) error {
	_, err := w.Write(jsonListHeader)
	for i, m := range metaList {
		if err != nil {
			return err
		}
		if i > 0 {
			_, err = w.Write(jsonListSep)
			if err != nil {
				return err
			}
		}
		zn, err1 := prepareZettel(m)
		if err1 != nil {
			return err1
		}
		err = writeZettel(w, zn)
	}
	if err == nil {
		_, err = w.Write(jsonListFooter)
	}
	return err
}

func writeContent(w io.Writer, zn *ast.ZettelNode, format string, env *encoder.Environment) error {
	enc := encoder.Create(format, env)
	if enc == nil {
		return adapter.ErrNoSuchFormat
	}

	_, err := enc.WriteContent(w, zn)
	return err
}

func writeMeta(
	w io.Writer, m *meta.Meta, format string, env *encoder.Environment) error {
	enc := encoder.Create(format, env)
	if enc == nil {
		return adapter.ErrNoSuchFormat
	}

	_, err := enc.WriteMeta(w, m)
	return err
}

func encodeJSONData(w http.ResponseWriter, data interface{}, addHeader bool) error {
func encodeJSONData(w io.Writer, data interface{}) error {
	w.Header().Set(adapter.ContentType, format2ContentType("json"))
	enc := json.NewEncoder(w)
	enc.SetEscapeHTML(false)
	return enc.Encode(data)
}

func writeMetaList(w http.ResponseWriter, m *meta.Meta, metaList []*meta.Meta) error {
func (api *API) writeMetaList(w http.ResponseWriter, m *meta.Meta, metaList []*meta.Meta) error {
	outData := jsonMetaList{
		ID:   m.Zid.String(),
		URL:  adapter.NewURLBuilder('z').SetZid(m.Zid).String(),
		URL:  api.NewURLBuilder('z').SetZid(m.Zid).String(),
		Meta: m.Map(),
		List: make([]jsonMeta, len(metaList)),
	}
	for i, m := range metaList {
		outData.List[i].ID = m.Zid.String()
		outData.List[i].URL = adapter.NewURLBuilder('z').SetZid(m.Zid).String()
		outData.List[i].URL = api.NewURLBuilder('z').SetZid(m.Zid).String()
		outData.List[i].Meta = m.Map()
	}
	w.Header().Set(adapter.ContentType, format2ContentType("json"))
	return encodeJSONData(w, outData, true)
	return encodeJSONData(w, outData)
}

Changes to web/adapter/api/login.go.

12
13
14
15
16
17
18
19
20

21
22
23
24
25
26
27

28
29

30
31
32
33
34

35
36
37
38

39
40
41
42

43
44
45
46
47
48
49
50
51
52
53
54











55
56
57
58
59
60





61
62
63

64
65
66

67
68
69
70




71
72
73

74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

91
92
93
94


95
96
97
98
99


100
101
102
103

104
105
106
107

108
109

110
111
112
113
114
115

116
117
12
13
14
15
16
17
18


19
20
21

22
23
24

25
26

27
28
29
30
31

32




33




34












35
36
37
38
39
40
41
42
43
44
45
46





47
48
49
50
51



52



53




54
55
56
57
58


59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

76
77
78


79
80
81
82
83


84
85
86
87
88

89
90
91
92

93


94
95
96
97
98
99

100
101
102







-
-
+


-



-
+

-
+




-
+
-
-
-
-
+
-
-
-
-
+
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+

-
-
-
-
-
+
+
+
+
+
-
-
-
+
-
-
-
+
-
-
-
-
+
+
+
+

-
-
+
















-
+


-
-
+
+



-
-
+
+



-
+



-
+
-
-
+





-
+


package api

import (
	"encoding/json"
	"net/http"
	"time"

	"zettelstore.de/z/auth/token"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakePostLoginHandlerAPI creates a new HTTP handler to authenticate the given user via API.
func MakePostLoginHandlerAPI(auth usecase.Authenticate) http.HandlerFunc {
func (api *API) MakePostLoginHandlerAPI(ucAuth usecase.Authenticate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		if !startup.WithAuth() {
		if !api.withAuth() {
			w.Header().Set(adapter.ContentType, format2ContentType("json"))
			writeJSONToken(w, "freeaccess", 24*366*10*time.Hour)
			return
		}
		_, apiDur := startup.TokenLifetime()
		var token []byte
		authenticateViaJSON(auth, w, r, apiDur)
	}
}

		if ident, cred := retrieveIdentCred(r); ident != "" {
func authenticateViaJSON(
	auth usecase.Authenticate,
	w http.ResponseWriter,
	r *http.Request,
			var err error
	authDuration time.Duration,
) {
	token, err := authenticateForJSON(auth, w, r, authDuration)
	if err != nil {
		adapter.ReportUsecaseError(w, err)
		return
	}
	if token == nil {
		w.Header().Set("WWW-Authenticate", `Bearer realm="Default"`)
		http.Error(w, "Authentication failed", http.StatusUnauthorized)
		return
	}
			token, err = ucAuth.Run(r.Context(), ident, cred, api.tokenLifetime, auth.KindJSON)
			if err != nil {
				adapter.ReportUsecaseError(w, err)
				return
			}
		}
		if len(token) == 0 {
			w.Header().Set("WWW-Authenticate", `Bearer realm="Default"`)
			http.Error(w, "Authentication failed", http.StatusUnauthorized)
			return
		}

	w.Header().Set(adapter.ContentType, format2ContentType("json"))
	writeJSONToken(w, string(token), authDuration)
}

func authenticateForJSON(
		w.Header().Set(adapter.ContentType, format2ContentType("json"))
		writeJSONToken(w, string(token), api.tokenLifetime)
	}
}

	auth usecase.Authenticate,
	w http.ResponseWriter,
	r *http.Request,
func retrieveIdentCred(r *http.Request) (string, string) {
	authDuration time.Duration,
) ([]byte, error) {
	ident, cred, ok := adapter.GetCredentialsViaForm(r)
	if ident, cred, ok := adapter.GetCredentialsViaForm(r); ok {
	if !ok {
		if ident, cred, ok = r.BasicAuth(); !ok {
			return nil, nil
		}
		return ident, cred
	}
	if ident, cred, ok := r.BasicAuth(); ok {
		return ident, cred
	}
	token, err := auth.Run(r.Context(), ident, cred, authDuration, token.KindJSON)
	return token, err
	return "", ""
}

func writeJSONToken(w http.ResponseWriter, token string, lifetime time.Duration) {
	je := json.NewEncoder(w)
	je.Encode(struct {
		Token   string `json:"access_token"`
		Type    string `json:"token_type"`
		Expires int    `json:"expires_in"`
	}{
		Token:   token,
		Type:    "Bearer",
		Expires: int(lifetime / time.Second),
	})
}

// MakeRenewAuthHandler creates a new HTTP handler to renew the authenticate of a user.
func MakeRenewAuthHandler() http.HandlerFunc {
func (api *API) MakeRenewAuthHandler() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		auth := session.GetAuthData(ctx)
		if auth == nil || auth.Token == nil || auth.User == nil {
		authData := api.getAuthData(ctx)
		if authData == nil || len(authData.Token) == 0 || authData.User == nil {
			adapter.BadRequest(w, "Not authenticated")
			return
		}
		totalLifetime := auth.Expires.Sub(auth.Issued)
		currentLifetime := auth.Now.Sub(auth.Issued)
		totalLifetime := authData.Expires.Sub(authData.Issued)
		currentLifetime := authData.Now.Sub(authData.Issued)
		// If we are in the first quarter of the tokens lifetime, return the token
		if currentLifetime*4 < totalLifetime {
			w.Header().Set(adapter.ContentType, format2ContentType("json"))
			writeJSONToken(w, string(auth.Token), totalLifetime-currentLifetime)
			writeJSONToken(w, string(authData.Token), totalLifetime-currentLifetime)
			return
		}

		// Toke is a little bit aged. Create a new one
		// Token is a little bit aged. Create a new one
		_, apiDur := startup.TokenLifetime()
		token, err := token.GetToken(auth.User, apiDur, token.KindJSON)
		token, err := api.getToken(authData.User)
		if err != nil {
			adapter.ReportUsecaseError(w, err)
			return
		}
		w.Header().Set(adapter.ContentType, format2ContentType("json"))
		writeJSONToken(w, string(token), apiDur)
		writeJSONToken(w, string(token), api.tokenLifetime)
	}
}

Changes to web/adapter/encoding.go.

13
14
15
16
17
18
19
20
21
22
23
24
25



26
27
28
29
30
31
32
13
14
15
16
17
18
19

20
21



22
23
24
25
26
27
28
29
30
31







-


-
-
-
+
+
+








import (
	"context"
	"errors"
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/server"
)

// ErrNoSuchFormat signals an unsupported encoding format
var ErrNoSuchFormat = errors.New("no such format")

// FormatInlines returns a string representation of the inline slice.
func FormatInlines(is ast.InlineSlice, format string, env *encoder.Environment) (string, error) {
42
43
44
45
46
47
48

49
50
51
52
53
54
55
56
57
58
59

60

61
62
63
64
65
66
67
68
69
70
71
72
73


74
75
76
77
78
79
80
81











82

83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109

110
111

112
113
114
115
116
117
118
119
120






















121
122
123
124



125
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

61
62
63
64
65
66
67
68
69
70
71
72


73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

94
95
96
97
98
99
100
101
102
103
104
105














106

107
108

109
110
111







112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133




134
135
136








+











+
-
+











-
-
+
+








+
+
+
+
+
+
+
+
+
+
+
-
+











-
-
-
-
-
-
-
-
-
-
-
-
-
-

-
+

-
+


-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
+
-
	}
	return content.String(), nil
}

// MakeLinkAdapter creates an adapter to change a link node during encoding.
func MakeLinkAdapter(
	ctx context.Context,
	b server.Builder,
	key byte,
	getMeta usecase.GetMeta,
	part, format string,
) func(*ast.LinkNode) ast.InlineNode {
	return func(origLink *ast.LinkNode) ast.InlineNode {
		origRef := origLink.Ref
		if origRef == nil {
			return origLink
		}
		if origRef.State == ast.RefStateBased {
			newLink := *origLink
			urlPrefix := b.GetURLPrefix()
			newRef := ast.ParseReference(startup.URLPrefix() + origRef.Value[1:])
			newRef := ast.ParseReference(urlPrefix + origRef.Value[1:])
			newRef.State = ast.RefStateHosted
			newLink.Ref = newRef
			return &newLink
		}
		if origRef.State != ast.RefStateZettel {
			return origLink
		}
		zid, err := id.Parse(origRef.URL.Path)
		if err != nil {
			panic(err)
		}
		_, err = getMeta.Run(index.NoEnrichContext(ctx), zid)
		if place.IsErrNotAllowed(err) {
		_, err = getMeta.Run(place.NoEnrichContext(ctx), zid)
		if errors.Is(err, &place.ErrNotAllowed{}) {
			return &ast.FormatNode{
				Code:    ast.FormatSpan,
				Attrs:   origLink.Attrs,
				Inlines: origLink.Inlines,
			}
		}
		var newRef *ast.Reference
		if err == nil {
			ub := b.NewURLBuilder(key).SetZid(zid)
			if part != "" {
				ub.AppendQuery("_part", part)
			}
			if format != "" {
				ub.AppendQuery("_format", format)
			}
			if fragment := origRef.URL.EscapedFragment(); fragment != "" {
				ub.SetFragment(fragment)
			}

			newRef = ast.ParseReference(adaptZettelReference(key, zid, part, format, origRef.URL.EscapedFragment()))
			newRef = ast.ParseReference(ub.String())
			newRef.State = ast.RefStateFound
		} else {
			newRef = ast.ParseReference(origRef.Value)
			newRef.State = ast.RefStateBroken
		}
		newLink := *origLink
		newLink.Ref = newRef
		return &newLink
	}
}

func adaptZettelReference(key byte, zid id.Zid, part, format, fragment string) string {
	u := NewURLBuilder(key).SetZid(zid)
	if part != "" {
		u.AppendQuery("_part", part)
	}
	if format != "" {
		u.AppendQuery("_format", format)
	}
	if fragment != "" {
		u.SetFragment(fragment)
	}
	return u.String()
}

// MakeImageAdapter creates an adapter to change an image node during encoding.
func MakeImageAdapter() func(*ast.ImageNode) ast.InlineNode {
func MakeImageAdapter(ctx context.Context, b server.Builder, getMeta usecase.GetMeta) func(*ast.ImageNode) ast.InlineNode {
	return func(origImage *ast.ImageNode) ast.InlineNode {
		if origImage.Ref == nil || origImage.Ref.State != ast.RefStateZettel {
		if origImage.Ref == nil {
			return origImage
		}
		newImage := *origImage
		zid, err := id.Parse(newImage.Ref.Value)
		if err != nil {
			panic(err)
		}
		newImage.Ref = ast.ParseReference(
			NewURLBuilder('z').SetZid(zid).AppendQuery("_part", "content").AppendQuery(
		switch origImage.Ref.State {
		case ast.RefStateInvalid:
			return createZettelImage(b, origImage, id.EmojiZid, ast.RefStateInvalid)
		case ast.RefStateZettel:
			zid, err := id.Parse(origImage.Ref.Value)
			if err != nil {
				panic(err)
			}
			_, err = getMeta.Run(place.NoEnrichContext(ctx), zid)
			if err != nil {
				return createZettelImage(b, origImage, id.EmojiZid, ast.RefStateBroken)
			}
			return createZettelImage(b, origImage, zid, ast.RefStateFound)
		}
		return origImage
	}
}

func createZettelImage(b server.Builder, origImage *ast.ImageNode, zid id.Zid, state ast.RefState) *ast.ImageNode {
	newImage := *origImage
	newImage.Ref = ast.ParseReference(
		b.NewURLBuilder('z').SetZid(zid).AppendQuery("_part", "content").AppendQuery("_format", "raw").String())
				"_format", "raw").String())
		newImage.Ref.State = ast.RefStateFound
		return &newImage
	}
	newImage.Ref.State = state
	return &newImage
}
}

Changes to web/adapter/request.go.

132
133
134
135
136
137
138
139

140
141
142
143
144
145
146


147
148

149
132
133
134
135
136
137
138

139
140






141
142


143
144







-
+

-
-
-
-
-
-
+
+
-
-
+

func getQueryKeys(forSearch bool) (string, string, string, string, string, string) {
	if forSearch {
		return "sort", "order", "offset", "limit", "negate", "s"
	}
	return "_sort", "_order", "_offset", "_limit", "_negate", "_s"
}

func setCleanedQueryValues(filter *search.Search, key string, values []string) *search.Search {
func setCleanedQueryValues(s *search.Search, key string, values []string) *search.Search {
	for _, val := range values {
		val = strings.TrimSpace(val)
		if len(val) > 0 && val[0] == '!' {
			filter = filter.AddExpr(key, val[1:], true)
		} else {
			filter = filter.AddExpr(key, val, false)
		}
		s = s.AddExpr(key, val)
	}
	}
	return filter
	return s
}

Changes to web/adapter/response.go.

8
9
10
11
12
13
14

15
16
17
18
19
20
21
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22







+







// under this license.
//-----------------------------------------------------------------------------

// Package adapter provides handlers for web requests.
package adapter

import (
	"errors"
	"fmt"
	"log"
	"net/http"

	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
)
44
45
46
47
48
49
50
51

52
53
54

55
56
57
58
59
60


61
62
63


64
65
66
45
46
47
48
49
50
51

52
53
54

55
56
57
58
59


60
61
62


63
64
65
66
67







-
+


-
+




-
-
+
+

-
-
+
+



	if err == place.ErrNotFound {
		return http.StatusNotFound, http.StatusText(http.StatusNotFound)
	}
	if err1, ok := err.(*place.ErrNotAllowed); ok {
		return http.StatusForbidden, err1.Error()
	}
	if err1, ok := err.(*place.ErrInvalidID); ok {
		return http.StatusBadRequest, fmt.Sprintf("Zettel-ID %q not appropriate in this context.", err1.Zid)
		return http.StatusBadRequest, fmt.Sprintf("Zettel-ID %q not appropriate in this context", err1.Zid)
	}
	if err1, ok := err.(*usecase.ErrZidInUse); ok {
		return http.StatusBadRequest, fmt.Sprintf("Zettel-ID %q already in use.", err1.Zid)
		return http.StatusBadRequest, fmt.Sprintf("Zettel-ID %q already in use", err1.Zid)
	}
	if err1, ok := err.(*ErrBadRequest); ok {
		return http.StatusBadRequest, err1.Text
	}
	if err == place.ErrStopped {
		return http.StatusInternalServerError, fmt.Sprintf("Zettelstore not operational: %v.", err)
	if errors.Is(err, place.ErrStopped) {
		return http.StatusInternalServerError, fmt.Sprintf("Zettelstore not operational: %v", err)
	}
	if err == place.ErrTimeout {
		return http.StatusLoopDetected, "Zettelstore operation took too long"
	if errors.Is(err, place.ErrConflict) {
		return http.StatusConflict, "Zettelstore operations conflicted"
	}
	return http.StatusInternalServerError, err.Error()
}

Deleted web/adapter/urlbuilder.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114


















































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package adapter provides handlers for web requests.
package adapter

import (
	"net/url"
	"strings"

	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
)

type urlQuery struct{ key, val string }

// URLBuilder should be used to create zettelstore URLs.
type URLBuilder struct {
	key      byte
	path     []string
	query    []urlQuery
	fragment string
}

// NewURLBuilder creates a new URLBuilder.
func NewURLBuilder(key byte) *URLBuilder {
	return &URLBuilder{key: key}
}

// Clone an URLBuilder
func (ub *URLBuilder) Clone() *URLBuilder {
	cpy := new(URLBuilder)
	cpy.key = ub.key
	if len(ub.path) > 0 {
		cpy.path = make([]string, 0, len(ub.path))
		cpy.path = append(cpy.path, ub.path...)
	}
	if len(ub.query) > 0 {
		cpy.query = make([]urlQuery, 0, len(ub.query))
		cpy.query = append(cpy.query, ub.query...)
	}
	cpy.fragment = ub.fragment
	return cpy
}

// SetZid sets the zettel identifier.
func (ub *URLBuilder) SetZid(zid id.Zid) *URLBuilder {
	if len(ub.path) > 0 {
		panic("Cannot add Zid")
	}
	ub.path = append(ub.path, zid.String())
	return ub
}

// AppendPath adds a new path element
func (ub *URLBuilder) AppendPath(p string) *URLBuilder {
	ub.path = append(ub.path, p)
	return ub
}

// AppendQuery adds a new query parameter
func (ub *URLBuilder) AppendQuery(key, value string) *URLBuilder {
	ub.query = append(ub.query, urlQuery{key, value})
	return ub
}

// ClearQuery removes all query parameters.
func (ub *URLBuilder) ClearQuery() *URLBuilder {
	ub.query = nil
	ub.fragment = ""
	return ub
}

// SetFragment stores the fragment
func (ub *URLBuilder) SetFragment(s string) *URLBuilder {
	ub.fragment = s
	return ub
}

// String produces a string value.
func (ub *URLBuilder) String() string {
	var sb strings.Builder

	sb.WriteString(startup.URLPrefix())
	if ub.key != '/' {
		sb.WriteByte(ub.key)
	}
	for _, p := range ub.path {
		sb.WriteByte('/')
		sb.WriteString(url.PathEscape(p))
	}
	if len(ub.fragment) > 0 {
		sb.WriteByte('#')
		sb.WriteString(ub.fragment)
	}
	for i, q := range ub.query {
		if i == 0 {
			sb.WriteByte('?')
		} else {
			sb.WriteByte('&')
		}
		sb.WriteString(q.key)
		sb.WriteByte('=')
		sb.WriteString(url.QueryEscape(q.val))
	}
	return sb.String()
}

Changes to web/adapter/webui/create_zettel.go.

12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

39
40
41
42
43
44

45
46
47

48
49
50
51
52
53
54
55
56

57
58
59
60
61
62

63
64
65

66
67
68
69
70
71
72
73

74
75
76
77
78
79
80

81
82
83
84

85
86
87

88
89
90

91
92
93

94
95
96

97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115

116
117
118
119
120
121
122

123
124
125
126
127
128
129
130

131
132
133
134


135
136

137
138
139


140
141
142
143
144
145
146
147
148

149
150
151
152
153

154
155
156
157

158
159
160
161

162
163

164
165
166

167
168
12
13
14
15
16
17
18

19
20
21
22
23

24
25
26
27
28

29
30
31
32




33

34
35
36
37

38
39
40

41
42
43
44
45
46




47

48
49
50
51

52
53
54

55
56
57
58
59
60



61


62
63
64
65

66
67
68
69

70
71
72

73
74
75

76
77
78

79
80
81

82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

101
102
103
104
105
106
107

108
109
110

111
112
113
114

115
116
117


118
119
120

121
122


123
124
125
126
127
128
129
130
131
132

133
134
135
136
137

138
139
140
141

142
143
144
145

146
147

148
149
150

151
152
153







-
+




-





-




-
-
-
-
+
-




-
+


-
+





-
-
-
-
+
-




-
+


-
+





-
-
-
+
-
-




-
+



-
+


-
+


-
+


-
+


-
+


















-
+






-
+


-




-
+


-
-
+
+

-
+

-
-
+
+








-
+




-
+



-
+



-
+

-
+


-
+


package webui

import (
	"context"
	"fmt"
	"net/http"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/index"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakeGetCopyZettelHandler creates a new HTTP handler to display the
// HTML edit view of a copied zettel.
func MakeGetCopyZettelHandler(
	te *TemplateEngine,
	getZettel usecase.GetZettel,
	copyZettel usecase.CopyZettel,
func (wui *WebUI) MakeGetCopyZettelHandler(getZettel usecase.GetZettel, copyZettel usecase.CopyZettel) http.HandlerFunc {
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getOrigZettel(ctx, w, r, getZettel, "Copy")
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		renderZettelForm(w, r, te, copyZettel.Run(origZettel), "Copy Zettel", "Copy Zettel")
		wui.renderZettelForm(w, r, copyZettel.Run(origZettel), "Copy Zettel", "Copy Zettel")
	}
}

// MakeGetFolgeZettelHandler creates a new HTTP handler to display the
// HTML edit view of a follow-up zettel.
func MakeGetFolgeZettelHandler(
	te *TemplateEngine,
	getZettel usecase.GetZettel,
	folgeZettel usecase.FolgeZettel,
func (wui *WebUI) MakeGetFolgeZettelHandler(getZettel usecase.GetZettel, folgeZettel usecase.FolgeZettel) http.HandlerFunc {
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getOrigZettel(ctx, w, r, getZettel, "Folge")
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		renderZettelForm(w, r, te, folgeZettel.Run(origZettel), "Folge Zettel", "Folgezettel")
		wui.renderZettelForm(w, r, folgeZettel.Run(origZettel), "Folge Zettel", "Folgezettel")
	}
}

// MakeGetNewZettelHandler creates a new HTTP handler to display the
// HTML edit view of a zettel.
func MakeGetNewZettelHandler(
	te *TemplateEngine,
	getZettel usecase.GetZettel,
func (wui *WebUI) MakeGetNewZettelHandler(getZettel usecase.GetZettel, newZettel usecase.NewZettel) http.HandlerFunc {
	newZettel usecase.NewZettel,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		origZettel, err := getOrigZettel(ctx, w, r, getZettel, "New")
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		m := origZettel.Meta
		title := parser.ParseInlines(input.NewInput(runtime.GetTitle(m)), meta.ValueSyntaxZmk)
		title := parser.ParseInlines(input.NewInput(config.GetTitle(m, wui.rtConfig)), meta.ValueSyntaxZmk)
		textTitle, err := adapter.FormatInlines(title, "text", nil)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		env := encoder.Environment{Lang: runtime.GetLang(m)}
		env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)}
		htmlTitle, err := adapter.FormatInlines(title, "html", &env)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		renderZettelForm(w, r, te, newZettel.Run(origZettel), textTitle, htmlTitle)
		wui.renderZettelForm(w, r, newZettel.Run(origZettel), textTitle, htmlTitle)
	}
}

func getOrigZettel(
	ctx context.Context,
	w http.ResponseWriter,
	r *http.Request,
	getZettel usecase.GetZettel,
	op string,
) (domain.Zettel, error) {
	if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" {
		return domain.Zettel{}, adapter.NewErrBadRequest(
			fmt.Sprintf("%v zettel not possible in format %q", op, format))
	}
	zid, err := id.Parse(r.URL.Path[1:])
	if err != nil {
		return domain.Zettel{}, place.ErrNotFound
	}
	origZettel, err := getZettel.Run(index.NoEnrichContext(ctx), zid)
	origZettel, err := getZettel.Run(place.NoEnrichContext(ctx), zid)
	if err != nil {
		return domain.Zettel{}, place.ErrNotFound
	}
	return origZettel, nil
}

func renderZettelForm(
func (wui *WebUI) renderZettelForm(
	w http.ResponseWriter,
	r *http.Request,
	te *TemplateEngine,
	zettel domain.Zettel,
	title, heading string,
) {
	ctx := r.Context()
	user := session.GetUser(ctx)
	user := wui.getUser(ctx)
	m := zettel.Meta
	var base baseData
	te.makeBaseData(ctx, runtime.GetLang(m), title, user, &base)
	te.renderTemplate(ctx, w, id.FormTemplateZid, &base, formZettelData{
	wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), title, user, &base)
	wui.renderTemplate(ctx, w, id.FormTemplateZid, &base, formZettelData{
		Heading:       heading,
		MetaTitle:     runtime.GetTitle(m),
		MetaTitle:     m.GetDefault(meta.KeyTitle, ""),
		MetaTags:      m.GetDefault(meta.KeyTags, ""),
		MetaRole:      runtime.GetRole(m),
		MetaSyntax:    runtime.GetSyntax(m),
		MetaRole:      config.GetRole(m, wui.rtConfig),
		MetaSyntax:    config.GetSyntax(m, wui.rtConfig),
		MetaPairsRest: m.PairsRest(false),
		IsTextContent: !zettel.Content.IsBinary(),
		Content:       zettel.Content.AsString(),
	})
}

// MakePostCreateZettelHandler creates a new HTTP handler to store content of
// an existing zettel.
func MakePostCreateZettelHandler(te *TemplateEngine, createZettel usecase.CreateZettel) http.HandlerFunc {
func (wui *WebUI) MakePostCreateZettelHandler(createZettel usecase.CreateZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zettel, hasContent, err := parseZettelForm(r, id.Invalid)
		if err != nil {
			te.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read form data"))
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read form data"))
			return
		}
		if !hasContent {
			te.reportError(ctx, w, adapter.NewErrBadRequest("Content is missing"))
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Content is missing"))
			return
		}

		newZid, err := createZettel.Run(r.Context(), zettel)
		newZid, err := createZettel.Run(ctx, zettel)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		redirectFound(w, r, adapter.NewURLBuilder('h').SetZid(newZid))
		redirectFound(w, r, wui.NewURLBuilder('h').SetZid(newZid))
	}
}

Changes to web/adapter/webui/delete_zettel.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28
29
30
31

32
33
34
35
36

37
38
39
40
41
42
43

44
45
46
47
48
49

50
51
52
53

54
55
56
57


58
59
60
61
62
63
64
65
66
67
68

69
70
71
72
73

74
75
76
77
78

79
80
81

82
83
11
12
13
14
15
16
17

18
19
20
21
22
23

24
25
26
27



28

29
30
31

32
33
34
35
36
37
38

39
40
41
42
43
44

45
46
47
48

49
50
51


52
53
54
55
56
57
58
59
60
61
62
63

64
65
66
67
68

69
70
71
72
73

74
75
76

77
78
79







-
+





-




-
-
-
+
-



-
+






-
+





-
+



-
+


-
-
+
+










-
+




-
+




-
+


-
+


// Package webui provides web-UI handlers for web requests.
package webui

import (
	"fmt"
	"net/http"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakeGetDeleteZettelHandler creates a new HTTP handler to display the
// HTML delete view of a zettel.
func MakeGetDeleteZettelHandler(
	te *TemplateEngine,
	getZettel usecase.GetZettel,
func (wui *WebUI) MakeGetDeleteZettelHandler(getZettel usecase.GetZettel) http.HandlerFunc {
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" {
			te.reportError(ctx, w, adapter.NewErrBadRequest(
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Delete zettel not possible in format %q", format)))
			return
		}

		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		zettel, err := getZettel.Run(ctx, zid)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}

		user := session.GetUser(ctx)
		user := wui.getUser(ctx)
		m := zettel.Meta
		var base baseData
		te.makeBaseData(ctx, runtime.GetLang(m), "Delete Zettel "+m.Zid.String(), user, &base)
		te.renderTemplate(ctx, w, id.DeleteTemplateZid, &base, struct {
		wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Delete Zettel "+m.Zid.String(), user, &base)
		wui.renderTemplate(ctx, w, id.DeleteTemplateZid, &base, struct {
			Zid       string
			MetaPairs []meta.Pair
		}{
			Zid:       zid.String(),
			MetaPairs: m.Pairs(true),
		})
	}
}

// MakePostDeleteZettelHandler creates a new HTTP handler to delete a zettel.
func MakePostDeleteZettelHandler(te *TemplateEngine, deleteZettel usecase.DeleteZettel) http.HandlerFunc {
func (wui *WebUI) MakePostDeleteZettelHandler(deleteZettel usecase.DeleteZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		if err := deleteZettel.Run(r.Context(), zid); err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		redirectFound(w, r, adapter.NewURLBuilder('/'))
		redirectFound(w, r, wui.NewURLBuilder('/'))
	}
}

Changes to web/adapter/webui/edit_zettel.go.

11
12
13
14
15
16
17
18

19
20
21
22
23
24
25
26
27
28
29
30
31

32
33
34
35
36

37
38
39
40

41
42

43
44
45
46
47

48
49
50
51
52

53
54
55
56


57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

72
73
74
75
76

77
78
79
80
81
82

83
84
85
86
87

88
89
90

91
92
11
12
13
14
15
16
17

18
19
20

21
22
23

24
25
26
27


28
29
30
31
32

33
34
35
36

37
38

39
40
41
42
43

44
45
46
47
48

49
50
51


52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

68
69
70
71
72

73
74
75
76
77
78

79
80
81
82
83

84
85
86

87
88
89







-
+


-



-




-
-
+




-
+



-
+

-
+




-
+




-
+


-
-
+
+














-
+




-
+





-
+




-
+


-
+


// Package webui provides web-UI handlers for web requests.
package webui

import (
	"fmt"
	"net/http"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/index"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakeEditGetZettelHandler creates a new HTTP handler to display the
// HTML edit view of a zettel.
func MakeEditGetZettelHandler(
	te *TemplateEngine, getZettel usecase.GetZettel) http.HandlerFunc {
func (wui *WebUI) MakeEditGetZettelHandler(getZettel usecase.GetZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		zettel, err := getZettel.Run(index.NoEnrichContext(ctx), zid)
		zettel, err := getZettel.Run(place.NoEnrichContext(ctx), zid)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}

		if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" {
			te.reportError(ctx, w, adapter.NewErrBadRequest(
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Edit zettel %q not possible in format %q", zid, format)))
			return
		}

		user := session.GetUser(ctx)
		user := wui.getUser(ctx)
		m := zettel.Meta
		var base baseData
		te.makeBaseData(ctx, runtime.GetLang(m), "Edit Zettel", user, &base)
		te.renderTemplate(ctx, w, id.FormTemplateZid, &base, formZettelData{
		wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Edit Zettel", user, &base)
		wui.renderTemplate(ctx, w, id.FormTemplateZid, &base, formZettelData{
			Heading:       base.Title,
			MetaTitle:     m.GetDefault(meta.KeyTitle, ""),
			MetaRole:      m.GetDefault(meta.KeyRole, ""),
			MetaTags:      m.GetDefault(meta.KeyTags, ""),
			MetaSyntax:    m.GetDefault(meta.KeySyntax, ""),
			MetaPairsRest: m.PairsRest(false),
			IsTextContent: !zettel.Content.IsBinary(),
			Content:       zettel.Content.AsString(),
		})
	}
}

// MakeEditSetZettelHandler creates a new HTTP handler to store content of
// an existing zettel.
func MakeEditSetZettelHandler(te *TemplateEngine, updateZettel usecase.UpdateZettel) http.HandlerFunc {
func (wui *WebUI) MakeEditSetZettelHandler(updateZettel usecase.UpdateZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		zettel, hasContent, err := parseZettelForm(r, zid)
		if err != nil {
			te.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read zettel form"))
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read zettel form"))
			return
		}

		if err := updateZettel.Run(r.Context(), zettel, hasContent); err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		redirectFound(w, r, adapter.NewURLBuilder('h').SetZid(zid))
		redirectFound(w, r, wui.NewURLBuilder('h').SetZid(zid))
	}
}

Changes to web/adapter/webui/get_info.go.

14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

50
51
52
53
54
55
56

57
58
59
60
61
62
63

64
65
66
67
68
69

70
71
72
73
74
75
76

77
78
79
80
81
82
83

84
85
86
87
88
89
90
91
92

93
94

95
96

97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132












133
134
135
136
137
138
139
140

141
142
143
144
145





146

147
148
149
150
151
152
153
154
155

156

157
158
159

160

161
162
163
164
165

166
167
168
169
170

171
172
173
174
175
176
177
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28

29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45



46


47
48
49
50

51
52
53
54
55
56
57

58
59
60
61
62
63

64
65
66
67
68
69
70

71
72
73
74
75
76
77

78
79
80
81
82
83
84
85
86

87
88

89


90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106

107
108
109
110
111
112
113
114












115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133

134
135
136
137
138
139
140
141
142
143
144

145
146
147
148
149
150
151
152
153
154
155

156
157


158
159
160
161
162
163
164

165
166
167
168
169

170
171
172
173
174
175
176
177







-
+







-

















-
-
-
+
-
-




-
+






-
+





-
+






-
+






-
+








-
+

-
+
-
-
+
















-
+







-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+







-
+





+
+
+
+
+
-
+









+
-
+

-
-
+

+




-
+




-
+







import (
	"fmt"
	"net/http"
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/collect"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/encfun"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

type metaDataInfo struct {
	Key   string
	Value string
}

type matrixElement struct {
	Text   string
	HasURL bool
	URL    string
}
type matrixLine struct {
	Elements []matrixElement
}

// MakeGetInfoHandler creates a new HTTP handler for the use case "get zettel".
func MakeGetInfoHandler(
	te *TemplateEngine,
	parseZettel usecase.ParseZettel,
func (wui *WebUI) MakeGetInfoHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc {
	getMeta usecase.GetMeta,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		q := r.URL.Query()
		if format := adapter.GetFormat(r, q, "html"); format != "html" {
			te.reportError(ctx, w, adapter.NewErrBadRequest(
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Zettel info not available in format %q", format)))
			return
		}

		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		zn, err := parseZettel.Run(ctx, zid, q.Get("syntax"))
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}

		summary := collect.References(zn)
		locLinks, extLinks := splitLocExtLinks(append(summary.Links, summary.Images...))

		lang := runtime.GetLang(zn.InhMeta)
		lang := config.GetLang(zn.InhMeta, wui.rtConfig)
		env := encoder.Environment{Lang: lang}
		pairs := zn.Meta.Pairs(true)
		metaData := make([]metaDataInfo, len(pairs))
		getTitle := makeGetTitle(ctx, getMeta, &env)
		for i, p := range pairs {
			var html strings.Builder
			writeHTMLMetaValue(&html, zn.Meta, p.Key, getTitle, &env)
			wui.writeHTMLMetaValue(&html, zn.Meta, p.Key, getTitle, &env)
			metaData[i] = metaDataInfo{p.Key, html.String()}
		}
		endnotes, err := formatBlocks(nil, "html", &env)
		if err != nil {
			endnotes = ""
		}

		textTitle := encfun.MetaAsText(zn.InhMeta, meta.KeyTitle)
		user := session.GetUser(ctx)
		user := wui.getUser(ctx)
		var base baseData
		te.makeBaseData(ctx, lang, textTitle, user, &base)
		wui.makeBaseData(ctx, lang, textTitle, user, &base)
		canCopy := base.CanCreate && !zn.Content.IsBinary()
		te.renderTemplate(ctx, w, id.InfoTemplateZid, &base, struct {
		wui.renderTemplate(ctx, w, id.InfoTemplateZid, &base, struct {
			Zid          string
			WebURL       string
			ContextURL   string
			CanWrite     bool
			EditURL      string
			CanFolge     bool
			FolgeURL     string
			CanCopy      bool
			CopyURL      string
			CanRename    bool
			RenameURL    string
			CanDelete    bool
			DeleteURL    string
			MetaData     []metaDataInfo
			HasLinks     bool
			HasLocLinks  bool
			LocLinks     []string
			LocLinks     []localLink
			HasExtLinks  bool
			ExtLinks     []string
			ExtNewWindow string
			Matrix       []matrixLine
			Endnotes     string
		}{
			Zid:          zid.String(),
			WebURL:       adapter.NewURLBuilder('h').SetZid(zid).String(),
			ContextURL:   adapter.NewURLBuilder('j').SetZid(zid).String(),
			CanWrite:     te.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:      adapter.NewURLBuilder('e').SetZid(zid).String(),
			CanFolge:     base.CanCreate && !zn.Content.IsBinary(),
			FolgeURL:     adapter.NewURLBuilder('f').SetZid(zid).String(),
			CanCopy:      canCopy,
			CopyURL:      adapter.NewURLBuilder('c').SetZid(zid).String(),
			CanRename:    te.canRename(ctx, user, zn.Meta),
			RenameURL:    adapter.NewURLBuilder('b').SetZid(zid).String(),
			CanDelete:    te.canDelete(ctx, user, zn.Meta),
			DeleteURL:    adapter.NewURLBuilder('d').SetZid(zid).String(),
			WebURL:       wui.NewURLBuilder('h').SetZid(zid).String(),
			ContextURL:   wui.NewURLBuilder('j').SetZid(zid).String(),
			CanWrite:     wui.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:      wui.NewURLBuilder('e').SetZid(zid).String(),
			CanFolge:     base.CanCreate,
			FolgeURL:     wui.NewURLBuilder('f').SetZid(zid).String(),
			CanCopy:      base.CanCreate && !zn.Content.IsBinary(),
			CopyURL:      wui.NewURLBuilder('c').SetZid(zid).String(),
			CanRename:    wui.canRename(ctx, user, zn.Meta),
			RenameURL:    wui.NewURLBuilder('b').SetZid(zid).String(),
			CanDelete:    wui.canDelete(ctx, user, zn.Meta),
			DeleteURL:    wui.NewURLBuilder('d').SetZid(zid).String(),
			MetaData:     metaData,
			HasLinks:     len(extLinks)+len(locLinks) > 0,
			HasLocLinks:  len(locLinks) > 0,
			LocLinks:     locLinks,
			HasExtLinks:  len(extLinks) > 0,
			ExtLinks:     extLinks,
			ExtNewWindow: htmlAttrNewWindow(len(extLinks) > 0),
			Matrix:       infoAPIMatrix(zid),
			Matrix:       wui.infoAPIMatrix(zid),
			Endnotes:     endnotes,
		})
	}
}

type localLink struct {
	Valid bool
	Zid   string
}

func splitLocExtLinks(links []*ast.Reference) (locLinks, extLinks []string) {
func splitLocExtLinks(links []*ast.Reference) (locLinks []localLink, extLinks []string) {
	if len(links) == 0 {
		return nil, nil
	}
	for _, ref := range links {
		if ref.State == ast.RefStateSelf {
			continue
		}
		if ref.IsZettel() {
			continue
		}
		} else if ref.IsExternal() {
		if ref.IsExternal() {
			extLinks = append(extLinks, ref.String())
		} else {
			locLinks = append(locLinks, ref.String())
			continue
		}
		locLinks = append(locLinks, localLink{ref.IsValid(), ref.String()})
	}
	return locLinks, extLinks
}

func infoAPIMatrix(zid id.Zid) []matrixLine {
func (wui *WebUI) infoAPIMatrix(zid id.Zid) []matrixLine {
	formats := encoder.GetFormats()
	defFormat := encoder.GetDefaultFormat()
	parts := []string{"zettel", "meta", "content"}
	matrix := make([]matrixLine, 0, len(parts))
	u := adapter.NewURLBuilder('z').SetZid(zid)
	u := wui.NewURLBuilder('z').SetZid(zid)
	for _, part := range parts {
		row := make([]matrixElement, 0, len(formats)+1)
		row = append(row, matrixElement{part, false, ""})
		for _, format := range formats {
			u.AppendQuery("_part", part)
			if format != defFormat {
				u.AppendQuery("_format", format)

Changes to web/adapter/webui/get_zettel.go.

13
14
15
16
17
18
19
20

21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

36
37
38
39
40

41
42
43
44
45
46
47

48
49
50
51

52
53
54


55
56
57
58

59
60
61
62
63
64

65
66
67
68
69
70

71
72
73
74
75

76
77
78
79

80
81

82
83
84

85
86

87
88
89

90
91
92
93
94
95
96
13
14
15
16
17
18
19

20
21
22
23
24
25
26
27

28
29
30




31
32
33
34
35

36
37
38
39
40
41
42

43
44
45
46

47
48


49
50
51
52
53

54
55
56
57
58
59

60
61
62
63
64
65

66
67
68
69
70

71
72
73
74

75
76

77
78
79

80
81

82
83


84
85
86
87
88
89
90
91







-
+







-



-
-
-
-
+




-
+






-
+



-
+

-
-
+
+



-
+





-
+





-
+




-
+



-
+

-
+


-
+

-
+

-
-
+








import (
	"bytes"
	"net/http"
	"strings"

	"zettelstore.de/z/ast"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/encoder/encfun"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakeGetHTMLZettelHandler creates a new HTTP handler for the use case "get zettel".
func MakeGetHTMLZettelHandler(
	te *TemplateEngine,
	parseZettel usecase.ParseZettel,
	getMeta usecase.GetMeta) http.HandlerFunc {
func (wui *WebUI) MakeGetHTMLZettelHandler(parseZettel usecase.ParseZettel, getMeta usecase.GetMeta) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		syntax := r.URL.Query().Get("syntax")
		zn, err := parseZettel.Run(ctx, zid, syntax)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}

		lang := runtime.GetLang(zn.InhMeta)
		lang := config.GetLang(zn.InhMeta, wui.rtConfig)
		envHTML := encoder.Environment{
			LinkAdapter:    adapter.MakeLinkAdapter(ctx, 'h', getMeta, "", ""),
			ImageAdapter:   adapter.MakeImageAdapter(),
			LinkAdapter:    adapter.MakeLinkAdapter(ctx, wui, 'h', getMeta, "", ""),
			ImageAdapter:   adapter.MakeImageAdapter(ctx, wui, getMeta),
			CiteAdapter:    nil,
			Lang:           lang,
			Xhtml:          false,
			MarkerExternal: runtime.GetMarkerExternal(),
			MarkerExternal: wui.rtConfig.GetMarkerExternal(),
			NewWindow:      true,
			IgnoreMeta:     map[string]bool{meta.KeyTitle: true, meta.KeyLang: true},
		}
		metaHeader, err := formatMeta(zn.InhMeta, "html", &envHTML)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		htmlTitle, err := adapter.FormatInlines(
			encfun.MetaAsInlineSlice(zn.InhMeta, meta.KeyTitle), "html", &envHTML)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		htmlContent, err := formatBlocks(zn.Ast, "html", &envHTML)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		textTitle := encfun.MetaAsText(zn.InhMeta, meta.KeyTitle)
		user := session.GetUser(ctx)
		user := wui.getUser(ctx)
		roleText := zn.Meta.GetDefault(meta.KeyRole, "*")
		tags := buildTagInfos(zn.Meta)
		tags := wui.buildTagInfos(zn.Meta)
		getTitle := makeGetTitle(ctx, getMeta, &encoder.Environment{Lang: lang})
		extURL, hasExtURL := zn.Meta.Get(meta.KeyURL)
		backLinks := formatBackLinks(zn.InhMeta, getTitle)
		backLinks := wui.formatBackLinks(zn.InhMeta, getTitle)
		var base baseData
		te.makeBaseData(ctx, lang, textTitle, user, &base)
		wui.makeBaseData(ctx, lang, textTitle, user, &base)
		base.MetaHeader = metaHeader
		canCopy := base.CanCreate && !zn.Content.IsBinary()
		te.renderTemplate(ctx, w, id.ZettelTemplateZid, &base, struct {
		wui.renderTemplate(ctx, w, id.ZettelTemplateZid, &base, struct {
			HTMLTitle     string
			CanWrite      bool
			EditURL       string
			Zid           string
			InfoURL       string
			RoleText      string
			RoleURL       string
106
107
108
109
110
111
112
113
114


115
116

117
118

119
120
121
122
123
124
125
126






127
128
129
130
131
132
133
101
102
103
104
105
106
107


108
109
110

111
112

113
114
115






116
117
118
119
120
121
122
123
124
125
126
127
128







-
-
+
+

-
+

-
+


-
-
-
-
-
-
+
+
+
+
+
+







			ExtURL        string
			ExtNewWindow  string
			Content       string
			HasBackLinks  bool
			BackLinks     []simpleLink
		}{
			HTMLTitle:     htmlTitle,
			CanWrite:      te.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:       adapter.NewURLBuilder('e').SetZid(zid).String(),
			CanWrite:      wui.canWrite(ctx, user, zn.Meta, zn.Content),
			EditURL:       wui.NewURLBuilder('e').SetZid(zid).String(),
			Zid:           zid.String(),
			InfoURL:       adapter.NewURLBuilder('i').SetZid(zid).String(),
			InfoURL:       wui.NewURLBuilder('i').SetZid(zid).String(),
			RoleText:      roleText,
			RoleURL:       adapter.NewURLBuilder('h').AppendQuery("role", roleText).String(),
			RoleURL:       wui.NewURLBuilder('h').AppendQuery("role", roleText).String(),
			HasTags:       len(tags) > 0,
			Tags:          tags,
			CanCopy:       canCopy,
			CopyURL:       adapter.NewURLBuilder('c').SetZid(zid).String(),
			CanFolge:      base.CanCreate && !zn.Content.IsBinary(),
			FolgeURL:      adapter.NewURLBuilder('f').SetZid(zid).String(),
			FolgeRefs:     formatMetaKey(zn.InhMeta, meta.KeyFolge, getTitle),
			PrecursorRefs: formatMetaKey(zn.InhMeta, meta.KeyPrecursor, getTitle),
			CanCopy:       base.CanCreate && !zn.Content.IsBinary(),
			CopyURL:       wui.NewURLBuilder('c').SetZid(zid).String(),
			CanFolge:      base.CanCreate,
			FolgeURL:      wui.NewURLBuilder('f').SetZid(zid).String(),
			FolgeRefs:     wui.formatMetaKey(zn.InhMeta, meta.KeyFolge, getTitle),
			PrecursorRefs: wui.formatMetaKey(zn.InhMeta, meta.KeyPrecursor, getTitle),
			ExtURL:        extURL,
			HasExtURL:     hasExtURL,
			ExtNewWindow:  htmlAttrNewWindow(envHTML.NewWindow && hasExtURL),
			Content:       htmlContent,
			HasBackLinks:  len(backLinks) > 0,
			BackLinks:     backLinks,
		})
158
159
160
161
162
163
164
165

166
167
168

169
170
171
172
173
174
175
176
177
178

179
180
181

182
183
184
185
186
187

188
189
190
191
192
193
194
195
196
197
198
199

200
201
202
203
204
205
206
207
208
153
154
155
156
157
158
159

160
161
162

163
164
165
166
167
168
169
170
171
172

173
174
175

176
177
178
179
180
181

182
183
184
185
186
187
188
189
190
191
192
193

194
195
196
197
198
199
200
201
202
203







-
+


-
+









-
+


-
+





-
+











-
+









	_, err := enc.WriteMeta(&content, m)
	if err != nil {
		return "", err
	}
	return content.String(), nil
}

func buildTagInfos(m *meta.Meta) []simpleLink {
func (wui *WebUI) buildTagInfos(m *meta.Meta) []simpleLink {
	var tagInfos []simpleLink
	if tags, ok := m.GetList(meta.KeyTags); ok {
		ub := adapter.NewURLBuilder('h')
		ub := wui.NewURLBuilder('h')
		tagInfos = make([]simpleLink, len(tags))
		for i, tag := range tags {
			tagInfos[i] = simpleLink{Text: tag, URL: ub.AppendQuery("tags", tag).String()}
			ub.ClearQuery()
		}
	}
	return tagInfos
}

func formatMetaKey(m *meta.Meta, key string, getTitle getTitleFunc) string {
func (wui *WebUI) formatMetaKey(m *meta.Meta, key string, getTitle getTitleFunc) string {
	if _, ok := m.Get(key); ok {
		var buf bytes.Buffer
		writeHTMLMetaValue(&buf, m, key, getTitle, nil)
		wui.writeHTMLMetaValue(&buf, m, key, getTitle, nil)
		return buf.String()
	}
	return ""
}

func formatBackLinks(m *meta.Meta, getTitle getTitleFunc) []simpleLink {
func (wui *WebUI) formatBackLinks(m *meta.Meta, getTitle getTitleFunc) []simpleLink {
	values, ok := m.GetList(meta.KeyBack)
	if !ok || len(values) == 0 {
		return nil
	}
	result := make([]simpleLink, 0, len(values))
	for _, val := range values {
		zid, err := id.Parse(val)
		if err != nil {
			continue
		}
		if title, found := getTitle(zid, "text"); found > 0 {
			url := adapter.NewURLBuilder('h').SetZid(zid).String()
			url := wui.NewURLBuilder('h').SetZid(zid).String()
			if title == "" {
				result = append(result, simpleLink{Text: val, URL: url})
			} else {
				result = append(result, simpleLink{Text: title, URL: url})
			}
		}
	}
	return result
}

Changes to web/adapter/webui/home.go.

9
10
11
12
13
14
15

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

34
35
36
37

38
39
40

41
42
43

44
45
46
47
48
49
50

51
52
53
54


55
56
57

58
59
9
10
11
12
13
14
15
16
17
18


19
20
21


22
23
24
25
26
27
28
29

30
31
32
33

34
35
36

37
38
39

40
41
42
43
44
45
46

47
48
49


50
51
52
53

54
55
56







+


-
-



-
-








-
+



-
+


-
+


-
+






-
+


-
-
+
+


-
+


//-----------------------------------------------------------------------------

// Package webui provides web-UI handlers for web requests.
package webui

import (
	"context"
	"errors"
	"net/http"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

type getRootStore interface {
	// GetMeta retrieves just the meta data of a specific zettel.
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
}

// MakeGetRootHandler creates a new HTTP handler to show the root URL.
func MakeGetRootHandler(te *TemplateEngine, s getRootStore) http.HandlerFunc {
func (wui *WebUI) MakeGetRootHandler(s getRootStore) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		if r.URL.Path != "/" {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}
		homeZid := runtime.GetHomeZettel()
		homeZid := wui.rtConfig.GetHomeZettel()
		if homeZid != id.DefaultHomeZid {
			if _, err := s.GetMeta(ctx, homeZid); err == nil {
				redirectFound(w, r, adapter.NewURLBuilder('h').SetZid(homeZid))
				redirectFound(w, r, wui.NewURLBuilder('h').SetZid(homeZid))
				return
			}
			homeZid = id.DefaultHomeZid
		}
		_, err := s.GetMeta(ctx, homeZid)
		if err == nil {
			redirectFound(w, r, adapter.NewURLBuilder('h').SetZid(homeZid))
			redirectFound(w, r, wui.NewURLBuilder('h').SetZid(homeZid))
			return
		}
		if place.IsErrNotAllowed(err) && startup.WithAuth() && session.GetUser(ctx) == nil {
			redirectFound(w, r, adapter.NewURLBuilder('a'))
		if errors.Is(err, &place.ErrNotAllowed{}) && wui.authz.WithAuth() && wui.getUser(ctx) == nil {
			redirectFound(w, r, wui.NewURLBuilder('a'))
			return
		}
		redirectFound(w, r, adapter.NewURLBuilder('h'))
		redirectFound(w, r, wui.NewURLBuilder('h'))
	}
}

Changes to web/adapter/webui/htmlmeta.go.

9
10
11
12
13
14
15

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

35
36
37

38
39
40
41
42
43

44
45
46

47
48
49
50
51
52
53
54

55
56
57
58
59
60
61
62
63

64
65
66

67
68
69
70
71
72
73
74
75
76
77
78

79
80

81
82

83
84
85
86
87
88
89
90
91
92
93
94

95
96
97
98
99
100
101
102
103
104
105
106

107
108
109
110

111
112
113
114
115
116
117
118
119
120
121

122
123
124
125
126

127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142

143
144
145
146
147

148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167


168
169
170

171
172
173
174
175

176
177
178
179
180
181
182
183
184
185
186
187

188
189
190

191
192
193
194
195
196
197
198
199

200
201

202
203
204
205
206
207
208
209
210
211
212
213
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

25
26
27
28
29
30
31
32
33

34
35
36

37
38
39
40
41
42

43
44
45

46
47
48
49
50
51
52
53

54
55
56
57
58
59
60
61
62

63
64
65

66
67
68
69
70
71
72
73
74
75
76
77

78
79

80
81

82
83
84
85
86
87
88
89
90
91
92
93

94
95
96
97
98
99
100
101
102
103



104

105


106


107
108
109
110
111
112
113
114

115
116
117
118
119

120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135

136
137
138
139
140

141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159


160
161
162
163

164
165
166
167
168

169
170
171
172
173
174
175
176
177
178
179
180

181



182
183
184
185
186
187
188
189
190

191
192

193
194
195
196
197
198
199
200
201
202
203
204
205







+








-









-
+


-
+





-
+


-
+







-
+








-
+


-
+











-
+

-
+

-
+











-
+









-
-
-
+
-

-
-
+
-
-








-
+




-
+















-
+




-
+


















-
-
+
+


-
+




-
+











-
+
-
-
-
+








-
+

-
+












//-----------------------------------------------------------------------------

// Package webui provides web-UI handlers for web requests.
package webui

import (
	"context"
	"errors"
	"fmt"
	"io"
	"net/url"
	"time"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/index"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/strfun"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
)

var space = []byte{' '}

func writeHTMLMetaValue(w io.Writer, m *meta.Meta, key string, getTitle getTitleFunc, env *encoder.Environment) {
func (wui *WebUI) writeHTMLMetaValue(w io.Writer, m *meta.Meta, key string, getTitle getTitleFunc, env *encoder.Environment) {
	switch kt := m.Type(key); kt {
	case meta.TypeBool:
		writeHTMLBool(w, key, m.GetBool(key))
		wui.writeHTMLBool(w, key, m.GetBool(key))
	case meta.TypeCredential:
		writeCredential(w, m.GetDefault(key, "???c"))
	case meta.TypeEmpty:
		writeEmpty(w, m.GetDefault(key, "???e"))
	case meta.TypeID:
		writeIdentifier(w, m.GetDefault(key, "???i"), getTitle)
		wui.writeIdentifier(w, m.GetDefault(key, "???i"), getTitle)
	case meta.TypeIDSet:
		if l, ok := m.GetList(key); ok {
			writeIdentifierSet(w, l, getTitle)
			wui.writeIdentifierSet(w, l, getTitle)
		}
	case meta.TypeNumber:
		writeNumber(w, m.GetDefault(key, "???n"))
	case meta.TypeString:
		writeString(w, m.GetDefault(key, "???s"))
	case meta.TypeTagSet:
		if l, ok := m.GetList(key); ok {
			writeTagSet(w, key, l)
			wui.writeTagSet(w, key, l)
		}
	case meta.TypeTimestamp:
		if ts, ok := m.GetTime(key); ok {
			writeTimestamp(w, ts)
		}
	case meta.TypeURL:
		writeURL(w, m.GetDefault(key, "???u"))
	case meta.TypeWord:
		writeWord(w, key, m.GetDefault(key, "???w"))
		wui.writeWord(w, key, m.GetDefault(key, "???w"))
	case meta.TypeWordSet:
		if l, ok := m.GetList(key); ok {
			writeWordSet(w, key, l)
			wui.writeWordSet(w, key, l)
		}
	case meta.TypeZettelmarkup:
		writeZettelmarkup(w, m.GetDefault(key, "???z"), env)
	case meta.TypeUnknown:
		writeUnknown(w, m.GetDefault(key, "???u"))
	default:
		strfun.HTMLEscape(w, m.GetDefault(key, "???w"), false)
		fmt.Fprintf(w, " <b>(Unhandled type: %v, key: %v)</b>", kt, key)
	}
}

func writeHTMLBool(w io.Writer, key string, val bool) {
func (wui *WebUI) writeHTMLBool(w io.Writer, key string, val bool) {
	if val {
		writeLink(w, key, "true", "True")
		wui.writeLink(w, key, "true", "True")
	} else {
		writeLink(w, key, "false", "False")
		wui.writeLink(w, key, "false", "False")
	}
}

func writeCredential(w io.Writer, val string) {
	strfun.HTMLEscape(w, val, false)
}

func writeEmpty(w io.Writer, val string) {
	strfun.HTMLEscape(w, val, false)
}

func writeIdentifier(w io.Writer, val string, getTitle func(id.Zid, string) (string, int)) {
func (wui *WebUI) writeIdentifier(w io.Writer, val string, getTitle func(id.Zid, string) (string, int)) {
	zid, err := id.Parse(val)
	if err != nil {
		strfun.HTMLEscape(w, val, false)
		return
	}
	title, found := getTitle(zid, "text")
	switch {
	case found > 0:
		if title == "" {
			fmt.Fprintf(
				w, "<a href=\"%v\">%v</a>",
				adapter.NewURLBuilder('h').SetZid(zid), zid,
			fmt.Fprintf(w, "<a href=\"%v\">%v</a>", wui.NewURLBuilder('h').SetZid(zid), zid)
			)
		} else {
			fmt.Fprintf(
				w, "<a href=\"%v\" title=\"%v\">%v</a>",
			fmt.Fprintf(w, "<a href=\"%v\" title=\"%v\">%v</a>", wui.NewURLBuilder('h').SetZid(zid), title, zid)
				adapter.NewURLBuilder('h').SetZid(zid), title, zid,
			)
		}
	case found == 0:
		fmt.Fprintf(w, "<s>%v</s>", val)
	case found < 0:
		io.WriteString(w, val)
	}
}

func writeIdentifierSet(w io.Writer, vals []string, getTitle func(id.Zid, string) (string, int)) {
func (wui *WebUI) writeIdentifierSet(w io.Writer, vals []string, getTitle func(id.Zid, string) (string, int)) {
	for i, val := range vals {
		if i > 0 {
			w.Write(space)
		}
		writeIdentifier(w, val, getTitle)
		wui.writeIdentifier(w, val, getTitle)
	}
}

func writeNumber(w io.Writer, val string) {
	strfun.HTMLEscape(w, val, false)
}

func writeString(w io.Writer, val string) {
	strfun.HTMLEscape(w, val, false)
}

func writeUnknown(w io.Writer, val string) {
	strfun.HTMLEscape(w, val, false)
}

func writeTagSet(w io.Writer, key string, tags []string) {
func (wui *WebUI) writeTagSet(w io.Writer, key string, tags []string) {
	for i, tag := range tags {
		if i > 0 {
			w.Write(space)
		}
		writeLink(w, key, tag, tag)
		wui.writeLink(w, key, tag, tag)
	}
}

func writeTimestamp(w io.Writer, ts time.Time) {
	io.WriteString(w, ts.Format("2006-01-02&nbsp;15:04:05"))
}

func writeURL(w io.Writer, val string) {
	u, err := url.Parse(val)
	if err != nil {
		strfun.HTMLEscape(w, val, false)
		return
	}
	fmt.Fprintf(w, "<a href=\"%v\">", u)
	strfun.HTMLEscape(w, val, false)
	io.WriteString(w, "</a>")
}

func writeWord(w io.Writer, key, word string) {
	writeLink(w, key, word, word)
func (wui *WebUI) writeWord(w io.Writer, key, word string) {
	wui.writeLink(w, key, word, word)
}

func writeWordSet(w io.Writer, key string, words []string) {
func (wui *WebUI) writeWordSet(w io.Writer, key string, words []string) {
	for i, word := range words {
		if i > 0 {
			w.Write(space)
		}
		writeWord(w, key, word)
		wui.writeWord(w, key, word)
	}
}
func writeZettelmarkup(w io.Writer, val string, env *encoder.Environment) {
	title, err := adapter.FormatInlines(parser.ParseMetadata(val), "html", env)
	if err != nil {
		strfun.HTMLEscape(w, val, false)
		return
	}
	io.WriteString(w, title)
}

func writeLink(w io.Writer, key, value, text string) {
func (wui *WebUI) writeLink(w io.Writer, key, value, text string) {
	fmt.Fprintf(
		w, "<a href=\"%v?%v=%v\">",
		adapter.NewURLBuilder('h'), url.QueryEscape(key), url.QueryEscape(value))
	fmt.Fprintf(w, "<a href=\"%v?%v=%v\">", wui.NewURLBuilder('h'), url.QueryEscape(key), url.QueryEscape(value))
	strfun.HTMLEscape(w, text, false)
	io.WriteString(w, "</a>")
}

type getTitleFunc func(id.Zid, string) (string, int)

func makeGetTitle(ctx context.Context, getMeta usecase.GetMeta, env *encoder.Environment) getTitleFunc {
	return func(zid id.Zid, format string) (string, int) {
		m, err := getMeta.Run(index.NoEnrichContext(ctx), zid)
		m, err := getMeta.Run(place.NoEnrichContext(ctx), zid)
		if err != nil {
			if place.IsErrNotAllowed(err) {
			if errors.Is(err, &place.ErrNotAllowed{}) {
				return "", -1
			}
			return "", 0
		}
		astTitle := parser.ParseMetadata(m.GetDefault(meta.KeyTitle, ""))
		title, err := adapter.FormatInlines(astTitle, format, env)
		if err == nil {
			return title, 1
		}
		return "", 1
	}
}

Changes to web/adapter/webui/lists.go.

15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45
46
47

48
49

50
51

52
53
54
55
56
57

58
59
60
61
62
63




64
65

66
67
68
69
70

71
72
73
74
75
76
77
78
79
80
81
82
83

84
85
86
87
88
89
90
91
92
93

94
95
96

97
98
99

100
101
102


103
104
105
106
107
108
109
15
16
17
18
19
20
21

22
23
24

25
26
27
28
29

30
31
32
33

34

35
36
37
38
39
40
41
42

43
44

45
46

47
48
49
50
51


52
53
54
55



56
57
58
59
60

61
62
63
64
65

66
67
68
69
70
71
72
73
74





75

76
77
78
79
80
81
82
83

84
85
86

87
88
89

90
91


92
93
94
95
96
97
98
99
100







-



-





-




-
+
-








-
+

-
+

-
+




-
-
+



-
-
-
+
+
+
+

-
+




-
+








-
-
-
-
-
+
-








-
+


-
+


-
+

-
-
+
+







	"context"
	"net/http"
	"net/url"
	"sort"
	"strconv"
	"strings"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/index"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/search"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakeListHTMLMetaHandler creates a HTTP handler for rendering the list of
// zettel as HTML.
func MakeListHTMLMetaHandler(
func (wui *WebUI) MakeListHTMLMetaHandler(
	te *TemplateEngine,
	listMeta usecase.ListMeta,
	listRole usecase.ListRole,
	listTags usecase.ListTags,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		query := r.URL.Query()
		switch query.Get("_l") {
		case "r":
			renderWebUIRolesList(w, r, te, listRole)
			wui.renderRolesList(w, r, listRole)
		case "t":
			renderWebUITagsList(w, r, te, listTags)
			wui.renderTagsList(w, r, listTags)
		default:
			renderWebUIZettelList(w, r, te, listMeta)
			wui.renderZettelList(w, r, listMeta)
		}
	}
}

func renderWebUIZettelList(
	w http.ResponseWriter, r *http.Request, te *TemplateEngine, listMeta usecase.ListMeta) {
func (wui *WebUI) renderZettelList(w http.ResponseWriter, r *http.Request, listMeta usecase.ListMeta) {
	query := r.URL.Query()
	s := adapter.GetSearch(query, false)
	ctx := r.Context()
	title := listTitleSearch("Filter", s)
	renderWebUIMetaList(
		ctx, w, te, title, s, func(s *search.Search) ([]*meta.Meta, error) {
	title := wui.listTitleSearch("Filter", s)
	wui.renderMetaList(
		ctx, w, title, s,
		func(s *search.Search) ([]*meta.Meta, error) {
			if !s.HasComputedMetaKey() {
				ctx = index.NoEnrichContext(ctx)
				ctx = place.NoEnrichContext(ctx)
			}
			return listMeta.Run(ctx, s)
		},
		func(offset int) string {
			return newPageURL('h', query, offset, "_offset", "_limit")
			return wui.newPageURL('h', query, offset, "_offset", "_limit")
		})
}

type roleInfo struct {
	Text string
	URL  string
}

func renderWebUIRolesList(
	w http.ResponseWriter,
	r *http.Request,
	te *TemplateEngine,
	listRole usecase.ListRole,
func (wui *WebUI) renderRolesList(w http.ResponseWriter, r *http.Request, listRole usecase.ListRole) {
) {
	ctx := r.Context()
	roleList, err := listRole.Run(ctx)
	if err != nil {
		adapter.ReportUsecaseError(w, err)
		return
	}

	roleInfos := make([]roleInfo, 0, len(roleList))
	for _, r := range roleList {
	for _, role := range roleList {
		roleInfos = append(
			roleInfos,
			roleInfo{r, adapter.NewURLBuilder('h').AppendQuery("role", r).String()})
			roleInfo{role, wui.NewURLBuilder('h').AppendQuery("role", role).String()})
	}

	user := session.GetUser(ctx)
	user := wui.getUser(ctx)
	var base baseData
	te.makeBaseData(ctx, runtime.GetDefaultLang(), runtime.GetSiteName(), user, &base)
	te.renderTemplate(ctx, w, id.RolesTemplateZid, &base, struct {
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base)
	wui.renderTemplate(ctx, w, id.RolesTemplateZid, &base, struct {
		Roles []roleInfo
	}{
		Roles: roleInfos,
	})
}

type countInfo struct {
117
118
119
120
121
122
123
124
125
126
127
128

129
130
131
132
133
134

135
136
137
138

139
140
141

142
143
144
145
146
147
148
108
109
110
111
112
113
114





115

116
117
118
119

120
121
122
123

124
125
126

127
128
129
130
131
132
133
134







-
-
-
-
-
+
-




-
+



-
+


-
+







	count int
	Count string
	Size  string
}

var fontSizes = [...]int{75, 83, 100, 117, 150, 200}

func renderWebUITagsList(
	w http.ResponseWriter,
	r *http.Request,
	te *TemplateEngine,
	listTags usecase.ListTags,
func (wui *WebUI) renderTagsList(w http.ResponseWriter, r *http.Request, listTags usecase.ListTags) {
) {
	ctx := r.Context()
	iMinCount, _ := strconv.Atoi(r.URL.Query().Get("min"))
	tagData, err := listTags.Run(ctx, iMinCount)
	if err != nil {
		te.reportError(ctx, w, err)
		wui.reportError(ctx, w, err)
		return
	}

	user := session.GetUser(ctx)
	user := wui.getUser(ctx)
	tagsList := make([]tagInfo, 0, len(tagData))
	countMap := make(map[int]int)
	baseTagListURL := adapter.NewURLBuilder('h')
	baseTagListURL := wui.NewURLBuilder('h')
	for tag, ml := range tagData {
		count := len(ml)
		countMap[count]++
		tagsList = append(
			tagsList,
			tagInfo{tag, baseTagListURL.AppendQuery("tags", tag).String(), count, "", ""})
		baseTagListURL.ClearQuery()
160
161
162
163
164
165
166
167

168
169
170
171
172
173
174

175
176
177
178
179
180
181
182
183
184
185
186

187
188
189
190
191
192
193

194
195
196

197
198
199
200
201
202
203



204
205

206
207
208
209
210

211
212
213
214
215
216

217
218
219
220
221

222
223
224
225
226
227
228
229
230

231
232
233

234
235
236
237
238
239
240
241

242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257



258
259
260
261
262
263
264
265

266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282



283
284
285
286
287
288
289
290
291

292
293
294
295
296
297
298

299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315

316
317
318
319
320


321
322

323
324
325
326
327


328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346

347
348

349
350
351
352
353
354
355
356
357
358
359
360


361
362
363
364

365
366
367
368
369

370
371

372
373
374
375
376


377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393

394
395
396
397
146
147
148
149
150
151
152

153
154
155
156
157
158
159

160
161
162
163
164
165
166
167
168
169
170
171

172

173
174
175
176
177
178
179
180
181

182
183
184
185




186
187
188
189

190
191
192
193
194

195
196
197
198
199
200

201
202
203
204
205

206
207
208
209
210
211
212
213
214

215
216
217

218
219
220
221
222
223
224
225

226
227
228
229
230
231
232
233
234
235
236
237
238
239



240
241
242
243
244
245
246
247
248
249

250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265


266
267
268
269
270
271
272
273
274
275
276

277
278
279
280
281
282
283

284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300

301
302
303
304


305
306
307

308
309
310
311


312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331

332
333

334
335
336
337
338
339
340
341
342
343
344


345
346
347
348
349

350
351
352
353
354

355
356

357
358
359
360


361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378

379
380
381
382
383







-
+






-
+











-
+
-






+


-
+



-
-
-
-
+
+
+

-
+




-
+





-
+




-
+








-
+


-
+







-
+













-
-
-
+
+
+







-
+















-
-
+
+
+








-
+






-
+
















-
+



-
-
+
+

-
+



-
-
+
+


















-
+

-
+










-
-
+
+



-
+




-
+

-
+



-
-
+
+
















-
+




	for i := 0; i < len(tagsList); i++ {
		count := tagsList[i].count
		tagsList[i].Count = strconv.Itoa(count)
		tagsList[i].Size = strconv.Itoa(countMap[count])
	}

	var base baseData
	te.makeBaseData(ctx, runtime.GetDefaultLang(), runtime.GetSiteName(), user, &base)
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base)
	minCounts := make([]countInfo, 0, len(countList))
	for _, c := range countList {
		sCount := strconv.Itoa(c)
		minCounts = append(minCounts, countInfo{sCount, base.ListTagsURL + "&min=" + sCount})
	}

	te.renderTemplate(ctx, w, id.TagsTemplateZid, &base, struct {
	wui.renderTemplate(ctx, w, id.TagsTemplateZid, &base, struct {
		ListTagsURL string
		MinCounts   []countInfo
		Tags        []tagInfo
	}{
		ListTagsURL: base.ListTagsURL,
		MinCounts:   minCounts,
		Tags:        tagsList,
	})
}

// MakeSearchHandler creates a new HTTP handler for the use case "search".
func MakeSearchHandler(
func (wui *WebUI) MakeSearchHandler(
	te *TemplateEngine,
	ucSearch usecase.Search,
	getMeta usecase.GetMeta,
	getZettel usecase.GetZettel,
) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		query := r.URL.Query()
		ctx := r.Context()
		s := adapter.GetSearch(query, true)
		if s == nil {
			redirectFound(w, r, adapter.NewURLBuilder('h'))
			redirectFound(w, r, wui.NewURLBuilder('h'))
			return
		}

		ctx := r.Context()
		title := listTitleSearch("Search", s)
		renderWebUIMetaList(
			ctx, w, te, title, s, func(s *search.Search) ([]*meta.Meta, error) {
		title := wui.listTitleSearch("Search", s)
		wui.renderMetaList(
			ctx, w, title, s, func(s *search.Search) ([]*meta.Meta, error) {
				if !s.HasComputedMetaKey() {
					ctx = index.NoEnrichContext(ctx)
					ctx = place.NoEnrichContext(ctx)
				}
				return ucSearch.Run(ctx, s)
			},
			func(offset int) string {
				return newPageURL('f', query, offset, "offset", "limit")
				return wui.newPageURL('f', query, offset, "offset", "limit")
			})
	}
}

// MakeZettelContextHandler creates a new HTTP handler for the use case "zettel context".
func MakeZettelContextHandler(te *TemplateEngine, getContext usecase.ZettelContext) http.HandlerFunc {
func (wui *WebUI) MakeZettelContextHandler(getContext usecase.ZettelContext) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}
		q := r.URL.Query()
		dir := usecase.ParseZCDirection(q.Get("dir"))
		depth := getIntParameter(q, "depth", 5)
		limit := getIntParameter(q, "limit", 200)
		metaList, err := getContext.Run(ctx, zid, dir, depth, limit)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		metaLinks, err := buildHTMLMetaList(metaList)
		metaLinks, err := wui.buildHTMLMetaList(metaList)
		if err != nil {
			adapter.InternalServerError(w, "Build HTML meta list", err)
			return
		}

		depths := []string{"2", "3", "4", "5", "6", "7", "8", "9", "10"}
		depthLinks := make([]simpleLink, len(depths))
		depthURL := adapter.NewURLBuilder('j').SetZid(zid)
		depthURL := wui.NewURLBuilder('j').SetZid(zid)
		for i, depth := range depths {
			depthURL.ClearQuery()
			switch dir {
			case usecase.ZettelContextBackward:
				depthURL.AppendQuery("dir", "backward")
			case usecase.ZettelContextForward:
				depthURL.AppendQuery("dir", "forward")
			}
			depthURL.AppendQuery("depth", depth)
			depthLinks[i].Text = depth
			depthLinks[i].URL = depthURL.String()
		}
		var base baseData
		user := session.GetUser(ctx)
		te.makeBaseData(ctx, runtime.GetDefaultLang(), runtime.GetSiteName(), user, &base)
		te.renderTemplate(ctx, w, id.ContextTemplateZid, &base, struct {
		user := wui.getUser(ctx)
		wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base)
		wui.renderTemplate(ctx, w, id.ContextTemplateZid, &base, struct {
			Title   string
			InfoURL string
			Depths  []simpleLink
			Start   simpleLink
			Metas   []simpleLink
		}{
			Title:   "Zettel Context",
			InfoURL: adapter.NewURLBuilder('i').SetZid(zid).String(),
			InfoURL: wui.NewURLBuilder('i').SetZid(zid).String(),
			Depths:  depthLinks,
			Start:   metaLinks[0],
			Metas:   metaLinks[1:],
		})
	}
}

func getIntParameter(q url.Values, key string, minValue int) int {
	val, ok := adapter.GetInteger(q, key)
	if !ok || val < 0 {
		return minValue
	}
	return val
}

func renderWebUIMetaList(
	ctx context.Context, w http.ResponseWriter, te *TemplateEngine,
func (wui *WebUI) renderMetaList(
	ctx context.Context,
	w http.ResponseWriter,
	title string,
	s *search.Search,
	ucMetaList func(sorter *search.Search) ([]*meta.Meta, error),
	pageURL func(int) string) {

	var metaList []*meta.Meta
	var err error
	var prevURL, nextURL string
	if lps := runtime.GetListPageSize(); lps > 0 {
	if lps := wui.rtConfig.GetListPageSize(); lps > 0 {
		if s.GetLimit() < lps {
			s.SetLimit(lps + 1)
		}

		metaList, err = ucMetaList(s)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		if offset := s.GetOffset(); offset > 0 {
			offset -= lps
			if offset < 0 {
				offset = 0
			}
			prevURL = pageURL(offset)
		}
		if len(metaList) >= s.GetLimit() {
			nextURL = pageURL(s.GetOffset() + lps)
			metaList = metaList[:len(metaList)-1]
		}
	} else {
		metaList, err = ucMetaList(s)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
	}
	user := session.GetUser(ctx)
	metas, err := buildHTMLMetaList(metaList)
	user := wui.getUser(ctx)
	metas, err := wui.buildHTMLMetaList(metaList)
	if err != nil {
		te.reportError(ctx, w, err)
		wui.reportError(ctx, w, err)
		return
	}
	var base baseData
	te.makeBaseData(ctx, runtime.GetDefaultLang(), runtime.GetSiteName(), user, &base)
	te.renderTemplate(ctx, w, id.ListTemplateZid, &base, struct {
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), wui.rtConfig.GetSiteName(), user, &base)
	wui.renderTemplate(ctx, w, id.ListTemplateZid, &base, struct {
		Title       string
		Metas       []simpleLink
		HasPrevNext bool
		HasPrev     bool
		PrevURL     string
		HasNext     bool
		NextURL     string
	}{
		Title:       title,
		Metas:       metas,
		HasPrevNext: len(prevURL) > 0 || len(nextURL) > 0,
		HasPrev:     len(prevURL) > 0,
		PrevURL:     prevURL,
		HasNext:     len(nextURL) > 0,
		NextURL:     nextURL,
	})
}

func listTitleSearch(prefix string, s *search.Search) string {
func (wui *WebUI) listTitleSearch(prefix string, s *search.Search) string {
	if s == nil {
		return runtime.GetSiteName()
		return wui.rtConfig.GetSiteName()
	}
	var sb strings.Builder
	sb.WriteString(prefix)
	if s != nil {
		sb.WriteString(": ")
		s.Print(&sb)
	}
	return sb.String()
}

func newPageURL(key byte, query url.Values, offset int, offsetKey, limitKey string) string {
	urlBuilder := adapter.NewURLBuilder(key)
func (wui *WebUI) newPageURL(key byte, query url.Values, offset int, offsetKey, limitKey string) string {
	ub := wui.NewURLBuilder(key)
	for key, values := range query {
		if key != offsetKey && key != limitKey {
			for _, val := range values {
				urlBuilder.AppendQuery(key, val)
				ub.AppendQuery(key, val)
			}
		}
	}
	if offset > 0 {
		urlBuilder.AppendQuery(offsetKey, strconv.Itoa(offset))
		ub.AppendQuery(offsetKey, strconv.Itoa(offset))
	}
	return urlBuilder.String()
	return ub.String()
}

// buildHTMLMetaList builds a zettel list based on a meta list for HTML rendering.
func buildHTMLMetaList(metaList []*meta.Meta) ([]simpleLink, error) {
	defaultLang := runtime.GetDefaultLang()
func (wui *WebUI) buildHTMLMetaList(metaList []*meta.Meta) ([]simpleLink, error) {
	defaultLang := wui.rtConfig.GetDefaultLang()
	metas := make([]simpleLink, 0, len(metaList))
	for _, m := range metaList {
		var lang string
		if val, ok := m.Get(meta.KeyLang); ok {
			lang = val
		} else {
			lang = defaultLang
		}
		title, _ := m.Get(meta.KeyTitle)
		env := encoder.Environment{Lang: lang, Interactive: true}
		htmlTitle, err := adapter.FormatInlines(parser.ParseMetadata(title), "html", &env)
		if err != nil {
			return nil, err
		}
		metas = append(metas, simpleLink{
			Text: htmlTitle,
			URL:  adapter.NewURLBuilder('h').SetZid(m.Zid).String(),
			URL:  wui.NewURLBuilder('h').SetZid(m.Zid).String(),
		})
	}
	return metas, nil
}

Changes to web/adapter/webui/login.go.

9
10
11
12
13
14
15
16
17
18
19
20
21

22
23
24
25
26
27
28
29
30

31
32

33
34
35
36

37
38
39


40
41
42
43
44
45
46
47
48
49

50
51
52


53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81















82
83
84
85
86





87
88

89
90
91
92
93
94
95
96
97


98
99
9
10
11
12
13
14
15

16

17


18

19
20
21

22
23
24

25
26

27
28
29
30

31
32


33
34
35
36
37
38
39
40
41
42
43

44
45


46
47
48
49



























50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65




66
67
68
69
70
71

72
73








74
75
76
77







-

-

-
-
+
-



-



-
+

-
+



-
+

-
-
+
+









-
+

-
-
+
+


-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+

-
-
-
-
+
+
+
+
+

-
+

-
-
-
-
-
-
-
-
+
+


//-----------------------------------------------------------------------------

// Package webui provides web-UI handlers for web requests.
package webui

import (
	"context"
	"fmt"
	"net/http"
	"time"

	"zettelstore.de/z/auth/token"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/auth"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakeGetLoginHandler creates a new HTTP handler to display the HTML login view.
func MakeGetLoginHandler(te *TemplateEngine) http.HandlerFunc {
func (wui *WebUI) MakeGetLoginHandler() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		renderLoginForm(session.ClearToken(r.Context(), w), w, te, false)
		wui.renderLoginForm(wui.clearToken(r.Context(), w), w, false)
	}
}

func renderLoginForm(ctx context.Context, w http.ResponseWriter, te *TemplateEngine, retry bool) {
func (wui *WebUI) renderLoginForm(ctx context.Context, w http.ResponseWriter, retry bool) {
	var base baseData
	te.makeBaseData(ctx, runtime.GetDefaultLang(), "Login", nil, &base)
	te.renderTemplate(ctx, w, id.LoginTemplateZid, &base, struct {
	wui.makeBaseData(ctx, wui.rtConfig.GetDefaultLang(), "Login", nil, &base)
	wui.renderTemplate(ctx, w, id.LoginTemplateZid, &base, struct {
		Title string
		Retry bool
	}{
		Title: base.Title,
		Retry: retry,
	})
}

// MakePostLoginHandlerHTML creates a new HTTP handler to authenticate the given user.
func MakePostLoginHandlerHTML(te *TemplateEngine, auth usecase.Authenticate) http.HandlerFunc {
func (wui *WebUI) MakePostLoginHandlerHTML(ucAuth usecase.Authenticate) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		if !startup.WithAuth() {
			redirectFound(w, r, adapter.NewURLBuilder('/'))
		if !wui.authz.WithAuth() {
			redirectFound(w, r, wui.NewURLBuilder('/'))
			return
		}
		htmlDur, _ := startup.TokenLifetime()
		authenticateViaHTML(te, auth, w, r, htmlDur)
	}
}

func authenticateViaHTML(
	te *TemplateEngine,
	auth usecase.Authenticate,
	w http.ResponseWriter,
	r *http.Request,
	authDuration time.Duration,
) {
	ctx := r.Context()
	ident, cred, ok := adapter.GetCredentialsViaForm(r)
	if !ok {
		te.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read login form"))
		return
	}
	token, err := auth.Run(ctx, ident, cred, authDuration, token.KindHTML)
	if err != nil {
		te.reportError(ctx, w, err)
		return
	}
	if token == nil {
		renderLoginForm(session.ClearToken(ctx, w), w, te, true)
		return
	}
		ctx := r.Context()
		ident, cred, ok := adapter.GetCredentialsViaForm(r)
		if !ok {
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read login form"))
			return
		}
		token, err := ucAuth.Run(ctx, ident, cred, wui.tokenLifetime, auth.KindHTML)
		if err != nil {
			wui.reportError(ctx, w, err)
			return
		}
		if token == nil {
			wui.renderLoginForm(wui.clearToken(ctx, w), w, true)
			return
		}

	session.SetToken(w, token, authDuration)
	redirectFound(w, r, adapter.NewURLBuilder('/'))
}

		wui.setToken(w, token)
		redirectFound(w, r, wui.NewURLBuilder('/'))
	}
}

// MakeGetLogoutHandler creates a new HTTP handler to log out the current user
func MakeGetLogoutHandler(te *TemplateEngine) http.HandlerFunc {
func (wui *WebUI) MakeGetLogoutHandler() http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" {
			te.reportError(r.Context(), w, adapter.NewErrBadRequest(
				fmt.Sprintf("Logout not possible in format %q", format)))
			return
		}

		session.ClearToken(r.Context(), w)
		redirectFound(w, r, adapter.NewURLBuilder('/'))
		wui.clearToken(r.Context(), w)
		redirectFound(w, r, wui.NewURLBuilder('/'))
	}
}

Changes to web/adapter/webui/rename_zettel.go.

12
13
14
15
16
17
18
19

20
21
22
23
24
25
26
27
28
29
30
31

32
33
34
35
36

37
38
39
40
41
42

43
44
45
46
47

48
49
50
51
52

53
54
55


56
57
58
59
60
61
62
63
64
65
66

67
68
69
70
71

72
73
74
75
76

77
78
79
80
81

82
83
84
85
86

87
88
89
90
91

92
93
94

95
96
12
13
14
15
16
17
18

19
20
21
22
23
24

25
26
27
28


29
30
31
32
33

34
35
36
37
38
39

40
41
42
43
44

45
46
47
48
49

50
51


52
53
54
55
56
57
58
59
60
61
62
63

64
65
66
67
68

69
70
71
72
73

74
75
76
77
78

79
80
81
82
83

84
85
86
87
88

89
90
91

92
93
94







-
+





-




-
-
+




-
+





-
+




-
+




-
+

-
-
+
+










-
+




-
+




-
+




-
+




-
+




-
+


-
+


package webui

import (
	"fmt"
	"net/http"
	"strings"

	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/place"
	"zettelstore.de/z/usecase"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

// MakeGetRenameZettelHandler creates a new HTTP handler to display the
// HTML rename view of a zettel.
func MakeGetRenameZettelHandler(
	te *TemplateEngine, getMeta usecase.GetMeta) http.HandlerFunc {
func (wui *WebUI) MakeGetRenameZettelHandler(getMeta usecase.GetMeta) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		zid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		m, err := getMeta.Run(ctx, zid)
		if err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}

		if format := adapter.GetFormat(r, r.URL.Query(), "html"); format != "html" {
			te.reportError(ctx, w, adapter.NewErrBadRequest(
			wui.reportError(ctx, w, adapter.NewErrBadRequest(
				fmt.Sprintf("Rename zettel %q not possible in format %q", zid.String(), format)))
			return
		}

		user := session.GetUser(ctx)
		user := wui.getUser(ctx)
		var base baseData
		te.makeBaseData(ctx, runtime.GetLang(m), "Rename Zettel "+zid.String(), user, &base)
		te.renderTemplate(ctx, w, id.RenameTemplateZid, &base, struct {
		wui.makeBaseData(ctx, config.GetLang(m, wui.rtConfig), "Rename Zettel "+zid.String(), user, &base)
		wui.renderTemplate(ctx, w, id.RenameTemplateZid, &base, struct {
			Zid       string
			MetaPairs []meta.Pair
		}{
			Zid:       zid.String(),
			MetaPairs: m.Pairs(true),
		})
	}
}

// MakePostRenameZettelHandler creates a new HTTP handler to rename an existing zettel.
func MakePostRenameZettelHandler(te *TemplateEngine, renameZettel usecase.RenameZettel) http.HandlerFunc {
func (wui *WebUI) MakePostRenameZettelHandler(renameZettel usecase.RenameZettel) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := r.Context()
		curZid, err := id.Parse(r.URL.Path[1:])
		if err != nil {
			te.reportError(ctx, w, place.ErrNotFound)
			wui.reportError(ctx, w, place.ErrNotFound)
			return
		}

		if err = r.ParseForm(); err != nil {
			te.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read rename zettel form"))
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Unable to read rename zettel form"))
			return
		}
		if formCurZid, err1 := id.Parse(
			r.PostFormValue("curzid")); err1 != nil || formCurZid != curZid {
			te.reportError(ctx, w, adapter.NewErrBadRequest("Invalid value for current zettel id in form"))
			wui.reportError(ctx, w, adapter.NewErrBadRequest("Invalid value for current zettel id in form"))
			return
		}
		newZid, err := id.Parse(strings.TrimSpace(r.PostFormValue("newzid")))
		if err != nil {
			te.reportError(ctx, w, adapter.NewErrBadRequest(fmt.Sprintf("Invalid new zettel id %q", newZid)))
			wui.reportError(ctx, w, adapter.NewErrBadRequest(fmt.Sprintf("Invalid new zettel id %q", newZid)))
			return
		}

		if err := renameZettel.Run(r.Context(), curZid, newZid); err != nil {
			te.reportError(ctx, w, err)
			wui.reportError(ctx, w, err)
			return
		}
		redirectFound(w, r, adapter.NewURLBuilder('h').SetZid(newZid))
		redirectFound(w, r, wui.NewURLBuilder('h').SetZid(newZid))
	}
}

Changes to web/adapter/webui/response.go.

10
11
12
13
14
15
16
17

18
19
20

21
22
10
11
12
13
14
15
16

17
18
19

20
21
22







-
+


-
+



// Package webui provides web-UI handlers for web requests.
package webui

import (
	"net/http"

	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/server"
)

func redirectFound(w http.ResponseWriter, r *http.Request, ub *adapter.URLBuilder) {
func redirectFound(w http.ResponseWriter, r *http.Request, ub server.URLBuilder) {
	http.Redirect(w, r, ub.String(), http.StatusFound)
}

Deleted web/adapter/webui/template.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324




































































































































































































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package webui provides web-UI handlers for web requests.
package webui

import (
	"bytes"
	"context"
	"log"
	"net/http"
	"sync"

	"zettelstore.de/z/auth/policy"
	"zettelstore.de/z/auth/token"
	"zettelstore.de/z/collect"
	"zettelstore.de/z/config/runtime"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/index"
	"zettelstore.de/z/input"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/place/change"
	"zettelstore.de/z/template"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/session"
)

type templatePlace interface {
	CanCreateZettel(ctx context.Context) bool
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
	CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool
	AllowRenameZettel(ctx context.Context, zid id.Zid) bool
	CanDeleteZettel(ctx context.Context, zid id.Zid) bool
}

// TemplateEngine is the way to render HTML templates.
type TemplateEngine struct {
	place         templatePlace
	templateCache map[id.Zid]*template.Template
	mxCache       sync.RWMutex
	policy        policy.Policy

	stylesheetURL string
	homeURL       string
	listZettelURL string
	listRolesURL  string
	listTagsURL   string
	withAuth      bool
	loginURL      string
	searchURL     string
}

// NewTemplateEngine creates a new TemplateEngine.
func NewTemplateEngine(mgr place.Manager, pol policy.Policy) *TemplateEngine {
	te := &TemplateEngine{
		place:  mgr,
		policy: pol,

		stylesheetURL: adapter.NewURLBuilder('z').SetZid(
			id.BaseCSSZid).AppendQuery("_format", "raw").AppendQuery(
			"_part", "content").String(),
		homeURL:       adapter.NewURLBuilder('/').String(),
		listZettelURL: adapter.NewURLBuilder('h').String(),
		listRolesURL:  adapter.NewURLBuilder('h').AppendQuery("_l", "r").String(),
		listTagsURL:   adapter.NewURLBuilder('h').AppendQuery("_l", "t").String(),
		withAuth:      startup.WithAuth(),
		loginURL:      adapter.NewURLBuilder('a').String(),
		searchURL:     adapter.NewURLBuilder('f').String(),
	}
	te.observe(change.Info{Reason: change.OnReload, Zid: id.Invalid})
	mgr.RegisterObserver(te.observe)
	return te
}

func (te *TemplateEngine) observe(ci change.Info) {
	te.mxCache.Lock()
	if ci.Reason == change.OnReload || ci.Zid == id.BaseTemplateZid {
		te.templateCache = make(map[id.Zid]*template.Template, len(te.templateCache))
	} else {
		delete(te.templateCache, ci.Zid)
	}
	te.mxCache.Unlock()
}

func (te *TemplateEngine) cacheSetTemplate(zid id.Zid, t *template.Template) {
	te.mxCache.Lock()
	te.templateCache[zid] = t
	te.mxCache.Unlock()
}

func (te *TemplateEngine) cacheGetTemplate(zid id.Zid) (*template.Template, bool) {
	te.mxCache.RLock()
	t, ok := te.templateCache[zid]
	te.mxCache.RUnlock()
	return t, ok
}

func (te *TemplateEngine) canCreate(ctx context.Context, user *meta.Meta) bool {
	m := meta.New(id.Invalid)
	return te.policy.CanCreate(user, m) && te.place.CanCreateZettel(ctx)
}

func (te *TemplateEngine) canWrite(
	ctx context.Context, user, meta *meta.Meta, content domain.Content) bool {
	return te.policy.CanWrite(user, meta, meta) &&
		te.place.CanUpdateZettel(ctx, domain.Zettel{Meta: meta, Content: content})
}

func (te *TemplateEngine) canRename(ctx context.Context, user, m *meta.Meta) bool {
	return te.policy.CanRename(user, m) && te.place.AllowRenameZettel(ctx, m.Zid)
}

func (te *TemplateEngine) canDelete(ctx context.Context, user, m *meta.Meta) bool {
	return te.policy.CanDelete(user, m) && te.place.CanDeleteZettel(ctx, m.Zid)
}

func (te *TemplateEngine) getTemplate(
	ctx context.Context, templateID id.Zid) (*template.Template, error) {
	if t, ok := te.cacheGetTemplate(templateID); ok {
		return t, nil
	}
	realTemplateZettel, err := te.place.GetZettel(ctx, templateID)
	if err != nil {
		return nil, err
	}
	t, err := template.ParseString(realTemplateZettel.Content.AsString(), nil)
	if err == nil {
		// t.SetErrorOnMissing()
		te.cacheSetTemplate(templateID, t)
	}
	return t, err
}

type simpleLink struct {
	Text string
	URL  string
}

type baseData struct {
	Lang           string
	MetaHeader     string
	StylesheetURL  string
	Title          string
	HomeURL        string
	WithUser       bool
	WithAuth       bool
	UserIsValid    bool
	UserZettelURL  string
	UserIdent      string
	UserLogoutURL  string
	LoginURL       string
	ListZettelURL  string
	ListRolesURL   string
	ListTagsURL    string
	CanCreate      bool
	NewZettelURL   string
	NewZettelLinks []simpleLink
	SearchURL      string
	Content        string
	FooterHTML     string
}

func (te *TemplateEngine) makeBaseData(
	ctx context.Context, lang, title string, user *meta.Meta, data *baseData) {
	var (
		newZettelLinks []simpleLink
		userZettelURL  string
		userIdent      string
		userLogoutURL  string
	)
	canCreate := te.canCreate(ctx, user)
	if canCreate {
		newZettelLinks = te.fetchNewTemplates(ctx, user)
	}
	userIsValid := user != nil
	if userIsValid {
		userZettelURL = adapter.NewURLBuilder('h').SetZid(user.Zid).String()
		userIdent = user.GetDefault(meta.KeyUserID, "")
		userLogoutURL = adapter.NewURLBuilder('a').SetZid(user.Zid).String()
	}

	data.Lang = lang
	data.StylesheetURL = te.stylesheetURL
	data.Title = title
	data.HomeURL = te.homeURL
	data.WithAuth = te.withAuth
	data.WithUser = data.WithAuth
	data.UserIsValid = userIsValid
	data.UserZettelURL = userZettelURL
	data.UserIdent = userIdent
	data.UserLogoutURL = userLogoutURL
	data.LoginURL = te.loginURL
	data.ListZettelURL = te.listZettelURL
	data.ListRolesURL = te.listRolesURL
	data.ListTagsURL = te.listTagsURL
	data.CanCreate = canCreate
	data.NewZettelLinks = newZettelLinks
	data.SearchURL = te.searchURL
	data.FooterHTML = runtime.GetFooterHTML()
}

// htmlAttrNewWindow eturns HTML attribute string for opening a link in a new window.
// If hasURL is false an empty string is returned.
func htmlAttrNewWindow(hasURL bool) string {
	if hasURL {
		return " target=\"_blank\" ref=\"noopener noreferrer\""
	}
	return ""
}

func (te *TemplateEngine) fetchNewTemplates(ctx context.Context, user *meta.Meta) []simpleLink {
	ctx = index.NoEnrichContext(ctx)
	menu, err := te.place.GetZettel(ctx, id.TOCNewTemplateZid)
	if err != nil {
		return nil
	}
	zn := parser.ParseZettel(menu, "")
	refs := collect.Order(zn)
	result := make([]simpleLink, 0, len(refs))
	for _, ref := range refs {
		zid, err := id.Parse(ref.URL.Path)
		if err != nil {
			continue
		}
		m, err := te.place.GetMeta(ctx, zid)
		if err != nil {
			continue
		}
		if !te.policy.CanRead(user, m) {
			continue
		}
		title := runtime.GetTitle(m)
		astTitle := parser.ParseInlines(input.NewInput(runtime.GetTitle(m)), meta.ValueSyntaxZmk)
		env := encoder.Environment{Lang: runtime.GetLang(m)}
		menuTitle, err := adapter.FormatInlines(astTitle, "html", &env)
		if err != nil {
			menuTitle, err = adapter.FormatInlines(astTitle, "text", nil)
			if err != nil {
				menuTitle = title
			}
		}
		result = append(result, simpleLink{
			Text: menuTitle,
			URL:  adapter.NewURLBuilder('g').SetZid(m.Zid).String(),
		})
	}
	return result
}

func (te *TemplateEngine) renderTemplate(
	ctx context.Context,
	w http.ResponseWriter,
	templateID id.Zid,
	base *baseData,
	data interface{}) {
	te.renderTemplateStatus(ctx, w, http.StatusOK, templateID, base, data)
}

func (te *TemplateEngine) reportError(ctx context.Context, w http.ResponseWriter, err error) {
	code, text := adapter.CodeMessageFromError(err)
	if code == http.StatusInternalServerError {
		log.Printf("%v: %v", text, err)
	}
	user := session.GetUser(ctx)
	var base baseData
	te.makeBaseData(ctx, "en", "Error", user, &base)
	te.renderTemplateStatus(ctx, w, code, id.ErrorTemplateZid, &base, struct {
		ErrorTitle string
		ErrorText  string
	}{
		ErrorTitle: http.StatusText(code),
		ErrorText:  text,
	})
}

func (te *TemplateEngine) renderTemplateStatus(
	ctx context.Context,
	w http.ResponseWriter,
	code int,
	templateID id.Zid,
	base *baseData,
	data interface{}) {

	bt, err := te.getTemplate(ctx, id.BaseTemplateZid)
	if err != nil {
		adapter.InternalServerError(w, "Unable to get base template", err)
		return
	}
	t, err := te.getTemplate(ctx, templateID)
	if err != nil {
		adapter.InternalServerError(w, "Unable to get template", err)
		return
	}
	if user := session.GetUser(ctx); user != nil {
		htmlLifetime, _ := startup.TokenLifetime()
		if tok, err1 := token.GetToken(user, htmlLifetime, token.KindHTML); err1 == nil {
			session.SetToken(w, tok, htmlLifetime)
		}
	}
	var content bytes.Buffer
	err = t.Render(&content, data)
	if err == nil {
		base.Content = content.String()
		w.Header().Set(adapter.ContentType, "text/html; charset=utf-8")
		w.WriteHeader(code)
		err = bt.Render(w, base)
	}
	if err != nil {
		log.Println("Unable to render template", err)
	}
}

Added web/adapter/webui/webui.go.





























































































































































































































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package webui provides web-UI handlers for web requests.
package webui

import (
	"bytes"
	"context"
	"log"
	"net/http"
	"sync"
	"time"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/collect"
	"zettelstore.de/z/config"
	"zettelstore.de/z/domain"
	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/encoder"
	"zettelstore.de/z/input"
	"zettelstore.de/z/kernel"
	"zettelstore.de/z/parser"
	"zettelstore.de/z/place"
	"zettelstore.de/z/template"
	"zettelstore.de/z/web/adapter"
	"zettelstore.de/z/web/server"
)

// WebUI holds all data for delivering the web ui.
type WebUI struct {
	ab       server.AuthBuilder
	authz    auth.AuthzManager
	rtConfig config.Config
	token    auth.TokenManager
	place    webuiPlace
	policy   auth.Policy

	templateCache map[id.Zid]*template.Template
	mxCache       sync.RWMutex

	tokenLifetime time.Duration
	stylesheetURL string
	homeURL       string
	listZettelURL string
	listRolesURL  string
	listTagsURL   string
	withAuth      bool
	loginURL      string
	searchURL     string
}

type webuiPlace interface {
	CanCreateZettel(ctx context.Context) bool
	GetZettel(ctx context.Context, zid id.Zid) (domain.Zettel, error)
	GetMeta(ctx context.Context, zid id.Zid) (*meta.Meta, error)
	CanUpdateZettel(ctx context.Context, zettel domain.Zettel) bool
	AllowRenameZettel(ctx context.Context, zid id.Zid) bool
	CanDeleteZettel(ctx context.Context, zid id.Zid) bool
}

// New creates a new WebUI struct.
func New(ab server.AuthBuilder, authz auth.AuthzManager, rtConfig config.Config, token auth.TokenManager,
	mgr place.Manager, pol auth.Policy) *WebUI {
	wui := &WebUI{
		ab:       ab,
		rtConfig: rtConfig,
		authz:    authz,
		token:    token,
		place:    mgr,
		policy:   pol,

		tokenLifetime: kernel.Main.GetConfig(kernel.WebService, kernel.WebTokenLifetimeHTML).(time.Duration),
		stylesheetURL: ab.NewURLBuilder('z').SetZid(
			id.BaseCSSZid).AppendQuery("_format", "raw").AppendQuery(
			"_part", "content").String(),
		homeURL:       ab.NewURLBuilder('/').String(),
		listZettelURL: ab.NewURLBuilder('h').String(),
		listRolesURL:  ab.NewURLBuilder('h').AppendQuery("_l", "r").String(),
		listTagsURL:   ab.NewURLBuilder('h').AppendQuery("_l", "t").String(),
		withAuth:      authz.WithAuth(),
		loginURL:      ab.NewURLBuilder('a').String(),
		searchURL:     ab.NewURLBuilder('f').String(),
	}
	wui.observe(place.UpdateInfo{Place: mgr, Reason: place.OnReload, Zid: id.Invalid})
	mgr.RegisterObserver(wui.observe)
	return wui
}

func (wui *WebUI) observe(ci place.UpdateInfo) {
	wui.mxCache.Lock()
	if ci.Reason == place.OnReload || ci.Zid == id.BaseTemplateZid {
		wui.templateCache = make(map[id.Zid]*template.Template, len(wui.templateCache))
	} else {
		delete(wui.templateCache, ci.Zid)
	}
	wui.mxCache.Unlock()
}

func (wui *WebUI) cacheSetTemplate(zid id.Zid, t *template.Template) {
	wui.mxCache.Lock()
	wui.templateCache[zid] = t
	wui.mxCache.Unlock()
}

func (wui *WebUI) cacheGetTemplate(zid id.Zid) (*template.Template, bool) {
	wui.mxCache.RLock()
	t, ok := wui.templateCache[zid]
	wui.mxCache.RUnlock()
	return t, ok
}

func (wui *WebUI) canCreate(ctx context.Context, user *meta.Meta) bool {
	m := meta.New(id.Invalid)
	return wui.policy.CanCreate(user, m) && wui.place.CanCreateZettel(ctx)
}

func (wui *WebUI) canWrite(
	ctx context.Context, user, meta *meta.Meta, content domain.Content) bool {
	return wui.policy.CanWrite(user, meta, meta) &&
		wui.place.CanUpdateZettel(ctx, domain.Zettel{Meta: meta, Content: content})
}

func (wui *WebUI) canRename(ctx context.Context, user, m *meta.Meta) bool {
	return wui.policy.CanRename(user, m) && wui.place.AllowRenameZettel(ctx, m.Zid)
}

func (wui *WebUI) canDelete(ctx context.Context, user, m *meta.Meta) bool {
	return wui.policy.CanDelete(user, m) && wui.place.CanDeleteZettel(ctx, m.Zid)
}

func (wui *WebUI) getTemplate(
	ctx context.Context, templateID id.Zid) (*template.Template, error) {
	if t, ok := wui.cacheGetTemplate(templateID); ok {
		return t, nil
	}
	realTemplateZettel, err := wui.place.GetZettel(ctx, templateID)
	if err != nil {
		return nil, err
	}
	t, err := template.ParseString(realTemplateZettel.Content.AsString(), nil)
	if err == nil {
		// t.SetErrorOnMissing()
		wui.cacheSetTemplate(templateID, t)
	}
	return t, err
}

type simpleLink struct {
	Text string
	URL  string
}

type baseData struct {
	Lang           string
	MetaHeader     string
	StylesheetURL  string
	Title          string
	HomeURL        string
	WithUser       bool
	WithAuth       bool
	UserIsValid    bool
	UserZettelURL  string
	UserIdent      string
	UserLogoutURL  string
	LoginURL       string
	ListZettelURL  string
	ListRolesURL   string
	ListTagsURL    string
	CanCreate      bool
	NewZettelURL   string
	NewZettelLinks []simpleLink
	SearchURL      string
	Content        string
	FooterHTML     string
}

func (wui *WebUI) makeBaseData(
	ctx context.Context, lang, title string, user *meta.Meta, data *baseData) {
	var (
		newZettelLinks []simpleLink
		userZettelURL  string
		userIdent      string
		userLogoutURL  string
	)
	canCreate := wui.canCreate(ctx, user)
	if canCreate {
		newZettelLinks = wui.fetchNewTemplates(ctx, user)
	}
	userIsValid := user != nil
	if userIsValid {
		userZettelURL = wui.NewURLBuilder('h').SetZid(user.Zid).String()
		userIdent = user.GetDefault(meta.KeyUserID, "")
		userLogoutURL = wui.NewURLBuilder('a').SetZid(user.Zid).String()
	}

	data.Lang = lang
	data.StylesheetURL = wui.stylesheetURL
	data.Title = title
	data.HomeURL = wui.homeURL
	data.WithAuth = wui.withAuth
	data.WithUser = data.WithAuth
	data.UserIsValid = userIsValid
	data.UserZettelURL = userZettelURL
	data.UserIdent = userIdent
	data.UserLogoutURL = userLogoutURL
	data.LoginURL = wui.loginURL
	data.ListZettelURL = wui.listZettelURL
	data.ListRolesURL = wui.listRolesURL
	data.ListTagsURL = wui.listTagsURL
	data.CanCreate = canCreate
	data.NewZettelLinks = newZettelLinks
	data.SearchURL = wui.searchURL
	data.FooterHTML = wui.rtConfig.GetFooterHTML()
}

// htmlAttrNewWindow eturns HTML attribute string for opening a link in a new window.
// If hasURL is false an empty string is returned.
func htmlAttrNewWindow(hasURL bool) string {
	if hasURL {
		return " target=\"_blank\" ref=\"noopener noreferrer\""
	}
	return ""
}

func (wui *WebUI) fetchNewTemplates(ctx context.Context, user *meta.Meta) []simpleLink {
	ctx = place.NoEnrichContext(ctx)
	menu, err := wui.place.GetZettel(ctx, id.TOCNewTemplateZid)
	if err != nil {
		return nil
	}
	zn := parser.ParseZettel(menu, "", wui.rtConfig)
	refs := collect.Order(zn)
	result := make([]simpleLink, 0, len(refs))
	for _, ref := range refs {
		zid, err := id.Parse(ref.URL.Path)
		if err != nil {
			continue
		}
		m, err := wui.place.GetMeta(ctx, zid)
		if err != nil {
			continue
		}
		if !wui.policy.CanRead(user, m) {
			continue
		}
		title := config.GetTitle(m, wui.rtConfig)
		astTitle := parser.ParseInlines(input.NewInput(title), meta.ValueSyntaxZmk)
		env := encoder.Environment{Lang: config.GetLang(m, wui.rtConfig)}
		menuTitle, err := adapter.FormatInlines(astTitle, "html", &env)
		if err != nil {
			menuTitle, err = adapter.FormatInlines(astTitle, "text", nil)
			if err != nil {
				menuTitle = title
			}
		}
		result = append(result, simpleLink{
			Text: menuTitle,
			URL:  wui.NewURLBuilder('g').SetZid(m.Zid).String(),
		})
	}
	return result
}

func (wui *WebUI) renderTemplate(
	ctx context.Context,
	w http.ResponseWriter,
	templateID id.Zid,
	base *baseData,
	data interface{}) {
	wui.renderTemplateStatus(ctx, w, http.StatusOK, templateID, base, data)
}

func (wui *WebUI) reportError(ctx context.Context, w http.ResponseWriter, err error) {
	code, text := adapter.CodeMessageFromError(err)
	if code == http.StatusInternalServerError {
		log.Printf("%v: %v", text, err)
	}
	user := wui.getUser(ctx)
	var base baseData
	wui.makeBaseData(ctx, meta.ValueLangEN, "Error", user, &base)
	wui.renderTemplateStatus(ctx, w, code, id.ErrorTemplateZid, &base, struct {
		ErrorTitle string
		ErrorText  string
	}{
		ErrorTitle: http.StatusText(code),
		ErrorText:  text,
	})
}

func (wui *WebUI) renderTemplateStatus(
	ctx context.Context,
	w http.ResponseWriter,
	code int,
	templateID id.Zid,
	base *baseData,
	data interface{}) {

	bt, err := wui.getTemplate(ctx, id.BaseTemplateZid)
	if err != nil {
		adapter.InternalServerError(w, "Unable to get base template", err)
		return
	}
	t, err := wui.getTemplate(ctx, templateID)
	if err != nil {
		adapter.InternalServerError(w, "Unable to get template", err)
		return
	}
	if user := wui.getUser(ctx); user != nil {
		if tok, err1 := wui.token.GetToken(user, wui.tokenLifetime, auth.KindHTML); err1 == nil {
			wui.setToken(w, tok)
		}
	}
	var content bytes.Buffer
	err = t.Render(&content, data)
	if err == nil {
		base.Content = content.String()
		w.Header().Set(adapter.ContentType, "text/html; charset=utf-8")
		w.WriteHeader(code)
		err = bt.Render(w, base)
	}
	if err != nil {
		log.Println("Unable to render template", err)
	}
}

func (wui *WebUI) getUser(ctx context.Context) *meta.Meta { return wui.ab.GetUser(ctx) }

// GetURLPrefix returns the configured URL prefix of the web server.
func (wui *WebUI) GetURLPrefix() string { return wui.ab.GetURLPrefix() }

// NewURLBuilder creates a new URL builder object with the given key.
func (wui *WebUI) NewURLBuilder(key byte) server.URLBuilder { return wui.ab.NewURLBuilder(key) }

func (wui *WebUI) clearToken(ctx context.Context, w http.ResponseWriter) context.Context {
	return wui.ab.ClearToken(ctx, w)
}
func (wui *WebUI) setToken(w http.ResponseWriter, token []byte) {
	wui.ab.SetToken(w, token, wui.tokenLifetime)
}

Deleted web/router/router.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111















































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package router provides a router for web requests.
package router

import (
	"net/http"
	"regexp"
)

type (
	methodHandler map[string]http.Handler
	routingTable  map[byte]methodHandler
)

// Router handles all routing for zettelstore.
type Router struct {
	minKey byte
	maxKey byte
	reURL  *regexp.Regexp
	tables [2]routingTable
	mux    *http.ServeMux
}

const (
	indexList   = 0
	indexZettel = 1
)

// NewRouter creates a new, empty router with the given root handler.
func NewRouter() *Router {
	router := &Router{
		minKey: 255,
		maxKey: 0,
		reURL:  regexp.MustCompile("^$"),
		mux:    http.NewServeMux(),
	}
	router.tables[indexList] = make(routingTable)
	router.tables[indexZettel] = make(routingTable)
	return router
}

func (rt *Router) addRoute(key byte, httpMethod string, handler http.Handler, index int) {
	// Set minKey and maxKey; re-calculate regexp.
	if key < rt.minKey || rt.maxKey < key {
		if key < rt.minKey {
			rt.minKey = key
		}
		if rt.maxKey < key {
			rt.maxKey = key
		}
		rt.reURL = regexp.MustCompile(
			"^/(?:([" + string(rt.minKey) + "-" + string(rt.maxKey) + "])(?:/(?:([0-9]{14})/?)?)?)$")
	}

	mh, hasKey := rt.tables[index][key]
	if !hasKey {
		mh = make(methodHandler)
		rt.tables[index][key] = mh
	}
	mh[httpMethod] = handler
	if httpMethod == http.MethodGet {
		if _, hasHead := rt.tables[index][key][http.MethodHead]; !hasHead {
			rt.tables[index][key][http.MethodHead] = handler
		}
	}
}

// AddListRoute adds a route for the given key and HTTP method to work with a list.
func (rt *Router) AddListRoute(key byte, httpMethod string, handler http.Handler) {
	rt.addRoute(key, httpMethod, handler, indexList)
}

// AddZettelRoute adds a route for the given key and HTTP method to work with a zettel.
func (rt *Router) AddZettelRoute(key byte, httpMethod string, handler http.Handler) {
	rt.addRoute(key, httpMethod, handler, indexZettel)
}

// Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics.
func (rt *Router) Handle(pattern string, handler http.Handler) {
	rt.mux.Handle(pattern, handler)
}

func (rt *Router) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	match := rt.reURL.FindStringSubmatch(r.URL.Path)
	if len(match) == 3 {
		key := match[1][0]
		index := indexZettel
		if match[2] == "" {
			index = indexList
		}
		if mh, ok := rt.tables[index][key]; ok {
			if handler, ok := mh[r.Method]; ok {
				r.URL.Path = "/" + match[2]
				handler.ServeHTTP(w, r)
				return
			}
			http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed)
			return
		}
	}
	rt.mux.ServeHTTP(w, r)
}

Added web/server/impl/http.go.















































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the Zettelstore web service.
package impl

import (
	"context"
	"net"
	"net/http"
	"time"
)

// Server timeout values
const (
	shutdownTimeout = 5 * time.Second
	readTimeout     = 5 * time.Second
	writeTimeout    = 10 * time.Second
	idleTimeout     = 120 * time.Second
)

// httpServer is a HTTP server.
type httpServer struct {
	http.Server
	waitStop chan struct{}
}

// initializeHTTPServer creates a new HTTP server object.
func (srv *httpServer) initializeHTTPServer(addr string, handler http.Handler) {
	if addr == "" {
		addr = ":http"
	}
	srv.Server = http.Server{
		Addr:    addr,
		Handler: handler,

		// See: https://blog.cloudflare.com/exposing-go-on-the-internet/
		ReadTimeout:  readTimeout,
		WriteTimeout: writeTimeout,
		IdleTimeout:  idleTimeout,
	}
	srv.waitStop = make(chan struct{})
}

// SetDebug enables debugging goroutines that are started by the server.
// Basically, just the timeout values are reset. This method should be called
// before running the server.
func (srv *httpServer) SetDebug() {
	srv.ReadTimeout = 0
	srv.WriteTimeout = 0
	srv.IdleTimeout = 0
}

// Run starts the web server, but does not wait for its completion.
func (srv *httpServer) Run() error {
	ln, err := net.Listen("tcp", srv.Addr)
	if err != nil {
		return err
	}

	go func() { srv.Serve(ln) }()
	return nil
}

// Stop the web server.
func (srv *httpServer) Stop() error {
	ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
	defer cancel()

	return srv.Shutdown(ctx)
}

Added web/server/impl/impl.go.




























































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the Zettelstore web service.
package impl

import (
	"context"
	"net/http"
	"time"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/web/server"
)

type myServer struct {
	server           httpServer
	router           httpRouter
	persistentCookie bool
	secureCookie     bool
}

// New creates a new web server.
func New(listenAddr, urlPrefix string, persistentCookie, secureCookie bool, auth auth.TokenManager) server.Server {
	srv := myServer{
		persistentCookie: persistentCookie,
		secureCookie:     secureCookie,
	}
	srv.router.initializeRouter(urlPrefix, auth)
	srv.server.initializeHTTPServer(listenAddr, &srv.router)
	return &srv
}

func (srv *myServer) Handle(pattern string, handler http.Handler) {
	srv.router.Handle(pattern, handler)
}
func (srv *myServer) AddListRoute(key byte, httpMethod string, handler http.Handler) {
	srv.router.addListRoute(key, httpMethod, handler)
}
func (srv *myServer) AddZettelRoute(key byte, httpMethod string, handler http.Handler) {
	srv.router.addZettelRoute(key, httpMethod, handler)
}
func (srv *myServer) SetUserRetriever(ur server.UserRetriever) {
	srv.router.ur = ur
}
func (srv *myServer) GetUser(ctx context.Context) *meta.Meta {
	if data := srv.GetAuthData(ctx); data != nil {
		return data.User
	}
	return nil
}
func (srv *myServer) NewURLBuilder(key byte) server.URLBuilder {
	return &URLBuilder{router: &srv.router, key: key}
}
func (srv *myServer) GetURLPrefix() string {
	return srv.router.urlPrefix
}

const sessionName = "zsession"

func (srv *myServer) SetToken(w http.ResponseWriter, token []byte, d time.Duration) {
	cookie := http.Cookie{
		Name:     sessionName,
		Value:    string(token),
		Path:     srv.GetURLPrefix(),
		Secure:   srv.secureCookie,
		HttpOnly: true,
		SameSite: http.SameSiteStrictMode,
	}
	if srv.persistentCookie && d > 0 {
		cookie.Expires = time.Now().Add(d).Add(30 * time.Second).UTC()
	}
	http.SetCookie(w, &cookie)
}

// ClearToken invalidates the session cookie by sending an empty one.
func (srv *myServer) ClearToken(ctx context.Context, w http.ResponseWriter) context.Context {
	if w != nil {
		srv.SetToken(w, nil, 0)
	}
	return updateContext(ctx, nil, nil)
}

// GetAuthData returns the full authentication data from the context.
func (srv *myServer) GetAuthData(ctx context.Context) *server.AuthData {
	data, ok := ctx.Value(ctxKeySession).(*server.AuthData)
	if ok {
		return data
	}
	return nil
}

type ctxKeyTypeSession struct{}

var ctxKeySession ctxKeyTypeSession

func updateContext(ctx context.Context, user *meta.Meta, data *auth.TokenData) context.Context {
	if data == nil {
		return context.WithValue(ctx, ctxKeySession, &server.AuthData{User: user})
	}
	return context.WithValue(
		ctx,
		ctxKeySession,
		&server.AuthData{
			User:    user,
			Token:   data.Token,
			Now:     data.Now,
			Issued:  data.Issued,
			Expires: data.Expires,
		})
}

func (srv *myServer) SetDebug()   { srv.server.SetDebug() }
func (srv *myServer) Run() error  { return srv.server.Run() }
func (srv *myServer) Stop() error { return srv.server.Stop() }

Added web/server/impl/router.go.














































































































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the Zettelstore web service.
package impl

import (
	"net/http"
	"regexp"
	"strings"

	"zettelstore.de/z/auth"
	"zettelstore.de/z/web/server"
)

type (
	methodHandler map[string]http.Handler
	routingTable  map[byte]methodHandler
)

// httpRouter handles all routing for zettelstore.
type httpRouter struct {
	urlPrefix   string
	auth        auth.TokenManager
	minKey      byte
	maxKey      byte
	reURL       *regexp.Regexp
	listTable   routingTable
	zettelTable routingTable
	ur          server.UserRetriever
	mux         *http.ServeMux
}

// initializeRouter creates a new, empty router with the given root handler.
func (rt *httpRouter) initializeRouter(urlPrefix string, auth auth.TokenManager) {
	rt.urlPrefix = urlPrefix
	rt.auth = auth
	rt.minKey = 255
	rt.maxKey = 0
	rt.reURL = regexp.MustCompile("^$")
	rt.mux = http.NewServeMux()
	rt.listTable = make(routingTable)
	rt.zettelTable = make(routingTable)
}

func (rt *httpRouter) addRoute(key byte, httpMethod string, handler http.Handler, table routingTable) {
	// Set minKey and maxKey; re-calculate regexp.
	if key < rt.minKey || rt.maxKey < key {
		if key < rt.minKey {
			rt.minKey = key
		}
		if rt.maxKey < key {
			rt.maxKey = key
		}
		rt.reURL = regexp.MustCompile(
			"^/(?:([" + string(rt.minKey) + "-" + string(rt.maxKey) + "])(?:/(?:([0-9]{14})/?)?)?)$")
	}

	mh, hasKey := table[key]
	if !hasKey {
		mh = make(methodHandler)
		table[key] = mh
	}
	mh[httpMethod] = handler
	if httpMethod == http.MethodGet {
		if _, hasHead := table[key][http.MethodHead]; !hasHead {
			table[key][http.MethodHead] = handler
		}
	}
}

// addListRoute adds a route for the given key and HTTP method to work with a list.
func (rt *httpRouter) addListRoute(key byte, httpMethod string, handler http.Handler) {
	rt.addRoute(key, httpMethod, handler, rt.listTable)
}

// addZettelRoute adds a route for the given key and HTTP method to work with a zettel.
func (rt *httpRouter) addZettelRoute(key byte, httpMethod string, handler http.Handler) {
	rt.addRoute(key, httpMethod, handler, rt.zettelTable)
}

// Handle registers the handler for the given pattern. If a handler already exists for pattern, Handle panics.
func (rt *httpRouter) Handle(pattern string, handler http.Handler) {
	rt.mux.Handle(pattern, handler)
}

func (rt *httpRouter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	if prefixLen := len(rt.urlPrefix); prefixLen > 1 {
		if len(r.URL.Path) < prefixLen || r.URL.Path[:prefixLen] != rt.urlPrefix {
			http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
			return
		}
		r.URL.Path = r.URL.Path[prefixLen-1:]
	}
	match := rt.reURL.FindStringSubmatch(r.URL.Path)
	if len(match) == 3 {
		key := match[1][0]
		table := rt.zettelTable
		if match[2] == "" {
			table = rt.listTable
		}
		if mh, ok := table[key]; ok {
			if handler, ok := mh[r.Method]; ok {
				r.URL.Path = "/" + match[2]
				handler.ServeHTTP(w, rt.addUserContext(r))
				return
			}
			http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed)
			return
		}
	}
	rt.mux.ServeHTTP(w, rt.addUserContext(r))
}

func (rt *httpRouter) addUserContext(r *http.Request) *http.Request {
	if rt.ur == nil {
		return r
	}
	k := auth.KindJSON
	t := getHeaderToken(r)
	if len(t) == 0 {
		k = auth.KindHTML
		t = getSessionToken(r)
	}
	if len(t) == 0 {
		return r
	}
	tokenData, err := rt.auth.CheckToken(t, k)
	if err != nil {
		return r
	}
	ctx := r.Context()
	user, err := rt.ur.GetUser(ctx, tokenData.Zid, tokenData.Ident)
	if err != nil {
		return r
	}
	return r.WithContext(updateContext(ctx, user, &tokenData))
}

func getSessionToken(r *http.Request) []byte {
	cookie, err := r.Cookie(sessionName)
	if err != nil {
		return nil
	}
	return []byte(cookie.Value)
}

func getHeaderToken(r *http.Request) []byte {
	h := r.Header["Authorization"]
	if h == nil {
		return nil
	}

	// “Multiple message-header fields with the same field-name MAY be
	// present in a message if and only if the entire field-value for that
	// header field is defined as a comma-separated list.”
	// — “Hypertext Transfer Protocol” RFC 2616, subsection 4.2
	auth := strings.Join(h, ", ")

	const prefix = "Bearer "
	// RFC 2617, subsection 1.2 defines the scheme token as case-insensitive.
	if len(auth) < len(prefix) || !strings.EqualFold(auth[:len(prefix)], prefix) {
		return nil
	}
	return []byte(auth[len(prefix):])
}

Added web/server/impl/urlbuilder.go.















































































































1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
//-----------------------------------------------------------------------------
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package impl provides the Zettelstore web service.
package impl

import (
	"net/url"
	"strings"

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/web/server"
)

type urlQuery struct{ key, val string }

// URLBuilder should be used to create zettelstore URLs.
type URLBuilder struct {
	router   *httpRouter
	key      byte
	path     []string
	query    []urlQuery
	fragment string
}

// Clone an URLBuilder
func (ub *URLBuilder) Clone() server.URLBuilder {
	cpy := new(URLBuilder)
	cpy.key = ub.key
	if len(ub.path) > 0 {
		cpy.path = make([]string, 0, len(ub.path))
		cpy.path = append(cpy.path, ub.path...)
	}
	if len(ub.query) > 0 {
		cpy.query = make([]urlQuery, 0, len(ub.query))
		cpy.query = append(cpy.query, ub.query...)
	}
	cpy.fragment = ub.fragment
	return cpy
}

// SetZid sets the zettel identifier.
func (ub *URLBuilder) SetZid(zid id.Zid) server.URLBuilder {
	if len(ub.path) > 0 {
		panic("Cannot add Zid")
	}
	ub.path = append(ub.path, zid.String())
	return ub
}

// AppendPath adds a new path element
func (ub *URLBuilder) AppendPath(p string) server.URLBuilder {
	ub.path = append(ub.path, p)
	return ub
}

// AppendQuery adds a new query parameter
func (ub *URLBuilder) AppendQuery(key, value string) server.URLBuilder {
	ub.query = append(ub.query, urlQuery{key, value})
	return ub
}

// ClearQuery removes all query parameters.
func (ub *URLBuilder) ClearQuery() server.URLBuilder {
	ub.query = nil
	ub.fragment = ""
	return ub
}

// SetFragment stores the fragment
func (ub *URLBuilder) SetFragment(s string) server.URLBuilder {
	ub.fragment = s
	return ub
}

// String produces a string value.
func (ub *URLBuilder) String() string {
	var sb strings.Builder

	sb.WriteString(ub.router.urlPrefix)
	if ub.key != '/' {
		sb.WriteByte(ub.key)
	}
	for _, p := range ub.path {
		sb.WriteByte('/')
		sb.WriteString(url.PathEscape(p))
	}
	if len(ub.fragment) > 0 {
		sb.WriteByte('#')
		sb.WriteString(ub.fragment)
	}
	for i, q := range ub.query {
		if i == 0 {
			sb.WriteByte('?')
		} else {
			sb.WriteByte('&')
		}
		sb.WriteString(q.key)
		sb.WriteByte('=')
		sb.WriteString(url.QueryEscape(q.val))
	}
	return sb.String()
}

Changes to web/server/server.go.

1
2

3
4
5
6
7
8
9
10
11

12
13
14
15
16
17
18
19
20
21
22
23
24
25
26









27
28
29
30



31
32
33
34
35
36








37


38
39



40
41
42
43
44




45
46
47


48
49
50
51
52
53
54
55







56
57
58
59
60
61




62
63
64
65
66
67
68
69
70





71


72
73
74
75


76
77
78


79
80
81
82
83
84
85








86
87
88


89
90
91



92
93






94
95
96



97
98
1

2
3
4
5
6
7
8
9
10

11
12
13
14
15

16



17





18
19
20
21
22
23
24
25
26




27
28
29
30





31
32
33
34
35
36
37
38
39
40
41


42
43
44





45
46
47
48



49
50








51
52
53
54
55
56
57
58
59




60
61
62
63



64
65




66
67
68
69
70

71
72
73



74
75



76
77







78
79
80
81
82
83
84
85


86
87
88



89
90
91


92
93
94
95
96
97
98


99
100
101

102

-
+








-
+




-

-
-
-

-
-
-
-
-
+
+
+
+
+
+
+
+
+
-
-
-
-
+
+
+

-
-
-
-
-
+
+
+
+
+
+
+
+

+
+
-
-
+
+
+
-
-
-
-
-
+
+
+
+
-
-
-
+
+
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+


-
-
-
-
+
+
+
+
-
-
-


-
-
-
-
+
+
+
+
+
-
+
+

-
-
-
+
+
-
-
-
+
+
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
-
-

+
+
-
-
-
+
+
+
-
-
+
+
+
+
+
+

-
-
+
+
+
-

//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
// Copyright (c) 2020-2021 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package server provides a web server.
// Package server provides the Zettelstore web service.
package server

import (
	"context"
	"log"
	"net/http"
	"os"
	"os/signal"
	"syscall"
	"time"
)

// Server timeout values
const (
	shutdownTimeout = 5 * time.Second

	"zettelstore.de/z/domain/id"
	"zettelstore.de/z/domain/meta"
)

// URLBuilder builds URLs.
type URLBuilder interface {
	// Clone an URLBuilder
	Clone() URLBuilder
	readTimeout     = 5 * time.Second
	writeTimeout    = 10 * time.Second
	idleTimeout     = 120 * time.Second
)

	// SetZid sets the zettel identifier.
	SetZid(zid id.Zid) URLBuilder

// Server is a HTTP server.
type Server struct {
	*http.Server
	waitShutdown chan struct{}
}
	// AppendPath adds a new path element
	AppendPath(p string) URLBuilder

	// AppendQuery adds a new query parameter
	AppendQuery(key, value string) URLBuilder

	// ClearQuery removes all query parameters.
	ClearQuery() URLBuilder

	// SetFragment stores the fragment
	SetFragment(s string) URLBuilder
// New creates a new HTTP server object.
func New(addr string, handler http.Handler) *Server {

	// String produces a string value.
	String() string
	if addr == "" {
		addr = ":http"
	}
	srv := &Server{
		Server: &http.Server{
}

// UserRetriever allows to retrieve user data based on a given zettel identifier.
type UserRetriever interface {
			Addr:    addr,
			Handler: handler,

	GetUser(ctx context.Context, zid id.Zid, ident string) (*meta.Meta, error)
}
			// See: https://blog.cloudflare.com/exposing-go-on-the-internet/
			ReadTimeout:  readTimeout,
			WriteTimeout: writeTimeout,
			IdleTimeout:  idleTimeout,
		},
		waitShutdown: make(chan struct{}),
	}
	return srv

// Router allows to state routes for various URL paths.
type Router interface {
	Handle(pattern string, handler http.Handler)
	AddListRoute(key byte, httpMethod string, handler http.Handler)
	AddZettelRoute(key byte, httpMethod string, handler http.Handler)
	SetUserRetriever(ur UserRetriever)
}

// SetDebug enables debugging goroutines that are started by the server.
// Basically, just the timeout values are reset. This method should be called
// before running the server.
func (srv *Server) SetDebug() {
// Builder allows to build new URLs for the web service.
type Builder interface {
	GetURLPrefix() string
	NewURLBuilder(key byte) URLBuilder
	srv.ReadTimeout = 0
	srv.WriteTimeout = 0
	srv.IdleTimeout = 0
}

// Run starts the web server and wait for its completion.
func (srv *Server) Run() error {
	waitInterrupt := make(chan os.Signal)
	waitError := make(chan error)
// Auth is.
type Auth interface {
	GetUser(context.Context) *meta.Meta
	SetToken(w http.ResponseWriter, token []byte, d time.Duration)

	signal.Notify(waitInterrupt, os.Interrupt, syscall.SIGTERM)
	// ClearToken invalidates the session cookie by sending an empty one.
	ClearToken(ctx context.Context, w http.ResponseWriter) context.Context

	go func() {
		select {
		case <-waitInterrupt:
	// GetAuthData returns the full authentication data from the context.
	GetAuthData(ctx context.Context) *AuthData
		case <-srv.waitShutdown:
		}
		ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
}

		defer cancel()

		log.Println("Stopping Zettelstore...")
		if err := srv.Shutdown(ctx); err != nil {
			waitError <- err
			return
		}
// AuthData stores all relevant authentication data for a context.
type AuthData struct {
	User    *meta.Meta
	Token   []byte
	Now     time.Time
	Issued  time.Time
	Expires time.Time
}
		waitError <- nil
	}()

// AuthBuilder is a Builder that also allows to execute authentication functions.
type AuthBuilder interface {
	if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		return err
	}
	Auth
	Builder
}
	return <-waitError
}

// Server is the main web server for accessing Zettelstore via HTTP.
type Server interface {
	Router
	Auth
	Builder

// Stop the web server.
func (srv *Server) Stop() {
	SetDebug()
	Run() error
	Stop() error
	close(srv.waitShutdown)
}

Deleted web/session/session.go.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166






































































































































































-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
//-----------------------------------------------------------------------------
// Copyright (c) 2020 Detlef Stern
//
// This file is part of zettelstore.
//
// Zettelstore is licensed under the latest version of the EUPL (European Union
// Public License). Please see file LICENSE.txt for your rights and obligations
// under this license.
//-----------------------------------------------------------------------------

// Package session provides utilities for using sessions.
package session

import (
	"context"
	"net/http"
	"strings"
	"time"

	"zettelstore.de/z/auth/token"
	"zettelstore.de/z/config/startup"
	"zettelstore.de/z/domain/meta"
	"zettelstore.de/z/usecase"
)

const sessionName = "zsession"

// SetToken sets the session cookie for later user identification.
func SetToken(w http.ResponseWriter, token []byte, d time.Duration) {
	cookie := http.Cookie{
		Name:     sessionName,
		Value:    string(token),
		Path:     startup.URLPrefix(),
		Secure:   startup.SecureCookie(),
		HttpOnly: true,
		SameSite: http.SameSiteStrictMode,
	}
	if startup.PersistentCookie() && d > 0 {
		cookie.Expires = time.Now().Add(d).Add(30 * time.Second).UTC()
	}
	http.SetCookie(w, &cookie)
}

// ClearToken invalidates the session cookie by sending an empty one.
func ClearToken(ctx context.Context, w http.ResponseWriter) context.Context {
	if w != nil {
		SetToken(w, nil, 0)
	}
	return updateContext(ctx, nil, nil)
}

// Handler enriches the request context with optional user information.
type Handler struct {
	next         http.Handler
	getUserByZid usecase.GetUserByZid
}

// NewHandler creates a new handler.
func NewHandler(next http.Handler, getUserByZid usecase.GetUserByZid) *Handler {
	return &Handler{
		next:         next,
		getUserByZid: getUserByZid,
	}
}

type ctxKeyType struct{}

var ctxKey ctxKeyType

// AuthData stores all relevant authentication data for a context.
type AuthData struct {
	User    *meta.Meta
	Token   []byte
	Now     time.Time
	Issued  time.Time
	Expires time.Time
}

// GetAuthData returns the full authentication data from the context.
func GetAuthData(ctx context.Context) *AuthData {
	data, ok := ctx.Value(ctxKey).(*AuthData)
	if ok {
		return data
	}
	return nil

}

// GetUser returns the user meta data from the context, if there is one. Else return nil.
func GetUser(ctx context.Context) *meta.Meta {
	if data := GetAuthData(ctx); data != nil {
		return data.User
	}
	return nil
}

func updateContext(
	ctx context.Context, user *meta.Meta, data *token.Data) context.Context {
	if data == nil {
		return context.WithValue(ctx, ctxKey, &AuthData{User: user})
	}
	return context.WithValue(
		ctx,
		ctxKey,
		&AuthData{
			User:    user,
			Token:   data.Token,
			Now:     data.Now,
			Issued:  data.Issued,
			Expires: data.Expires,
		})
}

// ServeHTTP processes one HTTP request.
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	k := token.KindJSON
	t := getHeaderToken(r)
	if t == nil {
		k = token.KindHTML
		t = getSessionToken(r)
	}
	if t == nil {
		h.next.ServeHTTP(w, r)
		return
	}
	tokenData, err := token.CheckToken(t, k)
	if err != nil {
		h.next.ServeHTTP(w, r)
		return
	}
	ctx := r.Context()
	user, err := h.getUserByZid.Run(ctx, tokenData.Zid, tokenData.Ident)
	if err != nil {
		h.next.ServeHTTP(w, r)
		return
	}
	h.next.ServeHTTP(w, r.WithContext(updateContext(ctx, user, &tokenData)))
}

func getSessionToken(r *http.Request) []byte {
	cookie, err := r.Cookie(sessionName)
	if err != nil {
		return nil
	}
	return []byte(cookie.Value)
}

func getHeaderToken(r *http.Request) []byte {
	h := r.Header["Authorization"]
	if h == nil {
		return nil
	}

	// “Multiple message-header fields with the same field-name MAY be
	// present in a message if and only if the entire field-value for that
	// header field is defined as a comma-separated list.”
	// — “Hypertext Transfer Protocol” RFC 2616, subsection 4.2
	auth := strings.Join(h, ", ")

	const prefix = "Bearer "
	// RFC 2617, subsection 1.2 defines the scheme token as case-insensitive.
	if len(auth) < len(prefix) || !strings.EqualFold(auth[:len(prefix)], prefix) {
		return nil
	}
	return []byte(auth[len(prefix):])
}

Changes to www/changes.wiki.

1
2



3
4











































5
6
7
8
9
10
11
1
2
3
4
5
6

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56


+
+
+

-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+







<title>Change Log</title>

<a name="0_0_14"></a>
<h2>Changes for Version 0.0.14 (pending)</h2>

<a name="0_0_13"></a>
<h2>Changes for Version 0.0.13 (pending)</h2>
<h2>Changes for Version 0.0.13 (2021-06-01)</h2>
  *  Startup configuration <tt>place-<em>X</em>-uri</tt> (where <em>X</em> is a
     number greater than zero) has been renamed to
     <tt>place-uri-<em>X</em></tt>.
     (breaking)
  *  Web server processes startup configuration <tt>url-prefix</tt>. There is
     no need for stripping the prefix by a front-end web server any more.
     (breaking: webui, api)
  *  Administrator console (only optional accessible locally). Enable it only
     on systems with a single user or with trusted users. It is disabled by
     default.
     (major: core)
  *  Remove visibility value &ldquo;simple-expert&rdquo; introduced in
     [#0_0_8|version 0.0.8]. It was too complicated, esp. authorization. There
     was a name collision with the &ldquo;simple&rdquo; directory place
     sub-type.
     (major)
  *  For security reasons, HTML blocks are not encoded as HTML if they contain
     certain snippets, such as <tt>&lt;script</tt> or <tt>&lt;iframe</tt>.
     These may be caused by using CommonMark as a zettel syntax.
     (major)
  *  Full-text search can be a prefix search or a search for equal words, in
     addition to the search whether a word just contains word of the search
     term.
     (minor: api, webui)
  *  Full-text search for URLs, with above additional operators.
     (minor: api, webui)
  *  Add system zettel about license, contributors, and dependencies (and their
     license).
     For a nicer layout of zettel identifier, the zettel about environment
     values and about runtime metrics got new zettel identifier. This affects
     only user that referenced those zettel.
     (minor)
  *  Local images that cannot be read (not found or no access rights) are
     substituted with the new default image, a spinning emoji.
     See [/file?name=place/constplace/emoji_spin.gif].
     (minor: webui)
  *  Add zettelmarkup syntax for a table row that should be ignored:
     <tt>|%</tt>. This allows to paste output of the administrator console into
     a zettel.
     (minor: zmk)
  *  Many smaller bug fixes and inprovements, to the software and to the
     documentation.

<a name="0_0_12"></a>
<h2>Changes for Version 0.0.12 (2021-04-16)</h2>
  *  Raise the per-process limit of open files on macOS to 1.048.576. This
     allows most macOS users to use at least 500.000 zettel. That should be
     enough for the near future.
     (major)

Changes to www/download.wiki.

1
2
3
4
5
6
7
8
9
10
11
12

13
14
15
16
17
18





19
20
21
22
23

24
25
1
2
3
4
5
6
7
8
9
10
11

12
13





14
15
16
17
18
19
20
21
22

23
24
25











-
+

-
-
-
-
-
+
+
+
+
+




-
+


<title>Download</title>
<h1>Download of Zettelstore Software</h1>
<h2>Foreword</h2>
  *  Zettelstore is free/libre open source software, licensed under EUPL-1.2-or-later.
  *  The software is provided as-is.
  *  There is no guarantee that it will not damage your system.
  *  However, it is in use by the main developer since March 2020 without any damage.
  *  It may be useful for you. It is useful for me.
  *  Take a look at the [https://zettelstore.de/manual/|manual] to know how to start and use it.

<h2>ZIP-ped Executables</h2>
Build: <code>v0.0.12</code> (2021-04-16).
Build: <code>v0.0.13</code> (2021-06-01).

  *  [/uv/zettelstore-0.0.12-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.0.12-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.0.12-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.0.12-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.0.12-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)
  *  [/uv/zettelstore-0.0.13-linux-amd64.zip|Linux] (amd64)
  *  [/uv/zettelstore-0.0.13-linux-arm.zip|Linux] (arm6, e.g. Raspberry Pi)
  *  [/uv/zettelstore-0.0.13-windows-amd64.zip|Windows] (amd64)
  *  [/uv/zettelstore-0.0.13-darwin-amd64.zip|macOS] (amd64)
  *  [/uv/zettelstore-0.0.13-darwin-arm64.zip|macOS] (arm64, aka Apple silicon)

Unzip the appropriate file, install and execute Zettelstore according to the manual.

<h2>Zettel for the manual</h2>
As a starter, you can download the zettel for the manual [/uv/manual-0.0.11.zip|here].
As a starter, you can download the zettel for the manual [/uv/manual-0.0.13.zip|here].
Just unzip the contained files and put them into your zettel folder or configure
a file place to read the zettel directly from the ZIP file.

Changes to www/index.wiki.

13
14
15
16
17
18
19
20

21
22
23
24
25
26
27






28
29
30
31
32
33
34


35
36
37


38
13
14
15
16
17
18
19

20
21






22
23
24
25
26
27
28
29
30
31



32
33
34


35
36
37







-
+

-
-
-
-
-
-
+
+
+
+
+
+




-
-
-
+
+

-
-
+
+

It is a live example of the zettelstore software, running in read-only mode.

The software, including the manual, is licensed under the
[/file?name=LICENSE.txt&ci=trunk|European Union Public License 1.2 (or later)].

[https://twitter.com/zettelstore|Stay tuned]&hellip;
<hr>
<h3>Latest Release: 0.0.12 (2021-04-16)</h3>
<h3>Latest Release: 0.0.13 (2021-06-01)</h3>
  *  [./download.wiki|Download]
  *  [./changes.wiki#0_0_12|Change Summary]
  *  [/timeline?p=version-0.0.12&bt=version-0.0.11&y=ci|Check-ins for version 0.0.12],
     [/vdiff?to=version-0.0.12&from=version-0.0.11|content diff]
  *  [/timeline?df=version-0.0.12&y=ci|Check-ins derived from the 0.0.12 release],
     [/vdiff?from=version-0.0.12&to=trunk|content diff]
  *  [./plan.wiki|Limitations and planned Improvements]
  *  [./changes.wiki#0_0_13|Change summary]
  *  [/timeline?p=version-0.0.13&bt=version-0.0.12&y=ci|Check-ins for version 0.0.13],
     [/vdiff?to=version-0.0.13&from=version-0.0.12|content diff]
  *  [/timeline?df=version-0.0.13&y=ci|Check-ins derived from the 0.0.13 release],
     [/vdiff?from=version-0.0.13&to=trunk|content diff]
  *  [./plan.wiki|Limitations and planned improvements]
  *  [/timeline?t=release|Timeline of all past releases]

<hr>
<h2>Build instructions</h2>
Just install [https://fossil-scm.org|Fossil],
[https://golang.org/dl/|Go] and some Go-based tools. Please read the
[./build.md|instructions] for details.
Just install [https://golang.org/dl/|Go] and some Go-based tools. Please read
the [./build.md|instructions] for details.

  *  [/dir?ci=trunk|Source Code]
  *  [/download|Download the source code] as a Tarball or a ZIP file
  *  [/dir?ci=trunk|Source code]
  *  [/download|Download the source code] as a tarball or a ZIP file
     (you must [/login|login] as user &quot;anonymous&quot;).

Changes to www/plan.wiki.

1

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21





22
23
24
25
26
27
28
29

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
-
+




















+
+
+
+
+








<title>Limitations and planned Improvements</title>
<title>Limitations and planned improvements</title>

Here is a list of some shortcomings of Zettelstore.
They are planned to be solved.

<h3>Serious limitations</h3>
  *  Content with binary data (e.g. a GIF, PNG, or JPG file) cannot be created
     nor modified via the standard web interface. As a workaround, you should
     place your file into the directory where your zettel are stored. Make sure
     that the file name starts with unique 14 digits that make up the zettel
     identifier.
  *  Automatic lists and transclusions are not supported in Zettelmarkup.
  *  &hellip;

<h3>Smaller limitations</h3>
  *  Quoted attribute values are not yet supported in Zettelmarkup:
     <code>{key="value with space"}</code>.
  *  The <tt>file</tt> sub-command currently does not support output format
     &ldquo;json&rdquo;.
  *  The horizontal tab character (<tt>U+0009</tt>) is not supported.
  *  Missing support for citation keys.
  *  Changing the content syntax is not reflected in file extension.
  *  File names with additional text besides the zettel identifier are not
     always preserved.
  *  Backspace character in links does not always work, esp. for <tt>\|</tt> or
     <tt>\]</tt>.
  *  &hellip;

<h3>Planned improvements</h3>
  *  Support for mathematical content is missing, e.g. <code>$$F(x) &=
     \\int^a_b \\frac{1}{3}x^3$$</code>.
  *  Render zettel in [https://pandoc.org|pandoc's] JSON version of
     their native AST to make pandoc an external renderer for Zettelstore.
  *  &hellip;