Compare commits

..

19 Commits

Author SHA1 Message Date
Tom Moor fa7f8d3592 fix: ExportDocumentTreeTask needs documentStructure 2025-05-07 19:38:30 -04:00
codegen-sh[bot] c598c61afe Add PromQL as a code highlighting option in the editor (#9146)
* Add PromQL as a code highlighting option in the editor

* Update code.ts

---------

Co-authored-by: codegen-sh[bot] <131295404+codegen-sh[bot]@users.noreply.github.com>
Co-authored-by: Tom Moor <tom@getoutline.com>
2025-05-07 19:11:47 -04:00
Tom Moor 68b07eb466 fix: withoutState scope should include state as fallback (#9145) 2025-05-07 18:49:33 -04:00
Tom Moor 06a149407a fix: withoutState scope should include state as fallback (#9144) 2025-05-07 09:00:42 -04:00
Tom Moor b9387734c7 perf: Remove documentStructure from default query select (#9141)
* perf: Remove documentStructure from default query select

* test
2025-05-07 07:47:57 -04:00
dependabot[bot] 810b7908e4 chore(deps): bump the aws group with 5 updates (#9136)
Bumps the aws group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [@aws-sdk/client-s3](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-s3) | `3.797.0` | `3.802.0` |
| [@aws-sdk/lib-storage](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/lib/lib-storage) | `3.797.0` | `3.802.0` |
| [@aws-sdk/s3-presigned-post](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/packages/s3-presigned-post) | `3.797.0` | `3.802.0` |
| [@aws-sdk/s3-request-presigner](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/packages/s3-request-presigner) | `3.797.0` | `3.802.0` |
| [@aws-sdk/signature-v4-crt](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/packages/signature-v4-crt) | `3.796.0` | `3.800.0` |


Updates `@aws-sdk/client-s3` from 3.797.0 to 3.802.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-s3/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.802.0/clients/client-s3)

Updates `@aws-sdk/lib-storage` from 3.797.0 to 3.802.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-storage/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.802.0/lib/lib-storage)

Updates `@aws-sdk/s3-presigned-post` from 3.797.0 to 3.802.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/packages/s3-presigned-post/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.802.0/packages/s3-presigned-post)

Updates `@aws-sdk/s3-request-presigner` from 3.797.0 to 3.802.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/packages/s3-request-presigner/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.802.0/packages/s3-request-presigner)

Updates `@aws-sdk/signature-v4-crt` from 3.796.0 to 3.800.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/packages/signature-v4-crt/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.800.0/packages/signature-v4-crt)

---
updated-dependencies:
- dependency-name: "@aws-sdk/client-s3"
  dependency-version: 3.802.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: aws
- dependency-name: "@aws-sdk/lib-storage"
  dependency-version: 3.802.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: aws
- dependency-name: "@aws-sdk/s3-presigned-post"
  dependency-version: 3.802.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: aws
- dependency-name: "@aws-sdk/s3-request-presigner"
  dependency-version: 3.802.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: aws
- dependency-name: "@aws-sdk/signature-v4-crt"
  dependency-version: 3.800.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: aws
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-06 08:02:14 -04:00
dependabot[bot] 6b76a898fa chore(deps): bump the babel group with 9 updates (#9139)
Bumps the babel group with 9 updates:

| Package | From | To |
| --- | --- | --- |
| [@babel/core](https://github.com/babel/babel/tree/HEAD/packages/babel-core) | `7.26.10` | `7.27.1` |
| [@babel/plugin-proposal-decorators](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-proposal-decorators) | `7.25.9` | `7.27.1` |
| [@babel/plugin-transform-class-properties](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-transform-class-properties) | `7.25.9` | `7.27.1` |
| [@babel/plugin-transform-destructuring](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-transform-destructuring) | `7.25.9` | `7.27.1` |
| [@babel/plugin-transform-regenerator](https://github.com/babel/babel/tree/HEAD/packages/babel-plugin-transform-regenerator) | `7.27.0` | `7.27.1` |
| [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env) | `7.26.9` | `7.27.1` |
| [@babel/preset-react](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-react) | `7.26.3` | `7.27.1` |
| [@babel/cli](https://github.com/babel/babel/tree/HEAD/packages/babel-cli) | `7.27.0` | `7.27.1` |
| [@babel/preset-typescript](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-typescript) | `7.27.0` | `7.27.1` |


Updates `@babel/core` from 7.26.10 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-core)

Updates `@babel/plugin-proposal-decorators` from 7.25.9 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-plugin-proposal-decorators)

Updates `@babel/plugin-transform-class-properties` from 7.25.9 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-plugin-transform-class-properties)

Updates `@babel/plugin-transform-destructuring` from 7.25.9 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-plugin-transform-destructuring)

Updates `@babel/plugin-transform-regenerator` from 7.27.0 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-plugin-transform-regenerator)

Updates `@babel/preset-env` from 7.26.9 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-preset-env)

Updates `@babel/preset-react` from 7.26.3 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-preset-react)

Updates `@babel/cli` from 7.27.0 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-cli)

Updates `@babel/preset-typescript` from 7.27.0 to 7.27.1
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.27.1/packages/babel-preset-typescript)

---
updated-dependencies:
- dependency-name: "@babel/core"
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: babel
- dependency-name: "@babel/plugin-proposal-decorators"
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: babel
- dependency-name: "@babel/plugin-transform-class-properties"
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: babel
- dependency-name: "@babel/plugin-transform-destructuring"
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: babel
- dependency-name: "@babel/plugin-transform-regenerator"
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: babel
- dependency-name: "@babel/preset-env"
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: babel
- dependency-name: "@babel/preset-react"
  dependency-version: 7.27.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: babel
- dependency-name: "@babel/cli"
  dependency-version: 7.27.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: babel
- dependency-name: "@babel/preset-typescript"
  dependency-version: 7.27.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: babel
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 21:05:41 -04:00
dependabot[bot] 8ba83e2173 chore(deps): bump react-medium-image-zoom from 5.2.13 to 5.2.14 (#9137)
Bumps [react-medium-image-zoom](https://github.com/rpearce/react-medium-image-zoom) from 5.2.13 to 5.2.14.
- [Release notes](https://github.com/rpearce/react-medium-image-zoom/releases)
- [Changelog](https://github.com/rpearce/react-medium-image-zoom/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rpearce/react-medium-image-zoom/compare/v5.2.13...v5.2.14)

---
updated-dependencies:
- dependency-name: react-medium-image-zoom
  dependency-version: 5.2.14
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 21:05:27 -04:00
dependabot[bot] 5a4b8c5faa chore(deps): bump validator and @types/validator (#9138)
Bumps [validator](https://github.com/validatorjs/validator.js) and [@types/validator](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/validator). These dependencies needed to be updated together.

Updates `validator` from 13.12.0 to 13.15.0
- [Release notes](https://github.com/validatorjs/validator.js/releases)
- [Changelog](https://github.com/validatorjs/validator.js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/validatorjs/validator.js/compare/13.12.0...13.15.0)

Updates `@types/validator` from 13.12.1 to 13.15.0
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/validator)

---
updated-dependencies:
- dependency-name: validator
  dependency-version: 13.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: "@types/validator"
  dependency-version: 13.15.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-05 21:05:14 -04:00
Tom Moor 3f8bdf7ac2 Refactor withMembershipScope (#9134) 2025-05-04 18:37:01 -04:00
Tom Moor 9c4b4f4989 fix: Chained scopes overwrite (#9133) 2025-05-04 22:16:38 +00:00
Hemachandar c5d534b2ad Add script to resolve existing collection index collisions (#8810)
* Add script to resolve existing collection index collisions

* Remove debug logging

---------

Co-authored-by: Tom Moor <tom.moor@gmail.com>
2025-05-04 16:12:09 -04:00
Tom Moor bed3d1078e fix: More guards against empty text nodes (#9132) 2025-05-04 20:11:02 +00:00
Tom Moor 83e87254c6 fix: Invisible JS error (#9131) 2025-05-04 13:59:21 +00:00
Tom Moor f576ddccfe chore: Add 10 more brand icons (#9130) 2025-05-04 13:18:37 +00:00
Tom Moor 0a674eacfa fix: Improve behavior when hitting backspace/delete with table cell selections (#9129) 2025-05-04 08:51:48 -04:00
Tom Moor ceac57bd64 Pick collection color based on existing collections (#9128) 2025-05-04 03:09:08 +00:00
Tom Moor 97f31e3f2a fix: Cannot create document through @mention on collection overview (#9127) 2025-05-03 22:13:54 -04:00
Tom Moor a06671e8ce OAuth provider (#8884)
This PR contains the necessary work to make Outline an OAuth provider including:

- OAuth app registration
- OAuth app management
- Private / public apps (Public in cloud only)
- Full OAuth 2.0 spec compatible authentication flow
- Granular scopes
- User token management screen in settings
- Associated API endpoints for programatic access
2025-05-03 19:40:18 -04:00
33 changed files with 1462 additions and 1090 deletions
+23 -5
View File
@@ -1,3 +1,4 @@
import uniq from "lodash/uniq";
import { observer } from "mobx-react";
import * as React from "react";
import { Controller, useForm } from "react-hook-form";
@@ -18,6 +19,7 @@ import Switch from "~/components/Switch";
import Text from "~/components/Text";
import useBoolean from "~/hooks/useBoolean";
import useCurrentTeam from "~/hooks/useCurrentTeam";
import useStores from "~/hooks/useStores";
import { EmptySelectValue } from "~/types";
const IconPicker = React.lazy(() => import("~/components/IconPicker"));
@@ -30,6 +32,26 @@ export interface FormData {
permission: CollectionPermission | undefined;
}
const useIconColor = (collection?: Collection) => {
const { collections } = useStores();
const hasMultipleCollections = collections.orderedData.length > 1;
const collectionColors = uniq(
collections.orderedData.map((c) => c.color).filter(Boolean)
) as string[];
const iconColor = React.useMemo(
() =>
collection?.color ??
// If all the existing collections have the same color, use that color,
// otherwise pick a random color from the palette
(hasMultipleCollections && collectionColors.length === 1
? collectionColors[0]
: randomElement(colorPalette)),
[collection?.color]
);
return iconColor;
};
export const CollectionForm = observer(function CollectionForm_({
handleSubmit,
collection,
@@ -42,11 +64,7 @@ export const CollectionForm = observer(function CollectionForm_({
const [hasOpenedIconPicker, setHasOpenedIconPicker] = useBoolean(false);
const iconColor = React.useMemo(
() => collection?.color ?? randomElement(colorPalette),
[collection?.color]
);
const iconColor = useIconColor(collection);
const fallbackIcon = <Icon value="collection" color={iconColor} />;
const {
@@ -6,15 +6,18 @@ import { toast } from "sonner";
import styled from "styled-components";
import { richExtensions } from "@shared/editor/nodes";
import { s } from "@shared/styles";
import { ProsemirrorHelper } from "@shared/utils/ProsemirrorHelper";
import { CollectionValidation } from "@shared/validations";
import Collection from "~/models/Collection";
import Document from "~/models/Document";
import Editor from "~/components/Editor";
import LoadingIndicator from "~/components/LoadingIndicator";
import Text from "~/components/Text";
import { withUIExtensions } from "~/editor/extensions";
import useCurrentUser from "~/hooks/useCurrentUser";
import usePolicy from "~/hooks/usePolicy";
import useStores from "~/hooks/useStores";
import Text from "./Text";
import { Properties } from "~/types";
const extensions = withUIExtensions(richExtensions);
@@ -22,8 +25,8 @@ type Props = {
collection: Collection;
};
function CollectionDescription({ collection }: Props) {
const { collections } = useStores();
function Overview({ collection }: Props) {
const { documents, collections } = useStores();
const { t } = useTranslation();
const user = useCurrentUser({ rejectOnEmpty: true });
const can = usePolicy(collection);
@@ -54,6 +57,24 @@ function CollectionDescription({ collection }: Props) {
[childOffsetHeight]
);
const onCreateLink = React.useCallback(
async (params: Properties<Document>) => {
const newDocument = await documents.create(
{
collectionId: collection.id,
data: ProsemirrorHelper.getEmptyDocument(),
...params,
},
{
publish: true,
}
);
return newDocument.url;
},
[collection, documents]
);
return (
<>
{collections.isSaving && <LoadingIndicator />}
@@ -65,6 +86,7 @@ function CollectionDescription({ collection }: Props) {
placeholder={`${t("Add a description")}`}
extensions={extensions}
maxLength={CollectionValidation.maxDescriptionLength}
onCreateLink={onCreateLink}
canUpdate={can.update}
readOnly={!can.update}
userId={user.id}
@@ -83,4 +105,4 @@ const Placeholder = styled(Text)`
min-height: 27px;
`;
export default observer(CollectionDescription);
export default observer(Overview);
+2 -2
View File
@@ -20,7 +20,6 @@ import Collection from "~/models/Collection";
import { Action } from "~/components/Actions";
import CenteredContent from "~/components/CenteredContent";
import { CollectionBreadcrumb } from "~/components/CollectionBreadcrumb";
import CollectionDescription from "~/components/CollectionDescription";
import Heading from "~/components/Heading";
import CollectionIcon from "~/components/Icons/CollectionIcon";
import InputSearchPage from "~/components/InputSearchPage";
@@ -46,6 +45,7 @@ import DropToImport from "./components/DropToImport";
import Empty from "./components/Empty";
import MembershipPreview from "./components/MembershipPreview";
import Notices from "./components/Notices";
import Overview from "./components/Overview";
import ShareButton from "./components/ShareButton";
const IconPicker = React.lazy(() => import("~/components/IconPicker"));
@@ -259,7 +259,7 @@ const CollectionScene = observer(function _CollectionScene() {
path={collectionPath(collection.path, CollectionPath.Overview)}
>
{hasOverview ? (
<CollectionDescription collection={collection} />
<Overview collection={collection} />
) : (
<Redirect
to={{
+18 -17
View File
@@ -48,18 +48,18 @@
"> 0.25%, not dead"
],
"dependencies": {
"@aws-sdk/client-s3": "3.797.0",
"@aws-sdk/lib-storage": "3.797.0",
"@aws-sdk/s3-presigned-post": "3.797.0",
"@aws-sdk/s3-request-presigner": "3.797.0",
"@aws-sdk/signature-v4-crt": "^3.796.0",
"@babel/core": "^7.26.10",
"@babel/plugin-proposal-decorators": "^7.25.9",
"@babel/plugin-transform-class-properties": "^7.25.9",
"@babel/plugin-transform-destructuring": "^7.25.9",
"@babel/plugin-transform-regenerator": "^7.27.0",
"@babel/preset-env": "^7.26.9",
"@babel/preset-react": "^7.26.3",
"@aws-sdk/client-s3": "3.803.0",
"@aws-sdk/lib-storage": "3.803.0",
"@aws-sdk/s3-presigned-post": "3.803.0",
"@aws-sdk/s3-request-presigner": "3.803.0",
"@aws-sdk/signature-v4-crt": "^3.803.0",
"@babel/core": "^7.27.1",
"@babel/plugin-proposal-decorators": "^7.27.1",
"@babel/plugin-transform-class-properties": "^7.27.1",
"@babel/plugin-transform-destructuring": "^7.27.1",
"@babel/plugin-transform-regenerator": "^7.27.1",
"@babel/preset-env": "^7.27.1",
"@babel/preset-react": "^7.27.1",
"@benrbray/prosemirror-math": "^0.2.2",
"@bull-board/api": "^6.7.10",
"@bull-board/koa": "^6.7.10",
@@ -208,7 +208,7 @@
"react-helmet-async": "^2.0.5",
"react-hook-form": "^7.54.2",
"react-i18next": "^12.3.1",
"react-medium-image-zoom": "5.2.13",
"react-medium-image-zoom": "5.2.14",
"react-merge-refs": "^2.1.1",
"react-portal": "^4.3.0",
"react-router-dom": "^5.3.4",
@@ -228,6 +228,7 @@
"sequelize": "^6.37.3",
"sequelize-cli": "^6.6.2",
"sequelize-encrypted": "^1.0.0",
"sequelize-strict-attributes": "^1.0.2",
"sequelize-typescript": "^2.1.6",
"slug": "^5.3.0",
"slugify": "^1.6.6",
@@ -248,7 +249,7 @@
"umzug": "^3.8.2",
"utility-types": "^3.11.0",
"uuid": "^8.3.2",
"validator": "13.12.0",
"validator": "13.15.0",
"vaul": "^1.1.2",
"vite": "^6.3.4",
"vite-plugin-pwa": "^0.21.2",
@@ -262,8 +263,8 @@
"zod": "^3.24.2"
},
"devDependencies": {
"@babel/cli": "^7.27.0",
"@babel/preset-typescript": "^7.27.0",
"@babel/cli": "^7.27.1",
"@babel/preset-typescript": "^7.27.1",
"@faker-js/faker": "^8.4.1",
"@relative-ci/agent": "^4.3.0",
"@testing-library/react": "^12.0.0",
@@ -328,7 +329,7 @@
"@types/tmp": "^0.2.6",
"@types/turndown": "^5.0.5",
"@types/utf8": "^3.0.3",
"@types/validator": "^13.12.1",
"@types/validator": "^13.15.0",
"@types/yauzl": "^2.10.3",
"@typescript-eslint/eslint-plugin": "^6.21.0",
"@typescript-eslint/parser": "^6.21.0",
+20 -20
View File
@@ -288,7 +288,7 @@ export class NotionConverter {
if (item.mention.type === "link_mention") {
return {
type: "text",
text: item.plain_text,
text: item.plain_text || item.mention.link_mention.href,
marks: [
{
type: "link",
@@ -302,7 +302,7 @@ export class NotionConverter {
if (item.mention.type === "link_preview") {
return {
type: "text",
text: item.plain_text,
text: item.plain_text || item.mention.link_preview.url,
marks: [
{
type: "link",
@@ -314,14 +314,14 @@ export class NotionConverter {
};
}
if (!item.plain_text) {
return undefined;
if (item.plain_text) {
return {
type: "text",
text: item.plain_text,
};
}
return {
type: "text",
text: item.plain_text,
};
return undefined;
}
if (item.type === "equation") {
@@ -336,20 +336,20 @@ export class NotionConverter {
};
}
if (!item.text.content) {
return undefined;
if (item.text.content) {
return {
type: "text",
text: item.text.content,
marks: [
...mapAttrs(),
...(item.text.link
? [{ type: "link", attrs: { href: item.text.link.url } }]
: []),
].filter(Boolean),
};
}
return {
type: "text",
text: item.text.content,
marks: [
...mapAttrs(),
...(item.text.link
? [{ type: "link", attrs: { href: item.text.link.url } }]
: []),
].filter(Boolean),
};
return undefined;
}
private static rich_text_to_plaintext(item: RichTextItemResponse) {
+19 -30
View File
@@ -1,4 +1,3 @@
import invariant from "invariant";
import { Op, WhereOptions } from "sequelize";
import isUUID from "validator/lib/isUUID";
import { UrlHelper } from "@shared/utils/UrlHelper";
@@ -22,8 +21,8 @@ type Props = {
type Result = {
document: Document;
share?: Share;
collection?: Collection | null;
share: Share | null;
collection: Collection | null;
};
export default async function loadDocument({
@@ -33,9 +32,9 @@ export default async function loadDocument({
user,
includeState,
}: Props): Promise<Result> {
let document;
let collection;
let share;
let document: Document | null = null;
let collection: Collection | null = null;
let share: Share | null = null;
if (!shareId && !(id && user)) {
throw AuthenticationError(`Authentication or shareId required`);
@@ -72,20 +71,7 @@ export default async function loadDocument({
where: whereClause,
include: [
{
// unscoping here allows us to return unpublished documents
model: Document.unscoped(),
include: [
{
model: User,
as: "createdBy",
paranoid: false,
},
{
model: User,
as: "updatedBy",
paranoid: false,
},
],
model: Document.scope("withDrafts"),
required: true,
as: "document",
},
@@ -129,14 +115,13 @@ export default async function loadDocument({
const canReadDocument = user && can(user, "read", document);
if (canReadDocument) {
// Cannot use document.collection here as it does not include the
// documentStructure by default through the relationship.
if (document.collectionId) {
collection = await Collection.findByPk(document.collectionId);
if (!collection) {
throw NotFoundError("Collection could not be found for document");
}
collection = await Collection.scope("withDocumentStructure").findByPk(
document.collectionId,
{
rejectOnEmpty: true,
}
);
}
return {
@@ -155,11 +140,15 @@ export default async function loadDocument({
// It is possible to disable sharing at the collection so we must check
if (document.collectionId) {
collection = await Collection.findByPk(document.collectionId);
collection = await Collection.scope("withDocumentStructure").findByPk(
document.collectionId,
{
rejectOnEmpty: true,
}
);
}
invariant(collection, "collection not found");
if (!collection.sharing) {
if (!collection?.sharing) {
throw AuthorizationError();
}
+17 -11
View File
@@ -1,4 +1,3 @@
import invariant from "invariant";
import { Transaction } from "sequelize";
import { createContext } from "@server/context";
import { traceFunction } from "@server/logging/tracing";
@@ -66,16 +65,21 @@ async function documentMover({
result.documents.push(document);
} else {
// Load the current and the next collection upfront and lock them
const collection = await Collection.findByPk(document.collectionId!, {
transaction,
lock: Transaction.LOCK.UPDATE,
paranoid: false,
});
const collection = await Collection.scope("withDocumentStructure").findByPk(
document.collectionId!,
{
transaction,
lock: Transaction.LOCK.UPDATE,
paranoid: false,
}
);
let newCollection = collection;
if (collectionChanged) {
if (collectionId) {
newCollection = await Collection.findByPk(collectionId, {
newCollection = await Collection.scope(
"withDocumentStructure"
).findByPk(collectionId, {
transaction,
lock: Transaction.LOCK.UPDATE,
});
@@ -144,12 +148,14 @@ async function documentMover({
if (collectionId) {
// Reload the collection to get relationship data
newCollection = await Collection.scope({
method: ["withMembership", user.id],
}).findByPk(collectionId, {
newCollection = await Collection.scope([
{
method: ["withMembership", user.id],
},
]).findByPk(collectionId, {
transaction,
rejectOnEmpty: true,
});
invariant(newCollection, "Collection not found");
result.collections.push(newCollection);
@@ -0,0 +1,29 @@
"use strict";
const { execFileSync } = require("child_process");
const path = require("path");
/** @type {import('sequelize-cli').Migration} */
module.exports = {
async up() {
if (
process.env.NODE_ENV === "test" ||
process.env.DEPLOYMENT === "hosted"
) {
return;
}
const scriptName = path.basename(__filename);
const scriptPath = path.join(
process.cwd(),
"build",
`server/scripts/${scriptName}`
);
execFileSync("node", [scriptPath], { stdio: "inherit" });
},
async down() {
// noop
},
};
+35 -29
View File
@@ -16,7 +16,7 @@ beforeEach(() => {
});
describe("#url", () => {
test("should return correct url for the collection", () => {
it("should return correct url for the collection", () => {
const collection = new Collection({
id: "1234",
});
@@ -25,7 +25,7 @@ describe("#url", () => {
});
describe("getDocumentParents", () => {
test("should return array of parent document ids", async () => {
it("should return array of parent document ids", async () => {
const parent = await buildDocument();
const document = await buildDocument();
const collection = await buildCollection({
@@ -41,7 +41,7 @@ describe("getDocumentParents", () => {
expect(result ? result[0] : undefined).toBe(parent.id);
});
test("should return array of parent document ids", async () => {
it("should return array of parent document ids", async () => {
const parent = await buildDocument();
const document = await buildDocument();
const collection = await buildCollection({
@@ -56,7 +56,7 @@ describe("getDocumentParents", () => {
expect(result?.length).toBe(0);
});
test("should not error if documentStructure is empty", async () => {
it("should not error if documentStructure is empty", async () => {
const parent = await buildDocument();
await buildDocument();
const collection = await buildCollection();
@@ -66,7 +66,7 @@ describe("getDocumentParents", () => {
});
describe("getDocumentTree", () => {
test("should return document tree", async () => {
it("should return document tree", async () => {
const document = await buildDocument();
const collection = await buildCollection({
documentStructure: [await document.toNavigationNode()],
@@ -76,7 +76,7 @@ describe("getDocumentTree", () => {
);
});
test("should return nested documents in tree", async () => {
it("should return nested documents in tree", async () => {
const parent = await buildDocument();
const document = await buildDocument();
const collection = await buildCollection({
@@ -99,7 +99,7 @@ describe("getDocumentTree", () => {
});
describe("#addDocumentToStructure", () => {
test("should add as last element without index", async () => {
it("should add as last element without index", async () => {
const collection = await buildCollection();
const id = uuidv4();
const newDocument = await buildDocument({
@@ -117,7 +117,7 @@ describe("#addDocumentToStructure", () => {
expect(collection.documentStructure!.length).toBe(1);
});
test("should add with an index", async () => {
it("should add with an index", async () => {
const collection = await buildCollection();
const id = uuidv4();
const newDocument = await buildDocument({
@@ -131,7 +131,7 @@ describe("#addDocumentToStructure", () => {
expect(collection.documentStructure![0].id).toBe(id);
});
test("should add as a child if with parent", async () => {
it("should add as a child if with parent", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -150,7 +150,7 @@ describe("#addDocumentToStructure", () => {
expect(collection.documentStructure![0].children[0].id).toBe(id);
});
test("should add as a child if with parent with index", async () => {
it("should add as a child if with parent with index", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -176,7 +176,7 @@ describe("#addDocumentToStructure", () => {
expect(collection.documentStructure![0].children[0].id).toBe(id);
});
test("should add the document along with its nested document(s)", async () => {
it("should add the document along with its nested document(s)", async () => {
const collection = await buildCollection();
const document = await buildDocument({
@@ -204,7 +204,7 @@ describe("#addDocumentToStructure", () => {
);
});
test("should add the document along with its archived nested document(s)", async () => {
it("should add the document along with its archived nested document(s)", async () => {
const collection = await buildCollection();
const document = await buildDocument({
@@ -237,7 +237,7 @@ describe("#addDocumentToStructure", () => {
);
});
describe("options: documentJson", () => {
test("should append supplied json over document's own", async () => {
it("should append supplied json over document's own", async () => {
const collection = await buildCollection();
const id = uuidv4();
const newDocument = await buildDocument({
@@ -268,7 +268,7 @@ describe("#addDocumentToStructure", () => {
});
describe("#updateDocument", () => {
test("should update root document's data", async () => {
it("should update root document's data", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -279,7 +279,7 @@ describe("#updateDocument", () => {
expect(collection.documentStructure![0].title).toBe("Updated title");
});
test("should update child document's data", async () => {
it("should update child document's data", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -297,7 +297,7 @@ describe("#updateDocument", () => {
newDocument.title = "Updated title";
await newDocument.save();
await collection.updateDocument(newDocument);
const reloaded = await Collection.findByPk(collection.id);
const reloaded = await collection.reload();
expect(reloaded!.documentStructure![0].children[0].title).toBe(
"Updated title"
);
@@ -305,7 +305,7 @@ describe("#updateDocument", () => {
});
describe("#removeDocument", () => {
test("should save if removing", async () => {
it("should save if removing", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -315,7 +315,7 @@ describe("#removeDocument", () => {
expect(collection.save).toBeCalled();
});
test("should remove documents from root", async () => {
it("should remove documents from root", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -331,7 +331,7 @@ describe("#removeDocument", () => {
expect(collectionDocuments.count).toBe(0);
});
test("should remove a document with child documents", async () => {
it("should remove a document with child documents", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -359,7 +359,7 @@ describe("#removeDocument", () => {
expect(collectionDocuments.count).toBe(0);
});
test("should remove a child document", async () => {
it("should remove a child document", async () => {
const collection = await buildCollection();
const document = await buildDocument({ collectionId: collection.id });
await collection.reload();
@@ -380,7 +380,7 @@ describe("#removeDocument", () => {
expect(collection.documentStructure![0].children.length).toBe(1);
// Remove the document
await collection.deleteDocument(newDocument);
const reloaded = await Collection.findByPk(collection.id);
const reloaded = await collection.reload();
expect(reloaded!.documentStructure!.length).toBe(1);
expect(reloaded!.documentStructure![0].children.length).toBe(0);
const collectionDocuments = await Document.findAndCountAll({
@@ -393,7 +393,7 @@ describe("#removeDocument", () => {
});
describe("#membershipUserIds", () => {
test("should return collection and group memberships", async () => {
it("should return collection and group memberships", async () => {
const team = await buildTeam();
const teamId = team.id;
// Make 6 users
@@ -464,47 +464,53 @@ describe("#membershipUserIds", () => {
});
describe("#findByPk", () => {
test("should return collection with collection Id", async () => {
it("should return collection with collection Id", async () => {
const collection = await buildCollection();
const response = await Collection.findByPk(collection.id);
expect(response!.id).toBe(collection.id);
});
test("should return collection when urlId is present", async () => {
it("should not return documentStructure by default", async () => {
const collection = await buildCollection();
const response = await Collection.findByPk(collection.id);
expect(() => response!.documentStructure).toThrow();
});
it("should return collection when urlId is present", async () => {
const collection = await buildCollection();
const id = `${slugify(collection.name)}-${collection.urlId}`;
const response = await Collection.findByPk(id);
expect(response!.id).toBe(collection.id);
});
test("should return collection when urlId is present, but missing slug", async () => {
it("should return collection when urlId is present, but missing slug", async () => {
const collection = await buildCollection();
const id = collection.urlId;
const response = await Collection.findByPk(id);
expect(response!.id).toBe(collection.id);
});
test("should return null when incorrect uuid type", async () => {
it("should return null when incorrect uuid type", async () => {
const collection = await buildCollection();
const response = await Collection.findByPk(collection.id + "-incorrect");
expect(response).toBe(null);
});
test("should return null when incorrect urlId length", async () => {
it("should return null when incorrect urlId length", async () => {
const collection = await buildCollection();
const id = `${slugify(collection.name)}-${collection.urlId}incorrect`;
const response = await Collection.findByPk(id);
expect(response).toBe(null);
});
test("should return null when no collection is found with uuid", async () => {
it("should return null when no collection is found with uuid", async () => {
const response = await Collection.findByPk(
"a9e71a81-7342-4ea3-9889-9b9cc8f667da"
);
expect(response).toBe(null);
});
test("should return null when no collection is found with urlId", async () => {
it("should return null when no collection is found with urlId", async () => {
const id = `${slugify("test collection")}-${randomstring.generate(15)}`;
const response = await Collection.findByPk(id);
expect(response).toBe(null);
+13
View File
@@ -37,6 +37,7 @@ import {
AllowNull,
BeforeCreate,
BeforeUpdate,
DefaultScope,
} from "sequelize-typescript";
import isUUID from "validator/lib/isUUID";
import type { CollectionSort, ProsemirrorData } from "@shared/types";
@@ -69,6 +70,11 @@ type AdditionalFindOptions = {
rejectOnEmpty?: boolean | Error;
};
@DefaultScope(() => ({
attributes: {
exclude: ["documentStructure"],
},
}))
@Scopes(() => ({
withAllMemberships: {
include: [
@@ -121,6 +127,12 @@ type AdditionalFindOptions = {
},
],
}),
withDocumentStructure: () => ({
attributes: {
// resets to include the documentStructure column
exclude: [],
},
}),
withMembership: (userId: string) => {
if (!userId) {
return {};
@@ -238,6 +250,7 @@ class Collection extends ParanoidModel<
@Column
maintainerApprovalRequired: boolean;
@Default(null)
@Column(DataType.JSONB)
documentStructure: NavigationNode[] | null;
+2 -5
View File
@@ -11,7 +11,6 @@ import {
buildUser,
buildGuestUser,
} from "@server/test/factories";
import Collection from "./Collection";
import UserMembership from "./UserMembership";
beforeEach(() => {
@@ -96,10 +95,8 @@ describe("#delete", () => {
await document.delete(user);
const [newDocument, newCollection] = await Promise.all([
Document.findByPk(document.id, {
paranoid: false,
}),
Collection.findByPk(collection.id),
document.reload({ paranoid: false }),
collection.reload(),
]);
expect(newDocument?.lastModifiedById).toEqual(user.id);
+56 -25
View File
@@ -15,6 +15,7 @@ import {
FindOptions,
WhereOptions,
EmptyResultError,
Sequelize,
} from "sequelize";
import {
ForeignKey,
@@ -71,12 +72,20 @@ import Length from "./validators/Length";
export const DOCUMENT_VERSION = 2;
// If content (JSON) is null then we still need to return the state column (BINARY)
// as it's used as a fallback for content deserialization for older documents.
// This can be removed if content is 100% backfilled.
const stateIfContentEmpty = Sequelize.literal(
`CASE WHEN document.content IS NULL THEN document.state ELSE NULL END AS state`
);
type AdditionalFindOptions = {
userId?: string;
includeState?: boolean;
rejectOnEmpty?: boolean | Error;
};
// @ts-expect-error Type 'Literal' is not assignable to type 'string | ProjectionAlias'.
@DefaultScope(() => ({
include: [
{
@@ -101,13 +110,14 @@ type AdditionalFindOptions = {
},
},
attributes: {
exclude: ["state"],
include: [stateIfContentEmpty],
},
}))
// @ts-expect-error Type 'Literal' is not assignable to type 'string | ProjectionAlias'.
@Scopes(() => ({
withoutState: {
attributes: {
exclude: ["state"],
include: [stateIfContentEmpty],
},
},
withCollection: {
@@ -121,7 +131,7 @@ type AdditionalFindOptions = {
withState: {
attributes: {
// resets to include the state column
exclude: [],
include: [],
},
},
withDrafts: {
@@ -162,11 +172,13 @@ type AdditionalFindOptions = {
return {
include: [
{
attributes: ["id", "permission", "sharing", "teamId", "deletedAt"],
model: userId
? Collection.scope({
method: ["withMembership", userId],
})
? Collection.scope([
"defaultScope",
{
method: ["withMembership", userId],
},
])
: Collection,
as: "collection",
paranoid,
@@ -414,10 +426,13 @@ class Document extends ArchivableModel<
return;
}
const collection = await Collection.findByPk(model.collectionId, {
transaction,
lock: Transaction.LOCK.UPDATE,
});
const collection = await Collection.scope("withDocumentStructure").findByPk(
model.collectionId,
{
transaction,
lock: Transaction.LOCK.UPDATE,
}
);
if (!collection) {
return;
}
@@ -438,7 +453,9 @@ class Document extends ArchivableModel<
}
return this.sequelize!.transaction(async (transaction: Transaction) => {
const collection = await Collection.findByPk(model.collectionId!, {
const collection = await Collection.scope(
"withDocumentStructure"
).findByPk(model.collectionId!, {
transaction,
lock: transaction.LOCK.UPDATE,
});
@@ -632,9 +649,13 @@ class Document extends ArchivableModel<
return uniq(membershipUserIds);
}
static withMembershipScope(userId: string, options?: FindOptions<Document>) {
static withMembershipScope(
userId: string,
options?: FindOptions<Document> & { includeDrafts?: boolean }
) {
return this.scope([
"defaultScope",
options?.includeDrafts ? "withDrafts" : "defaultScope",
"withoutState",
{
method: ["withViews", userId],
},
@@ -922,7 +943,9 @@ class Document extends ArchivableModel<
}
if (!this.template && this.collectionId) {
const collection = await Collection.findByPk(this.collectionId, {
const collection = await Collection.scope(
"withDocumentStructure"
).findByPk(this.collectionId, {
transaction,
lock: Transaction.LOCK.UPDATE,
});
@@ -989,10 +1012,13 @@ class Document extends ArchivableModel<
await this.sequelize.transaction(async (transaction: Transaction) => {
const collection = this.collectionId
? await Collection.findByPk(this.collectionId, {
transaction,
lock: transaction.LOCK.UPDATE,
})
? await Collection.scope("withDocumentStructure").findByPk(
this.collectionId,
{
transaction,
lock: transaction.LOCK.UPDATE,
}
)
: undefined;
if (collection) {
@@ -1023,10 +1049,13 @@ class Document extends ArchivableModel<
archive = async (user: User, options?: FindOptions) => {
const { transaction } = { ...options };
const collection = this.collectionId
? await Collection.findByPk(this.collectionId, {
transaction,
lock: transaction?.LOCK.UPDATE,
})
? await Collection.scope("withDocumentStructure").findByPk(
this.collectionId,
{
transaction,
lock: transaction?.LOCK.UPDATE,
}
)
: undefined;
if (collection) {
@@ -1047,7 +1076,7 @@ class Document extends ArchivableModel<
) => {
const { transaction } = { ...options };
const collection = collectionId
? await Collection.findByPk(collectionId, {
? await Collection.scope("withDocumentStructure").findByPk(collectionId, {
transaction,
lock: transaction?.LOCK.UPDATE,
})
@@ -1099,7 +1128,9 @@ class Document extends ArchivableModel<
let deleted = false;
if (!this.template && this.collectionId) {
const collection = await Collection.findByPk(this.collectionId!, {
const collection = await Collection.scope(
"withDocumentStructure"
).findByPk(this.collectionId!, {
transaction,
lock: transaction.LOCK.UPDATE,
paranoid: false,
+16 -18
View File
@@ -182,16 +182,16 @@ export default class SearchHelper {
},
];
return Document.withMembershipScope(user.id)
.scope("withDrafts")
.findAll({
where,
subQuery: false,
order: [["updatedAt", "DESC"]],
include,
offset,
limit,
});
return Document.withMembershipScope(user.id, {
includeDrafts: true,
}).findAll({
where,
subQuery: false,
order: [["updatedAt", "DESC"]],
include,
offset,
limit,
});
}
public static async searchCollectionsForUser(
@@ -264,14 +264,12 @@ export default class SearchHelper {
// Final query to get associated document data
const [documents, count] = await Promise.all([
Document.withMembershipScope(user.id)
.scope("withDrafts")
.findAll({
where: {
teamId: user.teamId,
id: map(results, "id"),
},
}),
Document.withMembershipScope(user.id, { includeDrafts: true }).findAll({
where: {
teamId: user.teamId,
id: map(results, "id"),
},
}),
results.length < limit && offset === 0
? Promise.resolve(results.length)
: countQuery,
@@ -171,7 +171,8 @@ export default abstract class ExportDocumentTreeTask extends ExportTask {
/**
* Generates a map of document urls to their path in the zip file.
*
* @param collections
* @param collections The collections to generate the path map for.
* @param format The format of the exported documents.
*/
private createPathMap(
collections: Collection[],
+7 -5
View File
@@ -44,11 +44,13 @@ export default abstract class ExportTask extends BaseTask<Props> {
? [fileOperation.collectionId]
: await user.collectionIds();
const collections = await Collection.findAll({
where: {
id: collectionIds,
},
});
const collections = await Collection.scope("withDocumentStructure").findAll(
{
where: {
id: collectionIds,
},
}
);
let filePath: string | undefined;
+5 -3
View File
@@ -140,9 +140,11 @@ router.post(
async (ctx: APIContext<T.CollectionsDocumentsReq>) => {
const { id } = ctx.input.body;
const { user } = ctx.state.auth;
const collection = await Collection.scope({
method: ["withMembership", user.id],
}).findByPk(id);
const collection = await Collection.scope([
{
method: ["withMembership", user.id],
},
]).findByPk(id);
authorize(user, "readDocument", collection);
@@ -977,7 +977,7 @@ describe("#documents.list", () => {
const res = await server.post("/api/documents.list", {
body: {
token: user.getJwtToken(),
collection: document.collectionId,
collectionId: document.collectionId,
},
});
const body = await res.json();
@@ -1013,7 +1013,7 @@ describe("#documents.list", () => {
const res = await server.post("/api/documents.list", {
body: {
token: user.getJwtToken(),
collection: collection.id,
collectionId: collection.id,
},
});
const body = await res.json();
+16 -12
View File
@@ -133,15 +133,19 @@ router.post(
// if a specific collection is passed then we need to check auth to view it
if (collectionId) {
where[Op.and].push({ collectionId: [collectionId] });
const collection = await Collection.scope({
method: ["withMembership", user.id],
}).findByPk(collectionId);
const collection = await Collection.scope([
sort === "index" ? "withDocumentStructure" : "defaultScope",
{
method: ["withMembership", user.id],
},
]).findByPk(collectionId);
authorize(user, "readDocument", collection);
// index sort is special because it uses the order of the documents in the
// collection.documentStructure rather than a database column
if (sort === "index") {
documentIds = (collection?.documentStructure || [])
documentIds = (collection.documentStructure || [])
.map((node) => node.id)
.slice(ctx.state.pagination.offset, ctx.state.pagination.limit);
where[Op.and].push({ id: documentIds });
@@ -535,14 +539,14 @@ router.post(
delete where.updatedAt;
}
const documents = await Document.withMembershipScope(user.id)
.scope("withDrafts")
.findAll({
where,
order: [[sort, direction]],
offset: ctx.state.pagination.offset,
limit: ctx.state.pagination.limit,
});
const documents = await Document.withMembershipScope(user.id, {
includeDrafts: true,
}).findAll({
where,
order: [[sort, direction]],
offset: ctx.state.pagination.offset,
limit: ctx.state.pagination.limit,
});
const data = await Promise.all(
documents.map((document) => presentDocument(ctx, document))
);
@@ -58,13 +58,13 @@ router.post(
const documentIds = memberships
.map((p) => p.documentId)
.filter(Boolean) as string[];
const documents = await Document.withMembershipScope(userId)
.scope("withDrafts")
.findAll({
where: {
id: documentIds,
},
});
const documents = await Document.withMembershipScope(userId, {
includeDrafts: true,
}).findAll({
where: {
id: documentIds,
},
});
const groups = uniqBy(
memberships.map((membership) => membership.group),
+5 -1
View File
@@ -54,7 +54,11 @@ router.post(
});
authorize(user, "read", document);
const collection = await document.$get("collection");
const collection = document.collectionId
? await Collection.scope("withDocumentStructure").findByPk(
document.collectionId
)
: undefined;
const parentIds = collection?.getDocumentParents(documentId);
const parentShare = parentIds
? await Share.scope({
@@ -0,0 +1,127 @@
import "./bootstrap";
import fractionalIndex from "fractional-index";
import { Sequelize, Transaction } from "sequelize";
import { Collection, Team } from "@server/models";
import { sequelize } from "@server/storage/database";
const limit = 100;
class CollectionIndexCollisionResolver {
private teamId: string;
private currDuplicateIndex: string | null = null;
private currDuplicateGroup: Collection[] = [];
private resolvedCollisionsCount: number = 0;
constructor(teamId: string) {
this.teamId = teamId;
}
public async process() {
await sequelize.transaction(async (transaction) => {
await this.processPage(0, transaction);
// edge case of last batch
await this.resolveDuplicates({ transaction });
});
}
private async processPage(
page: number,
transaction: Transaction
): Promise<void> {
console.log(
`Resolve collection index collisions for team ${this.teamId}… page ${page}`
);
const collections = await Collection.unscoped().findAll({
where: { teamId: this.teamId },
attributes: ["id", "index"],
limit,
offset: page * limit,
order: [
Sequelize.literal('"collection"."index" collate "C"'), // ensure duplicates are in sequential order
["updatedAt", "DESC"], // fallback as a tie breaker
],
lock: Transaction.LOCK.UPDATE,
transaction,
});
if (!collections.length) {
return;
}
let idx = 0;
while (idx < collections.length) {
const collection = collections[idx];
if (collection.index === this.currDuplicateIndex) {
// still in the same duplicate group.
this.currDuplicateGroup.push(collection);
} else {
// current collection index is different from the previous one; resolve duplicates, if applicable.
await this.resolveDuplicates({
nextCollection: collection,
transaction,
});
// reset the duplicate index and group.
this.currDuplicateIndex = collection.index;
this.currDuplicateGroup = [collection];
}
idx++;
}
return collections.length === limit
? this.processPage(page + 1, transaction)
: undefined;
}
private async resolveDuplicates({
nextCollection,
transaction,
}: {
nextCollection?: Collection;
transaction: Transaction;
}) {
if (this.currDuplicateGroup.length <= 1) {
// no action needed when there aren't more than 1 item in a group.
return;
}
let prevIndex = this.currDuplicateGroup[0].index;
const endIndex = nextCollection?.index ?? null;
// First collection in a duplicate group can retain its index.
for (let idx = 1; idx < this.currDuplicateGroup.length; idx++) {
const collection = this.currDuplicateGroup[idx];
const newIndex = fractionalIndex(prevIndex, endIndex);
console.log(`New index for collection ${collection.id} = ${newIndex}`);
collection.index = newIndex;
await collection.save({ silent: true, hooks: false, transaction });
prevIndex = newIndex;
}
this.resolvedCollisionsCount += this.currDuplicateGroup.length - 1;
}
}
export default async function main(exit = false) {
await Team.findAllInBatches<Team>({ batchLimit: 5 }, async (teams) => {
for (const team of teams) {
const resolver = new CollectionIndexCollisionResolver(team.id);
await resolver.process();
}
});
if (exit) {
process.exit(0);
}
}
// In the test suite we import the script rather than run via node CLI
if (process.env.NODE_ENV !== "test") {
void main(true);
}
+4 -1
View File
@@ -1,5 +1,6 @@
import path from "path";
import { InferAttributes, InferCreationAttributes } from "sequelize";
import sequelizeStrictAttributes from "sequelize-strict-attributes";
import { Sequelize } from "sequelize-typescript";
import { Umzug, SequelizeStorage, MigrationError } from "umzug";
import env from "@server/env";
@@ -23,7 +24,7 @@ export function createDatabaseInstance(
}
): Sequelize {
try {
return new Sequelize(databaseUrl, {
const instance = new Sequelize(databaseUrl, {
logging: (msg) =>
process.env.DEBUG?.includes("database") &&
Logger.debug("database", msg),
@@ -47,6 +48,8 @@ export function createDatabaseInstance(
},
schema,
});
sequelizeStrictAttributes(instance);
return instance;
} catch (error) {
Logger.fatal(
"Could not connect to database",
+4 -2
View File
@@ -310,7 +310,7 @@ export async function buildCollection(
overrides.permission = CollectionPermission.ReadWrite;
}
return Collection.create({
return Collection.scope("withDocumentStructure").create({
name: faker.lorem.words(2),
description: faker.lorem.words(4),
createdById: overrides.userId,
@@ -416,7 +416,9 @@ export async function buildDocument(
if (overrides.collectionId && overrides.publishedAt !== null) {
collection = collection
? await Collection.findByPk(overrides.collectionId)
? await Collection.scope("withDocumentStructure").findByPk(
overrides.collectionId
)
: undefined;
await collection?.addDocumentToStructure(document, 0);
@@ -191,7 +191,7 @@
"code": false,
"color": "default"
},
"plain_text": "http://github.com/outline/",
"plain_text": "",
"href": "http://github.com/outline/"
}
],
@@ -506,4 +506,4 @@
"color": "default"
}
}
]
]
-1
View File
@@ -11,7 +11,6 @@ export async function collectionIndexing(
where: {
teamId,
},
attributes: ["id", "index", "name", "teamId"],
transaction,
});
+56 -3
View File
@@ -1,6 +1,11 @@
import { GapCursor } from "prosemirror-gapcursor";
import { Node, NodeType } from "prosemirror-model";
import { Command, EditorState, TextSelection } from "prosemirror-state";
import { Node, NodeType, Slice } from "prosemirror-model";
import {
Command,
EditorState,
TextSelection,
Transaction,
} from "prosemirror-state";
import {
CellSelection,
addRow,
@@ -11,11 +16,16 @@ import {
addColumn,
deleteRow,
deleteColumn,
deleteTable,
} from "prosemirror-tables";
import { ProsemirrorHelper } from "../../utils/ProsemirrorHelper";
import { CSVHelper } from "../../utils/csv";
import { chainTransactions } from "../lib/chainTransactions";
import { getCellsInColumn, isHeaderEnabled } from "../queries/table";
import {
getCellsInColumn,
isHeaderEnabled,
isTableSelected,
} from "../queries/table";
import { TableLayout } from "../types";
import { collapseSelection } from "./collapseSelection";
@@ -544,3 +554,46 @@ export function moveOutOfTable(direction: 1 | -1): Command {
return false;
};
}
/**
* A command that deletes the entire table if all cells are selected.
*
* @returns The command
*/
export function deleteTableIfSelected(): Command {
return (state, dispatch): boolean => {
if (isTableSelected(state)) {
return deleteTable(state, dispatch);
}
return false;
};
}
export function deleteCellSelection(
state: EditorState,
dispatch?: (tr: Transaction) => void
): boolean {
const sel = state.selection;
if (!(sel instanceof CellSelection)) {
return false;
}
if (dispatch) {
const tr = state.tr;
const baseContent = tableNodeTypes(state.schema).cell.createAndFill()!
.content;
sel.forEachCell((cell, pos) => {
if (!cell.content.eq(baseContent)) {
tr.replace(
tr.mapping.map(pos + 1),
tr.mapping.map(pos + cell.nodeSize - 1),
new Slice(baseContent, 0, 0)
);
}
});
if (tr.docChanged) {
dispatch(tr);
return true;
}
}
return false;
}
+6
View File
@@ -198,6 +198,12 @@ export const codeLanguages: Record<string, CodeLanguage> = {
label: "Powershell",
loader: () => import("refractor/lang/powershell").then((m) => m.default),
},
promql: {
lang: "promql",
label: "PromQL",
// @ts-expect-error PromQL is not in types but exists
loader: () => import("refractor/lang/promql").then((m) => m.default),
},
protobuf: {
lang: "protobuf",
label: "Protobuf",
+1 -12
View File
@@ -5,12 +5,6 @@ import {
mathSchemaSpec,
} from "@benrbray/prosemirror-math";
import { PluginSimple } from "markdown-it";
import {
chainCommands,
deleteSelection,
selectNodeBackward,
joinBackward,
} from "prosemirror-commands";
import {
NodeSpec,
NodeType,
@@ -51,12 +45,7 @@ export default class Math extends Node {
keys({ type }: { type: NodeType }) {
return {
"Mod-Space": insertMathCmd(type),
Backspace: chainCommands(
deleteSelection,
mathBackspaceCmd,
joinBackward,
selectNodeBackward
),
Backspace: mathBackspaceCmd,
};
}
+11 -1
View File
@@ -24,8 +24,10 @@ import {
setTableAttr,
deleteColSelection,
deleteRowSelection,
deleteCellSelection,
moveOutOfTable,
createTableInner,
deleteTableIfSelected,
} from "../commands/table";
import { MarkdownSerializerState } from "../lib/markdown/serializer";
import { FixTablesPlugin } from "../plugins/FixTables";
@@ -96,8 +98,16 @@ export default class Table extends Node {
"Shift-Tab": goToNextCell(-1),
"Mod-Enter": addRowAndMoveSelection(),
"Mod-Backspace": chainCommands(
deleteCellSelection,
deleteColSelection(),
deleteRowSelection()
deleteRowSelection(),
deleteTableIfSelected()
),
Backspace: chainCommands(
deleteCellSelection,
deleteColSelection(),
deleteRowSelection(),
deleteTableIfSelected()
),
ArrowDown: moveOutOfTable(1),
ArrowUp: moveOutOfTable(-1),
+62 -19
View File
@@ -6,6 +6,30 @@ import {
selectedRect,
} from "prosemirror-tables";
/**
* Checks if the current selection is a column selection.
* @param state The editor state.
* @returns True if the selection is a column selection, false otherwise.
*/
export function isColSelection(state: EditorState): boolean {
if (state.selection instanceof CellSelection) {
return state.selection.isColSelection();
}
return false;
}
/**
* Checks if the current selection is a row selection.
* @param state The editor state.
* @returns True if the selection is a row selection, false otherwise.
*/
export function isRowSelection(state: EditorState): boolean {
if (state.selection instanceof CellSelection) {
return state.selection.isRowSelection();
}
return false;
}
export function getColumnIndex(state: EditorState): number | undefined {
if (state.selection instanceof CellSelection) {
if (state.selection.isColSelection()) {
@@ -62,13 +86,18 @@ export function getCellsInRow(index: number) {
};
}
/**
* Check if a specific column is selected in the editor.
*
* @param state The editor state
* @param index The index of the column to check
* @returns Boolean indicating if the column is selected
*/
export function isColumnSelected(index: number) {
return (state: EditorState): boolean => {
if (state.selection instanceof CellSelection) {
if (state.selection.isColSelection()) {
const rect = selectedRect(state);
return rect.left <= index && rect.right > index;
}
if (isColSelection(state)) {
const rect = selectedRect(state);
return rect.left <= index && rect.right > index;
}
return false;
@@ -106,28 +135,42 @@ export function isHeaderEnabled(
return true;
}
/**
* Check if a specific row is selected in the editor.
*
* @param state The editor state
* @param index The index of the row to check
* @returns Boolean indicating if the row is selected
*/
export function isRowSelected(index: number) {
return (state: EditorState): boolean => {
if (state.selection instanceof CellSelection) {
if (state.selection.isRowSelection()) {
const rect = selectedRect(state);
return rect.top <= index && rect.bottom > index;
}
if (isRowSelection(state)) {
const rect = selectedRect(state);
return rect.top <= index && rect.bottom > index;
}
return false;
};
}
/**
* Check if an entire table is selected in the editor.
*
* @param state The editor state
* @returns Boolean indicating if the table is selected
*/
export function isTableSelected(state: EditorState): boolean {
const rect = selectedRect(state);
if (state.selection instanceof CellSelection) {
const rect = selectedRect(state);
return (
rect.top === 0 &&
rect.left === 0 &&
rect.bottom === rect.map.height &&
rect.right === rect.map.width &&
!state.selection.empty &&
state.selection instanceof CellSelection
);
return (
rect.top === 0 &&
rect.left === 0 &&
rect.bottom === rect.map.height &&
rect.right === rect.map.width &&
!state.selection.empty
);
}
return false;
}
+2 -2
View File
@@ -172,8 +172,6 @@
"Deleting": "Deleting",
"Are you sure about that? Deleting the <em>{{collectionName}}</em> collection is permanent and cannot be restored, however all published documents within will be moved to the trash.": "Are you sure about that? Deleting the <em>{{collectionName}}</em> collection is permanent and cannot be restored, however all published documents within will be moved to the trash.",
"Also, <em>{{collectionName}}</em> is being used as the start view deleting it will reset the start view to the Home page.": "Also, <em>{{collectionName}}</em> is being used as the start view deleting it will reset the start view to the Home page.",
"Sorry, an error occurred saving the collection": "Sorry, an error occurred saving the collection",
"Add a description": "Add a description",
"Type a command or search": "Type a command or search",
"Choose a template": "Choose a template",
"Are you sure you want to permanently delete this entire comment thread?": "Are you sure you want to permanently delete this entire comment thread?",
@@ -618,6 +616,8 @@
"{{ groupsCount }} groups with access": "{{ groupsCount }} group with access",
"{{ groupsCount }} groups with access_plural": "{{ groupsCount }} groups with access",
"Archived by {{userName}}": "Archived by {{userName}}",
"Sorry, an error occurred saving the collection": "Sorry, an error occurred saving the collection",
"Add a description": "Add a description",
"Share": "Share",
"Overview": "Overview",
"Recently updated": "Recently updated",
+20
View File
@@ -7,6 +7,16 @@ import {
faWebAwesome,
faXTwitter,
faBluesky,
faGithub,
faGitlab,
faDiscord,
faDocker,
faCodepen,
faDropbox,
faPaypal,
faShopify,
faSwift,
faSlack,
} from "@fortawesome/free-brands-svg-icons";
import {
faBagShopping,
@@ -551,6 +561,16 @@ export class IconLibrary {
faPython,
faXTwitter,
faBluesky,
faGithub,
faGitlab,
faDiscord,
faDocker,
faCodepen,
faDropbox,
faPaypal,
faShopify,
faSwift,
faSlack,
].map((icon) => [
icon.iconName,
{
+847 -850
View File
File diff suppressed because it is too large Load Diff