2
0
Commit Graph

73 Commits

Author SHA1 Message Date
wxiaoguang
bf8ec00a2c Refactor "string truncate" (#32984) 2024-12-26 11:56:03 +08:00
Lunny Xiao
cfd40b7487 Move RepoTransfer from models to models/repo sub package (#32506)
`RepoTransfer` now is at models, but if we want to move it into `repo`
model, it will depend on `Team`. So this PR also makes repo model depend
on org model to make it possible. Just refactor, no code change.

- [x] Move `DeleteOrganization` from `models/organization` to service
layer
- [x] Move `AccessibleTeamReposEnv` to `models/repo`
- [x] Move `RepoTransfer` from `models` to `models/repo`
- [x] Merge `getUserTeamIDs` and `GetUserTeamIDs`, Merge `GetUserTeams`
and `getUserTeams`.
- [x] Remove `Team`'s `Repos []*repo_model.Repository` to avoid dependency recycle.
2024-12-18 11:44:16 +08:00
TheFox0x7
022d4e3e63 Enable tenv and testifylint rules (#32852)
Enables tenv and testifylint linters
closes: https://github.com/go-gitea/gitea/issues/32842
2024-12-15 10:41:29 +00:00
Lunny Xiao
9398090c30 Move GetFeeds to service layer (#32526)
Move GetFeeds from models to service layer, no code change.
2024-11-29 17:53:49 +00:00
wxiaoguang
96f31114f0 Fix markup render regression and fix some tests (#32640)
Fix #32639,
https://github.com/go-gitea/gitea/issues/32608#issuecomment-2497918210

By the way, fix some incorrect SQLs (use single quote but not double
quote)
2024-11-26 03:04:55 +08:00
wxiaoguang
46d7fd3273 Refactor markup render system (#32612)
This PR removes (almost) all path tricks, and introduces "renderhelper"
package.

Now we can clearly see the rendering behaviors for comment/file/wiki,
more details are in "renderhelper" tests.

Fix #31411 , fix #18592, fix #25632 and maybe more problems. (ps: fix
#32608 by the way)
2024-11-24 16:18:57 +08:00
wxiaoguang
f980fff00d Fix nil panic if repo doesn't exist (#32501)
fix  #32496
2024-11-14 12:17:58 +08:00
BoYanZh
75e2b38647 Perf: add extra index to notification table (#32395)
Index SQL: `CREATE INDEX u_s_uu ON notification(user_id, status,
updated_unix);`

The naming follows `action.go` in the same dir.

I am unsure which version I should add SQL to the migration folder, so I
have not modified it.

Fix #32390
2024-11-13 18:17:54 +00:00
Lunny Xiao
c208a550db Add new index for action to resolve the performance problem (#32333)
Fix #32224
2024-11-06 22:04:48 +00:00
Lunny Xiao
b4f02a8e95 Don't join repository when loading action table data (#32127) 2024-09-26 04:50:30 +00:00
Timon van der Berg
bd9472e169 Repo Activity: count new issues that were closed (#31776)
I'm new to go and contributing to gitea, your guidance is much
appreciated.

This is meant to solve https://github.com/go-gitea/gitea/issues/13309

Previously, closed issues would not be shown under new issues in the
activity tab, even if they were newly created.

changes:
* Split out newlyCreatedIssues from issuesForActivityStatement to count
  both currently open and closed issues.
* Use a seperate function to count active issues to prevent
double-counting issues after the above change.

Result is that new issues that have been closed are shown both under
"new" and "closed".

Signed-off-by: Timon van der Berg <tmnvanderberg@gmail.com>
2024-09-21 18:57:01 +00:00
Lunny Xiao
2d6711e995 Improve get feed with pagination (#31821)
Fix #31752

@somera

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-09-02 19:05:09 +00:00
wxiaoguang
2d48b88738 Refactor webhook (#31587)
A more complete fix for #31588

1. Make "generic" code more readable
2. Clarify HTML or Markdown for the payload content
2024-07-10 11:37:16 +00:00
kiatt210
c908400209 Fix web notification icon not updated once you read all notifications (#31447)
Fix #29065
Remove status filtering from GetUIDsAndNotificationCounts sql.

---------

Co-authored-by: kiatt210 <kiatt210@github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-06-23 15:50:10 +08:00
Lunny Xiao
078c246de9 Rename project board -> column to make the UI less confusing (#30170)
This PR split the `Board` into two parts. One is the struct has been
renamed to `Column` and the second we have a `Template Type`.

But to make it easier to review, this PR will not change the database
schemas, they are just renames. The database schema changes could be in
future PRs.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: yp05327 <576951401@qq.com>
2024-05-27 08:59:54 +00:00
wxiaoguang
4692f90224 Filter out duplicate action(activity) items for a repository (#30957)
Fix #20986
2024-05-14 13:47:03 +00:00
Lunny Xiao
0cc4cf6de3 Reduce unnecessary database queries on actions table (#30509) 2024-04-17 01:25:03 +00:00
Lunny Xiao
2d5cc1dcb0 Refactor more filterslice (#30370) 2024-04-10 04:18:41 +00:00
oliverpool
56bd83b895 Add container.FilterSlice function (#30339)
Many places have the following logic:
```go
func (jobs ActionJobList) GetRunIDs() []int64 {
	ids := make(container.Set[int64], len(jobs))
	for _, j := range jobs {
		if j.RunID == 0 {
			continue
		}
		ids.Add(j.RunID)
	}
	return ids.Values()
}
```

this introduces a `container.FilterMapUnique` function, which reduces
the code above to:
```go
func (jobs ActionJobList) GetRunIDs() []int64 {
	return container.FilterMapUnique(jobs, func(j *ActionRunJob) (int64, bool) {
		return j.RunID, j.RunID != 0
	})
}
```
2024-04-09 20:27:30 +08:00
Lunny Xiao
ad61a23ae6 Performance improvements for pull request list page (#29900)
This PR will avoid load pullrequest.Issue twice in pull request list
page. It will reduce x times database queries for those WIP pull
requests.

Partially fix #29585

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-03-21 14:13:08 +01:00
Lunny Xiao
e93db9bfdf Move notifications to a standalone file (#29930)
There is no code change. Just move notification list related
structs/functions from one file to another.
2024-03-20 10:28:35 +00:00
Lunny Xiao
06bb053f38 Add more stats tables (#29730)
Add `Tags`, `Branches` and `CommitStatus` to monitor/stats
2024-03-12 11:21:09 +00:00
Lunny Xiao
38c260ebf8 Some performance optimization on dashboard and issues page (#29010)
This PR do some loading speed optimization for feeds user interface
pages.
- Load action users batchly but not one by one.
- Load action repositories batchly but not one by one.
- Load action's Repo Owners batchly but not one by one.
- Load action's possible issues batchly but not one by one.
- Load action's possible comments batchly but not one by one.
2024-03-12 04:57:19 +00:00
wxiaoguang
8879ee8f7a Avoid issue info panic (#29625)
Fix #29624
2024-03-06 13:12:44 +00:00
wxiaoguang
d0bebbfed0 Rename Action.GetDisplayName to GetActDisplayName (#29540)
To avoid conflicting with User.GetDisplayName, because there is no data
type in template.

And it matches other methods like GetActFullName / GetActUserName
2024-03-02 15:30:18 +00:00
Lunny Xiao
23cf53bab4 Revert "Speed up loading the dashboard on mysql/mariadb (#28546)" (#29006)
This reverts commit 939db4b041. #28546 
Because it seems performance become worse.
2024-02-01 14:25:37 +08:00
Lunny Xiao
44093d524b Simplify how git repositories are opened (#28937)
## Purpose
This is a refactor toward building an abstraction over managing git
repositories.
Afterwards, it does not matter anymore if they are stored on the local
disk or somewhere remote.

## What this PR changes
We used `git.OpenRepository` everywhere previously.
Now, we should split them into two distinct functions:

Firstly, there are temporary repositories which do not change:

```go
git.OpenRepository(ctx, diskPath)
```

Gitea managed repositories having a record in the database in the
`repository` table are moved into the new package `gitrepo`:

```go
gitrepo.OpenRepository(ctx, repo_model.Repo)
```

Why is `repo_model.Repository` the second parameter instead of file
path?
Because then we can easily adapt our repository storage strategy.
The repositories can be stored locally, however, they could just as well
be stored on a remote server.

## Further changes in other PRs
- A Git Command wrapper on package `gitrepo` could be created. i.e.
`NewCommand(ctx, repo_model.Repository, commands...)`. `git.RunOpts{Dir:
repo.RepoPath()}`, the directory should be empty before invoking this
method and it can be filled in the function only. #28940
- Remove the `RepoPath()`/`WikiPath()` functions to reduce the
possibility of mistakes.

---------

Co-authored-by: delvh <dev.lh@web.de>
2024-01-27 21:09:51 +01:00
Lunny Xiao
939db4b041 Speed up loading the dashboard on mysql/mariadb (#28546)
Fixes #28155
2024-01-14 15:34:25 +01:00
wxiaoguang
6213439dca Refactor timeutil package (#28623)
1. make names more readable
2. remove unused FormatLong/FormatShort
3. use `FormatDate` instead of `Format "2006-01-02"`
2023-12-28 10:09:57 +00:00
Lunny Xiao
d455445979 Use db.Find instead of writing methods for every object (#28084)
For those simple objects, it's unnecessary to write the find and count
methods again and again.
2023-11-24 03:49:41 +00:00
Lunny Xiao
9d007ec8cc Refactor Find Sources and fix bug when view a user who belongs to an unactive auth source (#27798)
The steps to reproduce it.

First, create a new oauth2 source.
Then, a user login with this oauth2 source.
Disable the oauth2 source.
Visit users -> settings -> security, 500 will be displayed.
This is because this page only load active Oauth2 sources but not all
Oauth2 sources.
2023-11-03 01:41:00 +00:00
silverwind
acb0973f5f Upgrade to golangci-lint@v1.55.0 (#27756)
https://github.com/golangci/golangci-lint/releases/tag/v1.55.0
2023-10-24 02:54:59 +00:00
JakobDev
e38626c0bb Penultimate round of db.DefaultContext refactor (#27414)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-10-11 04:24:07 +00:00
JakobDev
762f1360aa Add Index to action.user_id (#27403)
Another Column that needs a Index. Found at
https://codeberg.org/forgejo/discussions/issues/61#issuecomment-1258744.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-10-03 21:41:25 -04:00
JakobDev
7c6ab9c798 Even more db.DefaultContext refactor (#27352)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
2023-10-03 10:30:41 +00:00
JakobDev
9f14c44f2f More db.DefaultContext refactor (#27265)
Part of #27065

This PR touches functions used in templates. As templates are not static
typed, errors are harder to find, but I hope I catch it all. I think
some tests from other persons do not hurt.
2023-09-29 12:12:54 +00:00
Lunny Xiao
f491fe30b2 make writing main test easier (#27270)
This PR removed `unittest.MainTest` the second parameter
`TestOptions.GiteaRoot`. Now it detects the root directory by current
working directory.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-28 01:38:53 +00:00
JakobDev
0373a96d4f Another round of db.DefaultContext refactor (#27103)
Part of #27065

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-09-25 13:17:37 +00:00
KN4CK3R
8dff65d0fe Quote table release in sql queries (#27205)
Fixes #27174

`release` is a reserved keyword in MySql. I can't reproduce the issue on
my setup and we have a test for that code but it seems there can be
setups where it fails.


82b615e3e0/tests/integration/repo_activity_test.go (L45-L46)
2023-09-23 12:57:39 +00:00
JakobDev
c248273cb6 Reduce usage of db.DefaultContext (#27073)
Part of #27065

This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on
this.

Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-14 17:09:32 +00:00
Lunny Xiao
f2c51e40f5 move repository deletion to service layer (#26948)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-08 04:51:15 +00:00
CaiCandong
ba584882b5 Refactor dashboard/feed.tmpl (#26956)
- ~~Refactor `ActionType` to `models/activities/action_type.go`~~
- Replace the magic number in `feed.tmlp` with `InAction`
2023-09-07 16:23:13 +02:00
wxiaoguang
08e1b1ca9f Bypass MariaDB performance bug of the "IN" sub-query, fix incorrect IssueIndex (#26279)
Close #26277
Fix #26285
2023-08-07 12:23:59 +02:00
Lunny Xiao
354c6aa46c Fix bug with sqlite load read (#26305)
Possible fix #26280
2023-08-05 10:40:27 +08:00
Lunny Xiao
8276326718 Add context parameter to some database functions (#26055)
To avoid deadlock problem, almost database related functions should be
have ctx as the first parameter.
This PR do a refactor for some of these functions.
2023-07-22 22:14:27 +08:00
silverwind
a374e384c4 Update tool dependencies, lock govulncheck and actionlint (#25655)
- Update all tool dependencies
- Lock `govulncheck` and `actionlint` to their latest tags

---------

Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-09 11:58:06 +00:00
wxiaoguang
b8ee3c1a0e Use a separate admin page to show global stats, remove actions stat (#25062)
Before, Gitea shows the database table stats on the `admin dashboard`
page.

It has some problems:

* `count(*)` is quite heavy. If tables have many records, this blocks
loading the admin page blocks for a long time
* Some users had even reported issues that they can't visit their admin
page because this page causes blocking or `50x error (reverse proxy
timeout)`
* The `actions` stat is not useful. The table is simply too large. Does
it really matter if it contains 1,000,000 rows or 9,999,999 rows?
* The translation `admin.dashboard.statistic_info` is difficult to
maintain.

So, this PR uses a separate page to show the stats and removes the
`actions` stat.


![image](https://github.com/go-gitea/gitea/assets/2114189/babf7c61-b93b-4a62-bfaa-22983636427e)

## ⚠️ BREAKING

The `actions` Prometheus metrics collector has been removed for the
reasons mentioned beforehand.
Please do not rely on its output anymore.
2023-06-03 22:03:41 +08:00
Lunny Xiao
f56a4d8d6d Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634)
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
> 
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
> 
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
> 
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
> 
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-05-26 01:04:48 +00:00
Zettat123
8f8d90b068 Fix commits pushed with deploy keys not shown in dashboard (#24521)
Fix #21324

In the current logic, if the `Actor` user is not an admin user, all
activities from private organizations won't be shown even if the `Actor`
user is a member of the organization.

As mentioned in the issue, when using deploy key to make a commit and
push, the activity's `act_user_id` will be the id of the organization so
the activity won't be shown to non-admin users because the visibility of
the organization is private.

9e43f56029/models/activities/action.go (L490-L503)

This PR improves this logic so the activities of private organizations
can be shown.
2023-05-10 04:14:58 +00:00
Yarden Shoham
9bf6f0c2de Localize activity heatmap (except tooltip) (#24131)
The calculation of the total sum is moved to the backend so a full HTML
string could be sent.


![image](https://user-images.githubusercontent.com/20454870/232112381-c11d896b-ba47-40f8-b2a3-71cf4b3208de.png)

- Closes #10669
- 2nd attempt (the first was in #21570)

---------

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-04-17 14:26:01 -04:00