2
0
Commit Graph

222 Commits

Author SHA1 Message Date
TheFox0x7
aece12cb17 Enable addtional linters (#34085)
enable mirror, usestdlibbars and perfsprint 
part of: https://github.com/go-gitea/gitea/issues/34083

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-04-01 10:14:01 +00:00
TheFox0x7
8292e6bd5b Enable testifylint rules (#34075)
enable testifylint rules disabled in:
https://github.com/go-gitea/gitea/pull/34054
2025-03-31 01:53:48 -04:00
TheFox0x7
0ecd3cfbcc enable staticcheck QFxxxx rules (#34064) 2025-03-29 17:32:28 -04:00
Lunny Xiao
695aa08943 Decouple Batch from git.Repository to simplify usage without requiring the creation of a Repository struct. (#34001)
No logic change
2025-03-27 17:35:48 +00:00
wxiaoguang
9e791cfa0e Fix incorrect code search indexer options (#33992)
Fix #33798

Co-authored-by: Giteabot <teabot@gitea.io>
2025-03-24 17:18:21 +01:00
charles
186a695766 Fix file name could not be searched if the file was not a text file when using the Bleve indexer (#33959)
Close #33828

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-21 19:00:02 +00:00
Andreas Svanberg
a0fc7a26c8 Allow filtering issues by any assignee (#33343)
This is the opposite of the "No assignee" filter, it will match all
issues that have at least one assignee.

Before
![Before
change](https://github.com/user-attachments/assets/4aea194b-9add-4a84-8d6b-61bfd8d9e58e)

After
![After change with any
filter](https://github.com/user-attachments/assets/99f1205d-ba9f-4a0a-a60b-cc1a0c0823fe)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-03-21 04:25:36 +00:00
Lunny Xiao
b0603fac8c Refactor functions to reduce repopath expose (#33892) 2025-03-16 03:14:56 +00:00
wxiaoguang
9b54305e87 Make SearchMode have default value and add comments (#33863)
* Make `SearchMode` have default value if it is empty
* Add some comments for the "match" queries
* Fix a copy-paste mistake in `buildMatchQuery` (`db.go`)
* Add missing `q.Analyzer = repoIndexerAnalyzer`, it is in old code,
although I do not see real difference ....
2025-03-14 18:06:31 +00:00
wxiaoguang
75f63b3969 Improve issue & code search (#33860)
Each "indexer" should provide the "search modes" they support by
themselves. And we need to remove the "fuzzy" search for code.
2025-03-13 11:07:48 +08:00
TheFox0x7
96bf5c832d Remove context from git struct (#33793)
Argument is moved from struct init in command run, which lets us remove
context from struct.
2025-03-04 11:56:11 -08:00
TheFox0x7
807c82b13b Use test context in tests and new loop system in benchmarks (#33648)
Replace all contexts in tests with go1.24 t.Context()

---------

Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-20 09:57:40 +00:00
Lunny Xiao
9b436f03eb Fix project issues list and counting (#33594)
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2025-02-17 05:14:56 +00:00
Darren Hoo
2b33533d7b enable literal string for code search (#33590)
Close: #33588

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2025-02-16 11:28:06 +01:00
Lunny Xiao
c3f783391c Fix unnecessary comment when moving issue on the same project column (#33496)
Fix #33482
2025-02-05 11:51:10 +08:00
TheFox0x7
f9eac1d007 Refactor older tests to use testify (#33140)
Refactor checks to use assert/require
Use require.Eventually for waiting in elastic and meilisearch tests
Use require to exit early instead of assert
2025-01-09 09:21:47 +08:00
wxiaoguang
230523a45f Fix bleve fuzziness search (#33078)
Close #31565
2025-01-03 00:32:02 +08:00
TheFox0x7
022d4e3e63 Enable tenv and testifylint rules (#32852)
Enables tenv and testifylint linters
closes: https://github.com/go-gitea/gitea/issues/32842
2024-12-15 10:41:29 +00:00
silverwind
4f63af733e Update golangci-lint to v1.62.2, fix issues (#32845)
Update it and fix new issues related to `redefines-builtin-id`
2024-12-15 02:31:07 +00:00
yp05327
f3e6a1664e Add is_archived option for issue indexer (#32735)
Try to fix #32697

Reason:
`is_archived` is already defined in the query options, but it is not
implemented in the indexer.
2024-12-12 07:33:31 +08:00
wxiaoguang
566f4bd078 Add label/author/assignee filters to the user/org home issue list (#32779)
Replace #26661, fix #25979

Not perfect, but usable and much better than before. Since it is quite
complex, I am not quite sure whether there would be any regression, if
any, I will fix in first time.

I have tested the related pages many times: issue list, milestone issue
list, project view, user issue list, org issue list.
2024-12-11 06:33:24 +00:00
wxiaoguang
96f31114f0 Fix markup render regression and fix some tests (#32640)
Fix #32639,
https://github.com/go-gitea/gitea/issues/32608#issuecomment-2497918210

By the way, fix some incorrect SQLs (use single quote but not double
quote)
2024-11-26 03:04:55 +08:00
Rowan Bohde
75262625b8 Reduce integration test overhead (#32475)
In profiling integration tests, I found a couple places where per-test
overhead could be reduced:

* Avoiding disk IO by synchronizing instead of deleting & copying test
Git repository data. This saves ~100ms per test on my machine
* When flushing queues in `PrintCurrentTest`, invoke `FlushWithContext`
in a parallel.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-11-14 19:28:46 +00:00
Bruno Sofiato
0438362ce3 Updated tokenizer to better matching when search for code snippets (#32261)
This PR improves the accuracy of Gitea's code search. 

Currently, Gitea does not consider statements such as
`onsole.log("hello")` as hits when the user searches for `log`. The
culprit is how both ES and Bleve are tokenizing the file contents (in
both cases, `console.log` is a whole token).

In ES' case, we changed the tokenizer to
[simple_pattern_split](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-simplepatternsplit-tokenizer.html#:~:text=The%20simple_pattern_split%20tokenizer%20uses%20a,the%20tokenization%20is%20generally%20faster.).
In such a case, tokens are words formed by digits and letters. In
Bleve's case, it employs a
[letter](https://blevesearch.com/docs/Tokenizers/) tokenizer.

Resolves #32220

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-11-06 20:51:20 +00:00
wxiaoguang
ce13ae2391 Update go dependencies (#32389) 2024-10-31 12:05:54 +00:00
Bruno Sofiato
15eec25e35 Allow code search by filename (#32210)
This is a large and complex PR, so let me explain in detail its changes.

First, I had to create new index mappings for Bleve and ElasticSerach as
the current ones do not support search by filename. This requires Gitea
to recreate the code search indexes (I do not know if this is a breaking
change, but I feel it deserves a heads-up).

I've used [this
approach](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/analysis-pathhierarchy-tokenizer.html)
to model the filename index. It allows us to efficiently search for both
the full path and the name of a file. Bleve, however, does not support
this out-of-box, so I had to code a brand new [token
filter](https://blevesearch.com/docs/Token-Filters/) to generate the
search terms.

I also did an overhaul in the `indexer_test.go` file. It now asserts the
order of the expected results (this is important since matches based on
the name of a file are more relevant than those based on its content).
I've added new test scenarios that deal with searching by filename. They
use a new repo included in the Gitea fixture.

The screenshot below depicts how Gitea shows the search results. It
shows results based on content in the same way as the current version
does. In matches based on the filename, the first seven lines of the
file contents are shown (BTW, this is how GitHub does it).


![image](https://github.com/user-attachments/assets/9d938d86-1a8d-4f89-8644-1921a473e858)

Resolves #32096

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-11 23:35:04 +00:00
Bruno Sofiato
4780ec4b24 Fixed race condition when deleting documents by repoId in ElasticSearch (#32185)
Resolves #32184

---------

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-10-03 16:03:36 +00:00
Bruno Sofiato
305b5ca44e Change the code search to sort results by relevance (#32134)
Resolves #32129

Signed-off-by: Bruno Sofiato <bruno.sofiato@gmail.com>
2024-09-28 20:13:55 +00:00
Lunny Xiao
f59bb31f67 Fix index too many file names bug (#31903)
Try to fix #31884
Fix #28584
2024-09-01 05:57:31 +00:00
Lunny Xiao
e05c454662 Refactor the usage of batch catfile (#31754)
When opening a repository, it will call `ensureValidRepository` and also
`CatFileBatch`. But sometimes these will not be used until repository
closed. So it's a waste of CPU to invoke 3 times git command for every
open repository.

This PR removed all of these from `OpenRepository` but only kept
checking whether the folder exists. When a batch is necessary, the
necessary functions will be invoked.
2024-08-20 17:04:57 +00:00
Kemal Zebari
5e72d2a3dc Properly filter issue list given no assignees filter (#31522)
Quick fix #31520. This issue is related to #31337.
2024-07-23 18:36:32 +00:00
Carsten Klein
27c8c2bec7 Allow searching issues by ID (#31479)
When you are entering a number in the issue search, you likely want the
issue with the given ID (code internal concept: issue index).
As such, when a number is detected, the issue with the corresponding ID
will now be added to the results.

Fixes #4479

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-07-17 00:49:05 +02:00
Lunny Xiao
dfb5956b24 Fix bug filtering issues which have no project (#31337)
Fix #31327
This is a quick patch to fix the bug.
Some parameters are using 0, some are using -1. I think it needs a
refactor to keep consistent. But that will be another PR.
2024-06-14 02:31:07 +00:00
Lunny Xiao
078c246de9 Rename project board -> column to make the UI less confusing (#30170)
This PR split the `Board` into two parts. One is the struct has been
renamed to `Column` and the second we have a `Template Type`.

But to make it easier to review, this PR will not change the database
schemas, they are just renames. The database schema changes could be in
future PRs.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: yp05327 <576951401@qq.com>
2024-05-27 08:59:54 +00:00
wxiaoguang
cbf29c0414 Fix bleve fuzziness (#30799)
Fix #30797
Fix #30317
2024-05-01 15:32:52 +03:00
silverwind
b4127d8b3b Fix tautological conditions (#30735)
As discovered by https://github.com/go-gitea/gitea/pull/30729.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-30 14:34:40 +02:00
Chongyi Zheng
7c761651d3 Resolve lint for unused parameter and unnecessary type arguments (#30750)
Resolve all cases for `unused parameter` and `unnecessary type
arguments`

Related: #30729

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2024-04-29 08:47:56 +00:00
Kemal Zebari
f96137f399 Perform Newest sort type correctly when sorting issues (#30644)
Should resolve #30642.

Before this commit, we were treating an empty `?sort=` query parameter
as the correct sorting type (which is to sort issues in descending order
by their created UNIX time). But when we perform `sort=latest`, we did
not include this as a type so we would sort by the most recently updated
when reaching the `default` switch statement block.

This commit fixes this by considering the empty string, "latest", and
just any other string that is not mentioned in the switch statement as
sorting by newest.
2024-04-23 15:10:01 +08:00
silverwind
df8d7d0779 Enable more revive linter rules (#30608)
Noteable additions:

- `redefines-builtin-id` forbid variable names that shadow go builtins
- `empty-lines` remove unnecessary empty lines that `gofumpt` does not
remove for some reason
- `superfluous-else` eliminate more superfluous `else` branches

Rules are also sorted alphabetically and I cleaned up various parts of
`.golangci.yml`.
2024-04-22 11:48:42 +00:00
wxiaoguang
cfcb837461 Render embedded code preview by permlink in markdown (#30234)
The permlink in markdown will be rendered as a code preview block, like GitHub

Co-authored-by: silverwind <me@silverwind.io>
2024-04-02 17:48:27 +00:00
Lunny Xiao
291241033c Use db.ListOptions directly instead of Paginator interface to make it easier to use and fix performance of /pulls and /issues (#29990)
This PR uses `db.ListOptions` instead of `Paginor` to make the code
simpler.
And it also fixed the performance problem when viewing /pulls or
/issues. Before the counting in fact will also do the search.

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: silverwind <me@silverwind.io>
2024-03-24 18:51:08 +00:00
wxiaoguang
031efb167f Support repo code search without setting up an indexer (#29998)
By using git's ability, end users (especially small instance users) do
not need to enable the indexer, they could also benefit from the code
searching feature.

Fix #29996


![image](https://github.com/go-gitea/gitea/assets/2114189/11b7e458-88a4-480d-b4d7-72ee59406dd1)


![image](https://github.com/go-gitea/gitea/assets/2114189/0fe777d5-c95c-4288-a818-0427680805b6)

---------

Co-authored-by: silverwind <me@silverwind.io>
2024-03-24 17:05:00 +01:00
6543
7fe297730c Determine fuzziness of bleve indexer by keyword length (#29706)
also bleve did match on fuzzy search and the other way around. this also fix that bug.
2024-03-23 16:45:13 +01:00
Lunny Xiao
4a361391e5 Use db.ListOptionsAll instead of db.ListOptions{ListAll: true} (#29995) 2024-03-22 13:53:52 +01:00
6543
d8d6b7cdc8 Meilisearch double quote on "match" query (#29740)
make `nonFuzzyWorkaround` unessesary

cc @Kerollmops
2024-03-16 13:19:41 +00:00
6543
65f01c9349 Refactor code_indexer to use an SearchOptions struct for PerformSearch (#29724)
similar to how it's already done for the issue_indexer


---
*Sponsored by Kithara Software GmbH*
2024-03-16 10:32:45 +00:00
6543
5d55e42114 Refactor to use optional.Option for issue index search option (#29739)
Signed-off-by: 6543 <6543@obermui.de>
2024-03-13 08:25:53 +00:00
Lunny Xiao
2131e8b246 Use repo object format name instead of detecting from git repository (#29702)
It's unnecessary to detect the repository object format from git
repository. Just use the repository's object format name.
2024-03-10 22:30:36 +01:00
6543
b8befb2404 Patch in exact search for meilisearch (#29671)
meilisearch does not have an search option to contorl fuzzynes per query
right now:
 - https://github.com/meilisearch/meilisearch/issues/1192
 - https://github.com/orgs/meilisearch/discussions/377
 - https://github.com/meilisearch/meilisearch/discussions/1096

so we have to create a workaround by post-filter the search result in
gitea until this is addressed.

For future works I added an option in backend only atm, to enable
fuzzynes for issue indexer too.
And also refactored the code so the fuzzy option is equal in logic to
code indexer


---
*Sponsored by Kithara Software GmbH*
2024-03-09 01:39:27 +00:00
yp05327
db58279684 Fix wrong line number in code search result (#29260)
Fix #29136

Before: The result is a table and all line numbers are all in one row.

After: Use a separate table column for the line numbers.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2024-03-06 07:24:43 +00:00