|
|
上周更新系统，看到Gitlab-CE有新版本，直接就更新了。今天写好代码要推送的时候，发现服务端竟然挂了，提示502错误。- u* g2 }4 d4 o8 c' L
登陆到服务器，运行 gitlab-ctl status 查看状态，组件都正常运行。唯一的意外是：ssh登录服务器时，会出现”LANG cannot change locale (en_US.utf8)” 的提示。暂时想不到其他原因，就先解决这个问题。
0 ^9 J7 h3 d3 }8 O0 r运行 locale -a 命令，发现系统中确实没有 en_US.utf8 的本地化设置文件。于是编辑 /etc/locale.gen 文件，取消 “en_US.UTF-8 UTF-8” 这一行的注释，运行 locale-gen ，生成 en_US.utf8 相关文件。
/ O4 N$ L5 J& L0 O7 P接着设置语言和本地化设置：) \" H: [) N% o
export LC_ALL=en_US.UTF-8
% \5 }" k3 m! k7 H4 F- `% n+ dexport LANG=en_US.UTF-8断开ssh重新登陆，bash的警告提示消失了。运行 gitlab-ctl restart 重启Gitlab，代码能正常推送，网站能打开，似乎问题都解决了。
7 L/ G6 e$ `8 }% N7 G. A但是好事多磨，过了没多久，发现网页都是500错误。虽然能通过git正常推送和拉去代码，但是网页用不了也挺烦人的。想到上个月为了节省服务器资源，把Premethus、Grafana等用不到的组件都关了，会不会是这个原因导致的？7 l+ y' T6 c' X7 m& P5 R
于是编辑 /etc/gitlab/gitlab.rb 文件，把组件又启用。运行 gitlab-ctl reconfigure 重新配置，意外发现了，脚本无法运行成功，出现了如下提示：
7 q1 S0 W* J) e4 }( V# E( m3 gRecipe: gitlab::database_migrations( P7 u5 U5 v7 M4 \8 G. [% D
* ruby_block[check remote PG version] action nothing (skipped due to action :nothing)
# ?0 E" \9 e; b * rails_migration[gitlab-rails] action run: W U6 L7 _5 S
* bash[migrate gitlab-rails database] action run* K: T+ I! Q( P2 Y# g
5 U) C' G! b* H9 q ================================================================================5 ?& q9 @; K( C2 `
Error executing action `run` on resource 'bash[migrate gitlab-rails database]'* Q- v& d! r5 F' A l
================================================================================
1 M3 K2 ?4 b% T8 V! T. S- B
4 G8 ^0 A9 q- S Mixlib::ShellOut::ShellCommandFailed
/ Y* }9 s! X, b+ |( K& W9 i ------------------------------------
7 _% `% B) v1 Z, X5 y Command execution failed. STDOUT/STDERR suppressed for sensitive resource! G$ z& f5 g4 f6 F( A/ J3 F" }
, ^& S/ ?4 f; e) P0 a
Resource Declaration:$ ^# n$ r7 n7 s0 S
---------------------% ^5 g z2 h, t2 @# {. c
suppressed sensitive resource output
* f6 m/ U2 @! \2 I' Z
: ^9 w# ], X4 p- Z Compiled Resource:
/ I$ S( Q' f9 S7 h Q' a ------------------
9 w3 W4 o% A- \4 A+ `; D/ R/ L suppressed sensitive resource output
( b, w( b) N7 e% {7 s% o% ?
/ D5 g b% d$ O! P' V% h/ a System Info:
8 Z" |4 ]2 d+ `0 a! F. T) o ------------8 Z, K7 O# D4 n g1 |
chef_version=15.17.4: Q7 g! Z. F- V& y( m# {
platform=ubuntu3 A# h% {, Y! t+ b4 N
platform_version=20.04: U' A) y6 R: @( N' w+ K3 A7 s
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
* z d' o. ~7 T. D, n+ U9 ]) P program_name=/opt/gitlab/embedded/bin/chef-client4 s4 W% i* G5 i0 }
executable=/opt/gitlab/embedded/bin/chef-client% ] C2 o& V2 {8 [) A, H
- c: G+ G) U9 Q9 U# \
4 U9 M. Q; E- D9 ]- w2 `& x4 r* D ================================================================================. J+ u- m. |9 d+ p# f6 z& t$ d
Error executing action `run` on resource 'rails_migration[gitlab-rails]'2 a9 o9 a, {1 x5 H
================================================================================
2 p1 P" G7 o* G 9 w+ j( R* Y- I, m0 J& ?
Mixlib::ShellOut::ShellCommandFailed
# g8 X: \5 ~6 G0 p/ J) a% P+ K ------------------------------------
/ x- o: i# {! z" S bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource
/ Z* w: X9 d. n: N0 ^6 V5 F K1 T) O, u% N# ~
Resource Declaration:( N5 o0 s3 A7 s9 H8 B
---------------------
( ~: _& L* C7 [. H) M2 u# v # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb3 J R2 ^1 M" A x
7 g9 Q! ~" g. G 51: rails_migration "gitlab-rails" do
0 J t1 E# ]; b 52: rake_task 'gitlab:db:configure'6 x2 n7 t0 _- I) `+ a
53: logfile_prefix 'gitlab-rails-db-migrate'
" Z: j5 C! y/ M! b( L/ i 54: helper migration_helper$ d: }) e/ L8 u; Y6 N
55:
; B) D; a( |" l# D! p0 P 56: environment env_variables
/ I/ }( O# O7 t, Q 57: dependent_services dependent_services
* P/ Z' _2 R4 W8 q# u/ a 58: notifies :run, "execute[clear the gitlab-rails cache]", :immediately
0 V' E: I- M. X: J% y 59: notifies :run, "ruby_block[check remote PG version]", :immediately1 O. ~3 I' N) d4 o# J8 s0 {1 @, ^5 p
60: . S1 N Z$ V& ~
61: only_if { migration_helper.attributes_node['auto_migrate'] }1 g1 {3 f) O8 j, h
62: end- B' D& l; @5 e5 u
1 a9 U$ b% J: ^7 g) i; C$ o1 ?! I Compiled Resource:( ^+ k4 w. A0 ^8 @3 X2 ~
------------------
4 d* o3 r$ }$ v # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:51:in `from_file'
; ]/ B$ c$ i/ C4 ~ / W4 i' B* M2 Q) @( [% u" ?
rails_migration("gitlab-rails") do
! _! W. |; c: i2 ?/ A action [:run]" W, }$ Y+ W& Z+ O
default_guard_interpreter :default: M! s5 g$ \+ v2 ]' y
declared_type :rails_migration
\0 D! I; p6 t. T3 l% _4 L& h cookbook_name "gitlab"7 ]6 [: I4 u+ R
recipe_name "database_migrations"& n/ r6 R* Q/ }) U6 p. N
rake_task "gitlab:db:configure"* @- Z$ Y8 [8 m. Q: L' [8 \- T# J
logfile_prefix "gitlab-rails-db-migrate"( e# n! U1 K5 { R0 X
helper "*sensitive value suppressed*"$ ~2 G; Y6 p# ]4 n8 A
environment "*sensitive value suppressed*"
+ w4 ^2 ^9 Q4 @- s dependent_services ["runit_service[puma]", "sidekiq_service[sidekiq]"]
! W; q9 V5 U3 I7 ?2 u only_if { #code block }
9 T) L- Z3 u/ |' v2 W" m# L end9 P' x7 O7 `9 V2 d- g& }6 z3 [
0 E8 ^7 U. r6 G1 M$ Y
System Info:) U0 m4 l u5 t& e
------------
% i+ ^9 i" `% a% A chef_version=15.17.4- u9 l* |' r, Y; A0 [* e' f
platform=ubuntu
$ F% m% ]( {+ f8 N2 p platform_version=20.042 B8 U1 d* Q" S4 _* ~9 P4 W
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]7 q6 k4 i' l+ ]6 _' @
program_name=/opt/gitlab/embedded/bin/chef-client* Q% j% Y- M% P
executable=/opt/gitlab/embedded/bin/chef-client
3 j# ]/ I. m9 B4 b, e, ] 8 r m# W' T. u2 t7 ~( y
Running handlers:
0 Y/ ~5 i' J! xThere was an error running gitlab-ctl reconfigure:
' x0 x2 {& a- u$ _# trails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource
3 Z& R) x/ B0 {, C& E9 s, CRunning handlers complete# y5 M p* o" t
Chef Infra Client failed. 0 resources updated in 21 seconds网上一查，发现和上次碰到的 Gitlab更新设置总是500错误 问题一样，又是Gitlab官方的锅。但是这次解决问题比之前更麻烦，步骤为：9 n- f+ g" X: s4 l+ J, t! Y( F
1. 运行 gitlab-rake db:migrate，出现错误没关系，中间会输出一行用于终止后台任务的命令；( D/ ~0 u' t4 ^
2. 复制这条命令或者上一步控制台输出的指令： gitlab-rake gitlab:background_migrations:finalize[ProjectNamespaces::BackfillProjectNamespaces,projects,id,'[null\,"up"]']。运行这条指令可能会失败，如果失败，请重新运行上面的指令，成功说明问题解决了；
" [; u) I1 `8 M+ M: D% n3. 再次运行 gitlab-rake db:migrate，确认不会再报错；8 \8 d3 K% C# E6 G# Z
4. 运行 gitlab-ctl reconfigure;
4 C" G6 k# F; a' ~, {' E& T; S4 X5. 重启Gitlab：gitlab-ctl restart。
( W+ k0 X" @& v& R1 D: L, S需要注意的是，这个bug是在Gitlab-CE 14.10.0中引入，在14.10.1版本被修复。如果运行过reconfigure但没解决错误，是无法直接升级到14.10.1版本的。) H- w* e: M2 p. e$ J" F
参考1. Gitlab-ctl reconfigure doesn’t work after gitlab omnibus updated
; H3 p" Y; A @2. gitlab-ctl reconfigure failing due to migration issue7 U _( V/ C7 c5 H9 W1 y x
AD：【加速器推荐】 搬瓦工官方代理服务Just My Socks，高速CN2 GIA线路流畅访问外网，被墙自动更换IP打赏赞(2) |
|
发表于 2022-9-11 20:40:14
