|
|
上周更新系统，看到Gitlab-CE有新版本，直接就更新了。今天写好代码要推送的时候，发现服务端竟然挂了，提示502错误。
g; }, U3 D4 y2 T i' t登陆到服务器，运行 gitlab-ctl status 查看状态，组件都正常运行。唯一的意外是：ssh登录服务器时，会出现”LANG cannot change locale (en_US.utf8)” 的提示。暂时想不到其他原因，就先解决这个问题。
; m; @9 j& I2 c4 J运行 locale -a 命令，发现系统中确实没有 en_US.utf8 的本地化设置文件。于是编辑 /etc/locale.gen 文件，取消 “en_US.UTF-8 UTF-8” 这一行的注释，运行 locale-gen ，生成 en_US.utf8 相关文件。
) Y U* s4 X8 d4 t& g接着设置语言和本地化设置：
) n/ Y* Q( [& u2 t; \) Oexport LC_ALL=en_US.UTF-8$ Z; e; |6 I+ [! m" g+ B
export LANG=en_US.UTF-8断开ssh重新登陆，bash的警告提示消失了。运行 gitlab-ctl restart 重启Gitlab，代码能正常推送，网站能打开，似乎问题都解决了。
) c& w. v" d) h4 ~, e# D# z但是好事多磨，过了没多久，发现网页都是500错误。虽然能通过git正常推送和拉去代码，但是网页用不了也挺烦人的。想到上个月为了节省服务器资源，把Premethus、Grafana等用不到的组件都关了，会不会是这个原因导致的？
9 G, _6 u/ [7 \, K9 M% ?& m于是编辑 /etc/gitlab/gitlab.rb 文件，把组件又启用。运行 gitlab-ctl reconfigure 重新配置，意外发现了，脚本无法运行成功，出现了如下提示：
2 i5 m# w; O3 U4 b) e3 G* a2 m7 ~/ ARecipe: gitlab::database_migrations* t o; m) X9 e6 L+ w; y
* ruby_block[check remote PG version] action nothing (skipped due to action :nothing)
, O$ ~2 O4 t0 b6 s+ o d) Z * rails_migration[gitlab-rails] action run
. g( t! X* ~# l1 ~% [, ~2 G * bash[migrate gitlab-rails database] action run: G+ X; E2 P* `7 @4 A) i
- C5 e4 e% S% _+ R6 z& _ ================================================================================
" m2 d1 e+ K( e Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
$ A5 n( e8 Y* d# Y' ?( ]" [7 C ================================================================================
+ H8 a3 t) B" O% z1 |: ^# i ' V2 S- w4 F' D4 E. \' ~- a$ V
Mixlib::ShellOut::ShellCommandFailed
# G5 l# C: W+ m3 S ------------------------------------ Q d2 O, K/ b p' k: D) {% Z9 T
Command execution failed. STDOUT/STDERR suppressed for sensitive resource
- d8 f' o7 n+ g0 b% a G$ W 5 o- ^! T' U) x5 M1 t, I
Resource Declaration:
8 G1 R" T* N* K6 M6 G ---------------------- @1 J+ {+ P9 X$ D7 l5 v
suppressed sensitive resource output
* E {6 Q# A" r! w: J) }2 R
% R- c; H9 x. S Compiled Resource:3 n" f7 [9 r& c
------------------ D; w- e& Q& d# ^7 Q
suppressed sensitive resource output
8 ?+ }( K$ {8 z" \; b 7 b9 b# i( p6 M: B" `8 m6 m
System Info:6 M( @2 R7 l! u; r9 w# L
------------- y+ D2 ~! V$ [) ~) _3 z/ O% e
chef_version=15.17.4
% I( \/ b* c9 _% s$ M; ` platform=ubuntu. n- Q, T" }* m: y6 R5 k
platform_version=20.04- P& `2 W, w4 {! I; b4 j- y- A, A
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]' S( g' ?1 P1 X! F$ a) z6 z$ L0 I
program_name=/opt/gitlab/embedded/bin/chef-client3 f% t; u- r1 [. G, x9 V$ D4 J
executable=/opt/gitlab/embedded/bin/chef-client
* i3 R7 q3 f- Z7 p1 { e' T
5 o5 U: v% z, S' L3 n8 e; S3 v 4 D* c0 M" u9 T9 |3 g/ d; P6 ?6 y
================================================================================
# Z( @, A; Z* U7 p) \2 R Error executing action `run` on resource 'rails_migration[gitlab-rails]'
8 V+ e4 L0 }: _0 T% o ================================================================================' v U* p. P( z
8 C+ }6 U5 |( v' m* I! r Mixlib::ShellOut::ShellCommandFailed6 A6 r' {, d5 S& l
------------------------------------1 j2 Y1 y' d U9 G
bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource4 Z) |+ L' g4 a% i% c/ I- T4 x# d
* Q9 Y5 x# _0 g1 l# C5 ?; \4 R( \ Resource Declaration:
, [ E% m/ O; n9 }/ [% e ---------------------/ n ?2 ~; I9 v& H3 u3 B' d
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb
) z$ L" E, v1 {2 n! l C $ o( D; r' h e" f W J
51: rails_migration "gitlab-rails" do) d% B3 l: F) y Y
52: rake_task 'gitlab:db:configure'
2 ~; ^/ A6 K+ {# L* J8 G, h2 Z! E* e 53: logfile_prefix 'gitlab-rails-db-migrate'+ Z/ r- ~7 M; A+ X) w
54: helper migration_helper1 @2 e; a% U5 X7 t# i
55: , b# o/ m+ M5 ?
56: environment env_variables) H5 q( f+ q& u" [# M9 R
57: dependent_services dependent_services
: N7 J5 v ^+ }' Y' M8 i* ` 58: notifies :run, "execute[clear the gitlab-rails cache]", :immediately) |- d& `$ o1 O, M( e9 o+ `4 h p
59: notifies :run, "ruby_block[check remote PG version]", :immediately
# q1 @- h) l& E8 G$ B, v9 n 60: 1 D+ q- R& W5 A: |
61: only_if { migration_helper.attributes_node['auto_migrate'] }
- m9 r& Q% h3 w3 K4 @2 W) i 62: end: v- T% c- H Z+ a
) ]5 U+ e+ K5 I( L Compiled Resource:
1 \& \& W, O; K% c) E1 i+ N! D ------------------ ~: b& A4 O$ e. D8 X
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:51:in `from_file'
, `" h, V( z% J9 W* `& |
( ~5 z0 p. c ~' [ rails_migration("gitlab-rails") do2 Q: o( Z6 I, L
action [:run]
) [3 U. y5 j$ o* n default_guard_interpreter :default5 `) |( {% R, |+ ?
declared_type :rails_migration
! u* S j, M+ ]- o2 \7 j- A cookbook_name "gitlab"2 i3 T. t, b" G% P$ y$ ^+ Y7 K
recipe_name "database_migrations"
( ?8 Q9 p3 A" ^, T" d rake_task "gitlab:db:configure"
# z9 m9 G3 G9 ~% ^ logfile_prefix "gitlab-rails-db-migrate"
' Q% M- |+ F, o helper "*sensitive value suppressed*"
( W% l5 w5 L; c! ~3 ^ environment "*sensitive value suppressed*"& n' F! Q( F" D! h0 y. f
dependent_services ["runit_service[puma]", "sidekiq_service[sidekiq]"]) r% v3 j0 j. o& Y N6 V
only_if { #code block }
# a8 ]# D& i1 q8 o& D* n Y& c end$ C7 |4 ^5 K6 [# H) z. |
: D. q. Q2 v5 g. e9 U System Info:# N, N; y* M" Y2 r* m5 B
------------8 c# G5 s% N. d- }/ W
chef_version=15.17.4
+ v7 d% H9 v7 S) `6 I platform=ubuntu# J! a0 ^# \" b! p3 O
platform_version=20.04
" X2 ?) O9 q8 `9 d% ^* a ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]- P3 X2 I. \2 K7 B
program_name=/opt/gitlab/embedded/bin/chef-client
% D/ e3 H. D( D* B executable=/opt/gitlab/embedded/bin/chef-client
0 H* l( x* \+ C0 h5 Z: E& g5 D: t
2 ]7 A: g ]0 D9 \8 p) V4 cRunning handlers:- H. p; }. A y- e
There was an error running gitlab-ctl reconfigure:3 L; V& {( |8 t# ?, S
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource
8 R+ e/ A9 n7 f) k! t7 X$ P1 GRunning handlers complete
& U/ i( B( q+ c" HChef Infra Client failed. 0 resources updated in 21 seconds网上一查，发现和上次碰到的 Gitlab更新设置总是500错误 问题一样，又是Gitlab官方的锅。但是这次解决问题比之前更麻烦，步骤为：9 z3 R, p6 i. C3 d( E
1. 运行 gitlab-rake db:migrate，出现错误没关系，中间会输出一行用于终止后台任务的命令；
0 B4 n7 K& D6 s2. 复制这条命令或者上一步控制台输出的指令： gitlab-rake gitlab:background_migrations:finalize[ProjectNamespaces::BackfillProjectNamespaces,projects,id,'[null\,"up"]']。运行这条指令可能会失败，如果失败，请重新运行上面的指令，成功说明问题解决了；
! _! D% p/ h0 M# k3. 再次运行 gitlab-rake db:migrate，确认不会再报错；8 b2 n' r6 t3 M/ [4 S' B2 G
4. 运行 gitlab-ctl reconfigure;" S/ t6 b/ }7 X, O+ O7 h6 V# r
5. 重启Gitlab：gitlab-ctl restart。
( C( R8 z8 ? m2 L需要注意的是，这个bug是在Gitlab-CE 14.10.0中引入，在14.10.1版本被修复。如果运行过reconfigure但没解决错误，是无法直接升级到14.10.1版本的。) G; x# y# l. g) a( o3 j: h% g
参考1. Gitlab-ctl reconfigure doesn’t work after gitlab omnibus updated
* h* ^3 R" L7 t( m* ^2. gitlab-ctl reconfigure failing due to migration issue- l1 I$ t6 k8 D5 m: S2 g9 x R U
AD：【加速器推荐】 搬瓦工官方代理服务Just My Socks，高速CN2 GIA线路流畅访问外网，被墙自动更换IP打赏赞(2) |
|
发表于 2022-9-11 20:40:14
