Wikitech labswiki https://wikitech.wikimedia.org/wiki/Main_Page MediaWiki 1.44.0-wmf.6 first-letter Media Special Talk User User talk Wikitech Wikitech talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk Obsolete Obsolete talk OfficeIT OfficeIT talk Tool Tool talk Nova Resource Nova Resource Talk Heira Heira Talk TimedText TimedText talk Module Module talk 498 3086 2253930 2252570 2024-12-12T10:47:06Z Stashbot 7414 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-5 2253930 wikitext text/x-wiki g081wp3dvkisa6wo8zx4cdp456aw5s0 2253932 2253930 2024-12-12T10:52:53Z Stashbot 7414 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-5 2253932 wikitext text/x-wiki qg4pot8gzj1nph22t0j6sovzh8w26b9 0 4108 2253607 2253535 2024-12-11T12:18:27Z ScheduleDeploymentBot 37566 Add [[gerrit:1102283]] to Thursday, December 12 UTC morning backport window 2253607 wikitext text/x-wiki nbiehls3qzssozbr3bcf8adibjzr6cn 2253810 2253607 2024-12-11T17:24:05Z TNTBot 39092 [supervised test] Updating deployment status 2253810 wikitext text/x-wiki skn353lwfilf2m8w0tf1hus8q8k4vp2 2253820 2253810 2024-12-11T17:40:08Z ScheduleDeploymentBot 37566 Add [[gerrit:1101937]] to Wednesday, December 11 UTC late backport window 2253820 wikitext text/x-wiki lxpj9vqvfzzw8rumha05ihw5flcfi83 2253858 2253820 2024-12-11T20:10:53Z ScheduleDeploymentBot 37566 Add [[gerrit:1102205]] to Wednesday, December 11 UTC late backport window 2253858 wikitext text/x-wiki 2azg78gzmn7fu5u3loq5ip21yun7u77 12 5747 2253885 2249675 2024-12-12T00:13:06Z Lucas Werkmeister 8093 kick out --backend=kubernetes, no longer needed since the [[News/2024 Toolforge Grid Engine deprecation|grid engine deprecation]] finished 2253885 wikitext text/x-wiki fvufmhmceq3yioo9xcbkogcu37288cq 0 7919 2253593 2253591 2024-12-11T12:05:00Z Stashbot 7414 mvolz@deploy2002: helmfile [staging] START helmfile.d/services/citoid: apply 2253593 wikitext text/x-wiki s53ult12vv6g4ht0hjcloiqz6t76y66 2253594 2253593 2024-12-11T12:05:03Z Stashbot 7414 mvolz@deploy2002: helmfile [staging] DONE helmfile.d/services/citoid: apply 2253594 wikitext text/x-wiki 0j8lj7ssybkgsgre7lnyy6y9mdl0x9e 2253595 2253594 2024-12-11T12:08:12Z Stashbot 7414 mvolz@deploy2002: helmfile [staging] START helmfile.d/services/citoid: apply 2253595 wikitext text/x-wiki 72q7xns0tfx6ypg2rte4jb3jlhv3yz9 2253596 2253595 2024-12-11T12:08:38Z Stashbot 7414 mvolz@deploy2002: helmfile [staging] DONE helmfile.d/services/citoid: apply 2253596 wikitext text/x-wiki k46x9ke60h14inbbc4cgclc96z74o4e 2253597 2253596 2024-12-11T12:11:04Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich: apply 2253597 wikitext text/x-wiki st6joggx75fe5a3vepywhwg48genwmw 2253598 2253597 2024-12-11T12:11:07Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich: apply 2253598 wikitext text/x-wiki 6whsao55o44upyqg0fkhl9r1oo83oxz 2253599 2253598 2024-12-11T12:11:30Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich: apply 2253599 wikitext text/x-wiki e04myg7amwxqmbplaf9u66w256imwlr 2253600 2253599 2024-12-11T12:11:36Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich: apply 2253600 wikitext text/x-wiki l1oza0md0ew5v2kpjc017dok8u2q7hv 2253601 2253600 2024-12-11T12:11:57Z Stashbot 7414 mvolz@deploy2002: helmfile [codfw] START helmfile.d/services/citoid: apply 2253601 wikitext text/x-wiki rr1ivbbjoazhjih0wkl2rta46g1ts46 2253602 2253601 2024-12-11T12:12:43Z Stashbot 7414 mvolz@deploy2002: helmfile [codfw] DONE helmfile.d/services/citoid: apply 2253602 wikitext text/x-wiki tnra8mye7rn17bj92dc0z22rlah4ccv 2253603 2253602 2024-12-11T12:12:58Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich: apply 2253603 wikitext text/x-wiki 70kj4tokdbymkmq1t492tt3j1r40435 2253604 2253603 2024-12-11T12:13:03Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-dump-rev-content-reconcile-enrich: apply 2253604 wikitext text/x-wiki 24yajeatgscytt520wiraqafh87b5y2 2253605 2253604 2024-12-11T12:14:40Z Stashbot 7414 mvolz@deploy2002: helmfile [eqiad] START helmfile.d/services/citoid: apply 2253605 wikitext text/x-wiki 5flyye5x5h23djphk527r1f6jl879cr 2253606 2253605 2024-12-11T12:15:13Z Stashbot 7414 mvolz@deploy2002: helmfile [eqiad] DONE helmfile.d/services/citoid: apply 2253606 wikitext text/x-wiki 08x8x00psdoly9whp3lz55uf6mp966h 2253608 2253606 2024-12-11T12:47:36Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: Upgrade GitLab Replica to new version 2253608 wikitext text/x-wiki 6f8nj9844ndlhqqm9a8pjvhg4rg5pty 2253609 2253608 2024-12-11T12:54:22Z Stashbot 7414 kartik@deploy2002: helmfile [staging] START helmfile.d/services/cxserver: apply 2253609 wikitext text/x-wiki d5el0b6lyfz6sxy7hk1mtqi2o2tnec9 2253610 2253609 2024-12-11T12:54:37Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1003.wikimedia.org with reason: Upgrade GitLab Replica to new version 2253610 wikitext text/x-wiki 7k7ua8m7c6z52frp6y0w6nd91ijb6bt 2253611 2253610 2024-12-11T12:54:45Z Stashbot 7414 kartik@deploy2002: helmfile [staging] DONE helmfile.d/services/cxserver: apply 2253611 wikitext text/x-wiki m4d6qwn99j587ujdjggcq1mvzjdiwtm 2253612 2253611 2024-12-11T12:57:01Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1004.wikimedia.org with reason: Upgrade GitLab Replica to new version 2253612 wikitext text/x-wiki nusprfi7kr35a1wlltuvgnhtyzp59fb 2253613 2253612 2024-12-11T12:59:46Z Stashbot 7414 kartik@deploy2002: helmfile [codfw] START helmfile.d/services/cxserver: apply 2253613 wikitext text/x-wiki d627w9oseqkiyy45xq13bot86sro4cb 2253614 2253613 2024-12-11T13:00:11Z Stashbot 7414 kartik@deploy2002: helmfile [codfw] DONE helmfile.d/services/cxserver: apply 2253614 wikitext text/x-wiki 0b3sh5fqkxqgx8ciempks1l20vgqrli 2253615 2253614 2024-12-11T13:00:39Z Stashbot 7414 kartik@deploy2002: helmfile [eqiad] START helmfile.d/services/cxserver: apply 2253615 wikitext text/x-wiki bt511gek5563hc2m2cw4kyajtxg5cl4 2253616 2253615 2024-12-11T13:01:11Z Stashbot 7414 kartik@deploy2002: helmfile [eqiad] DONE helmfile.d/services/cxserver: apply 2253616 wikitext text/x-wiki kx6nuc8scd7nqnt554cukwuqm6s4v7s 2253617 2253616 2024-12-11T13:04:16Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.gitlab.upgrade (exit_code=0) on GitLab host gitlab1004.wikimedia.org with reason: Upgrade GitLab Replica to new version 2253617 wikitext text/x-wiki fdrlau67nuox4jvt99d6okfx85fd45s 2253618 2253617 2024-12-11T13:04:25Z Stashbot 7414 kart_: Updated cxserver to 2024-12-10-132417-production (T369815) 2253618 wikitext text/x-wiki sa2nwcuhyazuvo480h76nekbf15u8lo 2253619 2253618 2024-12-11T13:08:04Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2253619 wikitext text/x-wiki kwyvn6ul6w4xvtp7jg6uyvxz2c1ysnp 2253620 2253619 2024-12-11T13:09:20Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2253620 wikitext text/x-wiki crs86m5vbptdqkogaxxep4utj6dcreh 2253621 2253620 2024-12-11T13:13:43Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.k8s.pool-depool-node depool for host kubernetes[2017,2021-2022,2024].codfw.wmnet 2253621 wikitext text/x-wiki nowj9lv5kjc958eee3w4dr6svhe56lk 2253622 2253621 2024-12-11T13:17:56Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2253622 wikitext text/x-wiki mxfh2puvvx5fcy1lep15v3ce67c2xh4 2253623 2253622 2024-12-11T13:18:02Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2253623 wikitext text/x-wiki bgfqb80uofs082wraqx5t38cwf8rvwr 2253624 2253623 2024-12-11T13:18:38Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubernetes[2017,2021-2022,2024].codfw.wmnet 2253624 wikitext text/x-wiki eoiyr7k9rc06zo4oac7ir7cwn094e3g 2253625 2253624 2024-12-11T13:19:30Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2253625 wikitext text/x-wiki g1d3q19v8utn2p43s640ntm322kucvg 2253626 2253625 2024-12-11T13:21:02Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.rename from kubernetes2017 to wikikube-worker2184 2253626 wikitext text/x-wiki h7kg4q87xo1thu2yv3k6mje7qc4vyie 2253627 2253626 2024-12-11T13:21:22Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253627 wikitext text/x-wiki m4cf543tmisaxsadbrqrjipcu5jl4vr 2253628 2253627 2024-12-11T13:25:20Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2017 to wikikube-worker2184 - jelto@cumin1002" 2253628 wikitext text/x-wiki dqif3t9pntysnzwund2au2w1yffs9wf 2253629 2253628 2024-12-11T13:25:42Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2017 to wikikube-worker2184 - jelto@cumin1002" 2253629 wikitext text/x-wiki 4wjlxcjwag21okzpisf4pdpga3anqkn 2253630 2253629 2024-12-11T13:25:44Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253630 wikitext text/x-wiki ttqt1bjkrsfjnlmbga0j2um3j5e5pcj 2253631 2253630 2024-12-11T13:25:46Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2184 2253631 wikitext text/x-wiki pdobhpmjsbbnba05oj7pzcl4bsvlgmu 2253632 2253631 2024-12-11T13:26:00Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2184 2253632 wikitext text/x-wiki mf4ovh76byh2nx8yvt7z1qesj6xm6u5 2253633 2253632 2024-12-11T13:26:16Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply 2253633 wikitext text/x-wiki iavvg2tl8ttwbxx1vo51mr1rvkm1iej 2253634 2253633 2024-12-11T13:26:38Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from kubernetes2017 to wikikube-worker2184 2253634 wikitext text/x-wiki 08kmazx8aa0hbycmh2a5x1izxwjfa6b 2253635 2253634 2024-12-11T13:27:17Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply 2253635 wikitext text/x-wiki a4nmg3dx73kiyetstz2k6ohfyzdeb92 2253636 2253635 2024-12-11T13:27:56Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.rename from kubernetes2021 to wikikube-worker2185 2253636 wikitext text/x-wiki d2s9zxwgirkbei97ug3n0bgndga0z70 2253637 2253636 2024-12-11T13:28:17Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253637 wikitext text/x-wiki 364tytsctnmvf8z43m35oc187eto21h 2253638 2253637 2024-12-11T13:31:51Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2021 to wikikube-worker2185 - jelto@cumin1002" 2253638 wikitext text/x-wiki se9h304gzyii6nr8l1ptg0jdnnsmvdp 2253639 2253638 2024-12-11T13:32:44Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2021 to wikikube-worker2185 - jelto@cumin1002" 2253639 wikitext text/x-wiki 1eddasva6no7l3xzm2r8zwgsmum50ib 2253640 2253639 2024-12-11T13:32:46Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253640 wikitext text/x-wiki 1cqcjkwlk90lv7y63avqo3twq5h2k1x 2253641 2253640 2024-12-11T13:32:48Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2185 2253641 wikitext text/x-wiki isodhv3t495aop07zxolt9sasqakvxh 2253642 2253641 2024-12-11T13:32:56Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2185 2253642 wikitext text/x-wiki 8pld49sjy2nm0uivw8frgq3h5h908qm 2253643 2253642 2024-12-11T13:33:35Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from kubernetes2021 to wikikube-worker2185 2253643 wikitext text/x-wiki dkangy2eg3znjea1njw5c94zoggdxd6 2253644 2253643 2024-12-11T13:34:20Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.rename from kubernetes2022 to wikikube-worker2186 2253644 wikitext text/x-wiki b7kqkma56p6bveipqie0bwunvpdnv7h 2253645 2253644 2024-12-11T13:34:41Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253645 wikitext text/x-wiki jtz1hm6z6mv5dprz2if6xh73ozjyya0 2253647 2253645 2024-12-11T13:39:31Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2022 to wikikube-worker2186 - jelto@cumin1002" 2253647 wikitext text/x-wiki nner338cfbq4jxiv8mzu4nbt8u3fqpv 2253648 2253647 2024-12-11T13:40:44Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2022 to wikikube-worker2186 - jelto@cumin1002" 2253648 wikitext text/x-wiki pxhjfi6ojqiye6029oo9lwl16kh1klb 2253649 2253648 2024-12-11T13:40:46Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253649 wikitext text/x-wiki 2ctpg06uhugkfodeq4bqiyfu4btkwbc 2253650 2253649 2024-12-11T13:40:48Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2186 2253650 wikitext text/x-wiki epxie2m6gota7sxxv6v10up2wx6nwvb 2253651 2253650 2024-12-11T13:40:56Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2186 2253651 wikitext text/x-wiki 4otknpjoj1bhmz758rboidvyzthztjd 2253652 2253651 2024-12-11T13:41:35Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from kubernetes2022 to wikikube-worker2186 2253652 wikitext text/x-wiki rpeujfb25dtm1v5e53w2qneq86pdss9 2253653 2253652 2024-12-11T13:42:24Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.rename from kubernetes2024 to wikikube-worker2187 2253653 wikitext text/x-wiki sh4a1yt4yrrpnvqeai35l83sqwsvfa7 2253654 2253653 2024-12-11T13:42:45Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253654 wikitext text/x-wiki 7f5gbp2uudr3n2tzsfte473xnzitbtg 2253655 2253654 2024-12-11T13:53:14Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2024 to wikikube-worker2187 - jelto@cumin1002" 2253655 wikitext text/x-wiki 7v8vt0efo5eh290twfym7sl801rym61 2253656 2253655 2024-12-11T13:53:36Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming kubernetes2024 to wikikube-worker2187 - jelto@cumin1002" 2253656 wikitext text/x-wiki jbwsc7dvvboajf5bto2ibsgw3mze8u3 2253657 2253656 2024-12-11T13:53:37Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253657 wikitext text/x-wiki 4614a1kfj6xcszcrt7ksvcawyht9ohf 2253658 2253657 2024-12-11T13:53:40Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2187 2253658 wikitext text/x-wiki 44wy724k6r6ak10c90j6spw8n4qe0ul 2253659 2253658 2024-12-11T13:53:52Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2187 2253659 wikitext text/x-wiki m8jxvwkkmkzhog5ohs5zma4hgaif3sg 2253660 2253659 2024-12-11T13:54:31Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from kubernetes2024 to wikikube-worker2187 2253660 wikitext text/x-wiki tdmdip2j0ine4751q6enpwd591jv9fo 2253661 2253660 2024-12-11T13:57:23Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.wipe-cache wikikube-worker2184.codfw.wmnet wikikube-worker2185.codfw.wmnet wikikube-worker2186.codfw.wmnet wikikube-worker2187.codfw.wmnet on all recursors 2253661 wikitext text/x-wiki h22co0jmppj7ihk6ocgk9wjcw53zclt 2253662 2253661 2024-12-11T13:57:27Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker2184.codfw.wmnet wikikube-worker2185.codfw.wmnet wikikube-worker2186.codfw.wmnet wikikube-worker2187.codfw.wmnet on all recursors 2253662 wikitext text/x-wiki fn5e8bj40dn70wnt5r73whx5nqobb4z 2253663 2253662 2024-12-11T14:00:27Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.reimage for host wikikube-worker2184.codfw.wmnet with OS bookworm 2253663 wikitext text/x-wiki s4dz16f4a6837uue0dy92e5a49d1b8l 2253664 2253663 2024-12-11T14:00:30Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.reimage for host wikikube-worker2185.codfw.wmnet with OS bookworm 2253664 wikitext text/x-wiki n0hy6afa0d5t4tvr31rsr9ovrv07lad 2253665 2253664 2024-12-11T14:00:32Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.reimage for host wikikube-worker2186.codfw.wmnet with OS bookworm 2253665 wikitext text/x-wiki 13uxyhu87o8gyntcezs232dutc8blm1 2253666 2253665 2024-12-11T14:00:34Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.reimage for host wikikube-worker2187.codfw.wmnet with OS bookworm 2253666 wikitext text/x-wiki shlt6wlsbhblv8ekbmwnchd4t2zmjtm 2253667 2253666 2024-12-11T14:00:38Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.move-vlan for host wikikube-worker2184 2253667 wikitext text/x-wiki rzq15rmpf4jaoeyxsszk4xvuznzke5e 2253668 2253667 2024-12-11T14:00:41Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.move-vlan for host wikikube-worker2186 2253668 wikitext text/x-wiki 7cn4t3une85l5tpp7j8lje08kzg34uf 2253669 2253668 2024-12-11T14:00:43Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.move-vlan for host wikikube-worker2187 2253669 wikitext text/x-wiki ofp9koiru3311owsl7thgiyg0kpksub 2253670 2253669 2024-12-11T14:00:51Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253670 wikitext text/x-wiki 8etiylrpcjtjyo7crx19p4ojzgptpzu 2253671 2253670 2024-12-11T14:04:51Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker2186 - jelto@cumin1002" 2253671 wikitext text/x-wiki nq62n3rzzb5hdimq1rujikyo0moupvp 2253672 2253671 2024-12-11T14:04:55Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker2186 - jelto@cumin1002" 2253672 wikitext text/x-wiki fc2qw3ixvswr4a8dev44gjrhbpz0v4b 2253673 2253672 2024-12-11T14:04:57Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253673 wikitext text/x-wiki auo2pobigycd7b1hgmvauhs1qxlfgue 2253674 2253673 2024-12-11T14:04:59Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.wipe-cache wikikube-worker2186.codfw.wmnet 180.48.192.10.in-addr.arpa 0.8.1.0.8.4.0.0.2.9.1.0.0.1.0.0.4.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253674 wikitext text/x-wiki gt4ubhdsebelgcqbvoa1rnb455vj678 2253675 2253674 2024-12-11T14:05:01Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker2186.codfw.wmnet 180.48.192.10.in-addr.arpa 0.8.1.0.8.4.0.0.2.9.1.0.0.1.0.0.4.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253675 wikitext text/x-wiki ra160kbx364quvlw1ckh19odcvx62di 2253676 2253675 2024-12-11T14:05:03Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2186 2253676 wikitext text/x-wiki o4fv8v4l5k7h9gnctxk5tczov7seyp1 2253677 2253676 2024-12-11T14:05:10Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2186 2253677 wikitext text/x-wiki d68nqf3z8kds57x6iydbo00z404hkc1 2253678 2253677 2024-12-11T14:05:12Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker2186 2253678 wikitext text/x-wiki 7ccunflf6imh96u3ucxioo4dhvrugi8 2253679 2253678 2024-12-11T14:05:30Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253679 wikitext text/x-wiki spiokxx7lpprc127wgf8igen6hqpypo 2253680 2253679 2024-12-11T14:05:42Z Stashbot 7414 samtar@deploy2002: Started scap sync-world: Backport for [[gerrit:1098045|Remove feature flag which controls wikibase item link location (T377809)]] 2253680 wikitext text/x-wiki lu4z9yfvy28dpxfyncmeuwdytyg8tj5 2253681 2253680 2024-12-11T14:06:12Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2253681 wikitext text/x-wiki pkep3xzbiwnpiwrrzgbd6qx2wly52cn 2253682 2253681 2024-12-11T14:06:36Z Stashbot 7414 btullis@deploy1003: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2253682 wikitext text/x-wiki a037xf61zb7k9n9xnyqb2hwmh0y1vvb 2253683 2253682 2024-12-11T14:07:52Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253683 wikitext text/x-wiki kbfh4iu9vkv8o3xbix4ftq1tefbywv1 2253684 2253683 2024-12-11T14:07:55Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.wipe-cache wikikube-worker2184.codfw.wmnet 41.32.192.10.in-addr.arpa 1.4.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253684 wikitext text/x-wiki oza9zgp5cuzdeof2cvysbesn6xo5yw4 2253685 2253684 2024-12-11T14:07:57Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker2184.codfw.wmnet 41.32.192.10.in-addr.arpa 1.4.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253685 wikitext text/x-wiki ical9k38w7ds7e2itee7e50xbwwaxga 2253686 2253685 2024-12-11T14:07:59Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2184 2253686 wikitext text/x-wiki 7megnc4jg5c20jbn9coz40legg3grkj 2253687 2253686 2024-12-11T14:08:26Z Stashbot 7414 btullis@cumin1002: START - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash rolling restart_daemons on A:apifeatureusage 2253687 wikitext text/x-wiki p6wlbnm5qlcbw8ex7x70769t9hsxsdl 2253688 2253687 2024-12-11T14:08:33Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253688 wikitext text/x-wiki 3r7ycc499kq3wi9yd5ow52jj9fcbxlo 2253689 2253688 2024-12-11T14:09:01Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2184 2253689 wikitext text/x-wiki glv1wfox4egv40j6vhkw8b895qjwjnf 2253690 2253689 2024-12-11T14:09:03Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker2184 2253690 wikitext text/x-wiki obd0ek9o9x2lwmjradqfk1nekprl7hl 2253691 2253690 2024-12-11T14:09:26Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.move-vlan for host wikikube-worker2185 2253691 wikitext text/x-wiki 27rb8y7y0t8yrt5ormskah5xdbg4kjq 2253692 2253691 2024-12-11T14:09:29Z Stashbot 7414 samtar@deploy2002: samtar, joelyrookewmde: Backport for [[gerrit:1098045|Remove feature flag which controls wikibase item link location (T377809)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) 2253692 wikitext text/x-wiki 07l5ivtxqb0wdqy59akreelaqc6xaxn 2253693 2253692 2024-12-11T14:10:57Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253693 wikitext text/x-wiki p8gg51el6o0bc3fw32fu5ew75trh91j 2253694 2253693 2024-12-11T14:10:59Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.wipe-cache wikikube-worker2187.codfw.wmnet 87.48.192.10.in-addr.arpa 7.8.0.0.8.4.0.0.2.9.1.0.0.1.0.0.4.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253694 wikitext text/x-wiki 90lfujz8ppl7fv78mofou1c0609vxuv 2253695 2253694 2024-12-11T14:11:01Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.netbox 2253695 wikitext text/x-wiki q1h09wsb30u7olxgrfakbwrpyqu5l6x 2253696 2253695 2024-12-11T14:11:04Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker2187.codfw.wmnet 87.48.192.10.in-addr.arpa 7.8.0.0.8.4.0.0.2.9.1.0.0.1.0.0.4.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253696 wikitext text/x-wiki rgwfjirxrf4fst9q5egufmi8gc7xsv6 2253697 2253696 2024-12-11T14:11:06Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2187 2253697 wikitext text/x-wiki r8vvnzfla327u8halw7kgnyarfnrr2g 2253698 2253697 2024-12-11T14:11:08Z Stashbot 7414 btullis@cumin1002: END (PASS) - Cookbook sre.apifeatureusage.roll-restart-reboot-logstash (exit_code=0) rolling restart_daemons on A:apifeatureusage 2253698 wikitext text/x-wiki t9dq1uqrm69igixv7y88cz4dl7hcd27 2253699 2253698 2024-12-11T14:11:12Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2187 2253699 wikitext text/x-wiki scorwey45p7w1dbwfyimpz06893g7cm 2253700 2253699 2024-12-11T14:11:14Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker2187 2253700 wikitext text/x-wiki q148lufjfudigmi7cbn21h6dobqyn4g 2253701 2253700 2024-12-11T14:12:01Z Stashbot 7414 samtar@deploy2002: samtar, joelyrookewmde: Continuing with sync 2253701 wikitext text/x-wiki 0hbrlog2jnbgs82z437ajvoegucuk94 2253702 2253701 2024-12-11T14:14:29Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker2185 - jelto@cumin1002" 2253702 wikitext text/x-wiki h95t2mdq3p9z981thw6d5fpsdfbg64y 2253703 2253702 2024-12-11T14:14:32Z Stashbot 7414 jhancock@cumin2002: START - Cookbook sre.hosts.reimage for host es1043.eqiad.wmnet with OS bookworm 2253703 wikitext text/x-wiki 4y0pba6uy0i7bd8y17d8w3qj1dr8mm8 2253704 2253703 2024-12-11T14:14:34Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker2185 - jelto@cumin1002" 2253704 wikitext text/x-wiki b5z8ckd28ngmwv4eff3telay8whem99 2253705 2253704 2024-12-11T14:14:36Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253705 wikitext text/x-wiki 2yujt7sirjq1d3bzgq6lea9tdp2rt9q 2253706 2253705 2024-12-11T14:14:38Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.dns.wipe-cache wikikube-worker2185.codfw.wmnet 89.32.192.10.in-addr.arpa 9.8.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253706 wikitext text/x-wiki 6scxrcx1kqx381kqk1ctww40w4sq5ef 2253707 2253706 2024-12-11T14:14:40Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker2185.codfw.wmnet 89.32.192.10.in-addr.arpa 9.8.0.0.2.3.0.0.2.9.1.0.0.1.0.0.3.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors 2253707 wikitext text/x-wiki cv7e9joj68dlo8ro48w6vvitadx63gf 2253708 2253707 2024-12-11T14:14:42Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2185 2253708 wikitext text/x-wiki ofdn8e08iebqudu1r0jhul2q6nbhdwa 2253709 2253708 2024-12-11T14:14:44Z Stashbot 7414 btullis@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on archiva1002.wikimedia.org with reason: Adding new disk 2253709 wikitext text/x-wiki 5tmemu4fuhznrnlhnogpk5ssocidwh5 2253710 2253709 2024-12-11T14:14:55Z Stashbot 7414 btullis@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on archiva1002.wikimedia.org with reason: Adding new disk 2253710 wikitext text/x-wiki ryr5hielib4pz2fm38u3047cm3a8km8 2253711 2253710 2024-12-11T14:15:26Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2185 2253711 wikitext text/x-wiki b1ijvvvc84twv5840lj9vh7ca23kzs1 2253712 2253711 2024-12-11T14:15:28Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker2185 2253712 wikitext text/x-wiki f5sg3ckfzx5qv4vg549oquzrc47fsw3 2253713 2253712 2024-12-11T14:18:13Z Stashbot 7414 samtar@deploy2002: Finished scap sync-world: Backport for [[gerrit:1098045|Remove feature flag which controls wikibase item link location (T377809)]] (duration: 12m 32s) 2253713 wikitext text/x-wiki 97d3p4tx2k9bgfcb36li5ycyyeuily6 2253714 2253713 2024-12-11T14:19:39Z Stashbot 7414 elukey@cumin1002: START - Cookbook sre.hosts.reimage for host ms-be2085.codfw.wmnet with OS bullseye 2253714 wikitext text/x-wiki tuc5l7tel1ett8nr4m18kgr1rol3lzg 2253715 2253714 2024-12-11T14:19:47Z Stashbot 7414 samtar@deploy2002: Started scap sync-world: Backport for [[gerrit:1102141|ve.ui.CodeMirror.v6: Use plugin callback to load the actual module (T374072)]], [[gerrit:1102142|styles: Avoid misalignments when line numbering is disabled (T381714)]] 2253715 wikitext text/x-wiki 4e3wb5pbefogyrc46u4i45jpv711rg0 2253716 2253715 2024-12-11T14:22:17Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2186.codfw.wmnet with reason: host reimage 2253716 wikitext text/x-wiki gq6rbizcwu6uitambbpgjo5mubwgznn 2253717 2253716 2024-12-11T14:23:02Z Stashbot 7414 samtar@deploy2002: samtar, func: Backport for [[gerrit:1102141|ve.ui.CodeMirror.v6: Use plugin callback to load the actual module (T374072)]], [[gerrit:1102142|styles: Avoid misalignments when line numbering is disabled (T381714)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) 2253717 wikitext text/x-wiki 77vnk07cefrckv9xny9y3r4rxvhefur 2253718 2253717 2024-12-11T14:25:05Z Stashbot 7414 samtar@deploy2002: samtar, func: Continuing with sync 2253718 wikitext text/x-wiki g2w8tdbewdgq7kssdqyny9zi1p7t5d5 2253720 2253718 2024-12-11T14:25:53Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2186.codfw.wmnet with reason: host reimage 2253720 wikitext text/x-wiki inpb2p38ozr2x7w4lt0kg24i3n2o7sd 2253721 2253720 2024-12-11T14:25:59Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2184.codfw.wmnet with reason: host reimage 2253721 wikitext text/x-wiki cp7blb96hh877rmldpsu4fjgp7gxc8l 2253722 2253721 2024-12-11T14:28:29Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2187.codfw.wmnet with reason: host reimage 2253722 wikitext text/x-wiki f0cf1m5avhai5a6yjlz6j6talymqndx 2253723 2253722 2024-12-11T14:28:37Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2184.codfw.wmnet with reason: host reimage 2253723 wikitext text/x-wiki 9md7jerc4jtknfrirsobtz4tqnoapge 2253724 2253723 2024-12-11T14:30:29Z Stashbot 7414 samtar@deploy2002: Finished scap sync-world: Backport for [[gerrit:1102141|ve.ui.CodeMirror.v6: Use plugin callback to load the actual module (T374072)]], [[gerrit:1102142|styles: Avoid misalignments when line numbering is disabled (T381714)]] (duration: 10m 42s) 2253724 wikitext text/x-wiki 652ugov6ugq8ql2f5cv8wmlfkvcahmh 2253725 2253724 2024-12-11T14:31:40Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2187.codfw.wmnet with reason: host reimage 2253725 wikitext text/x-wiki evbwef1yvr7p0gdu34cxz6zss9ttlzl 2253726 2253725 2024-12-11T14:32:29Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2185.codfw.wmnet with reason: host reimage 2253726 wikitext text/x-wiki di7dbt5w6owls7d0i9c7xe5c8f6fllb 2253727 2253726 2024-12-11T14:32:57Z Stashbot 7414 samtar@deploy2002: Started scap sync-world: Backport for [[gerrit:1101577|Add Atieno's public key]] 2253727 wikitext text/x-wiki q345g4tb5792vrgi8p16fhsl02ded53 2253728 2253727 2024-12-11T14:34:07Z Stashbot 7414 elukey@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2085.codfw.wmnet with reason: host reimage 2253728 wikitext text/x-wiki nggpdqstv0jq37roczghkaec94qzn04 2253730 2253728 2024-12-11T14:35:46Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2185.codfw.wmnet with reason: host reimage 2253730 wikitext text/x-wiki 3smr2902ajafeohgg8f6tkutchxjv3x 2253732 2253730 2024-12-11T14:36:18Z Stashbot 7414 samtar@deploy2002: arlolra, samtar: Backport for [[gerrit:1101577|Add Atieno's public key]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) 2253732 wikitext text/x-wiki lsfk4eg84anzcwj2cf6npxql20xkp3j 2253733 2253732 2024-12-11T14:36:22Z Stashbot 7414 samtar@deploy2002: arlolra, samtar: Continuing with sync 2253733 wikitext text/x-wiki th399vhgxrowkpyqy1nprnsc0cyzdtb 2253736 2253733 2024-12-11T14:39:19Z Stashbot 7414 elukey@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2085.codfw.wmnet with reason: host reimage 2253736 wikitext text/x-wiki hm0ab4wpphua32vrgp2zh5bmiod7n37 2253737 2253736 2024-12-11T14:41:45Z Stashbot 7414 samtar@deploy2002: Finished scap sync-world: Backport for [[gerrit:1101577|Add Atieno's public key]] (duration: 08m 47s) 2253737 wikitext text/x-wiki 6w6f3bxnh4bjz6jmp2enzdjrr5jf7eu 2253738 2253737 2024-12-11T14:42:08Z Stashbot 7414 TheresNoTime: done UTC afternoon backport window 2253738 wikitext text/x-wiki d6262sbr6ykpz502p8v5oiljlav2ren 2253739 2253738 2024-12-11T14:45:45Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2186.codfw.wmnet with OS bookworm 2253739 wikitext text/x-wiki 3e4df3rpch6rdy4fvdrfl4hob2uh4xt 2253740 2253739 2024-12-11T14:48:32Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2184.codfw.wmnet with OS bookworm 2253740 wikitext text/x-wiki km8dccozbnjw3o1hlqulvi37e4kkwv7 2253741 2253740 2024-12-11T14:51:48Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2187.codfw.wmnet with OS bookworm 2253741 wikitext text/x-wiki 5pi02h1bcin6wsrap47g9xj9x1ix6rn 2253744 2253741 2024-12-11T14:55:37Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2185.codfw.wmnet with OS bookworm 2253744 wikitext text/x-wiki i4mz6q5lmezalrd34zcumcrxuganywj 2253745 2253744 2024-12-11T14:56:25Z Stashbot 7414 jelto: homer 'lsw1-d5-codfw*' commit 'T377877' 2253745 wikitext text/x-wiki 3v0ilnnctq36ets7cydwh88qj4mx563 2253746 2253745 2024-12-11T14:56:57Z Stashbot 7414 klausman@deploy2002: helmfile [staging] START helmfile.d/services/api-gateway: apply 2253746 wikitext text/x-wiki m9iutqhmq3slr5m5h3zgznldq9pgyzv 2253747 2253746 2024-12-11T14:57:26Z Stashbot 7414 klausman@deploy2002: helmfile [staging] DONE helmfile.d/services/api-gateway: apply 2253747 wikitext text/x-wiki qhxdl010pf9cbm11oo01qtgb6gaevc9 2253748 2253747 2024-12-11T14:57:48Z Stashbot 7414 jelto: homer 'lsw1-c3-codfw*' commit 'T377877' 2253748 wikitext text/x-wiki rye73a63lbtaoqep2qkqgwlowozggf0 2253749 2253748 2024-12-11T14:58:16Z Stashbot 7414 jhancock@cumin2002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es1043.eqiad.wmnet with OS bookworm 2253749 wikitext text/x-wiki 4kgacja1ok4893ftq1gi570bgnnl5md 2253750 2253749 2024-12-11T15:00:00Z Stashbot 7414 jelto: homer 'lsw1-d3-codfw*' commit 'T377877' 2253750 wikitext text/x-wiki 4n8qom6uwfnzdgcuv8q8xzq42hmmwi6 2253751 2253750 2024-12-11T15:02:46Z Stashbot 7414 elukey@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2085.codfw.wmnet with OS bullseye 2253751 wikitext text/x-wiki eq2h96fjtp99bs2ucc332sydf1ia3mr 2253752 2253751 2024-12-11T15:02:56Z Stashbot 7414 jelto@cumin1002: START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2184-2187].codfw.wmnet 2253752 wikitext text/x-wiki trpo9lsgaiyzk082a9bhvrcap8i2iui 2253753 2253752 2024-12-11T15:03:00Z Stashbot 7414 jelto@cumin1002: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2184-2187].codfw.wmnet 2253753 wikitext text/x-wiki c39flg1rcib1wsa8ygiahkcb626h8q6 2253754 2253753 2024-12-11T15:13:05Z Stashbot 7414 elukey@deploy2002: Started deploy [docker-pkg/deploy@9305554]: Update to 4.0.3 2253754 wikitext text/x-wiki fek4b277a4s4wk2h6e9tjmebr0rcmoc 2253755 2253754 2024-12-11T15:13:35Z Stashbot 7414 elukey@deploy2002: Finished deploy [docker-pkg/deploy@9305554]: Update to 4.0.3 (duration: 00m 37s) 2253755 wikitext text/x-wiki piopl2e6x9gwn8r2aw1pptwjhgo70fz 2253756 2253755 2024-12-11T15:15:36Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on wdqs1025.eqiad.wmnet with reason: T376150 2253756 wikitext text/x-wiki 72odnjyv3ebc1135huibopch2kiif4w 2253757 2253756 2024-12-11T15:15:51Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on wdqs1025.eqiad.wmnet with reason: T376150 2253757 wikitext text/x-wiki iidz2u5imo5vakl9oi53l8o8qrbs3bm 2253759 2253757 2024-12-11T15:19:26Z Stashbot 7414 klausman@deploy2002: helmfile [codfw] START helmfile.d/services/api-gateway: apply 2253759 wikitext text/x-wiki 8awouql1fhfx72dy3fnhzm0tmrsubxv 2253760 2253759 2024-12-11T15:19:51Z Stashbot 7414 klausman@deploy2002: helmfile [codfw] DONE helmfile.d/services/api-gateway: apply 2253760 wikitext text/x-wiki agvc8jiaqzqmwcro2dw6w6xes4xavhn 2253761 2253760 2024-12-11T15:20:15Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply 2253761 wikitext text/x-wiki emh4gyuvv6ymrj88knvk0xdjwjw79ly 2253762 2253761 2024-12-11T15:21:43Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply 2253762 wikitext text/x-wiki f563pd0zbyputm3nuenoz0v4o4fy1gz 2253763 2253762 2024-12-11T15:22:31Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply 2253763 wikitext text/x-wiki 0mb128ezjeb326xti95e7i99ltq4cld 2253765 2253763 2024-12-11T15:23:49Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-videoscaler: apply 2253765 wikitext text/x-wiki 4p3cpwnibjdpehokftilfx3zj0vyg19 2253766 2253765 2024-12-11T15:23:55Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-videoscaler: apply 2253766 wikitext text/x-wiki i2s7b1zukixri6ctw5wi1y61p1zp0mg 2253767 2253766 2024-12-11T15:35:09Z Stashbot 7414 fabfur@cumin1002: conftool action : set/pooled=no; selector: name=cp3066.esams.wmnet 2253767 wikitext text/x-wiki k9ogsiwushdj2e9fe5nzqurv2i8lmea 2253768 2253767 2024-12-11T15:36:04Z Stashbot 7414 fabfur@cumin1002: conftool action : set/pooled=yes; selector: name=cp3066.esams.wmnet 2253768 wikitext text/x-wiki 6t0hlnti53snuttrneq6ppmkjyhcgo7 2253769 2253768 2024-12-11T15:36:31Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-videoscaler: apply 2253769 wikitext text/x-wiki a5nttdcystou2pox1ygbjb5yrb95c43 2253770 2253769 2024-12-11T15:36:37Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-videoscaler: apply 2253770 wikitext text/x-wiki huw809ap5en37717x24pcvjtlaggo90 2253771 2253770 2024-12-11T15:38:23Z Stashbot 7414 klausman@deploy2002: helmfile [eqiad] START helmfile.d/services/api-gateway: apply 2253771 wikitext text/x-wiki bzrc6vznjwnlolubpm1bwi73q0nfdg9 2253772 2253771 2024-12-11T15:38:47Z Stashbot 7414 klausman@deploy2002: helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply 2253772 wikitext text/x-wiki g9a0evlj52v0qpqvjih2o71ygc40kkx 2253773 2253772 2024-12-11T15:39:00Z Stashbot 7414 btullis@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on archiva1002.wikimedia.org with reason: Adding new disk 2253773 wikitext text/x-wiki 2xa3eisnr2qy5tl9gt9xkh5mc9efdet 2253774 2253773 2024-12-11T15:39:14Z Stashbot 7414 btullis@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on archiva1002.wikimedia.org with reason: Adding new disk 2253774 wikitext text/x-wiki hgzq0pc7x70fpb1nuxagzfp94i12is4 2253775 2253774 2024-12-11T15:57:58Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on db2135.codfw.wmnet with reason: maintenance 2253775 wikitext text/x-wiki d7o3j4ogzdcod04boclvatturr174y8 2253776 2253775 2024-12-11T15:58:12Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2135.codfw.wmnet with reason: maintenance 2253776 wikitext text/x-wiki pjxap8bpkq3yldwcp9giebnf8ghaxue 2253777 2253776 2024-12-11T16:10:43Z Stashbot 7414 otto@deploy2002: Started scap sync-world: Backport for [[gerrit:1063222|mediawiki.org/beacon/event/index.php - use EventBus->send (T353817)]] 2253777 wikitext text/x-wiki fhv8746rv7dkl22ogax0v318we1x7e3 2253780 2253777 2024-12-11T16:16:23Z Stashbot 7414 otto@deploy2002: otto: Backport for [[gerrit:1063222|mediawiki.org/beacon/event/index.php - use EventBus->send (T353817)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) 2253780 wikitext text/x-wiki iyyvfuqsk51hja4sjk8s5d53z4mz9w2 2253781 2253780 2024-12-11T16:16:38Z Stashbot 7414 otto@deploy2002: otto: Continuing with sync 2253781 wikitext text/x-wiki tenbz294v67qyh4fk944ua7hn8feq8x 2253782 2253781 2024-12-11T16:21:32Z Stashbot 7414 swfrench@deploy2002: helmfile [staging] START helmfile.d/services/shellbox-video: apply 2253782 wikitext text/x-wiki 8ttbwh9wurlomk8tufsyt6jmyqkhk51 2253783 2253782 2024-12-11T16:21:40Z Stashbot 7414 swfrench@deploy2002: helmfile [staging] DONE helmfile.d/services/shellbox-video: apply 2253783 wikitext text/x-wiki eojnrnah2f6xdfbcrasvvyi4x069xja 2253784 2253783 2024-12-11T16:22:19Z Stashbot 7414 otto@deploy2002: Finished scap sync-world: Backport for [[gerrit:1063222|mediawiki.org/beacon/event/index.php - use EventBus->send (T353817)]] (duration: 11m 36s) 2253784 wikitext text/x-wiki 6g7fqq8xpdcyt2w30rgo156vz6ztfia 2253785 2253784 2024-12-11T16:23:22Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.wdqs.restart 2253785 wikitext text/x-wiki np2xz89ye505j1u6izxcbpq4w7lptbm 2253786 2253785 2024-12-11T16:24:34Z Stashbot 7414 swfrench@deploy2002: helmfile [eqiad] START helmfile.d/services/shellbox-video: apply 2253786 wikitext text/x-wiki 4cndosknb14utork4fkxwy2xu9qbx8k 2253787 2253786 2024-12-11T16:24:41Z Stashbot 7414 swfrench@deploy2002: helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply 2253787 wikitext text/x-wiki 37e9bcj965okovx6cxnlhsssiy7bk1c 2253788 2253787 2024-12-11T16:25:45Z Stashbot 7414 swfrench@deploy2002: helmfile [codfw] START helmfile.d/services/shellbox-video: apply 2253788 wikitext text/x-wiki mrqt9o99cnv6pjomkz8ho29o6kwdbmp 2253789 2253788 2024-12-11T16:25:49Z Stashbot 7414 swfrench@deploy2002: helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply 2253789 wikitext text/x-wiki t914fa64k21eb21a11smikhrtxar5bk 2253790 2253789 2024-12-11T16:32:25Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.wdqs.restart 2253790 wikitext text/x-wiki rd9n97syzprtcxohcd6k4l547ha4rgl 2253791 2253790 2024-12-11T16:34:14Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) 2253791 wikitext text/x-wiki g4lyb2ed4sfow5uae66cqcuzehq8tcp 2253793 2253791 2024-12-11T16:35:20Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.wdqs.restart 2253793 wikitext text/x-wiki 1wyjom9jpvt055siruinkdrd1xddni0 2253798 2253793 2024-12-11T16:42:58Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) 2253798 wikitext text/x-wiki 1myw0wn12a3ahtoytex6mz9bdczgzro 2253800 2253798 2024-12-11T16:43:31Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.wdqs.restart 2253800 wikitext text/x-wiki 9b0x41313eah03z772h802ttgk8adoc 2253801 2253800 2024-12-11T16:46:11Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) 2253801 wikitext text/x-wiki md7w8lvgye6uuau7gzt2sim2pvej2n5 2253802 2253801 2024-12-11T16:47:20Z Stashbot 7414 bking@cumin2002: END (PASS) - Cookbook sre.wdqs.restart (exit_code=0) 2253802 wikitext text/x-wiki sy3xs846h49q7qcybc2zrvbm7ha7zlz 2253804 2253802 2024-12-11T16:48:36Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.wdqs.restart 2253804 wikitext text/x-wiki ne6ieqt87u99rk1hdwpfw4hl8nqowne 2253807 2253804 2024-12-11T17:09:38Z Stashbot 7414 cgoubert@cumin1002: START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[2047,2066,2085-2086,2180-2183].codfw.wmnet 2253807 wikitext text/x-wiki g42u0rqmlj5djxdny4urjpb1y3w48lx 2253808 2253807 2024-12-11T17:16:54Z Stashbot 7414 cgoubert@cumin1002: END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[2047,2066,2085-2086,2180-2183].codfw.wmnet 2253808 wikitext text/x-wiki 237fh3ada9wq8s5drvdwdaeurf8fm6u 2253809 2253808 2024-12-11T17:19:41Z Stashbot 7414 bking@cumin2002: END (FAIL) - Cookbook sre.wdqs.restart (exit_code=99) 2253809 wikitext text/x-wiki nk4epvj8o2kx5m7m5jghf99aje4kmr8 2253812 2253809 2024-12-11T17:31:54Z Stashbot 7414 bking@cumin2002: START - Cookbook sre.wdqs.restart 2253812 wikitext text/x-wiki bsz0jppy8scv8yhao82dgozhov9ckcc 2253813 2253812 2024-12-11T17:31:57Z Stashbot 7414 bking@cumin2002: END (ERROR) - Cookbook sre.wdqs.restart (exit_code=97) 2253813 wikitext text/x-wiki crrq3ag1cwuz932xesia4fjqcxklypp 2253814 2253813 2024-12-11T17:32:01Z Stashbot 7414 cgoubert@cumin1002: START - Cookbook sre.hosts.decommission for hosts wikikube-worker[2047,2066,2085-2086].codfw.wmnet 2253814 wikitext text/x-wiki fdp96oje4zmxdawzv7ezstr50m6bmnl 2253815 2253814 2024-12-11T17:37:47Z Stashbot 7414 oblivian@cumin1002: START - Cookbook sre.deploy.hiddenparma Hiddenparma deployment to the alerting hosts with reason: "UI improvements, add uncomitted changes warning - oblivian@cumin1002" 2253815 wikitext text/x-wiki 83yxmhe4bip52mkhh251m9ffjvz88pl 2253816 2253815 2024-12-11T17:37:49Z Stashbot 7414 oblivian@cumin1002: START - Cookbook sre.deploy.python-code hiddenparma to alert[1002,2002].wikimedia.org with reason: UI improvements, add uncomitted changes warning - oblivian@cumin1002 2253816 wikitext text/x-wiki lpz5q1edgr5i59cajv9v2z9amxzxu53 2253817 2253816 2024-12-11T17:38:20Z Stashbot 7414 oblivian@cumin1002: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) hiddenparma to alert[1002,2002].wikimedia.org with reason: UI improvements, add uncomitted changes warning - oblivian@cumin1002 2253817 wikitext text/x-wiki 60dsnw4php50q62vijesr0r5b18al41 2253818 2253817 2024-12-11T17:38:22Z Stashbot 7414 oblivian@cumin1002: END (PASS) - Cookbook sre.deploy.hiddenparma (exit_code=0) Hiddenparma deployment to the alerting hosts with reason: "UI improvements, add uncomitted changes warning - oblivian@cumin1002" 2253818 wikitext text/x-wiki 55bfh68zau45ws5wj24lot6e4859jru 2253821 2253818 2024-12-11T17:48:40Z Stashbot 7414 cgoubert@cumin1002: START - Cookbook sre.dns.netbox 2253821 wikitext text/x-wiki 5u9t1tidk59httl77ajiyu9929eaoay 2253822 2253821 2024-12-11T17:53:37Z Stashbot 7414 cgoubert@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: wikikube-worker[2047,2066,2085-2086].codfw.wmnet decommissioned, removing all IPs except the asset tag one - cgoubert@cumin1002" 2253822 wikitext text/x-wiki ddjys1s7g6ab2etmdwx0yjdfdnrn1bt 2253823 2253822 2024-12-11T17:54:16Z Stashbot 7414 cgoubert@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: wikikube-worker[2047,2066,2085-2086].codfw.wmnet decommissioned, removing all IPs except the asset tag one - cgoubert@cumin1002" 2253823 wikitext text/x-wiki pnhylal0wobagdq7wjoysyq90id15g0 2253824 2253823 2024-12-11T17:54:18Z Stashbot 7414 cgoubert@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253824 wikitext text/x-wiki kztwglbcy0skjybsilmaqcrd5tzoeqq 2253825 2253824 2024-12-11T17:54:20Z Stashbot 7414 cgoubert@cumin1002: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts wikikube-worker[2047,2066,2085-2086].codfw.wmnet 2253825 wikitext text/x-wiki cq31cm79evg51wk6q2mwtorx3bgvy8c 2253826 2253825 2024-12-11T17:55:02Z Stashbot 7414 claime: homer 'lsw1-a6-codfw' commit 'T379788' 2253826 wikitext text/x-wiki c62nt1eqoja3ehisruncq4syy2abtdc 2253827 2253826 2024-12-11T17:58:48Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253827 wikitext text/x-wiki b9o1dhx8m0mnu1ih5td6m2ym0f0q9hl 2253828 2253827 2024-12-11T17:58:51Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253828 wikitext text/x-wiki ehpb4zd26ee0xhiyrgh8jgt3885jvjo 2253829 2253828 2024-12-11T17:59:17Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253829 wikitext text/x-wiki trkalb2opbhunixw31jvtqda3uqo4lt 2253830 2253829 2024-12-11T17:59:20Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253830 wikitext text/x-wiki 10nkwq5dr5hew06dcffmt8ztc7g3808 2253831 2253830 2024-12-11T18:00:08Z Stashbot 7414 cgoubert@cumin1002: START - Cookbook sre.hosts.decommission for hosts wikikube-worker[2180-2183].codfw.wmnet 2253831 wikitext text/x-wiki awu7xwi3vlhabm80t2eex5lc1ebice2 2253832 2253831 2024-12-11T18:04:29Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253832 wikitext text/x-wiki 09wjn4eeovy6rmbjk7ovym6z7xh5jrc 2253833 2253832 2024-12-11T18:04:31Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253833 wikitext text/x-wiki gfzor4gw2hdpqtl92uea71iswh20zgn 2253834 2253833 2024-12-11T18:05:14Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253834 wikitext text/x-wiki iikfmslba3niyzogb7zmkxul2nvpy2g 2253835 2253834 2024-12-11T18:05:16Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253835 wikitext text/x-wiki dhtbohfcr68vj8g5eq5maniy91ks805 2253836 2253835 2024-12-11T18:05:43Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253836 wikitext text/x-wiki 1227p5n9rdcsumdxndymsn4ofporzkb 2253837 2253836 2024-12-11T18:05:45Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253837 wikitext text/x-wiki ga9f0dh8umj1e7f1dgp3o05er86blry 2253838 2253837 2024-12-11T18:06:15Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253838 wikitext text/x-wiki o9x0dxxx7kjm9ppcpvwuvmiz73xdst7 2253839 2253838 2024-12-11T18:06:18Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253839 wikitext text/x-wiki e34czvrv3z91pmgu1z70u5q58xqxshd 2253840 2253839 2024-12-11T18:10:12Z Stashbot 7414 cgoubert@cumin1002: START - Cookbook sre.dns.netbox 2253840 wikitext text/x-wiki 0xwq2sh52qoxa2gc480og8gnmk3lbyd 2253841 2253840 2024-12-11T18:15:02Z Stashbot 7414 cgoubert@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: wikikube-worker[2180-2183].codfw.wmnet decommissioned, removing all IPs except the asset tag one - cgoubert@cumin1002" 2253841 wikitext text/x-wiki t4e2wivw7giuzzqq2tubthr9is881el 2253842 2253841 2024-12-11T18:16:15Z Stashbot 7414 cgoubert@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: wikikube-worker[2180-2183].codfw.wmnet decommissioned, removing all IPs except the asset tag one - cgoubert@cumin1002" 2253842 wikitext text/x-wiki pjnkgedsrdaf0z53smh5xsx2hg9tcc8 2253843 2253842 2024-12-11T18:16:17Z Stashbot 7414 cgoubert@cumin1002: END (PASS) - Cookbook sre.dns.netbox (exit_code=0) 2253843 wikitext text/x-wiki 4v6ql7zov9tbbgqeu71qfmrz9wiav6a 2253844 2253843 2024-12-11T18:16:19Z Stashbot 7414 cgoubert@cumin1002: END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts wikikube-worker[2180-2183].codfw.wmnet 2253844 wikitext text/x-wiki 019xn9g8acpb6e1ralcvzcfr6ia5qeh 2253845 2253844 2024-12-11T18:17:07Z Stashbot 7414 claime: homer 'lsw1-c1-codfw*' commit 'T379788' 2253845 wikitext text/x-wiki r09i5v8cpn716bavbb58b552yv2xy05 2253846 2253845 2024-12-11T18:18:23Z Stashbot 7414 claime: homer 'lsw1-d6-codfw*' commit 'T379788' 2253846 wikitext text/x-wiki 8a5ot8rshnep5zq855fb7vjcvesbasx 2253859 2253846 2024-12-11T20:42:07Z Stashbot 7414 eevans@cumin1002: START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on aqs1014.eqiad.wmnet with reason: Hardware replacement 2253859 wikitext text/x-wiki b7bu6j36wc1739ydriv0j9avsuouffj 2253860 2253859 2024-12-11T20:42:22Z Stashbot 7414 eevans@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on aqs1014.eqiad.wmnet with reason: Hardware replacement 2253860 wikitext text/x-wiki fu7lm1yn2asley0omsffl82u46a5vae 2253862 2253860 2024-12-11T21:00:11Z Stashbot 7414 jclark@cumin1002: START - Cookbook sre.hosts.provision for host cloudelastic1012.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2253862 wikitext text/x-wiki r0qcw770jeo634a5kf1sbikwbvzstfk 2253863 2253862 2024-12-11T21:00:14Z Stashbot 7414 jclark@cumin1002: START - Cookbook sre.hosts.provision for host cloudelastic1011.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2253863 wikitext text/x-wiki t30q6u0ho276fphv5202fzgvvvfkvs0 2253864 2253863 2024-12-11T21:03:14Z Stashbot 7414 samtar@deploy2002: Started scap sync-world: Backport for [[gerrit:1101937|Enable AutoModerator on bnwiki (T381000)]] 2253864 wikitext text/x-wiki c0l6sb4f0vdhv6tjk74pd7m09pho5f8 2253866 2253864 2024-12-11T21:08:03Z Stashbot 7414 samtar@deploy2002: kgraessle, samtar: Backport for [[gerrit:1101937|Enable AutoModerator on bnwiki (T381000)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) 2253866 wikitext text/x-wiki 20d43fhebkoikreotfrgol5xonhzn8w 2253867 2253866 2024-12-11T21:09:00Z Stashbot 7414 samtar@deploy2002: kgraessle, samtar: Continuing with sync 2253867 wikitext text/x-wiki c0uygm48k4e4j2xis4nc2e9nntlz7h9 2253868 2253867 2024-12-11T21:10:33Z Stashbot 7414 jclark@cumin1002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudelastic1012.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2253868 wikitext text/x-wiki ltekgb07c5x68p4egretej66y6t1vbf 2253869 2253868 2024-12-11T21:10:39Z Stashbot 7414 jclark@cumin1002: END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudelastic1011.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART 2253869 wikitext text/x-wiki 30n9gsry3yc9vgxddbu51i1jrv9j4u4 2253870 2253869 2024-12-11T21:11:40Z Stashbot 7414 jclark@cumin1002: START - Cookbook sre.hosts.reimage for host cloudelastic1011.eqiad.wmnet with OS bullseye 2253870 wikitext text/x-wiki scli17otrvv3cy5idffbkl1kmj0cg01 2253871 2253870 2024-12-11T21:11:42Z Stashbot 7414 jclark@cumin1002: START - Cookbook sre.hosts.reimage for host cloudelastic1012.eqiad.wmnet with OS bullseye 2253871 wikitext text/x-wiki qajfw78w4ah20efmlk56eomgxd9ugma 2253872 2253871 2024-12-11T21:14:15Z Stashbot 7414 samtar@deploy2002: Finished scap sync-world: Backport for [[gerrit:1101937|Enable AutoModerator on bnwiki (T381000)]] (duration: 11m 01s) 2253872 wikitext text/x-wiki sh06bvxhcf4djv67cr6mqt9y53jeshs 2253873 2253872 2024-12-11T21:22:04Z Stashbot 7414 samtar@deploy2002: Started scap sync-world: Backport for [[gerrit:1102205|Follow-up I9df39fdcc: Convert missed 'this' to 'el' (T381741)]] 2253873 wikitext text/x-wiki am62qrxkfjxwoomgi7rw6hlms60j0sl 2253874 2253873 2024-12-11T21:22:37Z Stashbot 7414 jclark@cumin1002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudelastic1012.eqiad.wmnet with OS bullseye 2253874 wikitext text/x-wiki 1eqidjblxkc118l9pph9u10fykd5ncg 2253875 2253874 2024-12-11T21:22:44Z Stashbot 7414 jclark@cumin1002: END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudelastic1011.eqiad.wmnet with OS bullseye 2253875 wikitext text/x-wiki 812k23kjjatlfjcatkf4w6hgvv35u4s 2253876 2253875 2024-12-11T21:26:00Z Stashbot 7414 samtar@deploy2002: novemlinguae, samtar: Backport for [[gerrit:1102205|Follow-up I9df39fdcc: Convert missed 'this' to 'el' (T381741)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) 2253876 wikitext text/x-wiki 15cpact2ahs50dzynnrb818vtuocgs7 2253877 2253876 2024-12-11T21:26:53Z Stashbot 7414 samtar@deploy2002: novemlinguae, samtar: Continuing with sync 2253877 wikitext text/x-wiki 1l6komwegp0jjrybzs77adyt74gt4bs 2253878 2253877 2024-12-11T21:32:06Z Stashbot 7414 samtar@deploy2002: Finished scap sync-world: Backport for [[gerrit:1102205|Follow-up I9df39fdcc: Convert missed 'this' to 'el' (T381741)]] (duration: 10m 01s) 2253878 wikitext text/x-wiki 9l2mqr52m4wvoy30skrzg6q1ewr8txm 2253879 2253878 2024-12-11T21:33:37Z Stashbot 7414 TheresNoTime: done UTC late backport window 2253879 wikitext text/x-wiki 8q09wh475y1b3itodohn8p5mjty406z 2253880 2253879 2024-12-11T21:53:11Z Stashbot 7414 eileen: civicrm upgraded from ddda6d67 to 0d7f2866 2253880 wikitext text/x-wiki eb7zgxiuzs0ghk1sca9uwqaucabyjr0 2253881 2253880 2024-12-11T22:52:53Z Stashbot 7414 tzatziki: removing three files for legal compliance 2253881 wikitext text/x-wiki q3g7lmlhb3z5ssz6p83192woom4vml2 2253882 2253881 2024-12-11T23:02:11Z Stashbot 7414 tzatziki: removing 4 files for legal compliance 2253882 wikitext text/x-wiki 82hymrjipnwnlo4p2o566lifqg4gidc 2253883 2253882 2024-12-11T23:24:32Z Stashbot 7414 tzatziki: removing 7 files for legal compliance 2253883 wikitext text/x-wiki 08aiuv59mrq5s5d24bo32hk9pv1up5j 2253888 2253883 2024-12-12T07:52:56Z Stashbot 7414 moritzm: installing upx-ucl security updates 2253888 wikitext text/x-wiki trz5k1d97a3c96419vo4s10sedlw02c 2253889 2253888 2024-12-12T07:56:26Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253889 wikitext text/x-wiki 9p9n4gw07kida7697k5sl7o0ee7ifgt 2253890 2253889 2024-12-12T07:56:29Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253890 wikitext text/x-wiki ds6qxuij3986mut53l7lwgps8fyo2v0 2253891 2253890 2024-12-12T08:00:06Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253891 wikitext text/x-wiki a8o3pyy606ypv4v0i1cnl0ayh1n4mrq 2253892 2253891 2024-12-12T08:00:14Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253892 wikitext text/x-wiki aq9k9d6et7jkaic8qgdmruhrk0y8idr 2253893 2253892 2024-12-12T08:00:46Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on dbproxy1023.eqiad.wmnet with reason: maintenance 2253893 wikitext text/x-wiki 8ak3snsg26wdx10brib2s8aw3uwsguj 2253894 2253893 2024-12-12T08:00:52Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253894 wikitext text/x-wiki 4e621s9fxqkgtjtbzo36c4zcw9yjx7l 2253895 2253894 2024-12-12T08:00:55Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253895 wikitext text/x-wiki 4p4r6sbk0kkaax76bodx9v84je8h4b0 2253896 2253895 2024-12-12T08:01:00Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on dbproxy1023.eqiad.wmnet with reason: maintenance 2253896 wikitext text/x-wiki tw1anbc4tu6cdb2gbwhpavn1rug7sx3 2253897 2253896 2024-12-12T08:02:28Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253897 wikitext text/x-wiki sy92cogk07hgk776940yocnn1gnb8h9 2253898 2253897 2024-12-12T08:02:31Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253898 wikitext text/x-wiki e7adtgtb4xl3bfxgk109o9u0pbsrbps 2253899 2253898 2024-12-12T08:07:15Z Stashbot 7414 kartik@deploy2002: Started scap sync-world: Backport for [[gerrit:1102283|Translate: Enable message group subscription for 7 wikis (T372386)]] 2253899 wikitext text/x-wiki fsjihrp13pd2b5ubmbmjkpy49hk2t9z 2253900 2253899 2024-12-12T08:11:20Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on dbproxy1024.eqiad.wmnet with reason: maintenance 2253900 wikitext text/x-wiki so5o18d4duagirp5e4qq1ia6lvp37it 2253901 2253900 2024-12-12T08:11:22Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on dbproxy1024.eqiad.wmnet with reason: maintenance 2253901 wikitext text/x-wiki 8y55rbk7i3a9mfgsx9ehffajh6u95f5 2253902 2253901 2024-12-12T08:12:17Z Stashbot 7414 kartik@deploy2002: kartik, abi: Backport for [[gerrit:1102283|Translate: Enable message group subscription for 7 wikis (T372386)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) 2253902 wikitext text/x-wiki em9ktux834wxfrr8mxofwy3moliwdor 2253903 2253902 2024-12-12T08:21:10Z Stashbot 7414 kartik@deploy2002: kartik, abi: Continuing with sync 2253903 wikitext text/x-wiki k6x4j1qarvrjsf9bzd2z9auoceq3g5t 2253904 2253903 2024-12-12T08:23:44Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253904 wikitext text/x-wiki nw77h0tu064sek6k72tq7uaym60v6pz 2253905 2253904 2024-12-12T08:23:47Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253905 wikitext text/x-wiki 7eurzi60rqiar4y0aat3q40s4bzjntj 2253906 2253905 2024-12-12T08:27:21Z Stashbot 7414 kartik@deploy2002: Finished scap sync-world: Backport for [[gerrit:1102283|Translate: Enable message group subscription for 7 wikis (T372386)]] (duration: 20m 05s) 2253906 wikitext text/x-wiki 73h7iju5f6prz9k0bt433x2s9dgwu1a 2253908 2253906 2024-12-12T08:43:17Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on dbproxy1026.eqiad.wmnet with reason: maintenance 2253908 wikitext text/x-wiki rjluz6wetklrmwlt2p5qsgcj7vgy002 2253909 2253908 2024-12-12T08:43:30Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on dbproxy1026.eqiad.wmnet with reason: maintenance 2253909 wikitext text/x-wiki 1y2i6kqlw2ywgxbvojinupxcqipa8wt 2253910 2253909 2024-12-12T08:54:23Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253910 wikitext text/x-wiki czbldhrer7zy27d5uyokcmcj8ro2ktd 2253911 2253910 2024-12-12T08:54:26Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253911 wikitext text/x-wiki dshvvrrn1ldlztelugbqneo39oem9cx 2253912 2253911 2024-12-12T09:02:23Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on dbproxy1027.eqiad.wmnet with reason: maintenance 2253912 wikitext text/x-wiki oyo0o5nfh77zpt6p1tobcuo3727ad7t 2253913 2253912 2024-12-12T09:02:37Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on dbproxy1027.eqiad.wmnet with reason: maintenance 2253913 wikitext text/x-wiki lrkqzvdt9kpdsqxrxcxibq32lbg789q 2253914 2253913 2024-12-12T09:17:34Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253914 wikitext text/x-wiki 6m402vm82ag567z46xk39o0bv82cpps 2253915 2253914 2024-12-12T09:17:39Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253915 wikitext text/x-wiki 7qlh6n2s52og7310jo1dlkrsi487pfa 2253916 2253915 2024-12-12T09:18:49Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253916 wikitext text/x-wiki 5l1wk72hxwr6n5p5aj6j7fbypvhjh80 2253917 2253916 2024-12-12T09:18:54Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253917 wikitext text/x-wiki 6p980tk5dfy2qob4a65rrgcbti45wkc 2253918 2253917 2024-12-12T09:22:38Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253918 wikitext text/x-wiki mjc906eatqupnuiq1y6l15pdpa140sb 2253919 2253918 2024-12-12T09:22:43Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253919 wikitext text/x-wiki bmgn3o5j3xyzb8fjs4o2id0ebu1tho3 2253920 2253919 2024-12-12T10:00:38Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2253920 wikitext text/x-wiki bfa1vebuq6na9ivy24e5n0islzpqll9 2253921 2253920 2024-12-12T10:00:53Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2253921 wikitext text/x-wiki rf2u725u7w3i5ktlgw2tvx0m1kvwwl9 2253922 2253921 2024-12-12T10:34:31Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253922 wikitext text/x-wiki iw0xd6r1ytfgzihrmfjj8u2gjr4zfjd 2253923 2253922 2024-12-12T10:34:36Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253923 wikitext text/x-wiki hbcjshwzy3fsjp46edbbvrmyycj3uay 2253924 2253923 2024-12-12T10:36:27Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253924 wikitext text/x-wiki l377bchr1pdjh9lmchlu87q3po5db3r 2253925 2253924 2024-12-12T10:36:32Z Stashbot 7414 gmodena@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply 2253925 wikitext text/x-wiki eqw73xi2mxnhhqpzv4l5h7nqqjhako7 2253926 2253925 2024-12-12T10:43:39Z Stashbot 7414 cgoubert@deploy2002: helmfile [eqiad] START helmfile.d/services/shellbox-video: apply 2253926 wikitext text/x-wiki sptnljtahslbxaxjnyxakhi4ygf3a4a 2253927 2253926 2024-12-12T10:43:48Z Stashbot 7414 cgoubert@deploy2002: helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply 2253927 wikitext text/x-wiki b756kn7yk1n55uskml83bd23gz1myc2 2253928 2253927 2024-12-12T10:43:57Z Stashbot 7414 cgoubert@deploy2002: helmfile [codfw] START helmfile.d/services/shellbox-video: apply 2253928 wikitext text/x-wiki pu9vfmlfghag66il5hvgnj3mypvv45l 2253929 2253928 2024-12-12T10:44:28Z Stashbot 7414 cmooney@cumin1002: START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: update Homer wmf-plugin - cmooney@cumin1002 2253929 wikitext text/x-wiki 353aj8spbq5lwk0mvo3uq0qse0eubu3 2253931 2253929 2024-12-12T10:48:40Z Stashbot 7414 cmooney@cumin1002: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: update Homer wmf-plugin - cmooney@cumin1002 2253931 wikitext text/x-wiki cjw2yc77ebwzg2so65gcgi96q2mrvxx 2253933 2253931 2024-12-12T11:01:11Z Stashbot 7414 cgoubert@deploy2002: helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply 2253933 wikitext text/x-wiki lbaeuktngb8kbb5qsf1b94ce0ovpsw9 2253934 2253933 2024-12-12T11:01:53Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-videoscaler: apply 2253934 wikitext text/x-wiki 24or29uo9aby6urc5il95hiz142l89z 2253935 2253934 2024-12-12T11:02:00Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-videoscaler: apply 2253935 wikitext text/x-wiki 8l3w5nnbmsqignil8t8igng4y2j81ya 2253936 2253935 2024-12-12T11:02:30Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-videoscaler: apply 2253936 wikitext text/x-wiki 66ok00fsfaeybznivy937bpmv97ktnu 2253937 2253936 2024-12-12T11:02:36Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-videoscaler: apply 2253937 wikitext text/x-wiki kvq83ssyg1926f1shn57qsa73v4808i 2253938 2253937 2024-12-12T11:03:23Z Stashbot 7414 cmooney@cumin1002: START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: update Homer wmf-plugin - cmooney@cumin1002 2253938 wikitext text/x-wiki nck9ziy7qxytdb3y3mxy0i6z50b4cas 2253939 2253938 2024-12-12T11:03:45Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] START helmfile.d/services/mw-videoscaler: apply 2253939 wikitext text/x-wiki sqsk156nvvvqxvuse58z8d62lbldjhf 2253940 2253939 2024-12-12T11:03:50Z Stashbot 7414 hnowlan@deploy1003: helmfile [eqiad] DONE helmfile.d/services/mw-videoscaler: apply 2253940 wikitext text/x-wiki n5wmufxuzu544xd74ib7vqzsijjada8 2253941 2253940 2024-12-12T11:04:03Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] START helmfile.d/services/mw-videoscaler: apply 2253941 wikitext text/x-wiki c6wslk9b4jnyuexwciyug96q3zpv2l0 2253942 2253941 2024-12-12T11:04:15Z Stashbot 7414 hnowlan@deploy1003: helmfile [codfw] DONE helmfile.d/services/mw-videoscaler: apply 2253942 wikitext text/x-wiki j4u1ampt4tp48s9sofp6nytaevwm59q 2253943 2253942 2024-12-12T11:04:57Z Stashbot 7414 cmooney@cumin1002: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: update Homer wmf-plugin - cmooney@cumin1002 2253943 wikitext text/x-wiki 7mku9gx4yt9xt2mog5vn3drihexupt1 2253944 2253943 2024-12-12T11:07:17Z Stashbot 7414 cmooney@cumin1002: START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: update Homer wmf-plugin - cmooney@cumin1002 2253944 wikitext text/x-wiki joik1tnw0w9gebnc8klsq1s0qpjg88m 2253945 2253944 2024-12-12T11:07:28Z Stashbot 7414 cgoubert@deploy2002: helmfile [codfw] START helmfile.d/services/shellbox-video: apply 2253945 wikitext text/x-wiki fpbxez3el7www4c20yfz07sw821r9qh 2253946 2253945 2024-12-12T11:07:31Z Stashbot 7414 cgoubert@deploy2002: helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply 2253946 wikitext text/x-wiki 1lbuexcsc8aaibeymefmwo580g6zzwc 2253947 2253946 2024-12-12T11:08:52Z Stashbot 7414 cmooney@cumin1002: END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: update Homer wmf-plugin - cmooney@cumin1002 2253947 wikitext text/x-wiki hpot8me9mji8b5j6oiuxzi9k8xk8dyg 2253949 2253947 2024-12-12T11:16:52Z Stashbot 7414 aqu@deploy2002: Started deploy [airflow-dags/analytics@0e18d4f]: Backfill webrequest actor label hourly 2024 12 2253949 wikitext text/x-wiki 2l6uqquzdoww2jhgcsec5llzm37lge2 2253950 2253949 2024-12-12T11:19:44Z Stashbot 7414 aqu@deploy2002: Finished deploy [airflow-dags/analytics@0e18d4f]: Backfill webrequest actor label hourly 2024 12 (duration: 02m 52s) 2253950 wikitext text/x-wiki oyt0aadlyjb54ngikv1v63she0bdb6c 2253951 2253950 2024-12-12T11:22:15Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on db2186.codfw.wmnet with reason: maintenance 2253951 wikitext text/x-wiki 7hpebj73ovfx1hebgnvb6gwg47blmfx 2253952 2253951 2024-12-12T11:22:29Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2186.codfw.wmnet with reason: maintenance 2253952 wikitext text/x-wiki mla5soy7dvm9updod3id2uhryb59uw8 2253953 2253952 2024-12-12T11:27:12Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on db2186.codfw.wmnet with reason: maintenance 2253953 wikitext text/x-wiki kpnkxb9n3tzslakjg8k5ht7syhu3qdp 2253954 2253953 2024-12-12T11:27:14Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2186.codfw.wmnet with reason: maintenance 2253954 wikitext text/x-wiki kjv2h2x6jwrm0ojwju4s8pqlvt70oar 2253955 2253954 2024-12-12T11:27:52Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on db2187.codfw.wmnet with reason: maintenance 2253955 wikitext text/x-wiki tvqmvrtcb1xkqpxfqzoe9mcqmp9lany 2253956 2253955 2024-12-12T11:28:06Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db2187.codfw.wmnet with reason: maintenance 2253956 wikitext text/x-wiki 7m87ymxh3r4xhessr2axxbxbrl477hw 2253957 2253956 2024-12-12T11:33:00Z Stashbot 7414 elukey@deploy2002: helmfile [codfw] START helmfile.d/admin 'sync'. 2253957 wikitext text/x-wiki 6oes5ft68p9j6ck2dde6pbhvaraddsg 2253958 2253957 2024-12-12T11:33:08Z Stashbot 7414 elukey@deploy2002: helmfile [codfw] DONE helmfile.d/admin 'sync'. 2253958 wikitext text/x-wiki 6u8i4wd8dp5yqb18atobfmtcfvcomah 2253959 2253958 2024-12-12T11:33:22Z Stashbot 7414 elukey@deploy2002: helmfile [eqiad] START helmfile.d/admin 'sync'. 2253959 wikitext text/x-wiki grq1o1au630x9lkc3qwp21ut1kpfxjc 2253960 2253959 2024-12-12T11:33:48Z Stashbot 7414 elukey@deploy2002: helmfile [eqiad] DONE helmfile.d/admin 'sync'. 2253960 wikitext text/x-wiki 3fpepbgt0t04rvke6ep1zropt1dzkah 2253961 2253960 2024-12-12T11:40:56Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. 2253961 wikitext text/x-wiki 0b7hcxmx6rxlmuf0gybivnjdmhh5lkv 2253962 2253961 2024-12-12T11:41:09Z Stashbot 7414 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 1:00:00 on db1154.eqiad.wmnet with reason: maintenance 2253962 wikitext text/x-wiki cq9kvniuh3wbj0sn38fo2tn4wd0bevx 2253963 2253962 2024-12-12T11:41:12Z Stashbot 7414 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1154.eqiad.wmnet with reason: maintenance 2253963 wikitext text/x-wiki g5rk4o8n2z81lb473g53ctozd96q7tx 2253964 2253963 2024-12-12T11:41:37Z Stashbot 7414 brouberol@deploy2002: helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. 2253964 wikitext text/x-wiki grp7ajgogyienxn7lbxi2k26ggad5pm 0 7942 2253861 2204549 2024-12-11T20:57:44Z SFrench-WMF 37799 /* Maintain the maintenance calendar */ Add link to T381680 in note about automation via script 2253861 wikitext text/x-wiki mab5o0tbxcoefnjt55exq72jex4vq3i 0 446332 2253865 2054845 2024-12-11T21:03:44Z RLazarus (WMF) 15215 /* One-off purge */ 2253865 wikitext text/x-wiki 7dtuodoriflmmw1sg3ocq3ihz00shsp 0 449160 2253884 2253459 2024-12-12T00:01:01Z Dexbot 30554 Bot: Updating the report 2253884 wikitext text/x-wiki 028ww80vysjdjsfvoh3uvh0xtl8hdf9 0 450294 2253735 2245840 2024-12-11T14:38:37Z Brouberol 37217 /* List of instances */ 2253735 wikitext text/x-wiki bs9d2i46vyfeswcbu8hzx0si2nzd0vb 2253764 2253735 2024-12-11T15:23:17Z BTullis (WMF) 25295 /* ml */ 2253764 wikitext text/x-wiki 6eg0jeoqdqkcj7wxuu5teh8idfcbuni 0 450684 2253758 2246773 2024-12-11T15:17:18Z Gmodena 20194 Add docs for python script execution and data retention 2253758 wikitext text/x-wiki 5yx553ydqtpxexduoido7hsvr4620y6 2 452238 2253795 2231796 2024-12-11T16:40:56Z TheresNoTime 6011 Replaced content with "{{deploy|type=config|gerrit=951042|title=IS: Enable Phonos on all projects|status=done|by=TheresNoTime|sal=https://sal.toolforge.org/log/u0SWRooBGiVuUzOdEfAd}}" 2253795 wikitext text/x-wiki m2tp0l74pypd3cd701tr2izbisb5dvr 10 452703 2253796 2252233 2024-12-11T16:41:39Z TheresNoTime 6011 add sal link 2253796 wikitext text/x-wiki 6sap55zzevdog72ic9horrtpotijskm 2 452734 2253719 2252238 2024-12-11T14:25:40Z TheresNoTime 6011 2253719 wikitext text/x-wiki 8sevonhoaycmsecqntw3rvexdby734t 2253731 2253719 2024-12-11T14:36:07Z TNTBot 39092 [supervised test] Updating deployment status 2253731 wikitext text/x-wiki 758ur46lo3iefpgfzu8jc8m57947hzr 2253734 2253731 2024-12-11T14:36:42Z TheresNoTime 6011 Reverted edit by [[Special:Contributions/TNTBot|TNTBot]] ([[User talk:TNTBot|talk]]) to last revision by [[User:TheresNoTime|TheresNoTime]] 2253719 wikitext text/x-wiki 8sevonhoaycmsecqntw3rvexdby734t 2253742 2253734 2024-12-11T14:52:25Z TNTBot 39092 [supervised test] Updating deployment status 2253742 wikitext text/x-wiki rrar3egquwu9korgieayuzc0cipgymq 2253743 2253742 2024-12-11T14:53:05Z TheresNoTime 6011 Reverted edit by [[Special:Contributions/TNTBot|TNTBot]] ([[User talk:TNTBot|talk]]) to last revision by [[User:TheresNoTime|TheresNoTime]] 2253719 wikitext text/x-wiki 8sevonhoaycmsecqntw3rvexdby734t 2253778 2253743 2024-12-11T16:12:16Z TheresNoTime 6011 2253778 wikitext text/x-wiki 4sf51cxisyr4gdhh6zlh0w5ee2tt6p5 2253779 2253778 2024-12-11T16:13:40Z TNTBot 39092 [supervised test] Updating deployment status 2253779 wikitext text/x-wiki 1afv8j8zucgdpb3tc3liexxniqu6y18 2253792 2253779 2024-12-11T16:35:03Z TheresNoTime 6011 Restored revision 2253743 by [[Special:Contributions/TheresNoTime|TheresNoTime]] ([[User talk:TheresNoTime|talk]]) (TwinkleGlobal) 2253792 wikitext text/x-wiki 8sevonhoaycmsecqntw3rvexdby734t 2253794 2253792 2024-12-11T16:35:56Z TNTBot 39092 [supervised test] Updating deployment status 2253794 wikitext text/x-wiki dy8dme1axhlvbb9v7kexwcmxsi8y0tr 2253799 2253794 2024-12-11T16:43:12Z TheresNoTime 6011 Reverted edit by [[Special:Contributions/TNTBot|TNTBot]] ([[User talk:TNTBot|talk]]) to last revision by [[User:TheresNoTime|TheresNoTime]] 2253792 wikitext text/x-wiki 8sevonhoaycmsecqntw3rvexdby734t 2253803 2253799 2024-12-11T16:47:45Z TNTBot 39092 [supervised test] Updating deployment status 2253803 wikitext text/x-wiki spxk3vab6bnmhrm8wbjziprk47qgume 2253805 2253803 2024-12-11T16:49:26Z TNTBot 39092 [supervised test] Updating deployment status 2253805 wikitext text/x-wiki 67iiqp9pj6hrjssppvmxxvlnsg98l25 2253806 2253805 2024-12-11T16:51:49Z TheresNoTime 6011 Reverted edits by [[Special:Contributions/TNTBot|TNTBot]] ([[User talk:TNTBot|talk]]) to last revision by [[User:TheresNoTime|TheresNoTime]] 2253792 wikitext text/x-wiki 8sevonhoaycmsecqntw3rvexdby734t 2253811 2253806 2024-12-11T17:25:30Z TheresNoTime 6011 2253811 wikitext text/x-wiki tt9e8ub2xijzqsbxegonps71ioopra9 2253819 2253811 2024-12-11T17:39:58Z TNTBot 39092 [supervised test] Updating deployment status 2253819 wikitext text/x-wiki 3y8cg4l95lhqgijl21bg3ghoeiigqdg 10 453357 2253797 2252234 2024-12-11T16:42:55Z TheresNoTime 6011 2253797 wikitext text/x-wiki c23arq4neunoddlhzsmpsc29vvjtybf 0 453693 2253886 2248716 2024-12-12T00:21:36Z Dreamy Jazz 31583 The 5 July 2024 link was wrong and went to the 4th of July. The correct link has no deploys listed. 2253886 wikitext text/x-wiki ho48rva2egostup56qok2hs7h969rbr 2253887 2253886 2024-12-12T00:23:13Z Dreamy Jazz 31583 11 Nov had no deployments 2253887 wikitext text/x-wiki dwsqi2i3z1pzsht5u7eoqvd42fp8045 0 453760 2253855 2133960 2024-12-11T18:35:19Z Ottomata 88 added Petr Pchelko's 2022 changeprop handoff walkthrough 2253855 wikitext text/x-wiki f3pjn2wrk4herybc2axekoe7ix1gcbk 0 455675 2253729 2247268 2024-12-11T14:34:45Z Brouberol 37217 /* DAGs deployment */ 2253729 wikitext text/x-wiki 3wfa1vq4t6p1zucjum1refoy5i3dq0u 0 456157 2253646 2251858 2024-12-11T13:38:32Z Brouberol 37217 /* Creating a new instance */ 2253646 wikitext text/x-wiki aqscgamykhjjggnpzu8m5c1hn38sbzs 0 456274 2253847 2252970 2024-12-11T18:21:09Z SSanchez-WMF 37857 Add page content 2253847 wikitext text/x-wiki 959rbtyubjzpmrdci755q68mqieuphc 2253848 2253847 2024-12-11T18:22:09Z SSanchez-WMF 37857 Apply paragraph style to button text 2253848 wikitext text/x-wiki tqxofb6qy53s5incqrpttp2xc7c55c0 2253850 2253848 2024-12-11T18:25:13Z SSanchez-WMF 37857 Add spacing around button 2253850 wikitext text/x-wiki tc297tpjec816f5mjuq0zz15g5jj098 2253851 2253850 2024-12-11T18:27:39Z SSanchez-WMF 37857 Add top and bottom margins to button 2253851 wikitext text/x-wiki ekxtdxed3sf74hy3d5ttl9f631qcx68 2253852 2253851 2024-12-11T18:29:25Z SSanchez-WMF 37857 Remove exclamation mark 2253852 wikitext text/x-wiki d6efsoheijmeok4gzez2thyktemdnum 2253856 2253852 2024-12-11T18:47:32Z SSanchez-WMF 37857 Improve question 3 2253856 wikitext text/x-wiki mawy8pk1ldihw8qeudiy67aupf886uy 2253948 2253856 2024-12-12T11:13:23Z SSanchez-WMF 37857 Update content 2253948 wikitext text/x-wiki pepkobkaz6dx71dji532a31txmlph9o 498 456280 2253592 2024-12-11T12:00:17Z Labslogbot 55 Auto update of instance info. 2253592 wikitext text/x-wiki ie68biivwzc0kzkc8igvvj5zbhn38wg 0 456281 2253849 2024-12-11T18:23:57Z SSanchez-WMF 37857 Add content 2253849 wikitext text/x-wiki 5ymovg70vl9f3hmd7ge6o6uaahehnaj 2253854 2253849 2024-12-11T18:34:24Z SSanchez-WMF 37857 Add space between questions 2253854 wikitext text/x-wiki dqcxsf09u41x8hvhp6dq3tete1wmuwn 2253857 2253854 2024-12-11T18:48:41Z SSanchez-WMF 37857 Improve question 3 2253857 wikitext text/x-wiki 1jxw141utl1x4s3yvowt9uk848mr6ru 0 456282 2253853 2024-12-11T18:30:12Z SSanchez-WMF 37857 Create empty ToolforgeUI/Feedback page 2253853 wikitext text/x-wiki phoiac9h4m842xq45sp7s6u21eteeq1 1 456283 2253907 2024-12-12T08:29:17Z SSanchez-WMF 37857 /* We need your feedback */ new section 2253907 wikitext text/x-wiki 2cs3anxz3pkhztqqncnk6ta6i881kjv