Skip to content

Mysql slave corrupted relay log

If you have a slave that has a corrupted relay log, it will show:
Last_Error: Could not parse relay log event entry. The possible reasons are: the master's binary log is corrupted (you can check this by running 'mysqlbinlog' on the binary log), the slave's relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master's or slave's MySQL code. If you want to check the master's binary log or slave's relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
in it’s mysqld.log & slave status.

This can happen because of a crash on the mysql slave, or a disk filling up, or something.

You can fix it by executing on the slave in question:
1. STOP SLAVE
2. SHOW SLAVE STATUS
3. Note down ‘Relay_Master_Log_File’ and ‘Exec_Master_Log_Pos’ entries.
4. RESET SLAVE
5. CHANGE MASTER TO ….. (use MASTER_LOG_FILE=relay_master_log_file and MASTER_LOG_POS=exec_master_log_pos from Step 3)
6. START SLAVE

For example, #5 might be something like: change master to master_log_file=”mysql-bin.001878″,master_log_pos=898514732;

Advertisements

IPA server & client commands

Not that this isn’t available elsewhere, but… just keeping it nicely stored somewhere for myself.

on ipa server:
ipa host-add host.example.com --password=MEEEEEP --force

on client:

ipa-client-install --hostname=host.example.com --domain=example.com --password=MEEEEEP --realm=EXAMPLE.COM --server=ipa.example.com --unattended --mkhomedir

—-

script:

#!/bin/bash
echo
echo "ipa host-add $1.example.com --password=MEEEEEP --force"
echo
echo
echo "ipa-client-install --hostname=$1.example.com --domain=example.com --password=MEEEEEP --realm=EXAMPLE.COM --server=ipa.example.com --unattended --mkhomedir"

echo

—-

Uninstall IPA

[root@solr05 ~]# ipa-client-install –uninstall -U
Unenrolling client from IPA server
Removing Kerberos service principals from /etc/krb5.keytab
Disabling client Kerberos and LDAP configurations
Restoring client configuration files

Change IPA password expiration for a bunch of users

Run this on a ipa server. It’ll work there..

1. Find users with passwords that expire before DATE below

ldapsearch -Y GSSAPI -h localhost -b "cn=users,cn=accounts,dc=example,dc=com" '(krbPasswordExpirationlsearch2

2. Change the actual expiration date for those users to date :1/1/2015.

grep dn lsearch2 > dn
IFS=$'\n'
for i in `cat dn`;do echo $i; echo "changetype: modify";echo "replace: krbpasswordexpiration";echo "krbpasswordexpiration: 20150101000000Z";echo ;done > test
ldapmodify -D "cn=Directory Manager" -W -h localhost -f test

Building FPM on RHEL

(If you have a version of FPM installed, you may want to remove & install the newest version before building…)

Building FPM:

gem install –install-dir /tmp/gems fpm
cd /tmp/gems/cache/

fpm -s gem -t rpm -d rpm-build -d rubygems -d ruby -d ruby-devel -d kernel-devel -d gcc -d gcc-c++ -d make -e fpm*.gem

Building other gems:

fpm -s gem -t rpm -d rubygems -d ruby *.gem

BlockDiag Network maps on RHEL

To install blockdiag:

yum install python-reportlab python-imaging
easy_install "blockdiag[pdf]"

Generate a pdf

nwdiag -Tpdf routing.diag -f /usr/share/fonts/dejavu/DejaVuSans.ttf

Generate a SVG

nwdiag -Tsvg routing.diag

Generate a png.  (Not as pretty, actually.)

nwdiag routing.diag

 

RHEL / Centos 6 Software Collections

So, there’s a new way to use multiple versions of languages, such as Ruby, on RHEL, without breaking the system.  I’ve got this down, so I thought I should share it.

In this case, it’s replacing the need for RVM, and moving to a much more standardized system.  I don’t love can’t stand having GCC installed on production servers, so this is much better.

SCL – Software Collections 🙂

The goal of this was to get a recent version of Fog working on RHEL 6 / Centos 6.

First:  get the SCL repo:

cd /etc/yum.repos.d

wget http://people.redhat.com/bkabrda/scl_ruby193.repo

That gives you the basics, but we also need to get some pieces for Fog, compiled for SCL. Thanks to the Katello Project, which is packaging Fog for use w/ Foreman, we can add their repo. You’ll have to make a repo for this, though:

http://fedorapeople.org/groups/katello/releases/yum/foreman-nightly/RHEL/6Server/x86_64/

–You could probably use that as your repo for ruby_193, even without the above scl repo.

This is what the repo file would look like:

[foreman-nightly]
name=Foreman Nightly
baseurl=http://fedorapeople.org/groups/katello/releases/yum/foreman-nightly/RHEL/6Server/x86_64/
enabled=1
gpgcheck=0

Now, install ruby_193:

yum install ruby193-ruby ruby193-rubygem-fog   ruby193-rubygem-rbvmomi

Now, you can use it:

scl enable ruby193 bash

which ruby

One thing, if you want to use ruby_193 inside scripts, you can use a stub for it:

/usr/bin/ruby193-ruby

Puppet sshkey + IPA + Centos = odd failure of ssh known_hosts

Recently saw this at work:

Server with IPA (freeipa.org), running puppet 3.X (3.2.1, but not really relevant.)  Centos 6.4.

Normally

Puppet’s sshkey type  creates /etc/ssh/ssh_known_hosts by default.

SSH defaults to checking /etc/ssh/ssh_known_hosts & /etc/ssh/ssh_known_hosts2, if GlobalKnownHostsFile isn’t set inside ssh_config.   RHEL / Centos doesn’t have it in the default config file, so it’s probably using the default for you.

All very good.

BTW- they could fix a long time sshkey bug: https://projects.puppetlabs.com/issues/4145 – it defaults to mode 600, which really needs to be 644 to be useful.

With IPA

If you’re using IPA, though, with SSSD, it adds this to your /etc/ssh/ssh_config:

GlobalKnownHostsFile /var/lib/sss/pubconf/known_hosts

This in turn disables the defaults, and it only checks /etc/ssh/ssh_known_hosts2. Not /etc/ssh/ssh_known_hosts.

Fix is to use a target parameter, for the sshkey, to make sure you write to /etc/ssh/ssh_known_hosts2.

Not sure how I’m going to get a bug opened on this. It’s not a puppet bug at all. It’s probably a bug for OpenSSH, but I don’t have a binary that they build, so I can’t submit a bug 😦

 

BTW: for those that are interested:  strace ssh HOSTNAME  was how I found the details.  Since it stops and waits for the human to agree to add the host key to known_hosts, it’s actually fairly easy to strace.