missing blob for item

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
umarzuki
Advanced member
Advanced member
Posts: 97
Joined: Sat Sep 13, 2014 12:46 am

missing blob for item

Postby umarzuki » Wed Apr 30, 2014 12:45 am

a user reported of "missing blob for item" when trying to view emails from 2012
ZCS version

$ zmcontrol -v

Release 8.0.6_GA_5922.RHEL6_64_20131203103753 RHEL6_64 NETWORK edition.

volumes info

$ zmvolume -l

Volume id: 1

name: message1

type: primaryMessage

path: /opt/zimbra/store

compressed: false

current: true
Volume id: 2

name: index1

type: index

path: /opt/zimbra/index

compressed: false

current: true
Volume id: 3

name: hsm-volume

type: secondaryMessage

path: /mnt/hsm

compressed: false

current: false
Volume id: 4

name: hsm-volume2

type: secondaryMessage

path: /mnt/zarchive/hsm

compressed: false

current: true

below is output when I tried running zmblobchk
$ zmblobchk start | grep 'item 2796'

Mailbox 84, item 2796, rev 4835, volume 3, /mnt/zarchive/hsm/0/84/msg/0/2796-4835.msg: file has incorrect revision.

Mailbox 358, item 2796, rev 4703, volume 3, /mnt/zarchive/hsm/0/358/msg/0/2796-4703.msg: file has incorrect revision.

Mailbox 626, item 2796, rev 5151, volume 3, /mnt/zarchive/hsm/0/626/msg/0/2796-5151.msg: file has incorrect revision.

Mailbox 646, item 2796, rev 5274, volume 3, /mnt/zarchive/hsm/0/646/msg/0/2796-5274.msg: file has incorrect revision.

Mailbox 941, item 2796, rev 5022, volume 3, /mnt/zarchive/hsm/0/941/msg/0/2796-5022.msg: file has incorrect revision.

Mailbox 1058, item 2796, rev 5187, volume 3, /mnt/zarchive/hsm/0/1058/msg/0/2796-5187.msg: file has incorrect revision.

Mailbox 1382, item 2796, rev 5212, volume 3, /mnt/zarchive/hsm/0/1382/msg/0/2796-5212.msg: file has incorrect revision.

Mailbox 1487, item 2796, rev 5171, volume 3, /mnt/zarchive/hsm/0/1487/msg/0/2796-5171.msg: file has incorrect revision.

Mailbox 1545, item 2796, rev 5198, volume 3, /mnt/zarchive/hsm/0/1545/msg/0/2796-5198.msg: file has incorrect revision.

Mailbox 1595, item 2796, rev 4748, volume 3, /mnt/zarchive/hsm/0/1595/msg/0/2796-4748.msg: file has incorrect revision.

Mailbox 1638, item 2796, rev 5205, volume 3, /mnt/zarchive/hsm/0/1638/msg/0/2796-5205.msg: file has incorrect revision.

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded

at java.util.Vector.(Vector.java:131)

at org.apache.xerces.dom.AttributeMap.setNamedItem(Unknown Source)

at org.apache.xerces.dom.DeferredElementNSImpl.synchronizeData(Unknown Source)

at org.apache.xerces.dom.ElementNSImpl.getNamespaceURI(Unknown Source)

at com.zimbra.common.soap.W3cDomUtil.needToFlattenElementContent(W3cDomUtil.java:234)

at com.zimbra.common.soap.W3cDomUtil.nodeToElement(W3cDomUtil.java:282)

at com.zimbra.common.soap.W3cDomUtil.toHierarchy(W3cDomUtil.java:328)

at com.zimbra.common.soap.W3cDomUtil.nodeToElement(W3cDomUtil.java:285)

at com.zimbra.common.soap.W3cDomUtil.toHierarchy(W3cDomUtil.java:328)

at com.zimbra.common.soap.W3cDomUtil.nodeToElement(W3cDomUtil.java:285)

at com.zimbra.common.soap.W3cDomUtil.toHierarchy(W3cDomUtil.java:328)

at com.zimbra.common.soap.W3cDomUtil.nodeToElement(W3cDomUtil.java:285)

at com.zimbra.common.soap.W3cDomUtil.toHierarchy(W3cDomUtil.java:328)

at com.zimbra.common.soap.W3cDomUtil.nodeToElement(W3cDomUtil.java:285)

at com.zimbra.common.soap.W3cDomUtil.toHierarchy(W3cDomUtil.java:328)

at com.zimbra.common.soap.W3cDomUtil.nodeToElement(W3cDomUtil.java:285)

at com.zimbra.common.soap.W3cDomUtil.nodeToElement(W3cDomUtil.java:280)

at com.zimbra.common.soap.W3cDomUtil.parseXML(W3cDomUtil.java:219)

at com.zimbra.common.soap.Element.parseXML(Element.java:629)

at com.zimbra.common.soap.SoapTransport.parseSoapResponse(SoapTransport.java:299)

at com.zimbra.common.soap.SoapHttpTransport.invoke(SoapHttpTransport.java:255)

at com.zimbra.common.soap.SoapHttpTransport.invoke(SoapHttpTransport.java:167)

at com.zimbra.common.soap.SoapTransport.invoke(SoapTransport.java:386)

at com.zimbra.common.soap.SoapTransport.invokeWithoutSession(SoapTransport.java:372)

at com.zimbra.cs.account.soap.SoapProvisioning.invokeRequest(SoapProvisioning.java:442)

at com.zimbra.cs.account.soap.SoapProvisioning.invoke(SoapProvisioning.java:450)

at com.zimbra.cs.store.file.BlobConsistencyUtil.checkMailbox(BlobConsistencyUtil.java:244)

at com.zimbra.cs.store.file.BlobConsistencyUtil.run(BlobConsistencyUtil.java:201)

at com.zimbra.cs.store.file.BlobConsistencyUtil.main(BlobConsistencyUtil.java:328)

I also tried to symlink to old /mnt/hsm (former mountpoint of hsm lun) but still same
$ ls -l /mnt/hsm/

total 0

lrwxrwxrwx 1 root root 18 Apr 30 13:02 hsm -> /mnt/zarchive/hsm/


umarzuki
Advanced member
Advanced member
Posts: 97
Joined: Sat Sep 13, 2014 12:46 am

missing blob for item

Postby umarzuki » Sun May 04, 2014 7:24 pm

Tried copying individual message with same message number as missing blob error for a user - still have missing blob message

I also tried copying synching from hsm to store - still have missing blob message
umarzuki
Advanced member
Advanced member
Posts: 97
Joined: Sat Sep 13, 2014 12:46 am

missing blob for item

Postby umarzuki » Mon Aug 25, 2014 12:01 am

if anybody is interested, this is how to fix it per mailbox basis
# su - zimbra
Find mailbox_id
$ zmprov gmi user1@domain.com

mailboxId: 692

quotaUsed: 1028382663
Find mboxgroup
$ expr 692 % 100

92
$ zmblobchk -m 692 start > /tmp/blob_user1


$ cat /tmp/blob_user1 | grep -i hsm | grep "incorrect revision" | awk -F '{print $4}' | sed -e 's/,//g' > /tmp/bloblist_user1
In this case I moved from secondary volume 3 to 4. Now to fix it in database:
$ for id in `cat /tmp/bloblist_user1 `; do mysql mboxgroup92 -e "update mail_item set locator=4 where locator=3 and mailbox_id=692 and id=$id" ; done

Return to “Administrators”

Who is online

Users browsing this forum: MSN [Bot] and 6 guests