csync2 "Received record packet of unknown type 83"

By: Gene Liverman user 30 Aug 2016 at 2:45 p.m. CDT

5 Responses
Gene Liverman gravatar
I have gotten a cluster setup but now am having trouble getting the files to sync from one server to the other. It appears every time a file needs to be sync'ed I get something like this: ``` Connecting to host gluu-idp-t02.westga.edu (SSL) ... Bound to 10.10.5.129:0 as gluu-idp-t01.westga.edu. Connect to 10.10.5.130:30865 (gluu-idp-t02.westga.edu). Updating /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml on gluu-idp-t02.westga.edu ... Received record packet of unknown type 83 File stays in dirty state. Try again later... Connection closed. Finished with 2 errors. ``` After seeing this, I went back and followed this part of the cluster guide that I had not done initially: 1. Comment out auto younger; string in csync2.cfg on both nodes to disable autoresolution of conflicts 2. Run # csync2 -crvvv -N gluu-idp-t01.westga.edu / on the 1st node 3. Run # csync2 -crvvv -N gluu-idp-t02.westga.edu / on the 2nd node 4. Previous commands did initial scan and filled metadata database. Now run # csync2 -xrvvv -N gluu-idp-t01.westga.edu / on the 1st node. That will try to sync files with the 2nd node, and most likely will fail to replicate all files due to some conflicts. 5. You should be now in a state of conflict, as certain files in directories to be synced differ between nodes and tool can't decide which to prefer. Run this # csync2 -frvvv -N gluu-idp-t01.westga.edu / on the 1st node to mark its files that are still in dirty state as the ones that will win any conflict next time. 6. Run # csync2 -xrvvv -N gluu-idp-t01.westga.edu / on the 1st node to complete your initial sync. Now all your 2nd node's directories covered by csync should be identical to the 1st node's. At the end of #6 I got this in my terminal: ``` [.... lots of stuff before this ....] Connecting to host gluu-idp-t02.westga.edu (SSL) ... Bound to 10.10.5.129:0 as gluu-idp-t01.westga.edu. Connect to 10.10.5.130:30865 (gluu-idp-t02.westga.edu). Local> SSL\n Peer> OK (activating_ssl).\n response from peer(<no file>): gluu-idp-t02.westga.edu [7] <- OK (activating_ssl). ASSERT: extensions.c:65 ASSERT: dn.c:310 ASSERT: dn.c:420 ASSERT: x509.c:533 ASSERT: extensions.c:65 ASSERT: gnutls_constate.c:586 ASSERT: gnutls_buffers.c:1104 ASSERT: gnutls_buffers.c:1104 ASSERT: extensions.c:65 ASSERT: gnutls_buffers.c:1104 ASSERT: gnutls_buffers.c:1104 ASSERT: extensions.c:65 sign handshake cert vrfy: picked RSA-SHA256 with SHA256 ASSERT: gnutls_buffers.c:1104 SQL: SELECT certdata FROM x509_cert WHERE peername = 'gluu-idp-t02.westga.edu' SQL Query finished. Peer x509 certificate is: 308201FB3082016402090098DFEDFCEF0BE3B9300D06092A864886F70D01010505003042310B30090603550406130258583115301306035504070C0C44656661756C742043697479311C301A060355040A0C1344656661756C7420$ 36F6D70616E79204C7464301E170D3136303833303138323535375A170D3138303432323138323535375A3042310B30090603550406130258583115301306035504070C0C44656661756C742043697479311C301A060355040A0C1344656661756C7420436F6D706$ 6E79204C746430819F300D06092A864886F70D010101050003818D0030818902818100B97E80E59F0F2A04737CCE7D8EA7212AA56B6819D40A367E32FCA1E6ED5A0AE380C5FD57E739870CB69ECABC3FCCEF42477770508C929F4F3057E5A193938E61DAED773219$ A4190984B268B883B8F1919E0CFCD0FA279899DCC34715714E9B89417CFA4B3F3A0658197931F2C0ADAF1DC0D6895E7A97ACDD01604363C8CF0270203010001300D06092A864886F70D01010505000381810034C8A0DDF0988BDEE2957084FC212F2D6D7FE6F0396$ 2879E884028D370709F76D210FAFC657358CE83098A45C62CA2A08AFEED48C991C543C39ABF444802FA5F3ED0F77B6C63EA36AC6C8A4307FC6A2754CC08C5CDF07CCFEE3BEFA3530627B3FEAAD42D584FC641EA553E3CEA0E23453E21282E69B8A54047D950F560D$ BDF Local> CONFIG \n Peer> OK (cmd_finished).\n response from peer(<no file>): gluu-idp-t02.westga.edu [1] <- OK (cmd_finished). check: /usr/local/etc/csync2/csync2.cfg 22, /usr/local/etc/csync2/ 22, 22. check: /opt/idp/ssl 9, /usr/local/etc/csync2/ 22, 1. check: /opt/idp/metadata/1A75F51273A410CF000265FFD9FC0006BA4AFFCA-sp-metadata.xml 18, /opt/idp/ 9, 9. check: /opt/idp/metadata/1A75F51273A410CF000265FFD9FC00067CA98BF9-sp-metadata.xml 18, /opt/idp/metadata/ 18, 18. check: /opt/idp/metadata/1A75F51273A410CF000265FFD9FC00067990496B-sp-metadata.xml 18, /opt/idp/metadata/ 18, 18. check: /opt/idp/metadata/1A75F51273A410CF00017EA62F19-idp-metadata.xml 18, /opt/idp/metadata/ 18, 18. [...... lots of lines like this .......] check: /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.xhtml 41, /opt/apache-tomcat-7.0.65/webapps/oxauth/ 41, 41. check: /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml 41, /opt/apache-tomcat-7.0.65/webapps/oxauth/ 41, 41. Dirty item /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml gluu-idp-t01.westga.edu 1 Local> HELLO gluu-idp-t01.westga.edu\n Peer> OK (cmd_finished).\n While syncing file /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml: response from peer(/opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml): gluu-idp-t02.westga.edu [1] <- OK (cmd_finished). Match (+): /opt/apache-tomcat-7.0.65/webapps/oxauth/*.xml on /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml Updating /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml on gluu-idp-t02.westga.edu ... Local> FLUSH yRRFSG3RiKWK8OhN298mkNM6USDVUvUMx3a4tPDqF3.18hUty1i9w79KtqaHvUCW /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml\n Peer> OK (cmd_finished).\n While syncing file /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml: response from peer(/opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml): gluu-idp-t02.westga.edu [1] <- OK (cmd_finished). Local> PATCH yRRFSG3RiKWK8OhN298mkNM6USDVUvUMx3a4tPDqF3.18hUty1i9w79KtqaHvUCW /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml\n ASSERT: gnutls_record.c:592 Received record packet of unknown type 83 ASSERT: gnutls_record.c:1096 ASSERT: gnutls_record.c:1179 ASSERT: gnutls_record.c:1431 Peer> While syncing file /opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml: response from peer(/opt/apache-tomcat-7.0.65/webapps/oxauth/authorize.page.xml): gluu-idp-t02.westga.edu [9] <- Connection closed. File stays in dirty state. Try again later... Local> BYE\n ASSERT: gnutls_record.c:465 ASSERT: gnutls_record.c:1406 Peer> response from peer(<no file>): gluu-idp-t02.westga.edu [9] <- Connection closed. ASSERT: gnutls_buffers.c:653 ASSERT: gnutls_record.c:1406 ASSERT: gnutls_record.c:334 SQL: SELECT command, logfile FROM action GROUP BY command, logfile SQL Query finished. SQL: COMMIT Connection closed. Finished with 2 errors. ``` I have not sync'ed anything from `/etc/certs/` yet as it didn't seem to be needed but can if need be. Thanks for your time!
CentOS72
closed

Answers

By Aliaksandr Samuseu staff 30 Aug 2016 at 3:22 p.m. CDT

Copy
Aliaksandr Samuseu gravatar
Hi, Gene. Please check [this recent ticket](https://support.gluu.org/integrations/errors-when-running-csync2-xv-in-clustered-configuration-of-gluuce-3119/#at14099) on the similar problem. It seems you've encountered a known issue and will need to add `-l` flag to the xinetd's csync's configuration Also, please check [my recent response](https://support.gluu.org/installation/dsreplication-initialize-fails-3027/#at14248) to you in your previous ticket, you could have configured it incorrectly. Best regards, Alex.

By Gene Liverman user 30 Aug 2016 at 3:37 p.m. CDT

Copy
Gene Liverman gravatar
Adding the `-l` and then rerunning `csync2 -xrvvv -N gluu-idp-t01.westga.edu /` got the initial sync to work... what does the `-l` do? I didn't see it in the man page. Also, does that option need to stay in the config for good?

By Aliaksandr Samuseu staff 30 Aug 2016 at 3:48 p.m. CDT

Copy
Aliaksandr Samuseu gravatar
It was a workaround one of the maintainers provided to an inquiry in their mail lists. AFAICR, it has something to do with some console output streams being mixed into the stream used by csync's sync protocol. That key prevents it from happening.

By Aliaksandr Samuseu staff 30 Aug 2016 at 4:01 p.m. CDT

Copy
Aliaksandr Samuseu gravatar
Btw, make sure that crond is running in the container when you'll start to automate it, you may need to set it to startup on each Gluu service's restart (the same way as you set any service to start up on boot).

By Gene Liverman user 02 Sep 2016 at 12:02 p.m. CDT

Copy
Gene Liverman gravatar
The `-l` did seem to fully resolve this. This ticket may now be closed. Thanks again!

Post an answer