A customer was seeing some users marked as inactive when using the All Connections search but when clicking through to the user’s profile they were active and active in communities and all over areas of Connections.

Looking into the database tables I found that the “state” of these users were correct, for example, in the EMPINST.GIVEN_NAME a particular user had a PROF_USRSTATE equalling 0 which means he’s active. In the EMPINST.EMPLOYEE table affected users had their email addresses which are normally removed when they are made inactive.

After some investigation I found that by simply activating them would mark them as active without any changes to the various tables in PEOPLEDB.

This got me thinking that the problem was an index issue and without knowing how many people were affected I suggested that the customer recreate the index. I provided them with steps of how to back it up, delete it from the file system and create a new one but even after the index created users were still showing as inactive.

Thankfully I had access to the Control Center and decided to look at all the PEOPLEDB tables, none were useful. I then started looking at the next logical database, HOMEPAGE. Interestingly, in the HOMEPAGE.PERSON table there is a column called STATE and the affected users had a value of 1 in that column. Running the following command changed the STATE to 0 and then searching for the user using the All Connections search showed him as active.

wsadmin.bat -lang jython -port 8879
execfile(“D:\IBM\WebSphere\AppServer\profiles\AppSrv01\config\bin_lc_admin\profilesAdmin.py”)
ProfilesService.activateUserByUserId(“E4BB9E9D-43D3-B5A4-8025-7433003EFACB”,email=”ben.williams@acme.com”, displayName=”Ben Williams”)

Going further I had to identify how many users were affected and the below query gave me the column values I needed to activate users who were marked inactive.

SELECT PERSON.DISPLAYNAME, PERSON.EXID, PERSON.USER_MAIL_LOWER FROM HOMEPAGE.PERSON AS PERSON WHERE PERSON.USER_MAIL_LOWER IS  NOT  NULL  AND PERSON.STATE = 1

The above query helped but there were still a number of users that were not in HOMEPAGE.PERSON and are in PEOPLEDB. These people were showing as inactive in the All Connections search BUT had never logged into Connections and hence their email addresses had not populated the HOMEPAGE database. These I had cross referenced manually as I don’t have the know how to build a query over different databases :(

There is a bit of history here. The customer is importing users manually via populate_from dn_file because they want to control who is being added until their Connections 4 environment has been signed off for production and a custom TDI assembly line has been created. A few months ago sync_all_dns was run accidentally which meant that a 1000 or so users had to be identified and then removed from Connections. I believe that this (in some) way caused these problems.

Last week I started seeing problems with the s2s federation with Google, errors like the following were appearing in the SystemOut.log.

[12/03/13 17:24:43:112 GMT] 0000003b LoggableInput 3 com.ibm.rtc.gateway.xmpp.util.LoggableInputStream read(byte b[], int off, int len) XMPP logging < : length=203 msg:<presence to=”ben.williams@chooseportal.com” from=”hoagieben@gmail.com” type=”error”><error code=”503″ type=”cancel”><service-unavailable xmlns=”urn:ietf:params:xml:ns:xmpp-stanzas”/></error></presence>

It looked like Google were building white lists with approved domains. A friendly guy in IBM Support told me that there have been some changes in the Google policies and IBM are trying to “reach out” to Google in order to get a formal response and rectify these problems.

I read an article posted today called Google expected to unify chat under the name Babble and am thinking whether this is the beginning of the end for Gateway and Google federation?

LDAP and Sametime doesn’t always sit well together. There are various things you can do to try and improve LDAP performance, many of which are documented in Best Practices for using LDAP with Lotus Sametime.

STResolve seems to be the main contributor to these problems especially with the latest version of the Notes client which wants to resolve the email address of each email in the view to see whether the user is on line. We all know that joebloggs@hotmail.com does not exist in your LDAP so how to stop the Community server sending this to LDAP only to be told that it doesn’t exist?

Well, up until this morning the only way I believed to do this was by way of desktop policies controlling the managed settings of your Notes client as detailed in Optimizing Name Lookup: Clients.

This morning I was sent a URL to Excluding certain domains from user and group directory lookups using Whitelists and Blacklist which says that ST_RESOLVE_BLACKLIST or ST_RESOLVE_WHITELIST can be added to the [CONFIG] section of the sametime.ini. This will effectively do what was previously possible in the client via the plugin_customization.ini.

The document was edited in May last year and when searching for the parameters only a few hits appear on Google so I wonder how well known they are?

I presume the parameter will stop the Community server passing the search filters via STResolve to LDAP but will not stop them being sent to the Community server in the first place. Nonetheless this should dramatically improve STResolve performance.

I will implement and see what happens in the STResolve*.txt trace files, hopefully I will see much less and my customers will be much happier!

 

 

A customer was having a problem with notifications sent to someone using a mobile device logged into an STProxy server. The name of the server was not “Server” as it is normally but rather a random other server. There were two approaches, continue fixing it or remove the “Server” name and replace it with the name of the recipient which personally sounded a far better option.

The (always) helpful Cormac O’Leary from the Sametime PMR team assisted and liaised with L3 and provided me with a new cumulative hot fix. Once installed I had to add  to edit stproxyconfig.xml, located in AppServer/profiles/<Profile_Name>/config/cells/<Cell_Name>/nodes/<Node_Name>/servers/STProxyServer/stproxyconfig.xml

Add the following values to the <configuration> element. If a <mobile> element is already present, add the <disableSystemNoficiations> element to that existing element.

<mobile>

<disableSystemNotifications>true</disableSystemNotifications>

</mobile>

Now when an IM is sent to a using on a mobile device the name of the announcement is not “Server” as it is currently but rather the recipient’s name.

new STProxy announcement

A customer had a particularly difficult and awkward problem. A number of user reported that they could not see the status updates of others users who were part of the same network.

This seemed to affect a handful of users in the following pattern. User A cannot see User B’s status updates whilst User C can see User B’s updates. The problem is associated with User B. To c0mplicate things further User A shares a network with User B but does not follow him whilst User C also shares a network but does follow User B. So if you follow User B you get to see their status updates in your Status Updates. You should not need to follow the user to see their status updates the fact that you’re sharing a network should be sufficient.

I logged a PMR which thankfully found it’s way into the ever helpful hands of David McCarthy. An ISSC colleague mentioned that this had been seen in IBM’s deployment of Connections (w3) by Luiz Benietz and others so getting to the bottom of this would kill two birds with one stone.

IBM had me extract data from PEOPLEDB and HOMEPAGE databases using the DB2 commands below to provide the PERSON_ID of the users within HOMEPAGE to make sure it matches with the PROF_GUID within PEOPLEDB which for User B is E833339D-AED2-425F-8600-64CEFD85A3A5.

Comparison of NR_NETWORK showed that the data in there was correct.

PEOPLEDB.EMPLOYEE
PROF_GUID = E833339D-AED2-425F-8600-64CEFD85A3A5

HOMEPAGE.PERSON
PERSON_ID = 0bdd7880-8d9a-4fe1-ab13-77c54bb429bc
EXID = E833339D-AED2-425F-8600-64CEFD85A3A5

Using the PERSON_ID I was then able to retrieve data from the HOMEPAGE.NR_NEWS_STATUS_NETWORK table which details all the status updates posted by that PERSON_ID which in the HOMEPAGE.NR_NEWS_STATUS_NETWORK table is identified by ACTOR_UUID. This table also shows the READER_ID (detailed below) as well as a BRIEF_DESC column which shows the first 200 (or so) characters of the status update.

db2 “select * from HOMEPAGE.NR_NEWS_STATUS_NETWORK where ACTOR_UUID = ‘0bdd7880-8d9a-4fe1-ab13-77c54bb429bc‘”

HOMEPAGE.NR_NEWS_STATUS_NETWORK
ACTOR_UUID = 0bdd7880-8d9a-4fe1-ab13-77c54bb429bc
READER_ID = PERSON_ID of the person that is consuming the status updates ie those listed in the READER_ID column should be able to see the status updates.

To find out the names of the people who can read the updates you will need to use the READER_ID from HOMEPAGE.NR_NEWS_STATUS_NETWORK and find out the name of the user from the HOMEPAGE.PERSON by using the READER_ID converted to PERSON_ID.

db2 “select DISPLAYNAME from HOMEPAGE.PERSON where PERSON_ID = ’714fbcd9-1740-477b-972d-a6456a35e4ea’”

I contacted the user and they could indeed see User B’s status updates.

To see how many people User B has in their network the following command was run which produced a value of about 140.

db2 “select count(*) from HOMEPAGE.NR_NETWORK where PERSON_ID = ‘0bdd7880-8d9a-4fe1-ab13-77c54bb429bc‘”

IBM asked for the following trace to be enabled whilst User B posted a status update and in the trace.log an exception was seen during the processing of one of the threads.

*=info: java.sql.*=all: com.ibatis.sqlmap.*=all: com.ibm.lconn.homepage.*=all: com.ibm.lconn.news.*=all

It was suggested that 3.0.1.1 be installed but that would not be possible. The following fixes were applied which aimed to improve that code path to handle the case where the use records in the Homepage database are out of sync.

3.0.1.0     LO63965     VBUT8LNH89     Enhancement to News aggregation service for better handling of exceptional cases where user records become inconsistent with Profiles

3.0.1.0     LO66468     SCRD8QCDDK     Profiles Sync issue. Too many requests made on Event consumption to Profiles colleague feed

Unfortunately the problem was still apparent after the iFixes so more debugging was enabled, this time com.ibm.lconn.news.*=all: com.ibm.lconn.events.*=all

Thankfully this time the trace.log provided much more valuable information.

[30/05/12 16:14:59:301 BST] 00000067 StatusUpdateS 3 com.ibm.lconn.news.service.impl.StatusUpdateService insertStatusUpdate Going to insert into db with a batch process this number of records: 144 and BUCKET_SIZE_BATCH_INSERT: 100
[ ... snip ...]
[30/05/12 16:14:59:308 BST] 00000067 NewsStatusNet 3 com.ibm.lconn.news.data.dao.impl.ibatis.NewsStatusNetworkDao insertBatch – doInSqlMapClient rowsaffected 100
[30/05/12 16:14:59:308 BST] 00000067 NewsStatusNet < com.ibm.lconn.news.data.dao.impl.ibatis.NewsStatusNetworkDao insertBatch RETURN 100
[30/05/12 16:14:59:312 BST] 00000067 StatusUpdateS < com.ibm.lconn.news.service.impl.StatusUpdateService insertBatch RETURN 100
[ Here - it's missing the insert for the remaining 44 records ]
[30/05/12 16:14:59:312 BST] 00000067 StatusUpdateS 3 com.ibm.lconn.news.service.impl.StatusUpdateService insertStatusUpdate Going to insert into db with a batch process this number of records: 3 and BUCKET_SIZE_BATCH_INSERT: 100″

The response from L3 was:

The root cause is a coding error in a routine performing batch insert. The issue should be reproducible for people in the following conditions:

  • The number of people in the intersection of the sets (colleagues, followers)  for the author of the status update is > 100
  • The number of colleagues – number of people in the intersection is > 100 (this is what happen for this very specific PMR)
  • The number of followers – number of people in the intersection is > 100.

So the problem we are seeing is because the user had more than 100 people in their network and/or followers and the code would not add the 44 others to the database. pressing further I asked for an iFix for this which is too complicated and the only option is to go to 3.0.1.1 CR1.

“Upon further review – the issue appears to be indirectly fixed in 3.0.1.1….We performed a code review in 3.0.1.1 – this code path was entirely re-written. The final section of the batch is inserted in the new code in 3.0.1.1.”

Up until this point the problem still hasn’t been resolved as updating Connections has not been planned yet but IBM support are fairly confident that 3.0.1.1 CR1 will iron out this. With regards to w3, David McCarthy was able to track down one user who reported the problem initially who now (after the upgrade) can see all status updates.

A customer was asking me about the daily and weekly digest notification emails that are sent out. Their query was going down a different route but to try and head off a potential question about editing the times that the notifications are sent I read around Customizing IBM Lotus Connections 3.0 email digests which was useful but did not give me the information I wanted.

Remembering that I had read something related to this in the last couple of months I went through my feeds and found that one of my ISSC colleagues Dave Hay had already blogged about it in a post IBM Connections 3.0.1 – Email Notifications

“There is no set ‘time’ for the digests, they are processed/delivered throughout the day.

The users in the system are split into 20 tranches.
Each hour, a task runs to pick up the next applicable tranche (i.e. one whose daily digest was not processed in last 24 hours, or whose weekly digest has not run in 168 hours).

You could change the time of the hour that the task runs, but this will not give the behaviour you are looking for.”

Thanks again Dave!

This was seen the other day when attempting to execute a Connections script once connected to wsadmin:

wsadmin>execfile(“/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/config/bin_lc_admin/wikisAdmin.py”)
WASX7015E: Exception running command: “execfile(“/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/config/bin_lc_admin/wikisAdmin.py”)”

; exception information:
 com.ibm.bsf.BSFException: exception from Jython:
Traceback (innermost last):
  File “<input>”, line 1, in ?
  File “/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/config/bin_lc_admin/wikisAdmin.py”, line 19, in ?
NameError: java

This is because the default langauge for wsadmin is JACL and the script is JYTHON.

Ensure when starting the wsadmin utility, “-lang jython” is added to the command line.

I was stuck on this for a little while today when trying to install TDI 7 (Tivoli Directory Integrator) as part of a Connections installation on (unsupported) CentOS.

After extracting C1IF4ML and running ./launchpad.sh up popped the installation web page but when selecting “Tivoli Directory Integrator 7.0 Installer” I was presented with the following error:

“Bundled JRE is not binary compatible with the host OS/Arch or it is corrupt. Testing bundled JRE failed.”

After a bit of reading around people suggested it was due to the /tmp directory not having adequate space for the installation, but there seemed to be enough based on IBM’s disk space requirements.

Eventually I ran the following command which worked after creating another temporary directory:

./install_tdiv70_linux_x86_64.bin -is:tempdir /opt/temp

Follow

Get every new post delivered to your Inbox.

Join 55 other followers