On Wed, Mar 18, 2015 at 09:45:29AM +0800, Ian Kent wrote:
On Tue, 2015-03-17 at 13:04 -0700, Paul B. Henson wrote:
> > From: Ian Kent
> > Sent: Tuesday, March 17, 2015 6:37 AM
> >
> > If an indirect map has the "browse" (or ghost) option the entire map
> [...]
> > Direct maps must always be read completely because the direct mount
>
> Ah, yes, we do not use the ghost option or direct maps, so I did not
> consider that use case.
LOL, and understandably with that many map entries.
>
> > I think we discussed this at the time and, given the cases, decided it
> > was best for sss to read the entire map and cache it since it might need
> > to be able to supply the entire map and can't know if that will be the
> > case.
>
> On the other hand, there are cases such as mine where not only is
> reading the entire map unnecessary but inadvisable and practically
> infeasible 8-/. For my deployment, using sssd to access autofs data in
> ldap rather than autofs accessing it directly would drastically
> increase the load on the LDAP server, which seems completely cross
> purpose to the existence of sssd. Currently there is sporadic load as
> individual entries (probably no more than 10-1000 or so in a given day,
> depending on the server) are looked up. With sssd, all 120000 entries
> would be pulled over on an ongoing basis throughout the day (I'm not
> sure how often it refreshes its local cache).
Indeed, IIRC my large map testing showed that somewhere less than 30000
is about the practical limit for reading the entire map. There would be
other side effects in autofs spending that much time reading the the
entire map too so I don't think it will ever be possible in your case.
>
> That's okay though, I don't really have any strong need to use sssd
> for autofs. I just thought it would be interesting to look into given
> the new RHEL6 support from the perspective of centralizing all LDAP
> lookups into one place. Assuming there are no planned changes to the
> native autofs ldap support, that meets our needs fine.
I can't change the way the lookup and read map works very much
specifically because of the problems really large maps bring. If there
are any changes made they must also cover the large map requirement.
But we haven't heard from Jakub yet, it may be possible to change sss to
lookup and read the map on demand rather than all at once. Not sure
though given I'm not familiar with the wider design of sss.
Sorry, the discussion fell out of my timezone :-)
Currently we implement:
setautomntent(map_name)
endautomntent(map_name)
That select and de-select a map to work with, then also:
getautomntent(map_name, cursor, max_entries)
which is a iterator-like interface that returns up to max_entries keys
starting from cursor and finally:
getautomntbyname(map_name, key_name)
that returns data of key_name inside map_name
I'm not sure if we can optimize getautomntent() to read the next entries
on demand, I think we need to fetch the whole map there. But it would be
entirely doable to add another LDAP request that would be invoked by
getautomntbyname() and would fetch the single key.
Would that provide more efficient API towards automounter?