«DiscoVeriNg HomelessNess Volume 13, Number 1 • 2011 U.S. Department of Housing and Urban Development | Office of Policy Development and Research ...»
In many organizations, very few numbers of staff members were attempting to log on sporadically throughout the year. This system of use is contrary to the purpose of information management systems, which are designed to create a virtual network of providers who maintain up-to-date information on the clients served and services available. It may be that HMIS use among the sampled organizations is not yet at a point where usage can be evaluated. Because this study did not use a random sample, it is not possible to generalize the limited use found in this study to other communities and organizations using HMIS. In this sample, however, results suggest that implementation studies might yield more complete data when larger numbers of people in more organizations are using the HMIS.
The most important finding in the univariate and bivariate analyses was the variability in HMIS use among organizations. This finding is particularly interesting, given the stated purposes of the technology. HMIS are intended to capture most client interactions so that the providers can maintain counts of homeless and services and can provide online referrals and manage cases (HUD, 2009). This intention requires staff members to have access to the system to use it regularly.
In organizations with 30 staff members where 20 of them provide direct client care, it is expected that most of the staff members would have HMIS licenses. Moreover, if the 20 direct-care staff members are working with clients daily, it is expected that each of those staff members would log on to the HMIS daily. This assumption means that client services are being recorded immediately in the HMIS. The information is available in the database for other organizations to view, and the organization providing the service has a current count of its clients.
The study revealed several organizations using the HMIS daily. In these organizations, large numbers of staff members were licensed to use the HMIS, and they logged on to the system multiple times a month throughout the year. Other organizations showed markedly different use patterns.
These organizations had similarly large client volumes and provided the same types of services, but the study found that only two or three staff members had logged on to the HMIS during the year, and they had done so only once or twice. A third type of organization evident in the study’s findings had a very small client base and a single HMIS user who logged on infrequently.
One possible explanation for the difference in use may have been a difference in services provided.
It is logical that organizations providing emergency shelter might interact with the HMIS differently than organizations providing permanent housing or ancillary services. Emergency shelter providers have large nightly client caseloads, as many as 200 per night, and provide short-term basic services, such as temporary shelter, food, and clothing. Permanent housing facilities may have smaller caseloads and provide more long-term, comprehensive services, such as mental and physical health care and substance-abuse counseling. The study, however, did not show statistically significant differences in use among the different types of homeless services. Instead,
differences in use were statistically significant based on the CoC. This finding reinforces the study’s theoretical argument that HMIS use is partly a function of community norms, which may be a function of organizational culture.
The results also support previous studies showing that multisite program evaluations that assess overall effectiveness often mask significant variability between sites (Becker et al., 2000; Seltzer,
1994) and that the influence of organizational attributes varies according to types of service providers (Sosin, 2001). The variability within a CoC may challenge efforts to coordinate service provision. This coordination often requires standardizing certain procedures across organizations, such as using the HMIS for a common intake procedure. Program planners, however, may only achieve standardized data collection and care coordination through adaptive organizational implementation procedures. One example is providing site-specific training that modifies the HMIS to the unique physical environment of each organization, its established business processes, and the unique needs of its users.
Limitations The study has several limitations that are common among organizational research, including small sample size, measurement ambiguities (Wilderom, Glunk, and Maslowski, 2000), and a nonrandom sample (Poertner, 2006). The small sample limited the study from including more covariates that arguably could have affected technology use, such as years of work experience and age.
In addition, it was challenging to identify one measure of usage that represents all types of interactions with the system, because organizations and staff members use the HMIS differently. This study chose to use logon attempts as a proxy indicator of use to maximally capture user access of the system. In the current study, the concept of HMIS use was developed in consultation with the staff members at the ETCEH. Individually, each alternative measure was ruled too exclusive.
Some staff members with HMIS licenses do not enter new client data at all. Other staff members enter only new client assessments and do not record case notes or services provided. Consider the following examples of usage behaviors. Case managers who work intensively with a small number of clients may log on only once or twice a week. When they log on, they may spend a large amount of time writing case notes or completing lengthy assessments about a single client. In contrast, organizations providing emergency shelter often employ overnight staff members. These staff members may be assigned a large number of paper-based client assessments and asked to transfer the information to the HMIS. They will log on nightly and enter 200 client assessments. Finally, some administrators log on once a month to run a report for funders or a board of directors.
The frequency of logon attempts was considered the most inclusive single measure of HMIS use, considering the variety of interaction patterns. Ideally, the study would have triangulated measures to capture usage as fully as possible. This triangulation was not considered possible at the time of the study, though, because of the implementation stage. Ironically, the study was designed to examine HMIS use, but it discovered that usage is so irregular that it poses significant challenges to measurement and study. Many staff members, who are trained to electronically record case notes and services provided, do not and are not required to do so by their organizations. Some organizations still use dual recordkeeping systems on paper and in the HMIS. Staff members record client interactions on paper and then transfer large volumes of paper-based assessments to the HMIS at
a single time. Consequently, adequate data were not available for some of these measures. One organization said it provided services to 2,000 clients annually, but it had entered data for only 10 clients into the HMIS during the previous year. This organization did not provide any reason for this disparity. It may be that this organization only began using the HMIS recently and had not had time to enter all of the clients. In addition, the organization may be overestimating the number of clients served. It is this sort of ambiguity and inaccuracy in data based on self-reported recollections that HMIS are designed to minimize.
Moreover, irregular usage distorts measures of system use. Having basic information for all clients stored in the HMIS does not mean that all staff members are logging on regularly or as required by their job responsibilities. Episodic HMIS data entry does mean that the client information is not consistently available in real-time for different organizations and case managers to access.
Thus, the measure of logon attempts in this study was used with the recognition of its limitations.
For instance, reliance on this measure may have overestimated use by some individuals who log on frequently but do not input large amounts of data. In contrast, it may have underestimated use by other individuals who log on infrequently but input large amounts of detailed data, such as case notes and lengthy assessments. The use of this proxy measure of HMIS use may explain why the study failed to find a direct relationship between organizational culture and technology use. For instance, most organizations may be participating at a minimum level, but organizations with specific culture profiles will be more likely to transfer all datakeeping to the electronic system quickly and comprehensively. Thus, a measure of data quality might have demonstrated a stronger relationship between organizational culture and technology use.
In addition, the study’s findings cannot be generalized to all homeless-services providers who are using information management systems in the United States. This study, which to date is the largest of its kind, included only 24 organizations in four CoCs across two states. Results may have overestimated overall levels of proficiency while underestimating rigidity and resistance. It seems logical that organizations willing to participate in research compared with those who declined would be more likely to value proficiency but would be less rigid and resistant. In addition, lack of participation by some organizations may have underestimated the variance among organizations in their use of the HMIS. Perhaps those organizations that chose not to participate are the few organizations that are choosing not to use an HMIS.
Finally, the small number of men compared with the number of women in the sample may have influenced the results. This disparity was inevitable, considering that women dominate nonprofit services. In fact, this sample is consistent with a national study of social workers, which found that 80 percent are female (Whitaker and Arrington, 2008). Still, the interaction effect between men and proficiency may have been overestimated due to the small number of men in the sample.
Implications This study considers the pace of implementation and serves as a template for future studies examining more nuanced questions of quality and substance in use. It offers an empirical glimpse into the reality of how organizations and staff members are using HMIS as a tool of service on a daily basis. Results stress the need for administrators to examine the goodness of fit between organizations and new technologies before implementing them. As Weisman et al. (2002: 63)
argue, “...the utility of technology…is in its day-to-day workability…” in the organizations. This study shows that interactions between individual-level and organizational-level characteristics can complicate implementation of broad-based systems such as HMIS. A formulaic and rigid approach to implementation may be unsuccessful and wasteful. This study confirms the complexity of the process of diffusion of technology, especially in human service organizations. It is the result of dynamic interactions among technology, individuals, and organization; successful implementation efforts must consider the three levels simultaneously.
Communities may have more success with HMIS implementation if they provide custom implementation strategies for organizations. It may be useful to conduct preliminary organizational culture audits such as was done in this study to understand the unique strengths and challenges of each organization. Some organizations will show high levels of proficiency with low levels of resistance to changing work practices. Other organizations will show low levels of proficiency and high levels of resistance. The latter organizations may benefit from more intensive training and support than that provided to the former organizations.
The study also suggests that organizations should reconsider how they are using the HMIS. Those organizations that are using disproportionate data entry systems are creating significant burdens for individual staff members. Moreover, they are not entering client services in the real time.
Instead, staff members maintain paper recordkeeping systems, and one or two staff members enter these paper records into an HMIS retroactively. Such a system prevents organizations from using the HMIS as a resource for care coordination. Data in the HMIS must be current so that case managers at different organizations can use the system as a source of information for past services and for current resources available when making decisions about future referrals.
In addition, the results suggest that organizational environments may be an area for intervention.
Glisson et al. (2008) have designed an intervention called ARC (Availability, Responsibility, and Continuity) that aims to improve the culture and climate of human service organizations. The goal is to improve the overall functionality of organizations—reduce employee turnover rates, increase morale, and enhance functionality such as lower resistance and higher proficiency. Results of a national pilot study with this intervention show positive results.
Ultimately, it is essential to understand what organizational culture and technology mean for clients who are receiving housing services. Although the results from the current study are limited, they represent a small component of a larger system, which can be revealed through subsequent research. This study begins to show how organizational culture can affect service provision by demonstrating that one aspect of culture—proficiency—appears to change how men are beginning to use a this service technology among homeless-services providers.
The study has implications for social policy as well as for practice and research. The target of the study—HMIS use—stems from a federal HUD mandate, and much of its funding comes from federal grants. HUD policymakers can use the study’s results to determine the degree to which the technology is being used and how organizational culture may affect access. This study begins to show that the homeless-services provider setting has a significant asset regarding HMIS use. In the sample, organizations showed higher than average levels of proficiency; they value competency and invite innovation if it improves client services. Staff members in proficient organizations